November 2007 - Darwin-dev - lists.apple.com

pthread_cancel and C++ destructors
by Ethan Tira-Thompson 29 Nov '07

29 Nov '07

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… This email sent to site_archiver(a)lists.apple.com test.cc

1 0

Re: dbm_store() returns -1, but no specific error is reported
by Kevin Van Vechten 29 Nov '07

29 Nov '07

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com On Nov 28, 2007, at 6:05 PM, Anton Rang wrote: The DBM implementation in Libsystem does indeed come from Libc. - Kevin _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… So __add_ovflpage is failing. But I'm not sure which source this is built from. I thought it was the BerkeleyDB-15 package, but that doesn't seem to have this function (it has __ham_add_ovflpage instead)...ah, we're working with LibC-548 instead. [Is the BerkeleyDB code shipped somehow?] The BerkeleyDB project produces a static library that is used by CyrusIMAP and OpenLDAP. It is not otherwise available on a Mac OS X system. This email sent to site_archiver(a)lists.apple.com

1 0

Re: dbm_store() returns -1, but no specific error is reported
by Jordan K. Hubbard 29 Nov '07

29 Nov '07

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com On Nov 28, 2007, at 6:05 PM, Anton Rang wrote: - Jordan _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… This looks like pretty old DBM code. I'm not sure I'd rely on it for larger databases. You could probably use the btree instead of hash code (using the dbopen interface) and at least get a larger file. But I'd still wonder if Apple's shipping a more up-to-date database somewhere.... Got it in one. This is VERY old DBM code with known flaws like this. Unfortunately, later DBM code has the sleepy cat "conditional license" which prohibits us from using it for lots of reasons I won't go into here, so that's the only DBM code available. There's always sqlite3... This email sent to site_archiver(a)lists.apple.com

1 0

Re: dbm_store() returns -1, but no specific error is reported
by Anton Rang 29 Nov '07

29 Nov '07

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com On Nov 28, 2007, at 10:45 AM, Christian Moen wrote: darwin-dev, I'm experiencing strange behaviour with DBM on a Mac OS X 10.5.1 (both on G5 and Intel) and it would be great to get some feedback from the list on what the problem might be. Running a quick dtrace script (attached) reveals that this fails due to an overflow page not being added: 1 <- memcmp 10001 1 -> __addel 1788 1 -> __add_ovflpage 1789 1 <- __add_ovflpage 7068 1 <- __addel 7067 1 <- hash_access 7005 1 <- dbm_store 7896 So, looking at http://www.opensource.apple.com/darwinsource/10.5/Libc-498/db/hash/hash_pag… -- Anton _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… This email sent to site_archiver(a)lists.apple.com So __add_ovflpage is failing. But I'm not sure which source this is built from. I thought it was the BerkeleyDB-15 package, but that doesn't seem to have this function (it has __ham_add_ovflpage instead)...ah, we're working with LibC-548 instead. [Is the BerkeleyDB code shipped somehow?] it appears that overflow_page() must be returning 0, indicating that there are no usable free pages in the database file. There are some strange (to my eye) limits on which pages can be used for overflow pages. This looks like pretty old DBM code. I'm not sure I'd rely on it for larger databases. You could probably use the btree instead of hash code (using the dbopen interface) and at least get a larger file. But I'd still wonder if Apple's shipping a more up-to-date database somewhere.... test.d

1 0

Re: Assembler Question
by Terry Lambert 29 Nov '07

29 Nov '07

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com -- Terry _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… On Nov 28, 2007, at 11:44 AM, John Scheiderer wrote: I want to learn Assembler on the Mac Intel platform, but I'm having a hard time finding any tutorials targeted for the Mac. I have an iMac G5 (which I know won't work), but I did setup a Darwin x86 PC that has the 'as' compiler, and I also installed 'nasm' (compiled successfully) from the most recent sources. The reason I installed 'nasm', is I noticed that Leopard now includes it in the Xcode install on my G5 machine. I found some Hello World examples on the web, but none seem to work - they compile and link ok, but get error messages like "Bus error", etc. As a test, I took an Objective-C program, and compiled it with gcc using the -S flag, which produced assembler code. When compiling the .s file using gcc, it produces an executable, so I suspect the examples I'm finding aren't error free. Search <http://developer.apple.com> for "Mac OS X ABI Function Call Guide". By your question, though, it sounds like you are a compiler writer, and that may not be quite enough. You are most likely missing the setup for your main program, which has to push the stack base then call into itself (effectively) si the ret is to the caller of _main(). You may also be unaware that your linkages to libc need to be implemented as jump table entries so that dyld, which is loaded before the code starts running and invokes the crt1 statup code (see the "cctools" project in the Darwin sources). This is easily visible in the assembly from a -S compile of a small "hello world" example that just outputs and exits. This email sent to site_archiver(a)lists.apple.com

1 0

Assembler Question
by John Scheiderer 28 Nov '07

28 Nov '07

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com Any help is appreciated! Thanks, John _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… This email sent to site_archiver(a)lists.apple.com I want to learn Assembler on the Mac Intel platform, but I'm having a hard time finding any tutorials targeted for the Mac. I have an iMac G5 (which I know won't work), but I did setup a Darwin x86 PC that has the 'as' compiler, and I also installed 'nasm' (compiled successfully) from the most recent sources. The reason I installed 'nasm', is I noticed that Leopard now includes it in the Xcode install on my G5 machine. I found some Hello World examples on the web, but none seem to work - they compile and link ok, but get error messages like "Bus error", etc. As a test, I took an Objective-C program, and compiled it with gcc using the -S flag, which produced assembler code. When compiling the .s file using gcc, it produces an executable, so I suspect the examples I'm finding aren't error free. smime.p7s

1 0

dbm_store() returns -1, but no specific error is reported
by Christian Moen 28 Nov '07

28 Nov '07

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com Dkim-signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:message-id:date:from:to:subject:mime-version:content-type:content-transfer-encoding:content-disposition; bh=wnO3+VLL/84OWqtPDop+UtL4q+J4DTzrhP4Gkycr/SA=; b=EMXCINCZ9HYd/RkrubsomJWEykI1eYbmNyDPbWwnOKXJtcyx2pqZ/3L+hEQsVO8TJB4hX98py9FEPJwYa7e2YUKd0gXvRCsDKfLYKoHvcfEMpnUqSlEU8enbPEEDDXsrg1sxFUot6UmHcdvEVvgDGImCKIKzN7VS9Xzb2fSQBlY= Domainkey-signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=received:message-id:date:from:to:subject:mime-version:content-type:content-transfer-encoding:content-disposition; b=on4BonV4ieZ+817h6VraNZOSKijwZY2rRwwfWHbV+aSXoaTz6Z2LabbAYoBJ0M+YrpTTvkqCT8Te2YkdCoDe+Y0j1PCTIIlKpKb0IGfaZ0rxcGJg8Wk+U48eHrCGmweg0dUNlyAQ5P/zcBSnA9XNtrZncrXRdB7Dy1vUaX3qygo= darwin-dev, I'm experiencing strange behaviour with DBM on a Mac OS X 10.5.1 (both on G5 and Intel) and it would be great to get some feedback from the list on what the problem might be. The program below is storing 200 byte records into a hash with an incrementing numeric string key using dbm_store(). However, at insert (and key) number 232783, dbm_store() returns -1, which indicates failure. However, errno is 0 and dbm_error() also returns 0, so the error itself seems unspecified. I believe the DBM implementation on Mac OS X is Berkeley DB (http://www.opensource.apple.com/darwinsource/10.5/BerkeleyDB-15/) I've heard that some people have had problems with databases becoming corrupt after millions of store and fetch operations under heavy loads with the implementations available at around year 2000. These people also recommend against using Berkeley DB in general. Feedback on the problem with the below program and the overall suitability of the Berkeley DB implementation in Leopard is mostly appreciated. Thanks. Christian #include <stdio.h> #include <string.h> #include <ndbm.h> #include <errno.h> #include <assert.h> #define NUM_RECORDS 250000 #define MAX_KEY_SIZE 10 #define MAX_VALUE_SIZE 200 int main (int argc, const char * argv[]) { DBM *db; int i; db = dbm_open("/tmp/testdb", O_RDWR | O_CREAT, 0660); assert(db != NULL); for (i = 0; i < NUM_RECORDS; i++) { char key[MAX_KEY_SIZE]; char value[MAX_VALUE_SIZE]; datum k; datum v; int status; memset(key, '\0', MAX_KEY_SIZE); snprintf(key, MAX_KEY_SIZE, "%d", i); k.dptr = key; k.dsize = strlen(key); memset(value, 'A', MAX_VALUE_SIZE); v.dptr = value; v.dsize = MAX_VALUE_SIZE; status = dbm_store(db, k, v, DBM_REPLACE); if (status != 0) { fprintf(stderr, "key: %s, status: %d, errno=%d, strerrno=%s, dbm_error=%d\n", k.dptr, status, errno, strerror(errno), dbm_error(db)); return 1; } } dbm_close(db); return 0; } _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… This email sent to site_archiver(a)lists.apple.com

1 0

Re: A change with the -n option for cp in Mac OS X 10.5
by Chris Page 28 Nov '07

28 Nov '07

site_archiver(a)lists.apple.com Delivered-To: Darwin-dev(a)lists.apple.com On Nov 6, 2007, at 06:19 AM, Eric Gorr wrote: -- Chris Page - Retrospective Electrical Engineer Never attach a 12V/1A power supply to a 3V/5mA chip; that releases the magic smoke that makes it go. _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… Still seems strange that it would be considered an error if the command does what it is told to do. Non-zero status values are not by definition errors. Several commands use non-zero values to provide additional information about what the command did. This email sent to site_archiver(a)lists.apple.com

1 0

Re: Resident Memory, Private Memory, Shared Memory (top, Activity Monitor and vmmap)
by Markus Hanauska 28 Nov '07

28 Nov '07

site_archiver(a)lists.apple.com Delivered-To: Darwin-dev(a)lists.apple.com On Wed, 2007-11-28, at 00:08, Jim Magee wrote: Hi Jim, The Private and Shared memory sizes: -- Best Regards, Markus Hanauska _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… This value as returned by task_info() indicates how many page-level pmap mappings are currently present in the task's address space. In other words, how many valid page table entries there are for this processes current use of its virtual address space. This also means this value is the most uninteresting one for me as developer. It does not take into account if a page is shared or private. It means that this value has no reference to the memory my process really needs. Basically it only says how many pages can I access without faulting the MMU, however faulting the MMU is nothing bad. A reasonable system will fault the MMU a couple of times a second and this has no negative effects (okay, it degrades performance, since mapping a physical page will take some processing time). IOW the sum of private and shared memory is a much better representation of my process demands than this value; this value would be only of interest if something like shared memory wasn't existing. I can maybe use this value to find out how many pages I have already "touched" so far, but is it really interesting to know that? If I touch a shared page, this value will increase, but the physical memory demand on my system will not necessarily raise, since the page might have already been resident in physical memory before I touched it, it just had no mapping into my process space. Are calculated by walking the virtual address space of a task, looking at the objects mapped there, the resident page counts in those objects, and the reference counts on those objects to try and approximate the number of pages that fall into each category. Isn't that a much better representation of the memory demands? I mean, if I load a library into my process space, even though I have not ever touched anything of it, some pages of this library will be backed up by physical memory - my process might not be the cause why that happened, but I keep the library in memory at that time (together with maybe other processes), so I'm responsible (among other processes) that this memory object can't be dumped and that these pages stay in memory at the time being. But if the objects are bigger than what we have mapped, we may not be able to make dead accurate accounting. Not sure what you are trying to say with that sentence. If an object is 200 pages, and 20 pages are in fact present in physical memory (180 are not), then the accounting is perfectly correct when it says 20 * page_size is the used memory of this object. It may not be my process causing these 20 pages to be in physical memory, but since this object is mapped into my address space, I keep the memory object itself alive and thus I keep the 20 pages alive and mapped to memory, don't I? So accounting these 20 pages to my process seems reasonable. Private and shared memory sizes are an even greater approximation than the resident memory size. But they return different info. The private and shared memory counts say "how many pages of the objects I have mapped are cached in memory", regardless of whether I've accessed them yet/lately, where the resident memory size says "how many page-table-entries does my process already have to quickly access those cached pages". Okay, maybe I asked the wrong question here. Maybe the question should rather be how to interpret these values. E.g. what does it mean if a process has very little resident memory, but huge shared and huge private memory? This means I have memory objects mapped into my address space, large ones, that have many physical memory pages assigned to them and that are either private or shared, however, I have not touched many of these pages lately or at all, right? So my process is not really using many of the pages of these objects, still, I keep these objects alive and since they are alive, a lot of pages are lost of memory as these objects need them to keep process-private or process-shared data in memory. A real live example, I load a big library, a huge one, that is in use by many other processes, too. The code segment of that library will be shared (COW). Since some other process uses this library heavily, many of these COW code pages are resident in physical memory (cached so to say, if you see all physical RAM just as a cache for the swap file or other mapped binary files). This will raise my shared memory value to a huge value. However, if I never access any of the pages of the code segment of the library, all this memory is not accounted towards my resident size. Is this interpretation more or less accurate? I think I understood the case above, but how to interpret a process that has a huge resident memory size, but shared and private memory together are much less? What would be a real live example for that? Why would this process have so many physical pages mapped into its virtual address space, if all the objects it has mapped there have so little physical pages? What kind of pages would that be? Doesn't every page mapped into my process space needs to belong to some memory object in the kernel? If so, how can I have more mapped pages than I have mapped pages in all mapped memory objects? This case seems impossible, but I'm almost positive, I have seen that happening before. One side question: A virtual page is just virtual. The opposite of a virtual page is... a physical one? If I run vmmap with -resident, it will print the virtual size of every object and the physical size of every object (resident size). For the resident size, does it play a role whether the page is currently really in memory or swapped? Even when being swapped, is the page still resident or virtual again? Or does it make a difference how it is swapped? E.g. whether it's anonymous memory swapped by the default pager or a mapped file swapped by the vnode pager. This email sent to site_archiver(a)lists.apple.com

1 0

Re: Why no crash reporter if I install some signal handler on Leopard
by zhanglin 28 Nov '07

28 Nov '07

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com Dkim-signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:cc:message-id:from:to:in-reply-to:content-type:content-transfer-encoding:mime-version:subject:date:references:x-mailer; bh=jE8z+eEk+FkcUQLn9aPKR6HDtyPJbCDbtczQ1K8iyLk=; b=EiBPCub1mEPcxDvH/fK+8qRqGQCz8qhmM5+3KWwrCbEAFPdJCs5OxABL58mrNZn0Ly09BJZ+SjAZ22oT/FG/QCBpXp3acwWQqkAgYNbgKZz1zCRyi/B/m9lqaK8Tnm78bsaiRrtWo417BJcMAfSkuAupzWcWi/orGEIze5ijfvQ= Domainkey-signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=received:cc:message-id:from:to:in-reply-to:content-type:content-transfer-encoding:mime-version:subject:date:references:x-mailer; b=HUVc+f+hRSAX8OHPyt6QCCyi6jxWjI+8QmqiqHIK4R7a4VrwjoZ3GctCWfKqpsnIQ7vZYUZEh1Ja+X2xxrzZ4GFWNe+wJOu6A5ThZvFwAmRTY/R3xp1Yyhg+qz6OkZV44GZCNJdzLCaLASJoQruY8mkXdC+x+pKYefCzInTTrf4= On Nov 28, 2007, at 9:23 AM, Terry Lambert wrote: The following will work, but it's not ideal: ======= #include <signal.h> #include <stdio.h> void sig_abort(int sig) { printf("Application has been abort\n"); abort(); } Very true. -- Terry _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/zhanglin.zhou%40gmail.com _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… Thanks, all. I change to install an exception handler rather signal handler with api InstallExceptionHandler(), and in my exception handler, I firstly invoke some other tool to collect information, and then return -1, so the crash report dialog will be shown. But it still exist a problem: does the InstallExceptionHandler only affect the current thread? If I crash in other thread, will my exception handler be invoked? On Nov 27, 2007, at 4:36 PM, Jim Magee wrote: On Nov 27, 2007, at 5:42 PM, Terry Lambert wrote: ... It's not ideal because you can't really rethrow the abort with the same stack, you will always show as being in the trap handler (the stack for a crash report looks something like this (on a PPC): It's also not ideal because printf() is not signal-handler-safe. ;-) This email sent to zhanglin.zhou(a)gmail.com This email sent to site_archiver(a)lists.apple.com

1 0