September 2004 - Darwin-dev

Re: using stat
by Hugh Hoover 12 Sep '04

12 Sep '04

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com You are confused about what this field contains. returning 0 is normal for that field. See chflags(2) for typical contents. From <sys/stat.h>: doh! RTFMMC (... More Carefully :) thanks - I was just SURE the man page said st_flags for the IF_XXX flags _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… On Sep 12, 2004, at 12:04, darwin-dev-request(a)lists.apple.com wrote: This email sent to site_archiver(a)lists.apple.com

1 0

Re: using stat
by Shantonu Sen 12 Sep '04

12 Sep '04

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com None of these are typically set for files or directories. You can determine if you have a directory by doing: if(S_ISDIR(stat_buf.st_mode)) Shantonu On Sep 12, 2004, at 1:27 AM, Hugh Hoover wrote: Any suggestions? _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… You are confused about what this field contains. returning 0 is normal for that field. See chflags(2) for typical contents. From <sys/stat.h>: /* * Definitions of flags stored in file flags word. * * Super-user and owner changeable flags. */ #define UF_SETTABLE 0x0000ffff /* mask of owner changeable flags */ #define UF_NODUMP 0x00000001 /* do not dump file */ #define UF_IMMUTABLE 0x00000002 /* file may not be changed */ #define UF_APPEND 0x00000004 /* writes to file may only append */ #define UF_OPAQUE 0x00000008 /* directory is opaque wrt. union */ /* * Super-user changeable flags. */ #define SF_SETTABLE 0xffff0000 /* mask of superuser changeable flags */ #define SF_ARCHIVED 0x00010000 /* file is archived */ #define SF_IMMUTABLE 0x00020000 /* file may not be changed */ #define SF_APPEND 0x00040000 /* writes to file may only append */ This is not Darwin specific... You may want to check out a unix program text on this matter for further information. Hi - I'm attempting to use stat on 10.3.5 and getting back strange results... The first items in the result stat_buf struct seems more or less correct, dev, inode, uid, gid, times, size, etc. The nlinks field seems suspicious, but the killer is the st_flags field - it's ALWAYS 0 - even for directories (which is why I'm doing the stat - to find out if I have a directory or not). I've checked the sys/stat.h header and everything appears to be in order there. I'm pretty sure I've got the right set of headers (for 10.3). TIA This email sent to site_archiver(a)lists.apple.com

1 0

global register variable clobbered by save/restore
by David N. Williams 12 Sep '04

12 Sep '04

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com Organization: University of Michigan User-agent: Mozilla/5.0 (Macintosh; U; PPC Mac OS X Mach-O; en-US; rv:1.4) Gecko/20030624 Netscape/7.1 I've had a problem with a rebuild of pfe-0.33.30 with gcc 3.3 under MacOS 10.3.4, which worked fine in 2.95.2 under MacOS 10.2.x. It seems to boil down to an illegal (I think) save/restore of one or more registers declared as global variables, which I've distilled into the example below. Among several register problems, my initial scan of Bugzilla doesn't show this, but maybe I missed it (or maybe I'm just misunderstanding something). The assembly output was produced by cc -S -c -Wall -no-cpp-precomp -O3 $1.c Other optimizations give the same problem. ----- testreg.c --------- #include <locale.h> register int *testreg asm ("r29"); int *ext_func (int *p); void test_reg_save_restore (int *p) { setlocale (LC_ALL, "C"); testreg = ext_func(p); } ------ testreg.s --------- .section __TEXT,__text,regular,pure_instructions .section __TEXT,__picsymbolstub1,symbol_stubs,pure_instructions,32 .data .cstring .align 2 LC0: .ascii "C\0" .section __TEXT,__text,regular,pure_instructions .align 2 .align 2 .globl _test_reg_save_restore .section __TEXT,__text,regular,pure_instructions .align 2 _test_reg_save_restore: mflr r4 bcl 20,31,"L00000000001$pb" "L00000000001$pb": stmw r28,-16(r1) mflr r31 addis r2,r31,ha16(LC0-"L00000000001$pb") stw r4,8(r1) mr r28,r3 stwu r1,-80(r1) la r4,lo16(LC0-"L00000000001$pb")(r2) li r3,0 bl L_setlocale$stub mr r3,r28 bl L_ext_func$stub lwz r2,88(r1) addi r1,r1,80 mr r29,r3 mtlr r2 lmw r28,-16(r1) blr [the rest snipped] ----------------------- Note that registers r28-r31 are saved and then restored on top of the load into the global variable register r29. If I read correctly, the 3.3 documentation says otherwise. Here's a quote from "Using the GNU Compiler Collection" in the "Defining Global Register Variables" section (3.3 documentation distributed by Apple): -------------- You can define a global register variable in GNU C like this: register int *foo asm ("a5"); Here a5 is the name of the register which should be used. Choose a register which is normally saved and restored by function calls on your machine, so that library routines will not clobber it. ... Defining a global register variable in a certain register reserves that register entirely for this use, at least within the current compilation. The register will not be allocated for any other purpose in the functions in the current compilation. The register will not be saved and restored by these functions. Stores into this register are never deleted even if they would appear to be dead, but references may be deleted or moved or simplified. -------------- Am I off base here? Thanks for any insights! -- David _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… This email sent to site_archiver(a)lists.apple.com

1 0

using stat
by Hugh Hoover 12 Sep '04

12 Sep '04

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com Any suggestions? TIA _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… Hi - I'm attempting to use stat on 10.3.5 and getting back strange results... The first items in the result stat_buf struct seems more or less correct, dev, inode, uid, gid, times, size, etc. The nlinks field seems suspicious, but the killer is the st_flags field - it's ALWAYS 0 - even for directories (which is why I'm doing the stat - to find out if I have a directory or not). I've checked the sys/stat.h header and everything appears to be in order there. I'm pretty sure I've got the right set of headers (for 10.3). This email sent to site_archiver(a)lists.apple.com

1 0

Re: pthread_join
by Marc Van Olmen 11 Sep '04

11 Sep '04

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com Resent-date: Sun, 12 Sep 2004 15:17:04 -0400 Resent-from: Marc Van Olmen <marc(a)sky4studios.be> Resent-message-id: <A6EE50FC-0418-11D9-BE45-00306570AF76(a)sky4studios.be> Resent-to: darwin-dev(a)lists.apple.com HI, thanks for you time again, mvo ps: so my function looks like this (with the 2 options): void* cOS_Thread::Entry(cOS_Thread* inCAPThread) { int anError; On Sep 11, 2004, at 12:16 PM, Marc Van Olmen wrote: Hi Paul Thanks for you reaction: You are leaking threads. You need to either detach or join them. If you are comparing how the two options perform, it may be better to not use the semaphore at all in the attached case and simply join the thread. I will keep you up to date on my progress, mvo However, if at some point you run into a case where you have an attached thread which you know for certain has finished executing and are finished tracking it, I think the most efficient thing to do to release the resource at this point is to detach it. There's no reason you can't detach an attached thread that has finished executing. _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… I can confirm that when I call pthread_detach at the end of the function entry there is no leak any more in the virtual memory space. I'm still surprised that I need to call pthread_detach myself at the end of function call, because couldn't find any remark of that in the book, and threads under OS X seems to be created by default in 'attach' mode. one remark: The detach-thread version with semaphore techniques uses less memory then the 'attach'-thread and pthread_join technique. void* theAnswer = NULL; if(inCAPThread->mThreadRoutine != NULL) { #if FW_SEMAPHORE_WAIT_TECHNIQUE inCAPThread->mWaitSemaphore->Lock(); #endif theAnswer = inCAPThread->mThreadRoutine(inCAPThread->mThreadParameter); #if FW_SEMAPHORE_WAIT_TECHNIQUE inCAPThread->mThreadFinished = true; inCAPThread->mWaitSemaphore->Notify(); inCAPThread->mWaitSemaphore->Unlock(); #else anError = pthread_detach(inCAPThread->mPThread); OS_ThrowIf(anError != 0, anError, "cOS_Thread::Entry: Could not detach the thread"); #endif } inCAPThread->mPThread = 0; return theAnswer; } On Sep 11, 2004, at 1:00 AM, darwin-development-request(a)lists.apple.com wrote: You are right here, in case of attached i don't use Semaphore technique. I gonna do some more testing with your hint about "detach" Also wanted to give one remark is that I'm doing QuickTime calls (but i use the EnterMoviesOnThread and Exit.... calls.... I also wanted to give one correction to my the code (and also matt code) I'm just doing this for the record so people that search archive: void* cOS_Thread::Entry(cOS_Thread* inCAPThread) { void* theAnswer = NULL; if(inCAPThread->mThreadRoutine != NULL) { #if FW_SEMAPHORE_WAIT_TECHNIQUE inCAPThread->mWaitSemaphore->Lock(); #endif theAnswer = inCAPThread->mThreadRoutine(inCAPThread->mThreadParameter); #if FW_SEMAPHORE_WAIT_TECHNIQUE inCAPThread->mThreadFinished = true; inCAPThread->mWaitSemaphore->Notify(); inCAPThread->mWaitSemaphore->Unlock(); #endif } inCAPThread->mPThread = 0; return theAnswer; } The signa/Notify l has to be in side the lock/unlock because if you are waiting for delete and the switch happens right after Unlock call... you delete the thread object (and also semaphore) before the notify code can be reached.... during testing this happened after 12 hours.... This email sent to site_archiver(a)lists.apple.com

1 0

Re: Using dead notification with mach ports
by Jim Magee 11 Sep '04

11 Sep '04

This is where you went astray. The clients wouldn't look the new message queue up in some global namespace, but instead it would be returned to them (privately) as part of the reply to this "connection request" RPC. So now the server does the following fo each new client in the "connection request" RPC. mach_port_t task = mach_task_self(); mach_port_t old_port; mach_port_allocate(task,MACH_PORT_RIGHT_RECEIVE,&clientPort) mach_port_move_member(task,clientPort,serverPort) mach_port_request_notification(task, clientPort, MACH_NOTIFY_NO_SENDERS, 1, serverPort, MACH_MSG_TYPE_MAKE_SEND_ONCE,&old_port); where serverPort will be used by the server in the mach_msg_server. Then clientPort has to be sent back to the client, how is this done? Only by sending the clientPort back? Or the client needs to get a copy of send right to that clientPort Mach message queue alloccated by the server? Sorry, I sent that early by mistake.... Here's an example MIG definition that assumes the latter approach: routine StephaneConnectToServer ( serverPort : mach_port_t; connectFlags : integer_t; out semaphorePort: semaphore_t; out connectionPort: mach_port_make_send_t ); Hope that helps, --Jim _______________________________________________ darwin-development mailing list | darwin-development(a)lists.apple.com Help/Unsubscribe/Archives: http://www.lists.apple.com/mailman/listinfo/darwin-development Do not post admin requests to the list. They will be ignored. On Sep 11, 2004, at 9:36 AM, Jim Magee wrote: On Sep 11, 2004, at 4:55 AM, Stiphane LETZ wrote: 4) client get this new private port from the server (same method as 2 ) mach_port_insert_right (task,clientPort,clientPort,MACH_MSG_TYPE_MAKE_SEND) I'm quite confused... Anyway, one of the 2 or 3 primary advantages of Mach IPC over other forms is the flexibility in passing other port rights in messages. Since it appears you are using MIG, you simply have to declare an "out" parameter (or two, if you count sending the semaphore as well) of type mach_port_t. That will arrange to have send rights transfered to the client (copied, to be explicit). But for the no-senders notification to fire, the server can't hold its own send right to that port (and you have given the server a send right with the mach_port_insert_right() call above). Instead, it must "move" the send right it created to the client. You can do this in one of two ways: You can declare the "out" parameter as type "mach_port_move_send_t" in the MIG definition or You could declare it of type "mach_port_make_send_t" and skip the call to mach_port_insert_right() altogether

1 0

Re: Using dead notification with mach ports
by Jim Magee 11 Sep '04

11 Sep '04

4) client get this new private port from the server (same method as 2 ) This is where you went astray. The clients wouldn't look the new message queue up in some global namespace, but instead it would be returned to them (privately) as part of the reply to this "connection request" RPC. So now the server does the following fo each new client in the "connection request" RPC. mach_port_t task = mach_task_self(); mach_port_t old_port; mach_port_allocate(task,MACH_PORT_RIGHT_RECEIVE,&clientPort) mach_port_move_member(task,clientPort,serverPort) mach_port_request_notification(task, clientPort, MACH_NOTIFY_NO_SENDERS, 1, serverPort, MACH_MSG_TYPE_MAKE_SEND_ONCE,&old_port); where serverPort will be used by the server in the mach_msg_server. Then clientPort has to be sent back to the client, how is this done? Only by sending the clientPort back? Or the client needs to get a copy of send right to that clientPort Mach message queue alloccated by the server? _______________________________________________ darwin-development mailing list | darwin-development(a)lists.apple.com Help/Unsubscribe/Archives: http://www.lists.apple.com/mailman/listinfo/darwin-development Do not post admin requests to the list. They will be ignored. On Sep 11, 2004, at 4:55 AM, Stiphane LETZ wrote: mach_port_insert_right (task,clientPort,clientPort,MACH_MSG_TYPE_MAKE_SEND) I'm quite confused...

1 0

Re: Using dead notification with mach ports
by Stéphane LETZ 11 Sep '04

11 Sep '04

>> 4) client get this new private port from the server (same method as 2 >> ) > > This is where you went astray. The clients wouldn't look the new > message queue up in some global namespace, but instead it would be > returned to them (privately) as part of the reply to this "connection > request" RPC. > >> So now the server does the following fo each new client in the "connection request" RPC. mach_port_t task = mach_task_self(); mach_port_t old_port; mach_port_allocate(task,MACH_PORT_RIGHT_RECEIVE,&clientPort) mach_port_insert_right(task,clientPort,clientPort,MACH_MSG_TYPE_MAKE_SEN D) mach_port_move_member(task,clientPort,serverPort) mach_port_request_notification(task, clientPort, MACH_NOTIFY_NO_SENDERS, 1, serverPort, MACH_MSG_TYPE_MAKE_SEND_ONCE,&old_port); where serverPort will be used by the server in the mach_msg_server. Then clientPort has to be sent back to the client, how is this done? Only by sending the clientPort back? Or the client needs to get a copy of send right to that clientPort Mach message queue alloccated by the server? I'm quite confused... I am also using inter-process server/client Mach semaphores for synchronization purpose. Right now, I was allocating the semaphore on the server side with "semaphore_create" then registering it in the bootstrap namespace using "bootstrap_register" with a name for each semaphore (associated with a given client). Then the client was using "bootstrap_look_up" to get its semaphore. This is working. But I guess registering the semaphores in the bootstrap namespace is not a very good idea, and since Mach semaphores are actually Mach ports, I guess semaphores could be handled exactly like the previous "private port" mechanism for server/client communication. But again I don't understand how the "private port" (or semaphore) allocated on the server side is given back to the client side in a way the client can used them. Thanks for any advice Stephane Letz _______________________________________________ darwin-development mailing list | darwin-development(a)lists.apple.com Help/Unsubscribe/Archives: http://www.lists.apple.com/mailman/listinfo/darwin-development Do not post admin requests to the list. They will be ignored.

1 0

Re: pthread_join
by Paul Forgey 10 Sep '04

10 Sep '04

You are leaking threads. You need to either detach or join them. On Sep 10, 2004, at 2:17 PM, Marc Van Olmen wrote: _______________________________________________ darwin-development mailing list | darwin-development(a)lists.apple.com Help/Unsubscribe/Archives: http://www.lists.apple.com/mailman/listinfo/darwin-development Do not post admin requests to the list. They will be ignored. If you are comparing how the two options perform, it may be better to not use the semaphore at all in the attached case and simply join the thread. However, if at some point you run into a case where you have an attached thread which you know for certain has finished executing and are finished tracking it, I think the most efficient thing to do to release the resource at this point is to detach it. There's no reason you can't detach an attached thread that has finished executing. I implemented both techniques just to try because the 'attached' implementation gave some trouble... In the 'attached' implementation, my virtual memory space keeps on growing.... after 8 hours it reach 4 GB and my program stops at 'pthread_create' with error: EAGAIN The number of threads that are there is not growing. The 'real memory' space is not growing... I used activity monitor to check this information here above... The only difference between the semaphore version is the code here below. I defined a macro FW_SEMAPHORE_WAIT_TECHNIQUE to switch easy between the 2 technique's is there something special I need to do when the Thread's ends in case of 'attached' thread? I read the o-reilly book pThread programming and couldn't find anything but of course i could missed something on this. Currently in mys code in the the Entry function I just put it the inCAPThread->mPThread = 0;

1 0

Re: pthread_join
by Marc Van Olmen 10 Sep '04

10 Sep '04

Matt, inCAPThread->mPThread = 0; regards, mvo void cOS_Thread::WaitToFinish ( ) { if (mPThread) { On Sep 9, 2004, at 4:10 PM, Matt Watson wrote: pthread_mutex_lock(&mMutex); gThread_B_Finished = true; pthread_mutex_unlock(&mMutex); pthread_cond_signal(&mCondVar); then in your "waiting" routine, do: pthread_mutex_lock(& mMutex); while (gThread_B_Finished == false) pthread_cond_wait(& mCondVar, & mMutex); pthread_mutex_unlock(& mMutex); _______________________________________________ darwin-development mailing list | darwin-development(a)lists.apple.com Help/Unsubscribe/Archives: http://www.lists.apple.com/mailman/listinfo/darwin-development Do not post admin requests to the list. They will be ignored. I implemented both techniques just to try because the 'attached' implementation gave some trouble... In the 'attached' implementation, my virtual memory space keeps on growing.... after 8 hours it reach 4 GB and my program stops at 'pthread_create' with error: EAGAIN The number of threads that are there is not growing. The 'real memory' space is not growing... I used activity monitor to check this information here above... The only difference between the semaphore version is the code here below. I defined a macro FW_SEMAPHORE_WAIT_TECHNIQUE to switch easy between the 2 technique's is there something special I need to do when the Thread's ends in case of 'attached' thread? I read the o-reilly book pThread programming and couldn't find anything but of course i could missed something on this. Currently in mys code in the the Entry function I just put it the my code is for 95% based on the core-audio public code CAPThread.h and CAPThread.cpp here the code ( the code is based on my classes i can send all the related code if necessary, like the semaphore class...) it will be about 6 files. / /----------------------------------------------------------------------- ----------------- // cOS_Thread::Start / /----------------------------------------------------------------------- ----------------- void cOS_Thread::Start() { if(mPThread == 0) { OSStatus theResult; pthread_attr_t theThreadAttributes; mThreadFinished = false; theResult = pthread_attr_init(&theThreadAttributes); OS_ThrowIf(theResult != 0, theResult, "cOS_Thread::Start: Thread attributes could not be created."); #if FW_SEMAPHORE_WAIT_TECHNIQUE theResult = pthread_attr_setdetachstate(&theThreadAttributes, PTHREAD_CREATE_DETACHED); OS_ThrowIf(theResult != 0, theResult, "cOS_Thread::Start: A thread could not be created in the detached state."); #endif theResult = pthread_create(&mPThread, &theThreadAttributes, (ThreadRoutine)cOS_Thread::Entry, this); OS_ThrowIf(theResult != 0, theResult, "cOS_Thread::Start: Could not create a thread."); pthread_attr_destroy(&theThreadAttributes); if(mTimeConstraintSet) { SetTimeConstraints(mPeriod, mComputation, mConstraint, mIsPreemptible); } else { SetPriority(mPriority, mFixedPriority); } } } / /----------------------------------------------------------------------- ----------------- // cOS_Thread::Entry / /----------------------------------------------------------------------- ----------------- void* cOS_Thread::Entry(cOS_Thread* inCAPThread) { void* theAnswer = NULL; if(inCAPThread->mThreadRoutine != NULL) { #if FW_SEMAPHORE_WAIT_TECHNIQUE inCAPThread->mWaitSemaphore->Lock(); #endif theAnswer = inCAPThread->mThreadRoutine(inCAPThread->mThreadParameter); inCAPThread->mThreadFinished = true; #if FW_SEMAPHORE_WAIT_TECHNIQUE inCAPThread->mWaitSemaphore->Unlock(); inCAPThread->mWaitSemaphore->Notify(); #endif } inCAPThread->mPThread = 0; return theAnswer; } / /----------------------------------------------------------------------- ----------------- // cOS_Thread::WaitToFinish / /----------------------------------------------------------------------- ----------------- #if FW_SEMAPHORE_WAIT_TECHNIQUE mWaitSemaphore->Lock(); while (mThreadFinished == false) { mWaitSemaphore->Wait(); } mWaitSemaphore->Unlock(); #else OS_Error anError = pthread_join(mPThread,NULL); OS_ThrowIf(anError != 0, anError, "cOS_Thread::WaitToFinish: Could not wait for a signal"); #endif FW_ASSERT(mThreadFinished == true); } } Would this be better (performance wise) then using an attached thread? You probably wouldn't notice the performance difference, since they use the same underlying mechanism. You'd do something like this at the end of your thread's body routine: And you'll want to add error checking appropriate for your application to the above code, of course. matt.

1 0