May 2003 - Darwin-kernel - lists.apple.com

Re: Threading contexts
by Jim Magee 23 May '03

23 May '03

On Thursday, May 22, 2003, at 1:21 AM, Wally Crooze wrote: Jim Magee wrote: Wally Crooze wrote: Continuing on from the locking and synchronization primitives... I have the need to be able to associate particular information with a thread which is specific to my kext project. In other OSes an interface is provided to allow information stored which doesn't effect the control of the thread. I found the following interfaces which set/retrieve thread_t->saved.misc: extern void thread_set_cont_arg(int); extern int thread_get_cont_arg(void); These aren't used anywhere in xnu, but thread_t->saved is a union and is used for other information. Are these interfaces expected to be used for miscellaneous thread information within my own code? Also, can I treat this int as a void* ? You can only use this to save information while you are causing the thread to block. Some such blocks can occur at continuation points (where the kernel stack is thrown away). These are the continuation arguments that you can use to build context back up when the thread resumes. If you aren't blocking the thread, you can't claim ownership of these fields. Does this mean that I can expect that at no time the thread is in my code these fields will be in use? And that they wont be effected in any blocking (i.e. locking/synchronisation) calls? The "rule" for use of these fields is that they only get used while you are preparing for and recovering from a thread blocking within your own code. If the thread blocks within code that you call, you can't assume that the fields will be left alone/restored. It's not a stack. But if you never call any code that can block, they (in theory) could be used by you. I guess that I must use a lookup table mechanism on the thread to retrieve my data. This will certainly effect the performance. Is there a reason why a generic pointer for use within kexts (or drivers) isn't available? A generic pointer does no good if any two layers of software need to use it. You need something akin to pthread_set_specific()/pthread_get_specific() with a key to identify _your_ pointer. That's pretty much what I was talking about us adding. So, the only reason it doesn't exist yet is that it just hasn't been implemented yet. I have not found any philosophical opposition to it. An example for this use is part of my sema implementation requires to store whether the thread was woken normally or by an (software) interrupt from my code. This is a situation where use of the cont_arg mechanism would be appropriate. But you might want to use the wait_result value for this indication (depending upon what sleep/wakeup mechanism you are using). Remember that unless you block threads as "uninterruptible", the signal/Mach thread_abort mechanisms may do the interruptions without your wakeup code ever getting involved. You must deal with those cases. They will result in a wait_result of THREAD_INTERRUPTED. If you want to "poke at" a thread without making it look like a signal, the THREAD_RESTART wait_result is commonly used. Is there an interface to "interrupt" a semaphore as opposed to waking it? Or is it easier to mimic this in software with my own flags? Again, this is heavily tied to whatever sleep/wakeup mechanism you employed in your semaphore implementation. But there is thread_abort_safely() which targets specific threads (regardless of what they are waiting on), and there is thread_wakup_prim()-derived wakup mechanism that takes a wait_result parameter to apply to each thread it wakes up. --Jim _______________________________________________ darwin-kernel mailing list | darwin-kernel(a)lists.apple.com Help/Unsubscribe/Archives: http://www.lists.apple.com/mailman/listinfo/darwin-kernel Do not post admin requests to the list. They will be ignored.

1 0

Re: Threading contexts
by Wally Crooze 22 May '03

22 May '03

Jim Magee wrote: Wally Crooze wrote: Continuing on from the locking and synchronization primitives... I have the need to be able to associate particular information with a thread which is specific to my kext project. In other OSes an interface is provided to allow information stored which doesn't effect the control of the thread. I found the following interfaces which set/retrieve thread_t->saved.misc: extern void thread_set_cont_arg(int); extern int thread_get_cont_arg(void); These aren't used anywhere in xnu, but thread_t->saved is a union and is used for other information. Are these interfaces expected to be used for miscellaneous thread information within my own code? Also, can I treat this int as a void* ? You can only use this to save information while you are causing the thread to block. Some such blocks can occur at continuation points (where the kernel stack is thrown away). These are the continuation arguments that you can use to build context back up when the thread resumes. If you aren't blocking the thread, you can't claim ownership of these fields. Does this mean that I can expect that at no time the thread is in my code these fields will be in use? And that they wont be effected in any blocking (i.e. locking/synchronisation) calls? Unfornately neither the Kernel Programming guide and Threading Architecture Technotes describe a method of associating an information context to the thread. That's because there isn't any such mechanism today. We are looking at providing a generalized version of thread-local-data for the kernel environment. But we don't have anything like that now. I guess that I must use a lookup table mechanism on the thread to retrieve my data. This will certainly effect the performance. Is there a reason why a generic pointer for use within kexts (or drivers) isn't available? An example for this use is part of my sema implementation requires to store whether the thread was woken normally or by an (software) interrupt from my code. This is a situation where use of the cont_arg mechanism would be appropriate. But you might want to use the wait_result value for this indication (depending upon what sleep/wakeup mechanism you are using). Remember that unless you block threads as "uninterruptible", the signal/Mach thread_abort mechanisms may do the interruptions without your wakeup code ever getting involved. You must deal with those cases. They will result in a wait_result of THREAD_INTERRUPTED. If you want to "poke at" a thread without making it look like a signal, the THREAD_RESTART wait_result is commonly used. Is there an interface to "interrupt" a semaphore as opposed to waking it? Or is it easier to mimic this in software with my own flags? -- Wally Crooze +-------------------------------------------------------+ | ,-_|\ Wally Crooze <wallycrooze(a)pobox.com> | | / \ | | \_,-._/ | | v | +-------------------------------------------------------+ _______________________________________________ darwin-kernel mailing list | darwin-kernel(a)lists.apple.com Help/Unsubscribe/Archives: http://www.lists.apple.com/mailman/listinfo/darwin-kernel Do not post admin requests to the list. They will be ignored.

1 0

Re: Threading contexts
by Jim Magee 21 May '03

21 May '03

On Tuesday, May 20, 2003, at 12:54 AM, Wally Crooze wrote: Continuing on from the locking and synchronization primitives... I have the need to be able to associate particular information with a thread which is specific to my kext project. In other OSes an interface is provided to allow information stored which doesn't effect the control of the thread. I found the following interfaces which set/retrieve thread_t->saved.misc: extern void thread_set_cont_arg(int); extern int thread_get_cont_arg(void); These aren't used anywhere in xnu, but thread_t->saved is a union and is used for other information. Are these interfaces expected to be used for miscellaneous thread information within my own code? Also, can I treat this int as a void* ? You can only use this to save information while you are causing the thread to block. Some such blocks can occur at continuation points (where the kernel stack is thrown away). These are the continuation arguments that you can use to build context back up when the thread resumes. If you aren't blocking the thread, you can't claim ownership of these fields. Unfornately neither the Kernel Programming guide and Threading Architecture Technotes describe a method of associating an information context to the thread. That's because there isn't any such mechanism today. We are looking at providing a generalized version of thread-local-data for the kernel environment. But we don't have anything like that now. An example for this use is part of my sema implementation requires to store whether the thread was woken normally or by an (software) interrupt from my code. This is a situation where use of the cont_arg mechanism would be appropriate. But you might want to use the wait_result value for this indication (depending upon what sleep/wakeup mechanism you are using). Remember that unless you block threads as "uninterruptible", the signal/Mach thread_abort mechanisms may do the interruptions without your wakeup code ever getting involved. You must deal with those cases. They will result in a wait_result of THREAD_INTERRUPTED. If you want to "poke at" a thread without making it look like a signal, the THREAD_RESTART wait_result is commonly used. --Jim _______________________________________________ darwin-kernel mailing list | darwin-kernel(a)lists.apple.com Help/Unsubscribe/Archives: http://www.lists.apple.com/mailman/listinfo/darwin-kernel Do not post admin requests to the list. They will be ignored.

1 0

Threading contexts
by Wally Crooze 20 May '03

20 May '03

Continuing on from the locking and synchronization primitives... I have the need to be able to associate particular information with a thread which is specific to my kext project. In other OSes an interface is provided to allow information stored which doesn't effect the control of the thread. I found the following interfaces which set/retrieve thread_t->saved.misc: extern void thread_set_cont_arg(int); extern int thread_get_cont_arg(void); These aren't used anywhere in xnu, but thread_t->saved is a union and is used for other information. Are these interfaces expected to be used for miscellaneous thread information within my own code? Also, can I treat this int as a void* ? Unfornately neither the Kernel Programming guide and Threading Architecture Technotes describe a method of associating an information context to the thread. An example for this use is part of my sema implementation requires to store whether the thread was woken normally or by an (software) interrupt from my code. Thanks. -- Wally Crooze +-------------------------------------------------------+ | ,-_|\ Wally Crooze <wallycrooze(a)pobox.com> | | / \ | | \_,-._/ | | v | +-------------------------------------------------------+ _______________________________________________ darwin-kernel mailing list | darwin-kernel(a)lists.apple.com Help/Unsubscribe/Archives: http://www.lists.apple.com/mailman/listinfo/darwin-kernel Do not post admin requests to the list. They will be ignored.

1 0

Re: Can I use OSEnqueueAtomic & OSDequeueAtomic from user mode?
by Quinn 19 May '03

19 May '03

At 11:10 -0700 18/5/03, Shawn Erickson wrote: Can I use OSEnqueueAtomic & OSDequeueAtomic from user mode? No. There are, however, numerous atomic operations available from various user space frameworks. The two that I'm familiar with are those in "DriverSynchronization.h" and "OpenTransport.h". Do not roll your own atomic operations using PowerPC lwarx and stwcx instructions. This recommendation, from an old technote, still applies. DTS recommends that developers avoid using the PowerPC Load Reserved and Store Conditional instructions. There are two reasons for this. Firstly, these instructions are inherently processor-specific and reduce the portability of your code. Secondly, the behavior of these instructions varies between PowerPC CPU types. Accommodating all these variations is tricky. These instructions do not provide much utility beyond that provided by the Open Transport and DriverServicesLib atomic routines, and Apple ensures that these atomic routines are updated to do the right thing in all cases. -- <http://developer.apple.com/technotes/tn/tn1137.html> S+E -- Quinn "The Eskimo!" <http://www.apple.com/developer/> Apple Developer Technical Support * Networking, Communications, Hardware _______________________________________________ darwin-kernel mailing list | darwin-kernel(a)lists.apple.com Help/Unsubscribe/Archives: http://www.lists.apple.com/mailman/listinfo/darwin-kernel Do not post admin requests to the list. They will be ignored.

1 0

Can I use OSEnqueueAtomic & OSDequeueAtomic from user mode?
by Shawn Erickson 18 May '03

18 May '03

Can I use OSEnqueueAtomic & OSDequeueAtomic from user mode? _______________________________________________ darwin-kernel mailing list | darwin-kernel(a)lists.apple.com Help/Unsubscribe/Archives: http://www.lists.apple.com/mailman/listinfo/darwin-kernel Do not post admin requests to the list. They will be ignored.

1 0

Re: shared memory
by Brian Tabone 14 May '03

14 May '03

Ruotger, Have a look at the mmap man page (Search for MAP_FIXED), there is an option to a mmap call that allows you to specify the starting address of the memory region to which you memory map a file. Since you can memory map anonomous files (IE files that don't exist on disk, basically just a shared memory segment), you could use mmap to create the fixed location shared memory segment that you use between your applications. Hope that helps, Brian Tabone On Wednesday, May 14, 2003, at 09:26AM, Ruotger Skupin <skupin(a)ableton.com> wrote: >Hi, > >I have a problem where one process is building data-structures/objects >with lots of pointers for a second process to use. Currently we do that >using shared memory. We map the shared memory in a way that it appeares >at the same memory range in both processes (for making the pointers >work). This works fine on windows and on MacOS 9 but I ran into various >trouble on MacOS X > >Is there a solution to allocate shared memory in two processes at the >same address? (about 50-100MB) > >Any ideas? > Roddi > >--- >Ruotger Skupin >Engine Development > >ableton schoenhauser allee 6/7 >10119 berlin germany www.ableton.com >_______________________________________________ >darwin-kernel mailing list | darwin-kernel(a)lists.apple.com >Help/Unsubscribe/Archives: http://www.lists.apple.com/mailman/listinfo/darwin-kernel >Do not post admin requests to the list. They will be ignored. _______________________________________________ darwin-kernel mailing list | darwin-kernel(a)lists.apple.com Help/Unsubscribe/Archives: http://www.lists.apple.com/mailman/listinfo/darwin-kernel Do not post admin requests to the list. They will be ignored.

1 0

Re: shared memory
by Quinn 14 May '03

14 May '03

At 16:26 +0200 14/5/03, Ruotger Skupin wrote: Is there a solution to allocate shared memory in two processes at the same address? (about 50-100MB) In general, no. Many of the shared memory APIs (for example, BSD's mmap and Mach's vm_map) allow you to specify the address where you want the memory mapped. However, there's no easy way to come up with an address to use. The memory map of a Mac OS X process is very dynamic and Apple has, and will continue to, shuffle it around to meet our goals (for example, the address range used to map frameworks changed between Mac OS X 10.1 and 10.2). If you know in advance all of the processes that need to use the memory, you could probe each of their memory maps (vm_region should work) to find an address that's currently free in all of the processes. If you want to allocate a range that's free in the current process and in other arbitrary processes that might connect to it, the best you can do is use heuristics that will be a future compatibility liability. My general advice is to use offsets from the base of the shared memory rather than pointers within your shared memory. With enough C++ hackery you can make these look like normal pointers. S+E -- Quinn "The Eskimo!" <http://www.apple.com/developer/> Apple Developer Technical Support * Networking, Communications, Hardware _______________________________________________ darwin-kernel mailing list | darwin-kernel(a)lists.apple.com Help/Unsubscribe/Archives: http://www.lists.apple.com/mailman/listinfo/darwin-kernel Do not post admin requests to the list. They will be ignored.

1 0

shared memory
by Ruotger Skupin 14 May '03

14 May '03

Hi, I have a problem where one process is building data-structures/objects with lots of pointers for a second process to use. Currently we do that using shared memory. We map the shared memory in a way that it appeares at the same memory range in both processes (for making the pointers work). This works fine on windows and on MacOS 9 but I ran into various trouble on MacOS X Is there a solution to allocate shared memory in two processes at the same address? (about 50-100MB) Any ideas? Roddi --- Ruotger Skupin Engine Development ableton schoenhauser allee 6/7 10119 berlin germany www.ableton.com _______________________________________________ darwin-kernel mailing list | darwin-kernel(a)lists.apple.com Help/Unsubscribe/Archives: http://www.lists.apple.com/mailman/listinfo/darwin-kernel Do not post admin requests to the list. They will be ignored.

1 0

Re: Locking and synchronization primitives
by Godfrey van der Linden 12 May '03

12 May '03

G'day Wally, This is a fairly common misconception. The fact that the Funnel enforces 'single' treading does not mean that you can't write a mutliple thread layer. Check out IOSerialFamily. The IOSerialBSDClient.cpp launches to funneled threads as well as at least one client thread and sometimes a driver thread. All of these threads interact quite happily in one instance of a serial port. Godfrey On Monday, May 12, 2003, at 01:31 US/Pacific, Wally Crooze wrote: This system will be multithreaded as it is required to do work in the background and keep synchronized with each other as well as normal vfs / vnode requests. So I will have to look more into sys/lock.h and read the TechNote. _______________________________________________ darwin-kernel mailing list | darwin-kernel(a)lists.apple.com Help/Unsubscribe/Archives: http://www.lists.apple.com/mailman/listinfo/darwin-kernel Do not post admin requests to the list. They will be ignored.

1 0