April 2008 - Darwin-dev - lists.apple.com

Synchronization primitives (was:Re: Where is atomicqueue?)
by Army Research Lab 29 Apr '08

29 Apr '08

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com Thread-index: Acip663m7D5QhBXeEd23XAAdT0T19A== Thread-topic: Synchronization primitives (was:Re: Where is atomicqueue?) User-agent: Microsoft-Entourage/11.4.0.080122 On 4/28/08 3:36 PM, "Terry Lambert" <tlambert(a)apple.com> wrote: > On Apr 28, 2008, at 4:31 AM, Army Research Lab wrote: >> On 4/24/08 3:17 PM, "Terry Lambert" <tlambert(a)apple.com> wrote: >>> Short answer: >>> >>> (1) It's protected by #ifdef __ppc__, so it's PPC only >>> >>> (2) It's in a header <libkern/OSAtomic.h>, so you should only >>> _expect_ >>> to be able to use it in the kernel; the fact that there are some >>> inline functions there notwithstanding >>> >>> (3) The atual implementaiton is as an external symbol in the Libkern >>> and System6.0 exports for PPC only >>> >>> So to get at it you will need to be linked against the kernel (you >>> will need to be either kernel code or a KEXT), you will need to be >>> running PPC-only, and you will need to declare an explicit dependency >>> for your KEXT on an exported symbol set or on the entire kernel. >> >> OK, are there spinlocks in user space? I want to try a couple of >> experiments with some code, just to see if pthread_mutex_lock() is >> better, >> or if spinlocking is better. > > If you use > > nm /usr/lib/libSystem.B.dylib > > you can probably answer these questions for yourself. Good point. And, yes, I see now that spin_lock(), spinlock_try(), and spin_unlock() are defined. I still haven't found the headers for them, but I figure that's just a matter of time (unless they aren't supported). > For things for which there are not exported prototypes in header > files, there is no explicit support for them. So technically, i386 has > OSAtomicEnqueue(), spin_lock(), et. al., but they are not supported, > as such. You can see their source code in the published Darwin Libc > sources. Understood. And if I can't find the headers for them, I won't bother with them. > The pthread_mutex() is going to be heavier weight than spin locks, but > spin locks spin, which means they will meaninglessly burn CPU cycles, > so "better" is really going to depend on your definition; if you care > about battery life and hold them a long time, the answer will be "no", > and if you are single CPU, the answer may be "they will just cause you > to deadlock". Any spin lock you use, or OSAtomicEnqueue, etc., is > probably not going to be desirable to use in any PTHREAD_SCOPE_SYSTEM > rather than PTHREAD_SCOPE_PROCESS object. I understand the problems with them; however, the case I plan on supporting generally locks for just long enough to write a value to a shared location, with the values being 32 or 64 bit ints. That was why I got interested in the various atomic functions. I know that they have a much higher likelihood of being optimized than using pthread lock/write/unlock. > I would recommend you restrict yourself to supported interfaces. I don't plan on doing otherwise. If I can't find a supported interface for it, then I'll have to find another way. > Also, if you want to take an atomicqueue discussion and go off talking > about spin locks, you probably ought to either change the subject to > reflect a general interest in synchronization primitives, or to > explicitly talk about that instead, or the information isn't going to > be indexed properly and someone coming after you is just going to ask > the same questions again. My apologies. Thanks, Cem Karan _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… This email sent to site_archiver(a)lists.apple.com

1 0

Re: Where is atomicqueue?
by Terry Lambert 28 Apr '08

28 Apr '08

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com Short answer: (1) It's protected by #ifdef __ppc__, so it's PPC only (3) The atual implementaiton is as an external symbol in the Libkern and System6.0 exports for PPC only So to get at it you will need to be linked against the kernel (you will need to be either kernel code or a KEXT), you will need to be running PPC-only, and you will need to declare an explicit dependency for your KEXT on an exported symbol set or on the entire kernel. If you use nm /usr/lib/libSystem.B.dylib you can probably answer these questions for yourself. I would recommend you restrict yourself to supported interfaces. -- Terry _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… On Apr 28, 2008, at 4:31 AM, Army Research Lab wrote: On 4/24/08 3:17 PM, "Terry Lambert" <tlambert(a)apple.com> wrote: (2) It's in a header <libkern/OSAtomic.h>, so you should only _expect_ to be able to use it in the kernel; the fact that there are some inline functions there notwithstanding OK, are there spinlocks in user space? I want to try a couple of experiments with some code, just to see if pthread_mutex_lock() is better, or if spinlocking is better. For things for which there are not exported prototypes in header files, there is no explicit support for them. So technically, i386 has OSAtomicEnqueue(), spin_lock(), et. al., but they are not supported, as such. You can see their source code in the published Darwin Libc sources. The pthread_mutex() is going to be heavier weight than spin locks, but spin locks spin, which means they will meaninglessly burn CPU cycles, so "better" is really going to depend on your definition; if you care about battery life and hold them a long time, the answer will be "no", and if you are single CPU, the answer may be "they will just cause you to deadlock". Any spin lock you use, or OSAtomicEnqueue, etc., is probably not going to be desirable to use in any PTHREAD_SCOPE_SYSTEM rather than PTHREAD_SCOPE_PROCESS object. Also, if you want to take an atomicqueue discussion and go off talking about spin locks, you probably ought to either change the subject to reflect a general interest in synchronization primitives, or to explicitly talk about that instead, or the information isn't going to be indexed properly and someone coming after you is just going to ask the same questions again. This email sent to site_archiver(a)lists.apple.com

1 0

Re: Where is atomicqueue?
by Army Research Lab 28 Apr '08

28 Apr '08

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com Thread-index: AcipI1XjlEwBqhUWEd2rrQAdT0T19A== Thread-topic: Where is atomicqueue? User-agent: Microsoft-Entourage/11.4.0.080122 On 4/24/08 3:17 PM, "Terry Lambert" <tlambert(a)apple.com> wrote: > On Apr 24, 2008, at 10:34 AM, Army Research Lab wrote: >> On 4/24/08 11:40 AM, "Shawn Erickson" <shawnce(a)gmail.com> wrote: >>> On Thu, Apr 24, 2008 at 8:30 AM, Shawn Erickson <shawnce(a)gmail.com> >>> wrote: >>>> On Thu, Apr 24, 2008 at 4:35 AM, Army Research Lab <ckaran(a)arl.army.mil >>>>> >>>> wrote: >>>>> I've been poking through the man pages, and discovered the docs for >>>>> <libkern/OSAtomic.h>; I see the documentation for atomic and for >>>>> spinlock, >>>>> but 'man barrier' also lists atomicqueue. Unfortunately, >>>>> although the man >>>>> pages claim that I should find it, it isn't there. I also can't >>>>> find >>>>> anything like it in /usr/include. Does it exist? >>>>> >>>>> BTW, I'm looking at all of this on a OS X 10.4.11 machine... >>>> >>>> >>>> <http://developer.apple.com/documentation/Cocoa/Conceptual/Multithreading/T >>>> hr >>>> eadSafety/chapter_5_section_6.html> >>>> >>>> [0:505] > find /usr/include -name "OSAtomic.h" >>>> /usr/include/libkern/OSAtomic.h >>>> >>>> That is on a 10.4.11 system with Xcode 2.5. >>> >>> Ah maybe you are talking about that header not containing the >>> definition for OSAtomic queue? >>> >>> From the 10.5 man page... >>> >>> "Most of these functions first appeared in Mac OS 10.4 (Tiger). The >>> "Orig" forms of the boolean operations, the "int", "long" and "ptr" >>> forms of compare-and-swap, and lockless enqueue/dequeue first >>> appeared >>> in Mac OS 10.5 (Leopard)." >>> >>> The atomic queue does exist on 10.4 when in the kernel. >> >> OK, maybe that is the problem then; I did look in >> /usr/include/libkern/OSAtomic.h before I posted, but there is just a >> comment >> about the queue, not the queue itself. Do I have to download the >> Darwin >> sources to get access to it? Note that I'm just curious here, the >> only >> thing I need is spinlock, and if I'm clever, I won't even need that. > > > Short answer: > > (1) It's protected by #ifdef __ppc__, so it's PPC only > > (2) It's in a header <libkern/OSAtomic.h>, so you should only _expect_ > to be able to use it in the kernel; the fact that there are some > inline functions there notwithstanding > > (3) The atual implementaiton is as an external symbol in the Libkern > and System6.0 exports for PPC only > > So to get at it you will need to be linked against the kernel (you > will need to be either kernel code or a KEXT), you will need to be > running PPC-only, and you will need to declare an explicit dependency > for your KEXT on an exported symbol set or on the entire kernel. OK, are there spinlocks in user space? I want to try a couple of experiments with some code, just to see if pthread_mutex_lock() is better, or if spinlocking is better. Thanks, Cem Karan _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… This email sent to site_archiver(a)lists.apple.com

1 0

DTrace Error in Instruments
by Achille 27 Apr '08

27 Apr '08

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com Dkim-signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:message-id:date:from:to:subject:mime-version:content-type:content-transfer-encoding:content-disposition; bh=U0SNomMjth1qE0+iekDGlgV1CjZJo6HaRBKBJopdnes=; b=TFir8oeKdhcxqOaOZrDNmvclfCCXQZSDVaYeX9yYVPbbauKZ2v4rZJ2uN0ZnYxZbIqLqutVM+Pk0IDaIh59Qn2Nlc7TJvePPh/mFct+xWROrmNorkp/+OAktVWuoyIejSS5HiEhb1oQj8Wv+/7GwwC85UBUONycEK2AGRb8QWWI= Domainkey-signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=message-id:date:from:to:subject:mime-version:content-type:content-transfer-encoding:content-disposition; b=ibHgvtFbMweMpQ6Xsw9YVV2VHgtEl+JJJNi3vFTI4pUvpZT5xhGxoW67A6ByPu+C0hQ4DDs5aLCKuo9rcoK3lu2M+7oOFZCaC0gcMrApAhqEjWmBBtzOX4cQatj9Xpx48bRiqCjGOUZEHKUiQzwpqTvfFjPKlGDDFAG9blMQVcI= When monitoring file activity for All Processes in Instruments I often get errors. This happens every 1 to 5 minutes. "DTrace Error error on enabled probe ID 5 (ID 17606: syscall::close:entry): invalid address (0xbffff378) in action #8" Screenshot: http://tinyurl.com/4eqtjk I was unable to find any discussion of this error on Leopard, but some people have seen this in solaris: http://www.opensolaris.org/jive/thread.jspa?messageID=10250 "> dtrace: error on enabled probe ID ... This is because you're doing a copyinstr from a memory address which has not yet been faulted in. This is known as a "first touch" fault, and is described in the Solaris Dynamic Tracing Guide..." This error happens when Time Machine is running. Is there any way to avoid it? _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… This email sent to site_archiver(a)lists.apple.com

1 0

Re: Source for L2TP extension?
by Andre-John Mas 26 Apr '08

26 Apr '08

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com Thank you, they are indeed there. Andre _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… On 26-Apr-08, at 17:20 , Allan Nathanson wrote: On Apr 26, 2008, at 5:02 PM, Andre-John Mas wrote: Are there any plans to make the source for the L2TP or the PPTP kernel extensions available? If they are available I don't see them explicitly listed on: http://www.opensource.apple.com/darwinsource/10.5.2/ Check the "ppp" project. This email sent to site_archiver(a)lists.apple.com

1 0

Re: Source for L2TP extension?
by Allan Nathanson 26 Apr '08

26 Apr '08

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com Check the "ppp" project. - Allan _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… On Apr 26, 2008, at 5:02 PM, Andre-John Mas wrote: Are there any plans to make the source for the L2TP or the PPTP kernel extensions available? If they are available I don't see them explicitly listed on: http://www.opensource.apple.com/darwinsource/10.5.2/ This email sent to site_archiver(a)lists.apple.com

1 0

Source for L2TP extension?
by Andre-John Mas 26 Apr '08

26 Apr '08

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com Hi, http://www.opensource.apple.com/darwinsource/10.5.2/ Andre _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… Are there any plans to make the source for the L2TP or the PPTP kernel extensions available? If they are available I don't see them explicitly listed on: This email sent to site_archiver(a)lists.apple.com

1 0

Writing code that deals with time values
by Terry Lambert 25 Apr '08

25 Apr '08

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com This will be my last posting on this topic. -- Here is a short primer for people interested in writing working code: Q: I want to use arbitrary dates on Mac OS X A: Use NSDate/CFDate 32 bit time_t Jan 1 1970 00:00 +/- 2^31 seconds (Y1902 - Y2038) 64 bit time_t Y1970 +/- 2^63 seconds (~ +/- 209 billion years) struct timespec same as time_t struct timeval same as time_t struct tm Y1900 +/- 2^31 years (~ +/- 2 billion years) -- Terry _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… Q: But I want my code to be portable between Linux, Mac OS X, and other UNIX operating systems; what interfaces can I use? A: You can use the POSIX C library functions and you can use the POSIX system functions, as long as you are aware of natural type restrictions and what POSIX does or doesn't guarantee in terms of implementation details Q: What are the natural types used by the POSIX C library functions? A: The POSIX C library functions use a mix of struct tm, struct timespec, struct timeval, and time_t Q: What are the natural types used by the POSIX system functions? A: The POSIX system functions use a mix of struct timespec, struct timeval, and time_t Q: What is the lowest common denominator - the "natural" type - for time values on POSIX systems? A: time_t, which is a signed value in seconds relative to the UNIX Epoch of January 1st 1970 Q: What are the restrictions on date ranges are supported by these types? A: It depends on the type, and whether the system is 32 or 64 bit: Q: Doesn't that mean that there's potentially a problem for UNIX systems around Y2038? A: Yes. Computer scientists are aware of the issue, and have known about it for many years. There is even a Wikipedia article on it. Q: So if I am using POSIX interfaces, I should use a 64 bit time_t or struct tm to represent large date ranges? A: struct tm *can* represent large date ranges, but conversion is ambiguous; use a 64 bit time_t where possible to avoid the ambiguity. Q: What if I don't want to use a 64 bit time_t , can I use a struct tm instead? A: Either limit the ranges in which your software is going to operate, or carry around your own functions; the POSIX C library function conversions are ambiguous; it's impossible to use struct tm and get consistent cross-platform results Q: Why? A: The POSIX interfaces which deal with struct tm, as specified, lack reflexivity under certain conditions. Q: What does that mean? A: There is no function to get the current time without conversion from a time_t, and the range of the conversion into and out of time_t is bounded by the range bounds on time_t, and some of the functions are inconsistent or insufficiently specific. This translates to not being able to trust the conversion routines in a couple of cases. Q: What if I disagree with the Mac OS X interpretation of a standards ambiguity? A: File a bug. Q: What are the ambiguous cases? A: You can't trust the conversion routines to be able to be called to convert, and then convert back a value, and give the original input back to you. This is both because some values you could put in a struct tm can't fit into a time_t (as an internal implementation detail), and because some of the conversion routines don't specify an Epoch date (strftime(3)). Also, because there is no method of obtaining the current time that doesn't transit a system interface, there's no way to get the current time as anything other than a time_t value. Q: So I can't get the current time as a struct tm? A: No. Not without calling one of gmtime(3), gmtime_r(3), localtime(3), or localtime_r(3) on the results of a call to time(2), which only returns a time_t. Q: So my code will always have the Y2038 bug? A: No. You can compile your code as 64 bit, which will give you a 64 bit time_t; this will prevent having the Y2038 bug. Q: How do I avoid POSIX interface ambiguity? A: Start by using 4 byte year+century fields instead of 2 byte year- within-century fields; avoid %y and %C as separate formatting characters. If you insist on using these format specifiers, the best you can achieve is changing your T2038 bug into a Y2069 bug instead, assuming you never care about the current time after Y2038. Q: What's the bottom line, if time_t is 32 bits on my system? A: Your code has the Y2038 bug, if it deals with the current time or does math on time using standard functions, such as difftime(3). Q: Are there any other alternatives? A: Yes. Carry around your own private non-POSIX date manipulation functions as part of your software. This will make your code behave consistently on all platforms. This email sent to site_archiver(a)lists.apple.com

1 0

Re: strptime cannot parse date older than 1938
by Philip Aker 25 Apr '08

25 Apr '08

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com On 08-04-25, at 09:53, Jamison Hope wrote: I guess FAQ #1 here http://docs.info.apple.com/article.html?artnum=19843 isn't quite right anymore. Philip Aker echo astwta@lvpc.dslh@nl | tr a-z@. p-za-o.@ Sent from my SE/30 _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… Remember back in 1999 when Apple was running ads talking about Y2K? "We may not get everything right, but at least we knew the century was going to end. Macintosh was designed to work perfectly with dates all the way up to the year 29,940. But have no fear. We’re already hard at work on the Y30K problem." That was referring to DateTimeUtils.h facilities (LongDateTime record in particular), not unix ones. Most of the functionality has now been replaced by CFDate et. al. but LongDateTime is needed for backwards compatibility to OS 9 with AppleScript and a few other technologies. This email sent to site_archiver(a)lists.apple.com

1 0

Sources Posted: iPhone SDK Beta 4
by William Siegrist 25 Apr '08

25 Apr '08

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com The darwin team has posted sources for the iPhone SDK Beta 4. <http://www.opensource.apple.com/darwinsource/iPhoneSDKBeta4/> Thank you. ---- William Siegrist Darwin Team _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… This email sent to site_archiver(a)lists.apple.com smime.p7s

1 0