May 2006 - Darwin-dev - lists.apple.com

Re: How does one get the OS Product name programatically?
by plumber Idraulico 11 May '06

11 May '06

site_archiver(a)lists.apple.com Delivered-To: Darwin-dev(a)lists.apple.com On 11/05/06, Brendan Creane <bcreane(a)yahoo.com> wrote: Here's a (reconstructed) copy of sw_vers.c (source from cached copy of http://gobsd.com/code/darwin/build/sw_vers.c) Please retain the license header when re-distributing source code, especially when the license explicitly tells you to. You'll notice that I wrote that source file, and I wasn't joking when I said there is no API for this. sw_vers.c uses CoreFoundation private SPI, and you should not use this approach in shipping applications. Copy-pasting the declarations from CFPriv.h into the source file will let you build without installing the private headers, but the standard warnings about using SPI apply. This email sent to plumber(a)gnu-darwin.org make your test file on /System/Library/CoreServices/ServerVersion.plist or /System/Library/CoreServices/SystemVersion.plist load plist copy plist into dict get keys of dict you needn't CFPriv to do this -- Finlay _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/plumber%40gnu- darwin.org Best Regards, Plum _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… This email sent to site_archiver(a)lists.apple.com

1 0

Re: How does one get the OS Product name programatically?
by Finlay Dobbie 11 May '06

11 May '06

site_archiver(a)lists.apple.com Delivered-To: Darwin-dev(a)lists.apple.com Domainkey-signature: a=rsa-sha1; q=dns; c=nofws; s=beta; d=gmail.com; h=received:message-id:date:from:reply-to:to:subject:cc:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references; b=lluogsIIQQVPzmVA8QeWbHwOblReDTjg94vFzfxR5k7aGu4TV/yNF6uiTBYfOCIKcDshVyY/ESG4YZUaifdlcjQao+kpLXA6FxovBVkSZF78E33QI45LVPZhSU58uXh1S15IBOKypVWv2VYLswYTrJBptGvnPSOpEy5HXg3RI4g= Here's a (reconstructed) copy of sw_vers.c (source from cached copy of http://gobsd.com/code/darwin/build/sw_vers.c) Please retain the license header when re-distributing source code, especially when the license explicitly tells you to. You'll notice that I wrote that source file, and I wasn't joking when I said there is no API for this. sw_vers.c uses CoreFoundation private SPI, and you should not use this approach in shipping applications. Copy-pasting the declarations from CFPriv.h into the source file will let you build without installing the private headers, but the standard warnings about using SPI apply. -- Finlay _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… On 11/05/06, Brendan Creane <bcreane(a)yahoo.com> wrote: This email sent to site_archiver(a)lists.apple.com

1 0

re: How does one get the OS Product name programatically?
by Brendan Creane 11 May '06

11 May '06

site_archiver(a)lists.apple.com Delivered-To: Darwin-dev(a)lists.apple.com Domainkey-signature: a=rsa-sha1; q=dns; c=nofws; s=s1024; d=yahoo.com; h=Message-ID:Received:Date:From:Subject:To:Cc:MIME-Version:Content-Type:Content-Transfer-Encoding; b=qk4XMNWBNYE7K6+ieYytr+Pus7mOJUuUSa0Njtq6pUb/H7WZaPU63ztGRCXrXxAK8J/fajQDHmJYpSnBNQNhRf7h1KBF458zyrddrbpGf4+gWcvWE5uwl+no4kXatSHXVv1yD55Mwm1qGgOdxGVpzqhywRE74rMhwpsROaxsV0k= ; Here's a (reconstructed) copy of sw_vers.c (source from cached copy of http://gobsd.com/code/darwin/build/sw_vers.c) CFDictionaryGetValue() is the method for retrieving the contents of the plists that you discovered. cheers, bcreane #include <CoreFoundation/CoreFoundation.h> CF_EXPORT const CFStringRef _kCFSystemVersionProductNameKey; CF_EXPORT const CFStringRef _kCFSystemVersionProductCopyrightKey; CF_EXPORT const CFStringRef _kCFSystemVersionProductVersionKey; CF_EXPORT const CFStringRef _kCFSystemVersionProductVersionExtraKey; CF_EXPORT const CFStringRef _kCFSystemVersionProductUserVisibleVersionKey; // For loginwindow; see 2987512 CF_EXPORT const CFStringRef _kCFSystemVersionBuildVersionKey; CF_EXPORT const CFStringRef _kCFSystemVersionProductVersionStringKey; // Localized string for the string "Version" CF_EXPORT const CFStringRef _kCFSystemVersionBuildStringKey; void usage(char *progname) { fprintf(stderr, "Usage: %s [-productName|-productVersion|-buildVersion]\n", progname); exit(1); } int main(int argc, char *argv[]) { CFDictionaryRef dict= NULL; CFStringRef str = NULL; char cstr[256]; dict = _CFCopyServerVersionDictionary(); if (dict == NULL) dict = _CFCopySystemVersionDictionary(); if (dict == NULL) exit(1); if (argc == 2) { if (!strcmp(argv[1], "-productName")) str = CFDictionaryGetValue(dict, _kCFSystemVersionProductNameKey); else if (!strcmp(argv[1], "-productVersion")) { /* On Darwin, we set MacOSXProductVersion to the corresponding OS X release. This is for compatibility with scripts that set MACOSX_DEPLOYMENT_TARGET based on sw_vers -productVersion */ str = CFDictionaryGetValue(dict, CFSTR("MacOSXProductVersion")); if (str == NULL) str = CFDictionaryGetValue(dict, _kCFSystemVersionProductVersionKey); } else if (!strcmp(argv[1], "-buildVersion")) str = CFDictionaryGetValue(dict, _kCFSystemVersionBuildVersionKey); else usage(argv[0]); CFRetain(str); } else if (argc == 1) { str = CFStringCreateWithFormat(NULL, NULL, CFSTR("ProductName: %@\n" "ProductVersion: %@\n" "BuildVersion: %@"), CFDictionaryGetValue(dict, _kCFSystemVersionProductNameKey), CFDictionaryGetValue(dict, _kCFSystemVersionProductVersionKey), CFDictionaryGetValue(dict, _kCFSystemVersionBuildVersionKey)); } else { usage(argv[0]); } if (!CFStringGetCString(str, cstr, sizeof(cstr), CFStringGetSystemEncoding())) exit(1); printf("%s\n", cstr); CFRelease(str); CFRelease(dict); return 0; } _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… This email sent to site_archiver(a)lists.apple.com

1 0

Re: Shared mmap and data consistency upon a crash
by Felix Schwarz 10 May '06

10 May '06

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com Am 10.05.2006 um 21:28 schrieb Michael Smith: Date: Wed, 10 May 2006 11:02:13 +0200 From: Felix Schwarz <felix.schwarz(a)iospirit-gmbh.de> Subject: Shared mmap and data consistency upon a crash To: darwin-dev(a)lists.apple.com Message-ID: <4E6C2A59-FEEF-4E6C-88F6-27ECEAE2BD92(a)iospirit-gmbh.de> Content-Type: text/plain; charset=US-ASCII; delsp=yes; format=flowed Hello, I am currently exploring the possibilites to use mmap to speed up some of my most critical I/O, mmap() is a convenience interface, not a performance interface. Thanks for the quick response! Some benchmarking gives me: Adding 30000 sample records à 8100 byte using mmap(): 0.72958 seconds Adding 30000 sample records à 8100 byte using read()/write()/lseek(): 1.24004 seconds but am wondering about the reliability of mmap and the durability of changes made to a file this way for one particular edge case: If 1) I mmap() a file using MAP_SHARED, PROT_READ and PROT_WRITE 2) I write into the returned address space 3) my app crashes prior to reaching munmap() and close() To me, it feels like you are trying to solve the wrong problems. Hope all that explains it a bit better. Felix _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… On May 10, 2006, at 12:03 PM, darwin-dev-request(a)lists.apple.com wrote: Stop right here. What makes you think that mmap will "speed up" your I/O? If you want to avoid blocking during I/O, consider the Posix asynchronous I/O system calls. If you want to avoid the copy in/out from kernel space, consider using the F_NOCACHE fcntl(2) option. If you control your caching behaviour and don't expect another process will access your file data, this is typically the most "performant" way to go. I maybe should have added what it is that I'm writing: I'm writing a specialized, server-less database link library (similiar to SQLite by concept, my link library has only *very* basic functionality, though). The idea behind using mmap() is to pull indexed records into and out of a file as fast as possible and take advantage of any caching by the VMM, especially regarding the table inside the file that keeps record of all the records it stores. Since I've read elsewhere you can't mmap() files that are located on a network-volume, I've also written a fallback using read()/write()/ lseek().. It appears, that, in both cases, the data in the file seems to be in sync when crashing. I could observe both a delay before any changes could be made to the file after an instant restart of the little test app plus the data that was written last could be reread correctly. With either method. However, I'd still love an official confirmation from a kernel developer (or someone who is literate in that part of the kernel) since - I don't want to base data security on assumptions or lucky observations that may turn out wrong or may change with the next release of OS X - I'd love to use the mmap() solution - .. and then, I'm also plain curious ;-) If your app crashes, all bets are off - assuming you don't know where it crashed, you have no guarantee as to the state of anything it was doing, regardless of whether it had files mapped or not. The way the data is organized in the file and the order in which its data is modified ensures its always in a consistent state - the actual modification to the table that keeps record of all records is written last and with one single memcpy() (or write() for the alternative version). Typically, speaking as a developer, I find it's better to ship products that don't crash, rather than ones that are resistant to internal damage caused by crashes. = Mike In general, I do share your view. The focus in software development should always be on producing clean code in the first place, not trying to fix bad quality afterwards in other, new code. Rest assured, I spend considerable time in my projects on ensuring elegance and cleanness in the code and stability in the build. However, a database link library is a special case (again, sorry for not previously specifying the nature of the project more clearly). You can make sure your code works rock-solid. But you don't know who will end up linking it into which kind of application. And you don't know which other APIs and link libraries play a role in that application. Take OS X graphics APIs for example. There are a lot of ways that a user can crash your application if it - for example - uses QuickTime or Quartz ImageIO to load images and those choke on a file. You can neither prevent nor fix this. But your user's data should stay in a consistent state nonetheless. This email sent to site_archiver(a)lists.apple.com

1 0

RE: Shared mmap and data consistency upon a crash
by Michael Smith 10 May '06

10 May '06

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com On May 10, 2006, at 12:03 PM, darwin-dev-request(a)lists.apple.com wrote: Date: Wed, 10 May 2006 11:02:13 +0200 From: Felix Schwarz <felix.schwarz(a)iospirit-gmbh.de> Subject: Shared mmap and data consistency upon a crash To: darwin-dev(a)lists.apple.com Message-ID: <4E6C2A59-FEEF-4E6C-88F6-27ECEAE2BD92(a)iospirit-gmbh.de> Content-Type: text/plain; charset=US-ASCII; delsp=yes; format=flowed Hello, I am currently exploring the possibilites to use mmap to speed up some of my most critical I/O, mmap() is a convenience interface, not a performance interface. but am wondering about the reliability of mmap and the durability of changes made to a file this way for one particular edge case: If 1) I mmap() a file using MAP_SHARED, PROT_READ and PROT_WRITE 2) I write into the returned address space 3) my app crashes prior to reaching munmap() and close() To me, it feels like you are trying to solve the wrong problems. = Mike _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… Stop right here. What makes you think that mmap will "speed up" your I/O? If you want to avoid blocking during I/O, consider the Posix asynchronous I/O system calls. If you want to avoid the copy in/out from kernel space, consider using the F_NOCACHE fcntl(2) option. If you control your caching behaviour and don't expect another process will access your file data, this is typically the most "performant" way to go. If your app crashes, all bets are off - assuming you don't know where it crashed, you have no guarantee as to the state of anything it was doing, regardless of whether it had files mapped or not. Typically, speaking as a developer, I find it's better to ship products that don't crash, rather than ones that are resistant to internal damage caused by crashes. This email sent to site_archiver(a)lists.apple.com

1 0

Re: How does one get the OS Product name programatically?
by Finlay Dobbie 10 May '06

10 May '06

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com Domainkey-signature: a=rsa-sha1; q=dns; c=nofws; s=beta; d=gmail.com; h=received:message-id:date:from:reply-to:to:subject:cc:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references; b=nbp1r2HzCN9VFptyPLo4iL5KtHML/rzhRBpwN19GkCGoQdEvxVU2Ig21l90ZDxFHLUQSOv6pgrJEYbj8i2bxGt1Q9Z7c1iLhk4rPbkyoK7VfNIJ+LIIQ1tGfQHflINddgYfjWi1HadE7W8ZEVeS+XmDNEyX1ssLgbY24sxWT1g8= >From posts on macos-x-server I see that "sw_vers –productName" will give me exactly what I am looking for (i.e. whether I am running Mac OS X or Mac OS X Server), but I'd like to get this within my program without having to shell out to a tool (it gets it from *some* API). There is no public API for this. As you ascertain, checking for ServerVersion.plist is a way of doing this that will work on all current released verisons of OS X Server. File a bug. -- Finlay _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… On 10/05/06, David Litwin <david_litwin(a)bigfix.com> wrote: This email sent to site_archiver(a)lists.apple.com

1 0

How does one get the OS Product name programatically?
by David Litwin 10 May '06

10 May '06

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com Thread-index: AcZ0YRDxigFhHK0TRuabVCUvKFQ8IA== Thread-topic: How does one get the OS Product name programatically? _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… This email sent to site_archiver(a)lists.apple.com

1 0

RE: Shared mmap and data consistency upon a crash
by Sanjay Mishra 10 May '06

10 May '06

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com Thread-index: AcZ0EJo6T4GhLlU7Q6+X9yyilMmXnQAAQvCQ Thread-topic: Shared mmap and data consistency upon a crash Hi Felix, You need to msync() to get your data into your file soon after you changed the data in the shared file. You have to certainly do it before the point where you assume probable crash lurks. --Sanjay -----Original Message----- From: darwin-dev-bounces+samishra=quark.com(a)lists.apple.com [mailto:darwin-dev-bounces+samishra=quark.com@lists.apple.com] On Behalf Of Felix Schwarz Sent: Wednesday, May 10, 2006 2:32 PM To: darwin-dev(a)lists.apple.com Subject: Shared mmap and data consistency upon a crash Hello, I am currently exploring the possibilites to use mmap to speed up some of my most critical I/O, but am wondering about the reliability of mmap and the durability of changes made to a file this way for one particular edge case: If 1) I mmap() a file using MAP_SHARED, PROT_READ and PROT_WRITE 2) I write into the returned address space 3) my app crashes prior to reaching munmap() and close() Pseudocode: // mmap file (1) p_m = fopen(..) mptr = mmap(..p_m..) // Modify the file (2) *mptr = 0xBEEF; // Crash here (3) *((int *) 0) = 1; // Memory would have been unmapped and the corresponding file closed here // However, this shall not be reached in this example munmap(); fclose(p_m); Will the data written in 2) actually show up in the file? Does mmap() take care that mapped memory and the file it is mapped to are synced in the event of a crash? Felix _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/samishra%40quark.com This email sent to samishra(a)quark.com _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… This email sent to site_archiver(a)lists.apple.com

1 0

Shared mmap and data consistency upon a crash
by Felix Schwarz 10 May '06

10 May '06

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com Hello, If 1) I mmap() a file using MAP_SHARED, PROT_READ and PROT_WRITE 2) I write into the returned address space 3) my app crashes prior to reaching munmap() and close() Pseudocode: // mmap file (1) p_m = fopen(..) mptr = mmap(..p_m..) // Modify the file (2) *mptr = 0xBEEF; // Crash here (3) *((int *) 0) = 1; Felix _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… I am currently exploring the possibilites to use mmap to speed up some of my most critical I/O, but am wondering about the reliability of mmap and the durability of changes made to a file this way for one particular edge case: // Memory would have been unmapped and the corresponding file closed here // However, this shall not be reached in this example munmap(); fclose(p_m); Will the data written in 2) actually show up in the file? Does mmap() take care that mapped memory and the file it is mapped to are synced in the event of a crash? This email sent to site_archiver(a)lists.apple.com

1 0

OpenDirectory: creating a LocalCachedUser?
by Brian Bergstrand 09 May '06

09 May '06

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Hi all, $ dscl /NDS/PROSOFT -authonly frodo.users Password: $ dscl /NDS/PROSOFT/frodo.users -read / PrimaryGroupID: c7590d23 AccountName: frodo UniqueID: c7590d23 NodePath: NDS PROSOFT frodo.users UserShell: /bin/bash AuthenticationAuthority: ;LocalCachedUser; GeneratedUID: 00D61774-D651-D811-B0B8-0004757FF7AD RecordType: Users RealName: Frodo Baggins $ What I need to do: Here's the DS Requests that I currently respond to: kOpenDirNode kCloseDirNode kGetDirNodeInfo kGetAttributeEntry kGetAttributeValue kDoDirNodeAuth kCloseAttributeList kCloseAttributeValueList kDoDirNodeAuthOnRecordType Any pointers would be greatly appreciated. Thanks. -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.3 (Darwin) iD8DBQFEYRIredHYW7bHtqIRAnkmAJ9NcXIs0E5SUSYrzxphG8P4JikJxwCfaR+u afPG31K36Dg8jnAJuwdxtEg= =Ur9A -----END PGP SIGNATURE----- _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… So I've got a DirectoryServices plugin going and I can do authentication to the remote DS server: How do I now turn this user record into a local cached user so that a user can actually login to the GUI using only the remote DS credentials? Create local user record. Create local user home directory. Mount network home directory for file access (this part is easy once I know where to do it). Also, regarding the UID/GID, I'm assuming a ID space of 0-2^31. In the above example, the UID/GID is a hash of the UUID string. Is this a valid # for Tiger/Panther or is the ID space limited to 2^16? As you can see I'm implementing the bare minimum to get auth working. This is not meant to provide full blown access to the remote DS. This email sent to site_archiver(a)lists.apple.com

1 0