May 2009 - Darwin-dev - lists.apple.com

Re: Use of hdiutil in a launchdaemon?
by Soren Spies 09 May '09

09 May '09

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com On May 8, 2009, at 16:46, Nathan Herring wrote: -- Soren Spies CoreOS::DiskImages _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… diskimages-helper[xxx:yyy] _checkJournalBeforeCopy: trying to DISABLE journaling on disk2s2 As Damien mentioned, the bootstrap contexts are different when running "in the background" (whether it be from 'cron', with 'nohup', or in a LaunchDaemon) so many subsystems that expect certain services or user contexts may not behave correctly. In this particular case, it looks like you're running into the fact that we use 'diskutil' to disable journaling during the copy phase of creating a disk image from a source folder. A workaround might be to pass '-fs HFS+' to 'hdiutil' instead of letting it default to matching the source filesystem (which sounds like it is HFS+J). This email sent to site_archiver(a)lists.apple.com

1 0

Re: Use of hdiutil in a launchdaemon?
by Damien Sorresso 08 May '09

08 May '09

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com Nathan, _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… This email sent to site_archiver(a)lists.apple.com On May 8, 2009, at 3:11 PM, Nathan Herring wrote: I have a script that builds, places files in correct places, and am trying to add a call to hdiutil to build a compressed disk image from the placed files. This works swimmingly at the command line, either in a login window session, or an ssh session. However, it hangs (or blocks on some nebulous, never-to-be-entered input) when it's run ultimately from a LaunchDaemon run from /Library/ LaunchDaemons running as me. Is there something (well known) I need to do specifically to fix this? And or some better way to diagnose what's going on here? You appear to be laboring under the assumption that changing your UID is sufficient to run as your user. This is not true on our platform. When running from a daemon, hdiutil is in a different bootstrap than running at a loginwindow or GUI session and thus has different capabilities. You should be running as an agent to be truly running in your user's context. -- Damien Sorresso BSD Engineering Apple Inc. smime.p7s

1 0

RE: Use of hdiutil in a launchdaemon?
by Nathan Herring 08 May '09

08 May '09

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com Thread-index: AcnQKeHlrztPz7XqS/uZTBcZfCOumgADJt5g Thread-topic: Use of hdiutil in a launchdaemon? I found a "-quiet" flag hidden among the arguments I hadn't seen, and so now with "-debug" and "-verbose" on, it appears it's hanging at a point where diskimages-helper[xxx:yyy] _checkJournalBeforeCopy: trying to DISABLE journaling on disk2s2 >From the command line, this eventually succeeds. From with the launchdaemon process, it never does. This is 10.5.6 9G55 (i386). -----Original Message----- From: darwin-dev-bounces+nathanh=microsoft.com(a)lists.apple.com [mailto:darwin-dev-bounces+nathanh=microsoft.com@lists.apple.com] On Behalf Of Nathan Herring Sent: Friday, May 08, 2009 3:11 PM To: Dev Darwin Subject: Use of hdiutil in a launchdaemon? I have a script that builds, places files in correct places, and am trying to add a call to hdiutil to build a compressed disk image from the placed files. This works swimmingly at the command line, either in a login window session, or an ssh session. However, it hangs (or blocks on some nebulous, never-to-be-entered input) when it's run ultimately from a LaunchDaemon run from /Library/LaunchDaemons running as me. Is there something (well known) I need to do specifically to fix this? And or some better way to diagnose what's going on here? Thx in advance, nh -- Nathan Herring com.microsoft.devdiv.clr.os/development _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… This email sent to site_archiver(a)lists.apple.com

1 0

Use of hdiutil in a launchdaemon?
by Nathan Herring 08 May '09

08 May '09

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com Thread-index: AcnQKeHlrztPz7XqS/uZTBcZfCOumg== Thread-topic: Use of hdiutil in a launchdaemon? I have a script that builds, places files in correct places, and am trying to add a call to hdiutil to build a compressed disk image from the placed files. This works swimmingly at the command line, either in a login window session, or an ssh session. However, it hangs (or blocks on some nebulous, never-to-be-entered input) when it's run ultimately from a LaunchDaemon run from /Library/LaunchDaemons running as me. Is there something (well known) I need to do specifically to fix this? And or some better way to diagnose what's going on here? Thx in advance, nh -- Nathan Herring com.microsoft.devdiv.clr.os/development _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… This email sent to site_archiver(a)lists.apple.com

1 0

Help on shoutcast player
by Arnab Ganguly 07 May '09

07 May '09

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com Dkim-signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:date:message-id:subject :from:to:content-type; bh=KCzbaENZQW/+Dq4qgeayc3+Cht4Dnzynu4cD/seU3eI=; b=o0uMSBJQrL+SuVBJS6gkjTXxQVDHVEQW/G71Z+SiPOYvO+r4+8c+Eo+ZzpopVqAuJ5 IylWOOh3gmPhqI89lq9Yu7OfdtigcDYEV8yFjYmJu3QRZwr8WD9LZSRu1tUwq8yles/Y 4hEKyPhN4Ty5p9iLgerIsDoHD5Ltb+uu5uhjI= Domainkey-signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:date:message-id:subject:from:to:content-type; b=WB1XyfyCQFOrSvsV+Mv8+u052BnlRb1i3MXAoOmLOf2WswjRAue/IlevdJmBeRJHWe rjGUZo6gVKhvutI8ZplEAQgOSDPzDf4il30urCHbFAvzut/64Gjzclvvb7Tx9QUYkSBb 6LYF8n1r2F4acVENZqRn+c6Mq1pdIp7agNcTw= _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… This email sent to site_archiver(a)lists.apple.com

1 0

Re: Best Technique for Watched folders ?
by Josh Graessley 05 May '09

05 May '09

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com -josh On May 5, 2009, at 2:48 PM, Mark Gilbert wrote: Thanks Jean-Daniel I did start looking at kqueue but it all seemed rather obscure. Cheers Mark Gilbert. On 5 May 2009, at 22:41, Jean-Daniel Dupas wrote: Le 5 mai 09 à 22:04, Mark Gilbert a écrit : Folks. The key issues seem to be: - How do we find them ? - bulk search, iteration ? - What happens as our folder gets full up....? _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/jgraessley%40apple.com This email sent to jgraessley(a)apple.com _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… kqueue really isn't that hard and you can use a kqueue with a CFSocket to get notifications on a run loop when the kqueue has an event ready to read. polling is a waste. kqueue will eliminate the need to poll. I think for my starting point I am going to call stat() on the parent folder, that seems pretty reliable to pick up any kind of change in the contents (unless I am kidding myself). I am going to poll that once every few secs. FSEvents looks interesting - thanks for that. I am thinking that for a single folder, which doesnt change much the stat() approach is probably gentle enough not to matter, and its going to save me learning all about FSEvents. Once I spot a change, I will start the hard work, I think that is the bit which is more exotic... For the 3rd time in my life I am about to start writing folder watching code. It's one of those things thats easy to do in a half hearted manner, but somewhat complicated to do in an efficient and comprehensive way. I thought I might canvas some ideas from other people before I embark on this next generation. - What are we watching for ? - new files, changed files ? renamed files ? - How do we know which ones have already been processed ? - mod date ? some kind of flag ? - How do we know when its safe to 'trigger' on a file which is being written into the folder ? - How do we do all this efficiently without hammering the file system or the CPU ? I have my own ideas on some of these things, but I wanted to see if anyone else had a strong point of view. In particular whether its a good plan to mark files in some way, and whether this can be keyed automatically into the discovery process. Also, what is the best discovery process ? I am sure Apple would tell us to use event notifications rather than polling the folder, but I have not had much luck figuring out what can be notified and how. I can give you an answer at least about notifications. You can use either the FSEvent API or kevent/kqueue API (or a launchd task and let launchd watching the folder for you). This email sent to site_archiver(a)lists.apple.com

1 0

Re: Best Technique for Watched folders ?
by Mark Gilbert 05 May '09

05 May '09

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com Thanks Jean-Daniel I did start looking at kqueue but it all seemed rather obscure. Cheers Mark Gilbert. On 5 May 2009, at 22:41, Jean-Daniel Dupas wrote: Le 5 mai 09 à 22:04, Mark Gilbert a écrit : Folks. The key issues seem to be: _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… I think for my starting point I am going to call stat() on the parent folder, that seems pretty reliable to pick up any kind of change in the contents (unless I am kidding myself). I am going to poll that once every few secs. FSEvents looks interesting - thanks for that. I am thinking that for a single folder, which doesnt change much the stat() approach is probably gentle enough not to matter, and its going to save me learning all about FSEvents. Once I spot a change, I will start the hard work, I think that is the bit which is more exotic... For the 3rd time in my life I am about to start writing folder watching code. It's one of those things thats easy to do in a half hearted manner, but somewhat complicated to do in an efficient and comprehensive way. I thought I might canvas some ideas from other people before I embark on this next generation. - What are we watching for ? - new files, changed files ? renamed files ? - How do we find them ? - bulk search, iteration ? - How do we know which ones have already been processed ? - mod date ? some kind of flag ? - How do we know when its safe to 'trigger' on a file which is being written into the folder ? - What happens as our folder gets full up....? - How do we do all this efficiently without hammering the file system or the CPU ? I have my own ideas on some of these things, but I wanted to see if anyone else had a strong point of view. In particular whether its a good plan to mark files in some way, and whether this can be keyed automatically into the discovery process. Also, what is the best discovery process ? I am sure Apple would tell us to use event notifications rather than polling the folder, but I have not had much luck figuring out what can be notified and how. I can give you an answer at least about notifications. You can use either the FSEvent API or kevent/kqueue API (or a launchd task and let launchd watching the folder for you). This email sent to site_archiver(a)lists.apple.com

1 0

Re: Best Technique for Watched folders ?
by Jean-Daniel Dupas 05 May '09

05 May '09

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com Le 5 mai 09 à 22:04, Mark Gilbert a écrit : Folks. The key issues seem to be: _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… For the 3rd time in my life I am about to start writing folder watching code. It's one of those things thats easy to do in a half hearted manner, but somewhat complicated to do in an efficient and comprehensive way. I thought I might canvas some ideas from other people before I embark on this next generation. - What are we watching for ? - new files, changed files ? renamed files ? - How do we find them ? - bulk search, iteration ? - How do we know which ones have already been processed ? - mod date ? some kind of flag ? - How do we know when its safe to 'trigger' on a file which is being written into the folder ? - What happens as our folder gets full up....? - How do we do all this efficiently without hammering the file system or the CPU ? I have my own ideas on some of these things, but I wanted to see if anyone else had a strong point of view. In particular whether its a good plan to mark files in some way, and whether this can be keyed automatically into the discovery process. Also, what is the best discovery process ? I am sure Apple would tell us to use event notifications rather than polling the folder, but I have not had much luck figuring out what can be notified and how. I can give you an answer at least about notifications. You can use either the FSEvent API or kevent/kqueue API (or a launchd task and let launchd watching the folder for you). This email sent to site_archiver(a)lists.apple.com

1 0

Best Technique for Watched folders ?
by Mark Gilbert 05 May '09

05 May '09

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com Folks. The key issues seem to be: Cheers Mark. _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… For the 3rd time in my life I am about to start writing folder watching code. It's one of those things thats easy to do in a half hearted manner, but somewhat complicated to do in an efficient and comprehensive way. I thought I might canvas some ideas from other people before I embark on this next generation. - What are we watching for ? - new files, changed files ? renamed files ? - How do we find them ? - bulk search, iteration ? - How do we know which ones have already been processed ? - mod date ? some kind of flag ? - How do we know when its safe to 'trigger' on a file which is being written into the folder ? - What happens as our folder gets full up....? - How do we do all this efficiently without hammering the file system or the CPU ? I have my own ideas on some of these things, but I wanted to see if anyone else had a strong point of view. In particular whether its a good plan to mark files in some way, and whether this can be keyed automatically into the discovery process. Also, what is the best discovery process ? I am sure Apple would tell us to use event notifications rather than polling the folder, but I have not had much luck figuring out what can be notified and how. Well, I hope someone out there has some strong views or experience on this topic, and is willing to share. This email sent to site_archiver(a)lists.apple.com

1 0

Re: Setting st_flags returned by stat
by Terry Lambert 03 May '09

03 May '09

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com -- Terry Will I be able to use ATTR_CMN_FLAGS on UFS volumes as well? Thanks. so I have to believe you are looking at your 10.4 system again. 2. If I do an lstat() on a file and take its st_flags: lstat(path, &statbuf); Is this right? -- Terry How do we set these flags back to a file? Should it be done using the chflags() function call? As Terry Lambert noted, you use the chflags() call to set flags on a file. However do not set undefined bits - only use the bits in /usr/include/sys/stat.h. Specifically these are: and if you're the super-user you can set: Keep in mind that most of these bits have defined meanings in the file system: IMMUTABLE and APPEND in particular. Bits such as NODUMP are not really used on MacOS X. --dominic _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… You have to ask if the volume type supports the attribute before attempting to use it. But yes, UFS supports the attribute; that's where HFS derived their implementation of the attribute from. On May 1, 2009, at 11:56 PM, rohan a wrote: On Sat, May 2, 2009 at 12:21 PM, rohan a <info1686(a)gmail.com> wrote: Atleast on my Mac 10.4 there does not seem to be any lchflags. I have only chflags() and fchflags() available to me. I will use getattrlist() and setattrlist() which provide ATTR_CMN_FLAGS to get/set the flags. On Sat, May 2, 2009 at 12:06 PM, rohan a <info1686(a)gmail.com> wrote: So doesn't Mac OS provide an API to work on symboic links rather than its target ? Something like an lstat(). On Sat, May 2, 2009 at 2:12 AM, Terry Lambert <tlambert(a)apple.com> wrote: On May 1, 2009, at 5:07 AM, rohan a wrote: Hello, Thanks for this. 1. However, man 2 stat says that st_flags is of type u_long, but chflags() takes as argument flags which is u_int. For 32 but systems, it doesn't matter. For 64 bit systems, it doesn't care because it's unsigned, so extension to long on LP64 won't change the values. On Leopard and later, it's defined as: /usr/include/sys/stat.h:int chflags(const char *, __uint32_t); and then use statbuf.st_flags in the chflags() call, to set the flags to another file: chflags(file, statbuf.st_flags); Careful; you are most likely operating on the link target, and not the symbolic link itself. It's only recently that you've been able to modify link attributes rather than link target attributes. Inferring tto much about symbolic links is a really bad idea, since it's FS dependent how they are implemented. What Might work in Leopard on HFS may not work in Tiger on HFS, and that may not be the same as some other filesystem (i.e. there were a number of filesystems that supported symbolic links as immediate files, meaning that all the attributes were inherited from the directory, since they only existed as a directory entry, and not as real files). Avoid writing code that assumes symbolic link semantics are the same between filesystem types. Yes, that means that if you back up on one system and restore on another, the symbolic links could act different, and in undesirable ways, unless you go out of your way to characterize behaviour on each FS type, and each version of an OS you might be archiving or dearchiving. This is why there are not 50,000 5 star backup/restore tools out there. On 5/1/09, Dominic Giampaolo <dbg(a)apple.com> wrote: The stat() / lstat() API returns st_flags which are "user defined flags for file". #define UF_NODUMP 0x00000001 /* do not dump file */ #define UF_IMMUTABLE 0x00000002 /* file may not be changed */ #define UF_APPEND 0x00000004 /* writes to file may only append */ #define UF_OPAQUE 0x00000008 /* directory is opaque wrt. union */ #define SF_ARCHIVED 0x00010000 /* file is archived */ #define SF_IMMUTABLE 0x00020000 /* file may not be changed */ #define SF_APPEND 0x00040000 /* writes to file may only append */ This email sent to site_archiver(a)lists.apple.com

1 0