April 2012 - Darwin-dev - lists.apple.com

Re: Globally, persistently identifying a file
by Nick Zitzmann 27 Apr '12

27 Apr '12

On Apr 27, 2012, at 12:06 PM, Tilghman Lesher wrote: > Here's another idea: instead of relying upon the file's inode, use > the containing directory's inode number and a combination of the inode > number and the name of the file. It will work fine if the Cocoa API > deletes and recreates the file. It will also work fine if the > containing directory is moved to another part of the filesystem. If > either the filename or the inode don't match, you can use that > information to update your database on each run. This especially > works well with applications, since applications are themselves > directories. And that will stop working once the file name changes. Nick Zitzmann <http://www.chronosnet.com/> _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: https://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.ap… This email sent to site_archiver(a)lists.apple.com

1 0

Re: Globally, persistently identifying a file
by Tilghman Lesher 27 Apr '12

27 Apr '12

On Fri, Apr 27, 2012 at 12:51 PM, Nick Zitzmann <nick(a)chronosnet.com> wrote: > > On Apr 27, 2012, at 11:29 AM, Jens Alfke wrote: > >> >> On Apr 26, 2012, at 1:36 PM, Nick Zitzmann wrote: >> >>> I know about bookmarks, but the bookmark data is not persistent between reboots despite still being resolvable between reboots. >> >> What does that mean? I haven’t used bookmarks myself, but if they’re based on aliases they’re persistent. After all, an alias file contains alias/bookmark data, and those work across reboots. >> >>> I need something that will not change under any circumstances other than deleting and re-creating the file/folder. This is so I can search for a value in a database corresponding to a file/folder. >> >> You probably don’t actually want that. Consider that most mechanisms for safely updating a file (including the ‘atomic’ modes of the Cocoa writeToFile: methods) end up deleting it and replacing it with a new file. >> >> It really sounds to me like an alias/bookmark is exactly what you’re looking for. > > It is what I'm looking for, but the problem is, bookmark data changes between reboots, which means I can't use the data for searching. > > I'm storing references to files in a database, and need to search the database to see if the database is aware of the existence of the file. Doing an in-memory search by resolving the bookmark data for each item in the database scales very poorly. Which means that the reference must not change if the file is renamed or moved, the underlying folder structure changes (other than the volume), the file is modified, the OS is rebooted, or the OS is updated. If the file is copied and the original is deleted, then that's okay. So far, the only things I've found that have stayed constant when everything else changes (even when the file is overwritten!), at least for HFS+ volumes, are the inode and volume numbers. And for remote volumes, I've found that the inode and file system creation date don't change. I just need to know whether I can rely on this or not for the purpose of searching. Here's another idea: instead of relying upon the file's inode, use the containing directory's inode number and a combination of the inode number and the name of the file. It will work fine if the Cocoa API deletes and recreates the file. It will also work fine if the containing directory is moved to another part of the filesystem. If either the filename or the inode don't match, you can use that information to update your database on each run. This especially works well with applications, since applications are themselves directories. -Tilghman _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: https://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.ap… This email sent to site_archiver(a)lists.apple.com

1 0

Re: Globally, persistently identifying a file
by Nick Zitzmann 27 Apr '12

27 Apr '12

On Apr 27, 2012, at 11:29 AM, Jens Alfke wrote: > > On Apr 26, 2012, at 1:36 PM, Nick Zitzmann wrote: > >> I know about bookmarks, but the bookmark data is not persistent between reboots despite still being resolvable between reboots. > > What does that mean? I haven’t used bookmarks myself, but if they’re based on aliases they’re persistent. After all, an alias file contains alias/bookmark data, and those work across reboots. > >> I need something that will not change under any circumstances other than deleting and re-creating the file/folder. This is so I can search for a value in a database corresponding to a file/folder. > > You probably don’t actually want that. Consider that most mechanisms for safely updating a file (including the ‘atomic’ modes of the Cocoa writeToFile: methods) end up deleting it and replacing it with a new file. > > It really sounds to me like an alias/bookmark is exactly what you’re looking for. It is what I'm looking for, but the problem is, bookmark data changes between reboots, which means I can't use the data for searching. I'm storing references to files in a database, and need to search the database to see if the database is aware of the existence of the file. Doing an in-memory search by resolving the bookmark data for each item in the database scales very poorly. Which means that the reference must not change if the file is renamed or moved, the underlying folder structure changes (other than the volume), the file is modified, the OS is rebooted, or the OS is updated. If the file is copied and the original is deleted, then that's okay. So far, the only things I've found that have stayed constant when everything else changes (even when the file is overwritten!), at least for HFS+ volumes, are the inode and volume numbers. And for remote volumes, I've found that the inode and file system creation date don't change. I just need to know whether I can rely on this or not for the purpose of searching. Nick Zitzmann <http://www.chronosnet.com/> _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: https://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.ap… This email sent to site_archiver(a)lists.apple.com

1 0

Re: Globally, persistently identifying a file
by Chris Suter 27 Apr '12

27 Apr '12

_______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: https://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.ap… This email sent to site_archiver(a)lists.apple.com

1 0

Re: Globally, persistently identifying a file
by comex 27 Apr '12

27 Apr '12

On Thu, Apr 26, 2012 at 6:48 PM, Nick Zitzmann <nick(a)chronosnet.com> wrote: >> On Thu, Apr 26, 2012 at 2:09 PM, Nick Zitzmann <nick(a)chronosnet.com> wrote: >>> I need to be able to identify a file globally and persistently (such that it still works after moving or overwriting a file, or re-mounting a volume, or after rebooting). File paths don't work, because even though they're technically unique, they break if the file or its underlying path components are renamed or moved. Carbon FSSpec data structures don't work, because they're not available to 64-bit apps. Cocoa NSURLFileResourceIdentifier objects don't work, because they aren't persistent across reboots. >> >> Are these your own filetypes that you're monitoring? > > They aren't. You could get a reasonable ID with getattrlist and ATTR_CMN_OBJPERMANENTID + ATTR_VOL_UUID, but you should try to ensure you're not seeing a reused object ID, perhaps by checking the file's creation date (ATTR_CMN_CRTIME or st_birthtime). _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: https://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.ap… This email sent to site_archiver(a)lists.apple.com

1 0

Re: Globally, persistently identifying a file
by jonathan＠mugginsoft.com 26 Apr '12

26 Apr '12

On 26 Apr 2012, at 23:07, Nick Zitzmann wrote: > > On Apr 26, 2012, at 3:02 PM, jonathan(a)mugginsoft.com wrote: > >> >> On 26 Apr 2012, at 21:36, Nick Zitzmann wrote: >> >>> >>> On Apr 26, 2012, at 2:24 PM, Brian Bergstrand wrote: >>> >>>> Prior to 10.6 you would use Alias records, but in 10.6 Apple added URL-based bookmarks (which are pretty much alias records with some new features). >>>> >>>> Cocoa: NSURL bookmarks >>>> >>>> C/C++: CFURL bookmarks >>> >>> I know about bookmarks, but the bookmark data is not persistent between reboots despite still being resolvable between reboots. I need something that will not change under any circumstances other than deleting and re-creating the file/folder. This is so I can search for a value in a database corresponding to a file/folder. >> >> Is it right to say that what are doing is uniquely identifying the file/folder content? >> If so, git uses hashes for this purpose, a fact that you probably know already. > > > I need to uniquely identify the file/folder, even if the contents change, so SHA/MD5/etc. hashes won't work. Is an object orientated interpretation of the problem helpful? We are trying to track a file or path object as a unique entity with an object orientated file system regardless of the file/path object's current path or content properties. If so then it's a question of identifying whether the OS X filesystem conforms to such a model or not, which probably brings us right back to the original question. Jonathan _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: https://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.ap… This email sent to site_archiver(a)lists.apple.com

1 0

Re: Globally, persistently identifying a file
by Nick Zitzmann 26 Apr '12

26 Apr '12

On Apr 26, 2012, at 4:44 PM, Stephen J. Butler wrote: > On Thu, Apr 26, 2012 at 2:09 PM, Nick Zitzmann <nick(a)chronosnet.com> wrote: >> I need to be able to identify a file globally and persistently (such that it still works after moving or overwriting a file, or re-mounting a volume, or after rebooting). File paths don't work, because even though they're technically unique, they break if the file or its underlying path components are renamed or moved. Carbon FSSpec data structures don't work, because they're not available to 64-bit apps. Cocoa NSURLFileResourceIdentifier objects don't work, because they aren't persistent across reboots. > > Are these your own filetypes that you're monitoring? They aren't. Nick Zitzmann <http://www.chronosnet.com/> _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: https://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.ap… This email sent to site_archiver(a)lists.apple.com

1 0

Re: Globally, persistently identifying a file
by Stephen J. Butler 26 Apr '12

26 Apr '12

On Thu, Apr 26, 2012 at 2:09 PM, Nick Zitzmann <nick(a)chronosnet.com> wrote: > I need to be able to identify a file globally and persistently (such that it still works after moving or overwriting a file, or re-mounting a volume, or after rebooting). File paths don't work, because even though they're technically unique, they break if the file or its underlying path components are renamed or moved. Carbon FSSpec data structures don't work, because they're not available to 64-bit apps. Cocoa NSURLFileResourceIdentifier objects don't work, because they aren't persistent across reboots. Are these your own filetypes that you're monitoring? I ask because you could attach a GUID to a file (maybe via an extended attribute) then write a MetaData importer that adds your attribute to the Spotlight database. Then finding your file again is as easy as querying Spotlight. However, this won't work if you need to track arbitrary filetypes since you can only have one importer per UTI. _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: https://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.ap… This email sent to site_archiver(a)lists.apple.com

1 0

Re: Globally, persistently identifying a file
by Sean McBride 26 Apr '12

26 Apr '12

>I need to uniquely identify the file/folder, even if the contents >change, so SHA/MD5/etc. hashes won't work. Maybe fsevents could help? I've never used them, but seem to recall that they provide unique ids. Sean _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: https://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.ap… This email sent to site_archiver(a)lists.apple.com

1 0

Re: Globally, persistently identifying a file
by Nick Zitzmann 26 Apr '12

26 Apr '12

On Apr 26, 2012, at 3:02 PM, jonathan(a)mugginsoft.com wrote: > > On 26 Apr 2012, at 21:36, Nick Zitzmann wrote: > >> >> On Apr 26, 2012, at 2:24 PM, Brian Bergstrand wrote: >> >>> Prior to 10.6 you would use Alias records, but in 10.6 Apple added URL-based bookmarks (which are pretty much alias records with some new features). >>> >>> Cocoa: NSURL bookmarks >>> >>> C/C++: CFURL bookmarks >> >> I know about bookmarks, but the bookmark data is not persistent between reboots despite still being resolvable between reboots. I need something that will not change under any circumstances other than deleting and re-creating the file/folder. This is so I can search for a value in a database corresponding to a file/folder. > > Is it right to say that what are doing is uniquely identifying the file/folder content? > If so, git uses hashes for this purpose, a fact that you probably know already. I need to uniquely identify the file/folder, even if the contents change, so SHA/MD5/etc. hashes won't work. Nick Zitzmann <http://www.chronosnet.com/> _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: https://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.ap… This email sent to site_archiver(a)lists.apple.com

1 0