Re: using FSevents for backup - is it reliable enough?
Re: using FSevents for backup - is it reliable enough?
- Subject: Re: using FSevents for backup - is it reliable enough?
- From: Stan Sieler <email@hidden>
- Date: Thu, 20 Jan 2011 16:34:15 -0800
Hi,
A note of explanation about my verbosity...
wow, I found finally people who understand development and file-systems ... thanks!
For nearly 20 years, I was used to being able to post highly technical questions
(and answers for others) about HP's MPE/iX operating system ...
it's good to find a mailing list with similar quality for the Mac! Thanks!
(That also means you'll see me making rookie mistakes sometimes, since I'm
new to the Mac as a developer :)
>> The advantage of "near" is that an app can walk up to a file and say "what attributes do you have?"
>
> Which is exactly what the CSBackupIsItemExcluded function does. Give it a file and it will tell you what the file's backup exclusion status is.
An API is nice to have, but it doesn't pertain to the philosophical discussion of where the data should really be stored.
(I.e., no matter how it's implemented, the API can can be written and have identical semantics.)
In a "near" implementation, the API's run-time cost is cheap. In a "far" one, it's more expensive (at best, it has to
search a file/tree/something, which can often cost more CPU/elapsed than looking at a near structure).
>> Knowing what the attributes are means that there is some chance that they can be carried over to
>> a copy/backup of the file.
>
> But since this an OS API, applications don't have to worry about it. Whether a copy retains the attribute is up to the OS and the kind of attribute. A backup file would never have this attribute, since a file with this attribute shouldn't be backed up.
That's a hard one. For some file attributes, the app dealing with the file can make better judgements about whether or not
to maintain, delete, or modify an attribute. For others, only the OS can. E.g., imagine an attribute which
is a boolean: have_I_been_backed_up. If I copy file FOO to file FUM, then I'd probably want that attribute reset.
On the other hand, if the file has a boolean attribute: needs_virus_check, then copying the file shouldn't
clear the flag on the new file. Thus, two different boolean attributes with two different "desired" actions ... what's
a cp program to do? :)
In a system where a smart app can at least *ask*: "what attributes does this file have?", then one has a chance
of teaching the app what to do with each kind of attribute.
Such asking is more efficient/easier/more-accurate if the data is kept near the file, not "far" (in an unlimited number
of separate databases).
Imagine a new attribute with a character string: run_this_program_when_file_is_deleted
...that could be used to implement conditional file scrubbing.
(Hmmm, I forgot where I was going with that example :)
>
> The application doesn't have to know anything, except to call CSBackupIsItemExcluded.
For backups, yes. But I was discussing the more global question of how to store / query file attributes in general.
And, in the case of a "cp" program, if cp called that API and checked the flag, what does it do with the information?
Call CSBackupSetItemExcluded on the new file? That may not be appropriate in some cases.
>> Thus, if they do something like rename a file or copy/backup a file,
>> they won't know that there might be metadata about the file in these various and sundry databases.
>
> Again, this is an implementation detail that's the responsibility of the OS. Whether a file retains the property when renamed depends on the kind of backup exclusion set for that file, so again it's not a decision for an application to make.
Not always ... I've already shown one example (has file been virus-checked) where the use of an attribute is at least
partially the responsibility of some program outside the traditional definition of "the OS".
One can imagine a system where an some attributes are "declared" with handlers that get invoked when
certain file operations occur (e.g., file delete, file rename, file copy, file open-for-read, file open-for-write).
...but then, that's an old idea (Burroughs MCP implemented it circa 1975, IIRC, and I proposed a version
of it for HP's MPE circa 1980).
>> Of course, it's possible to implement a system that merges the best aspects of
>> both (e.g., a registry of "far" databases, with an API to access them) ...
>
> I'm pretty sure that's exactly what's been implemented.
I'm completely positive that's not what's been implemented :)
We *know* that some attributes are implemented via the dot-underscore method, and we know that
some are implemented via (possibly hidden) files.
>> and
>> I'm sure that if that was done, and enforced, you'd be getting complaints from
>> people complaining that you've departed from prior examples :)
>
> I don't understand this statement.
Take /etc/fstab ... Mac OS seems to not use it ... it's non-existent on most Macs I've seen.
And, that's good ... it was a kludgy way of implementing important system information.
OTOH, since there was no good standard API for getting some of the information in /etc/fstab entries,
those programmers who depend on it are out of luck on the Mac. (Oddly, the "standard" API for
opening/reading /etc/fstab does exist on the Mac ... the man pages don't mention the incredibly
vital point that they're mostly useless :)
Or, for another example, if Apple was to delete the readdir() routines and implement a fancy
mechansim usable only from Objective C, I'd be complaining ... as would anyone maintaining
a portable program that walks directories (IIRC, there are enough systems without ftw that
opendir/readdir is still somewhat popular).
>> I've explored the "Core" routines only a little ... they seem ill-suited for calling from C code.
>
> I don't understand this statement either. Why would C functions be "ill-suited" from being called from C code? I'm pretty sure that the purpose of a C API is that it is well-suited to being called from C. ;)
Oops, my fault...sorry!
I'd taken a quick glance at them and thought they looked like the Objective C routines I've been looking at for iPhone/iPad development.
You're right ... the CoreBackup routines are APIs callable from C.
Although crude and probably the really wrong way to do it, I found that if I add:
CBCORE= /System/Library/Frameworks/CoreFoundation.framework/Versions/A/CoreFoundation /System/Library/Frameworks/Carbon.framework/Carbon
CBINC= -I /Developer/SDKs/MacOSX10.6.sdk/System/Library/Frameworks/CoreServices.framework/Versions/A/Frameworks/CarbonCore.framework/Versions/A/Headers
to my Makefile, I was able to create a simple C program to set the "backup exclude flag by compiling/linking:
$(CC) $(CFLAGS) $(CBINC) cbset.c $(CBCORE) -o cbset
Cool.
thanks,
Stan
_______________________________________________
Do not post admin requests to the list. They will be ignored.
Filesystem-dev mailing list (email@hidden)
Help/Unsubscribe/Update your Subscription:
This email sent to email@hidden