Re: Shared mmap and data consistency upon a crash
Re: Shared mmap and data consistency upon a crash
- Subject: Re: Shared mmap and data consistency upon a crash
- From: Michael Smith <email@hidden>
- Date: Thu, 11 May 2006 20:52:56 -0700
On May 11, 2006, at 12:02 PM, email@hidden wrote:
Send Darwin-dev mailing list submissions to
Date: Wed, 10 May 2006 23:38:20 +0200
From: Felix Schwarz <email@hidden>
Subject: Re: Shared mmap and data consistency upon a crash
To: email@hidden
Cc: Michael Smith <email@hidden>
Am 10.05.2006 um 21:28 schrieb Michael Smith:
I maybe should have added what it is that I'm writing: I'm writing a
specialized, server-less database link library (similiar to SQLite by
concept, my link library has only *very* basic functionality, though).
The idea behind using mmap() is to pull indexed records into and out
of a file as fast as possible and take advantage of any caching by
the VMM, especially regarding the table inside the file that keeps
record of all the records it stores.
Using the VM for caching in this case does have some advantages
if you're trying to keep your code simple. Bear in mind though that
you'll typically get LRU behaviour on cached data, so your indexes
will only stay resident to the extent that you keep them hot.
Since I've read elsewhere you can't mmap() files that are located on
a network-volume, I've also written a fallback using read()/write()/
lseek()..
You can, but there is no graceful recovery. You are better off using
pread/pwrite, since that will allow you to multi-thread portions of
your I/O code as well as saving you a system call per op.
Mmap will also give you issues once your database starts to get
large, while file I/O will tend to scale better.
It appears, that, in both cases, the data in the file seems to be in
sync when crashing. I could observe both a delay before
any changes could be made to the file after an instant restart of the
little test app plus the data that was written last could be
reread correctly. With either method.
Assuming the crash happens when the data in the object (either via
memory write or write system call) is consistent, you should be OK.
Note that if the app crashes in e.g. another *thread* while you are
in the middle of an update to your datastructures, all bets are off.
You can be pre-empted or terminated at any time, and the object
will be flushed to disk as-is.
This is one argument for using a journal and system calls; as a
general rule, writes to a disk file tend to be atomic - either the
write completes, or it doesn't.
However, I'd still love an official confirmation from a kernel
developer (or someone who is literate in that part of the kernel)
since
- I don't want to base data security on assumptions or lucky
observations that may turn out wrong or may change with the next
release of OS X
- I'd love to use the mmap() solution
- .. and then, I'm also plain curious ;-)
I'm not going to give you an "official" anything, but I think I've
outlined
the tradeoffs fairly. 8)
If your app crashes, all bets are off - assuming you don't know
where it crashed, you have no guarantee as to the state of anything
it was doing, regardless of whether it had files mapped or not.
The way the data is organized in the file and the order in which its
data is modified ensures its always in a consistent state - the
Note that you don't get any guarantees about what order things
are written to disk unless you explicitly sync the file.
This is mostly an issue when considering system crashes, though
the scenario above w.r.t. crashes on another thread needs to be
borne in mind as well.
HTH, and good luck!
= Mike
_______________________________________________
Do not post admin requests to the list. They will be ignored.
Darwin-dev mailing list (email@hidden)
Help/Unsubscribe/Update your Subscription:
This email sent to email@hidden