Re: process exits with bus error: forced to power-down reboot to fix filesystem
Re: process exits with bus error: forced to power-down reboot to fix filesystem
- Subject: Re: process exits with bus error: forced to power-down reboot to fix filesystem
- From: James Peach <email@hidden>
- Date: Fri, 9 Jul 2010 12:11:54 -0700
On 8 July 2010 14:35, Terry Lambert <email@hidden> wrote:
> On Jul 7, 2010, at 12:37 PM, Daniel Quinlan wrote:
>>
>> I'm not certain this is the right list for this problem, but it seems to
>> me to be a kernel problem of some sort.
>> It's rather long, but I've tried to be comprehensive. I've also submitted
>> this as a bug to the
>> apple bug reporter.
>>
>> We have a problem with a ring buffer program which memory maps a large
>> file.
>
> [ ... ]
>
>> We can evade the problem by initializing the buffer file before
>> starting the ring buffer program, but this is operationally difficult
>> -- and it hasn't been necessary on other platforms.
>>
>> The problem appears to be related to how quickly the ring buffer
>> program is writing to the buffer file -- if it's slow enough and the
>> buffer file is small enough, we typically don't see the problem. It
>> seems to need multiple threads writing in the operational program.
>
> HFS does not support sparse files, so you are waiting for it to allocate
> backing pages. On the other operating systems, the FS's you are using
> apparent support sparse files, and therefore the backing pages are being
> lazily allocated for the sparse file control structures, which are allocable
> quickly.
Some combination of fcntl(F_PREALLOCATE) and fsync(2) can probably be
used to close the race condition.
> Technically, you still have a race condition on those other machines, you
> are just lucky that the order you start writing is the same as the order in
> which the control structures are allocated. That is you win the race
> because your use is such that the control structures themselves are in place
> by the time you get to any region of the file governed by the control
> structures in question. It would really be best on all platforms for you to
> preallocate your file, or at least the control structures.
>
> On Mac OS X you can use UFS (if it's a very old Mac OS X) or UDF (if it's a
> modern one) since both those filesystem types support sparse files. This is
> still racy (as noted above), but you would be in the same boat as you are on
> the other filesystems.
>
> PS: You do not get an aggregate performance win for using a sparse file.
> You win on startup time, but what you win there, you lose in runtime, as
> the operations are stalled waiting for backing store allocation whenever you
> hit a file region that does not already have backing store pre-allocated.
> If runtime performance is more important than startup performance, you may
> want to take that into account and write a byte in the filesystem blocks on
> block boundaries after allocating them on the other operating systems.
>
> PPS: I would minimally put an interlock there, such as opening the file
> initially O_EXCL, to block the second process' open of the file to prevent
> the file being used by the second process before it's ready, no matter what
> else you do.
>
> -- Terry
> _______________________________________________
> Do not post admin requests to the list. They will be ignored.
> Darwin-kernel mailing list (email@hidden)
> Help/Unsubscribe/Update your Subscription:
>
> This email sent to email@hidden
>
--
James Peach | email@hidden
_______________________________________________
Do not post admin requests to the list. They will be ignored.
Darwin-kernel mailing list (email@hidden)
Help/Unsubscribe/Update your Subscription:
This email sent to email@hidden