Re: How to debug VM trouble?
Re: How to debug VM trouble?
- Subject: Re: How to debug VM trouble?
- From: "Michael Crawford" <email@hidden>
- Date: Thu, 24 Jan 2008 13:31:20 -0800
We are now thinking that the error we're seeing with our disk test
tool may actually be a bug in the tool. One of my coworkers is now
inspecting its source to see if he can find the problem.
We have a tool to test our RAID controllers that spawns some threads
that each write a file, then reads it back to check that what it gets
back is what it wrote.
If we spawn a lot of threads, each writing very large files using
large memory buffers, and let the tool run for a day or two,
eventually there will be a miscompare. This doesn't happen with
smaller files, fewer threads, or with more memory installed in the
Mac.
But when it does, there will also be a bunch of messages in the
system.log from other programs such as daemons, complaining for
example of getting ENOMEM from write() system calls.
Our theory is that we're running the kernel out of memory, so that it
eventually returns errors to our own system calls, but we don't check
the error code properly. The data is then either not read or not
written because the call failed.
While I do have to let the tool run for a long time, I am also able to
stimulate the bug using drives hooked to the motherboard SATA
controller in my dual G5, without our PCI card or device driver being
involved.
Unfortunately, our test tool has grown organically as it was ported
from platform to platform over the years - so it's not actually easy
to spot any problems. But such legacy code is likely the source of
our trouble.
We were suspecting for a while that the bug was in the OS X kernel.
We're going to do whatever it takes to find the real source of
trouble. If it turns out that we later conclude that it is in fact in
Apple's code, I'll be sure to file a bug report and post the number to
this thread.
Thanks for all your advice.
Mike Crawford
mdcrawford at gmail dot com
http://www.geometricvisions.com/ <-- Creative Commons Music Downloads
_______________________________________________
Do not post admin requests to the list. They will be ignored.
Darwin-kernel mailing list (email@hidden)
Help/Unsubscribe/Update your Subscription:
This email sent to email@hidden