Re: fread multiple files
Re: fread multiple files
- Subject: Re: fread multiple files
- From: Jens Alfke <email@hidden>
- Date: Fri, 25 Sep 2009 14:10:31 -0700
On Sep 25, 2009, at 1:29 PM, Tomasz Koziara wrote:
The post-processing process (single-thread) simply opens all 3 * N
files and keeps them open. Of those 3 * N files, only N store some
substantial data (are large), while the rest stores indexing and
labeling data. So basically what happens is:
For each time t
For each output file f
Seek (f, t)
Read (f)
Sounds like you're reading a lot of data, so I would expect that your
process spends most of its time blocked in disk I/O, which is what the
profiles show. It sounds like your access patterns are pretty random
too, which makes performance worse.
I don't know anything at all about MPI or XDR, so I don't think I can
give any advice here.
—Jens _______________________________________________
Do not post admin requests to the list. They will be ignored.
Darwin-dev mailing list (email@hidden)
Help/Unsubscribe/Update your Subscription:
This email sent to email@hidden