Re: fread multiple files
Re: fread multiple files
- Subject: Re: fread multiple files
- From: Michael Smith <email@hidden>
- Date: Fri, 25 Sep 2009 17:49:54 -0700
An MPI code I use outputs 3 * N files, where N is the number of
processors. Then I am using a serial code to post-process the results.
The post-processor is single-threaded and for every time instant reads
a chunk of data from each of 3 *N files. I noticed that most of the
time (nearly 50%) is spent in:
__spin_lock
pthread_mutex_lock
fread
This - the use of fread - is almost certainly your problem.
Your post-processor should not be using the stdio interfaces. It's probable that it's also reading small amounts of data at a time, which would account for the large amount of time spent inside the locked region.
If you fix your post-processor to use the read(2) interface, and buffer data locally rather than pulling it through stdio, you should see a dramatic improvement in performance.
= Mike
-- Ars longa, vita brevis, occasio praeceps, experimentum periculosum, iudicium difficile -- Hippocrates
|
_______________________________________________
Do not post admin requests to the list. They will be ignored.
Darwin-dev mailing list (email@hidden)
Help/Unsubscribe/Update your Subscription:
This email sent to email@hidden