William Groppe wrote:
>I pointed this little test class at a directory filled with 1 - 1.5Meg
>jpegs. About 95 of them. Here's what I wound up with:
>
>Min: 25ms (Faster then any other platform)
>Max: 8935ms (Slower by a factor of 5 then any other platform)
>Mean: 2030ms (Slower by half then any other platform)
>
>I've played with memory settings, from 64m to 1024m. I've turned on
>verbose garbage collection. Nothing seems to explain it.
Without seeing the full results, no one else can explain it either.
The Mean value is meaningless without knowing the deviation from the mean.
All it takes is a few outliers like 8935 ms to skew the mean significantly.
When you "played with memory settings", I assume that was the -Xmx option
for max heap-size. But the initial heap-size (-Xms) also has an effect,
especially if you know the program will be reading about 100 MB of files
and ignoring all their data.
A single instance of 8935 ms for a 1 MB file sounds suspiciously like it
got caught in a GC cycle, or some other deviation. It may also be related
to virtual memory.
Try running Activity Monitor on your app and see how much vm paging occurs.
You can also see how much I/O occurs, and other things that might be
useful. The tool is here on your HD:
/Applications/Utilities/Activity Monitor.app
Finally, I ran your program (actually my slight variation of it) on my
machine, and it consistently reads 100 KB files in 1 or 2 ms each, and
newly-created 1 MB files at about 12-13 ms each. The I/O speed for the 1
MB files comes in at 7-9 MB/s.
When I read 100 newly-created files, rather than the 25 I was using above,
it's still 12-13 ms each. This is consistent with them being cached.
When I run the app on a random collection of 78 JPEG files of 1.2-1.5 MB,
not previously read, it comes in consistently at about 3 MB/s +/-10%, which
is around 50 ms per file. Best is about 40 ms, worst about 60 ms.
When I rerun on the JPEG files, the speed goes up to the same 7-9 MB/s as
earlier, elapsed time per file in the 11-16 ms range.
At no time did I ever see even 100 ms per file, and the times I do see are
quite consistent for files of similar size.
When I run your EXACT original program (with its println()'s in the
original places), I get 12-15 ms each for reading the metered 1 MB files.
There are no outliers, no stutters, and very consistent results.
When I run your original app on disk-cached JPEGs, I get 17-23 ms per file.
Still no outliers, and nothing even approaching 100 ms.
When I run your original on uncached JPEGs, I see similar performance as
reported above with first-time JPEGs. The only file over 100 ms was 2.5 MB
in size, at 125 ms.
This was all done on a 500 MHz dual G4 with 1 GB RAM and 10.3.5, Java
1.4.2_05 with no -Xmx or -Xms. This is definitely an older machine, and
hardly a speed-demon when it comes to disk, RAM, or CPU.
Since you didn't say what your machine or OS or Java was, I don't know how
any of this compares. For all anyone knows, it's a virtual memory problem,
or could even be another program contesting for available CPU. My machine
was quiescent at the time, as observed in Activity Monitor.app.
The changes I made to your benchmark basically move the println() out of
the measured time interval, then also calculate and display a MB/s value
for each file. Other than moving the println(), this shouldn't change the
reported speeds or times significantly.
I also wrote a simple Java program to generate test-files of known size and
quantity, rather than relying on random collections of randomly-sized
JPEGs. That generated my precisely metered 100 KB and 1 MB files for the
first runs.
At this point, I strongly suggest that you simply profile your original
program (not your benchmark) using -Xprof or any other profiler, and find
out where *IT* is spending its time. Measure reality, not its supposed
surrogate.
If you're caching images, I further suggest using java.lang.ref.SoftReference.
If you insist on running a file I/O benchmark, then definitely move the
println() out of the measured interval and do some cleanup. Simple
cleanup, like reusing a single byte[] as large as the largest file, so you
don't trigger a GC from allocating a big byte[] only to throw it away once
the file is read. You should also clean up the output so it's more easily
readable (1 line per file), and display a MB/s or KB/s value. Interpreting
the output is hard right now with unnormalized "bytes read" and "ms" values
alone.
You should also have the app print informative system properties like
"os.name" "os.version" and "java.version" directly on the output stream, so
you can always see those things.
Benchmarks are subtle devils, and there is an art to writing them. This
often involves what NOT to do, or The Art of Avoiding Self-Deception. Tiny
errors in understanding or expectation can radically skew the reported
results. If you then believe those results, you will be mislead.
Your benchmark is supposed to be a stand-in for your real program, but I'm
not sure it is in this case.
If a benchmark isn't an accurate stand-in for the thing you really care
about, then that benchmark is useless.
-- GG
_______________________________________________
Do not post admin requests to the list. They will be ignored.
Java-dev mailing list (email@hidden)
Help/Unsubscribe/Update your Subscription:
http://lists.apple.com/mailman/options/java-dev/email@hidden
This email sent to email@hidden
