RE: DataOutputStream performance under OS X

• Subject: RE: DataOutputStream performance under OS X
• From: Don Werder <email@hidden>
• Date: Wed, 30 Jul 2003 11:37:45 -0700

This might actually be a case where the file IO implementation on OS X is
actually "better" than other platforms. The fact that unbuffered IO takes
longer on a Mac is an indicator to me that the native file system isn't
doing a lot of buffering itself. This is actually a *good thing*.

This is actually a basic design issue. If you write to a file, in any
language on any platform, you have two options. You can actually create a
memory data structure (usually a block or sector) and actually write it to
the storage device when the "write" command is issued. This ensures your
data is now safely stored on your external media. OR, the second way, is to
buffer it, so when you call "write", your data doesn't actually go to the
storage right away. The buffer accumulates data until it is full, then is
written to the media.

By way of analogy, the differences in the two models is like a car and a
plane; a car lets you get in alone and drive wherever you want; a plane has
to accumulate enough passengers before a trip can be made. Both methods have
strengths and weaknesses. You can control your arrival and departure times
if you are driving a car; with a plane you have to pick a pre-arranged time.
But if you want to go to SF from LA, a plane will get more people there
faster and cheaper than if each of those passengers drove their own cars.

Using Java buffered streams makes sense when the data is not critical enough
to warrant having to be flushed to external storage after every write.
Usually, it's much more efficient to just use a buffer but you have to be
aware that your data isn't necessarily on disk after you call "write". Some
types of files by nature MUST be flushed after every write - transaction
logs, database recovery files, etc. If you have to ensure that your data is
safely stored when you write, you must not use a buffer.

Now, the problem I have had flushing files on Windows is even if I get a
native file descriptor (FileOutputStream.getFD) and call sync, it doesn't
guarantee that the file contents are actually sync'd! This is suppose to be
the case, but I periodically run into situations when I try to access the
file immediately after I have sync'd it, and the file is either not
available, or the contents don't match what I just wrote. I have to "wait"
before the native file system flushes its own internal caches.

So, I am inclined to think that OS X is really writing bytes at a time to
the external storage when you call write ( subject to minimum file block
caching of course) so this seems like a much "safer" implementation than
the Windows file system. If you want fast file writes and can live with the
fact that your data may only exist in RAM immediately after a write, use
Buffered IO. Most applications do fine with buffered IO; as I mentioned
before, unbuffered IO should only be used if that's what you really need for
a particular solution.

Rob

-----Original Message-----
From: Jussi Hagman [mailto:email@hidden]
Sent: Tuesday, July 29, 2003 8:19 PM
To: email@hidden
Subject: DataOutputStream performance under OS X


Hi,

Quite a some time ago I bumped into performance problems with
DataOutputStream and made a small program to test it against
BufferedOutputStream. I have run the test both under OS X and Linux.

My program is just a simple loop writing six files 40kB each
on the disk
using either DataOutputStream or BufferedOutputStream.

The source and a makefile to build and run the test programs is
available at http://infa.abo.fi/~jhagman/speedtest.tar.gz

The results I got are following:

DataOutputStream BufferedOutputStream
iMac 19.7s 1.0s
PC1 5.5s 1.5s
PC2 1.3s 0.7s


Machine configurations:
iMac, G3/350MHz, OS X 10.2.6, Java 1.4.1_01-58.1
PC1, 2xPPro 200MHz, Linux 2.2.22, Java Linux_JDK_1.2.2_RC4
PC2, 2xP3 700MHz, Linux 2.2.16, Java 1.4.0-b92

>

Of course the machine configurations are not comparable and they were
not under same load. I would like to know is why DataOutputStream takes
20 times more time to operate than BufferedOutputStream under OS X? On
Linux machines the difference is much smaller. On OS X (at least on a
low end machine) this renders the DataOutputStream completely useless.


--
Jussi Hagman CS in Ebo Akademi
University
Studentbyn 4 D 33 email@hidden
20540 Ebo email@hidden
Finland
_______________________________________________
java-dev mailing list | email@hidden
Help/Unsubscribe/Archives:
http://www.lists.apple.com/mailman/listinfo/java>-dev
Be sure
to read the FAQ
http://developer.apple.com/java/faq/ before posting
Do not post admin requests to the list. They will be ignored.

_______________________________________________
java-dev mailing list | email@hidden
Help/Unsubscribe/Archives: http://www.lists.apple.com/mailman/listinfo/java-dev
Be sure to read the FAQ http://developer.apple.com/java/faq/ before posting
Do not post admin requests to the list. They will be ignored.

• Prev by Date: Re: Codewarrior v8.3 and java2(1.4.1)
• Next by Date: Issues with modal dialogs
• Previous by thread: Re: DataOutputStream performance under OS X
• Next by thread: RE: DataOutputStream performance under OS X
• Index(es):
  • Date
  • Thread