Mailing Lists: Apple Mailing Lists
Image of Mac OS face in stamp
 
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

RE: DataOutputStream performance under OS X

>-----Original Message-----
>From: Don Werder [mailto:email@hidden]

>>Are you saying that the other non-OS X operating systems are
>automatically buffering when reading and writing? What is the cause
>in the improvement in the output times reported by Hagman on the
>Linux machines when comparing DataOutputStream and
>BufferedOutputStream? Is this additional buffering?
>Is DataOutputStream actually writing to disk and BufferOutputStream
>is writing to the buffer? Is that what each time is measuring?
>
>

All file systems that write to external media have *some* type of minimal
buffering going on. In fact, there are buffers on the actual hard drive that
receive data on their IO ports, and wait for them to fill up before sending
them to the physical media. For example, I don't know of any consumer hard
drives that let you write a single byte to a specific address on a disk
platter. The closest you can get is probably writing a 2000 byte block of
data. So if you really just want to write a byte, you'll have to either pad
the other 1999 bytes with dummy values (to fill out your block), OR *wait*
until your other writing commands have written an additional 1999 bytes, OR
READ the block of data from the disk you want to change, write your single
byte, then write that entire block back to the disk.

The degree of buffering is totally up to the native OS and the types of
storage devices being used.

I was speaking to the issue of my personal experiences with file IO on
Windows2000. The Windows OS has its own file buffers, so when you write to a
Windows file using Windows file APIs (as is the case for the actual File
IO implementation for Java on that platform) you're not writing to physical
media, you're writing to the Windows file buffer. This may be one reason why
"raw" transfer times APPEAR to be better on Windows. The OS will determine
when to flush that buffer to physical media. You are supposed to be able to
tell the OS "hey, flush your file buffer now please"; I'm just saying *my*
personal experience is that even when you ask for that, sometimes it takes a
bit of time before it happens. That wait may only be on the order of seconds
mind you, but a second is an eternity from the CPU's point of view. So it's
only meaningful to compare two platforms if you know how long it takes for
your bytes in memory to actually show up on the hard drive media.

This native file system buffering is in addition to any kind of buffer you
create in Java for your IO. So even if you don't use Java buffered IO, your
file writes, through the OS, are being buffered. If you use Java buffered
IO, you now have multiple buffers in play.

I'm not at all suggesting file buffering is a bad thing; quite the contrary,
it's a wonderful thing. It lets you make efficient use of the large
transfer latency inherent in physical storage devices. The time required to
seek and spin the drive heads is the limiting factor in data throughput, so
writing large buffers helps keep the total transfer overhead to a minimum.

In regard to your question, it's very hard to compare unbuffered writing
times of various platforms to each other. In fact, it's almost meaningless.
The amount of variables you'd have to take into account and standardize for
accurate correlation are the transfer rate of the hard drives, the
throughput on the data bus, the size of the native file system cache, the
rules for when the OS flushes its caches (do they always wait until they're
full before flushing, or will they flush a partly full cache after a certain
time interval..) Etc.


Rob
_______________________________________________
java-dev mailing list | email@hidden
Help/Unsubscribe/Archives: http://www.lists.apple.com/mailman/listinfo/java-dev
Be sure to read the FAQ http://developer.apple.com/java/faq/ before posting
Do not post admin requests to the list. They will be ignored.


Visit the Apple Store online or at retail locations.
1-800-MY-APPLE

Contact Apple | Terms of Use | Privacy Policy

Copyright © 2007 Apple Inc. All rights reserved.