Mailing Lists: Apple Mailing Lists

Image of Mac OS face in stamp
 
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: Mac OS X File IO Performance Issue?



I forgot to add my Windows scores:

Using IOTest:
    Write: 40MB/s
    Read: 450MB/s

Using NIOTest:
    Write: 40MB/s
    Read: 292MB/s

Its official: NIO sucks.

Not only is it slower, but its also more complicated and you have to write more code. NIO == selfFlagellation. (perhaps charset conversion is okay?)

On 29 Jul 2005, at 18:10, Brendon McLean wrote:

Sebastian,

Those figures are the time it took to write and read 50MB. So your results were 23MB/s write and about 250MB/s read.

I've change the program to use NIO just for interests sake and the results are quite surprising. Firstly, there's really no improvement at all on the write speed. Perhaps its is being hard drive limited? Secondly, the reads: using normal IO I get between 80MB/s and 120MB/s. But with NIO the reads drop to between 40MB/s and 60MB/s. NIO is slower?!

Both tests attached (and updated to show the MB/s as output). BTW, you may want to clear you /private/tmp directory after this test ;)

Interestingly, I've also learnt a thing or two about Activity Monitor. It seems the File system cache lives in the blue part of the pie chart. Each time you run this test, you'll see a chunk of the green give way to blue. When you run out green, both the write and read speeds drop. Moving the files to the Trash has no effect. If you use up all your Free mem (green) then empty the trash, the blue will virtually dissapear leaving you with a lot of green. This really has no bearing on anything, it just interested me so I thought I'd pass it on.

NIO Test
-------------
import java.io.*;
import java.nio.channels.FileChannel;
import java.nio.ByteBuffer;

/**
* Created by IntelliJ IDEA.
* User: brendon
* Date: Jul 29, 2005
* Time: 5:09:55 PM
* To change this template use File | Settings | File Templates.
*/
public class NIOTest {
    private static final long testFileSize = 50 * 1024 * 1024;
    private static final int BUF_SIZE = 8192;
    private static final byte[] chunk = new byte[BUF_SIZE];
    private static final double NANO = 1000.0 * 1000.0 * 1000.0;

    static {
        for (int i = 0; i < chunk.length; i++) {
            chunk[i] = (byte) (Math.random() * 255);
        }
    }

    public static void main(String[] args) throws IOException {
        File writeFile = File.createTempFile("vmBench", ".writeFile");
        FileOutputStream fos = new FileOutputStream(writeFile);
        FileChannel writeChannel = fos.getChannel();

        long testStart = System.nanoTime();
        writeChunkOfData(writeChannel);
        long testFinish = System.nanoTime();

System.out.println("Write: " + (50 / ((testFinish - testStart) / NANO)) + " MB/s");

        File readFile = new File(writeFile.getPath());
        FileInputStream fis = new FileInputStream(readFile);
        FileChannel readChannel = fis.getChannel();

        testStart = System.nanoTime();
        readChunkOfData(readChannel);
        testFinish = System.nanoTime();

System.out.println("Read: " + (50 / ((testFinish - testStart) / NANO)) + " MB/s");
}


private static void readChunkOfData(FileChannel bis) throws IOException {
ByteBuffer byteBuffer = ByteBuffer.allocate(8192);
while (bis.read(byteBuffer) != -1) {
byteBuffer.clear();
};
}


private static void writeChunkOfData(FileChannel fileChannel) throws IOException {
ByteBuffer byteBuffer = ByteBuffer.allocateDirect(BUF_SIZE);
long bytesWritten = 0;
while (bytesWritten < testFileSize) {
int len = (int) Math.min(BUF_SIZE, (testFileSize - bytesWritten));
byteBuffer.put(chunk, 0, len);
byteBuffer.flip();
bytesWritten += fileChannel.write(byteBuffer);
byteBuffer.clear();
}
}
}




IOTest
-----------
import java.io.*;

public class IOTest {
    private static final long testFileSize = 50 * 1024 * 1024;
    private static final int BUF_SIZE = 8192;
    private static final byte[] chunk = new byte[BUF_SIZE];
    private static final double NANO = 1000.0 * 1000.0 * 1000.0;

    static {
        for (int i = 0; i < chunk.length; i++) {
            chunk[i] = (byte) (Math.random() * 255);
        }
    }

    public static void main(String[] args) throws IOException {
        File writeFile = File.createTempFile("vmBench", ".writeFile");
        FileOutputStream fos = new FileOutputStream(writeFile);
        BufferedOutputStream bos = new BufferedOutputStream(fos);

        long testStart = System.nanoTime();
        writeChunkOfData(bos);
        long testFinish = System.nanoTime();

System.out.println("Write: " + (50 / ((testFinish - testStart) / NANO)) + " MB/s");

        File readFile = new File(writeFile.getPath());
        FileInputStream fis = new FileInputStream(readFile);
        BufferedInputStream bis = new BufferedInputStream(fis);

        testStart = System.nanoTime();
        readChunkOfData(bis);
        testFinish = System.nanoTime();

System.out.println("Read: " + (50 / ((testFinish - testStart) / NANO)) + " MB/s");
}


private static void readChunkOfData(BufferedInputStream bis) throws IOException {
byte[] readBuffer = new byte[BUF_SIZE];
while (bis.read(readBuffer) != -1);
}


private static void writeChunkOfData(BufferedOutputStream bos) throws IOException {
long bytesWritten = 0;
while (bytesWritten < testFileSize) {
int len = (int) Math.min(BUF_SIZE, (testFileSize - bytesWritten));
bos.write(chunk, 0, len);
bytesWritten += len;
}
bos.flush();
}
}


_______________________________________________
Do not post admin requests to the list. They will be ignored.
Java-dev mailing list (email@hidden)
Help/Unsubscribe/Update your Subscription:
http://lists.apple.com/mailman/options/java-dev/apple-java-dev% 40twistedprotein.com


This email sent to email@hidden


_______________________________________________ Do not post admin requests to the list. They will be ignored. Java-dev mailing list (email@hidden) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/java-dev/email@hidden

This email sent to email@hidden
References: 
 >RE: Mac OS X File IO Performance Issue? (From: "Rick Genter" <email@hidden>)
 >Re: Mac OS X File IO Performance Issue? (From: Brendon McLean <email@hidden>)
 >Re: Mac OS X File IO Performance Issue? (From: Sebastian Hagedorn <email@hidden>)
 >Re: Mac OS X File IO Performance Issue? (From: Brendon McLean <email@hidden>)



Visit the Apple Store online or at retail locations.
1-800-MY-APPLE

Contact Apple | Terms of Use | Privacy Policy

Copyright © 2007 Apple Inc. All rights reserved.