Optimizing writes of big files for specific hardware?
Optimizing writes of big files for specific hardware?
- Subject: Optimizing writes of big files for specific hardware?
- From: Jay Reynolds Freeman <email@hidden>
- Date: Fri, 03 Jul 2009 12:46:48 -0700
I am not sure whether this is a Cocoa question or a Darwin one.
I will take the discussion elsewhere if need be. Please advise.
I have an app whose initialization includes writing a huge file
to disk -- think GigaBytes, or even tens of GigaBytes. I am
doing this in the context of setting up a large area of shared
memory with mmap, so the big write has to happen at
initialization, and it is agonizingly slow.
At the moment I am actually just using the C++ library "write"
to do the writes, in a tight loop with a large buffer (50 MByte)
full of zeros.
Is there a way to optimize? I have enough scar tissue to know
that attempts to optimize without knowing exactly what you are
doing are often counterproductive, but in a perfect world,
appropriate techniques might involve sizing buffers correctly
in terms of disk track size, and using enough threads to make
sure the disk controller was kept busy with actual writes to
the hardware instead of waiting for the drive to spin.
The parameters for adjusting that kind of algorithm would of
course vary from computer to computer, so my app would have to
determine them at run-time.
I am loath to go down this rathole without a reasonable suspicion
that there is cheese at the bottom. Any advice or experience
out there? I did check the web, to no avail.
In a still more perfect world, Apple would already have provided
functionality to optimize large file writes ... but NSFileHandle's
"writeData:"; offers no hint that any such thing has been done.
And FWIW, I am building using Xcode 3.1.3, and using the LLVM
GCC 4.2 compiler, to run on MacOS 10.5.
-- Jay Reynolds Freeman
---------------------
email@hidden
http://web.mac.com/jay_reynolds_freeman (personal web site)
_______________________________________________
Cocoa-dev mailing list (email@hidden)
Please do not post admin requests or moderator comments to the list.
Contact the moderators at cocoa-dev-admins(at)lists.apple.com
Help/Unsubscribe/Update your Subscription:
This email sent to email@hidden