Re: Size limit of NSData or NSFileHandle?
Re: Size limit of NSData or NSFileHandle?
- Subject: Re: Size limit of NSData or NSFileHandle?
- From: Ivan Kourtev <email@hidden>
- Date: Mon, 10 Apr 2006 14:32:49 -0400
Greg:
I tried your approach -- consecutive calls to - [NSFileHandle
availableData] -- and it indeed worked, so I got the file in two
chunks. I also find another workaround:
fileData = [[NSData alloc] initWithContentsOfFile:filename
options:NSUncachedRead | NSMappedRead error:&err];
which initialized the data with the entire file size of 310 MB.
This clearly indicates that either the documentation for -
[NSFileHandle availableData] or its implementation is wrong, quote
from Apple's Docs (see quote following): "Discussion: If the receiver
is a file, returns the data obtained by reading the file from the
file pointer to the end of the file. If the receiver is a
communications channel, reads up to a buffer of data and returns it;
if no data is available, the method blocks. Returns an empty data
object if the end of file is reached. Raises
NSFileHandleOperationException if attempts to determine file-handle
type fail or if attempts to read from the file or channel fail."
--
ivan
On Apr 10, 2006, at 10:21 AM, Greg Herlihy wrote:
The maximum amount of virtual address space available to an OS X
app is 4
GB. The maximum size of a file on an HFS+ volume is over 4 billion
times
larger than 4 GB (the precise limit is 2^63 bytes, a number greater
than the
estimated number of atoms in the entire universe as it so happens).
Therefore it is reasonable to expect that some files may be too
large for
their entire contents to be resident in an OS X app's memory at one
time. So
when reading a file, a program should not assume that the first read
operation has read the file's entire contents - instead the program
should
keep on reading the file's contents until no more data is left to
be read.
Here is some code to demonstrate this technique:
NSFileHandle *fh = [NSFileHandle fileHandleForReadingAtPath:
@"BigFile.txt"];
assert(fh != NULL);
unsigned len = 0;
do
{
NSData * data = [fh availableData];
len = [data length];
printf("data len: %d offset: %lld\n", len, [fh offsetInFile]);
// release the data before reading more
// or the consequences will be quite unpleasant
[data release];
} while (len != 0);
Greg
On 4/9/06 1:33 PM, "Ivan Kourtev" <email@hidden> wrote:
Hi,
Is anyone aware of a size limit in NSData (I am aware of the supposed
2GB size limit of NSData according to the docs) or NSFileHandle?
I am using the - [NSFileHandle availableData] method to get all the
data of a file in one NSData object. But it seems that the maximum
amount of data I'll get back is 268,435,456 bytes -- it happens to be
precisely 2^28, by the way. I tried both (A) NSData *fileData =
[inFile availableData]; and (B) NSData *fileData = [inFile
readDataOfLength:fileSize]; but no luck either way.
I use this way of reading the entire file contents at once for many
files, and for the first time today noticed that it broke on a very
large file of about 310 MB in size.
I see no exceptions being raised. Is this a bug, or is someone aware
of this behavior? Couldn't find any information on this...
Thanks,
-- ivan
_______________________________________________
Do not post admin requests to the list. They will be ignored.
Cocoa-dev mailing list (email@hidden)
Help/Unsubscribe/Update your Subscription:
This email sent to email@hidden
_______________________________________________
Do not post admin requests to the list. They will be ignored.
Cocoa-dev mailing list (email@hidden)
Help/Unsubscribe/Update your Subscription:
This email sent to email@hidden