Re: Read lines from very large text file
Re: Read lines from very large text file
- Subject: Re: Read lines from very large text file
- From: Seth Willits <email@hidden>
- Date: Mon, 2 Feb 2009 18:02:35 -0800
On Feb 2, 2009, at 5:53 PM, Joar Wingfors wrote:
I am wondering what the best way to read a text file, line by
line, when the file size is much larger than available memory.
Use mmap. Scan through the bytes to find line ranges, and create
strings from there. Make sure it's deallocated when you're done
with it, then move on to the next chunk.
Everything I've seen in this thread so far skimps on one important
detail: If you're just looking at the raw data, how do you know how
to interpret it? If it's a known fixed character width ecoding or if
you don't care about representing it as readable text (as in the
case of Hex Fiend), then no problem, but what if that's not the case?
Before opening the file, either determine, guess, or be told what the
encoding is. With that encoding, convert your delimiter string into
raw bytes, then do byte-for-byte comparison on the file to find
occurrences of that delimiter.
--
Seth Willits
_______________________________________________
Cocoa-dev mailing list (email@hidden)
Please do not post admin requests or moderator comments to the list.
Contact the moderators at cocoa-dev-admins(at)lists.apple.com
Help/Unsubscribe/Update your Subscription:
This email sent to email@hidden