Re: Read lines from very large text file
Re: Read lines from very large text file
- Subject: Re: Read lines from very large text file
- From: Michael Ash <email@hidden>
- Date: Tue, 3 Feb 2009 00:55:33 -0500
On Mon, Feb 2, 2009 at 8:53 PM, Joar Wingfors <email@hidden> wrote:
>
> On Feb 2, 2009, at 1:49 PM, Seth Willits wrote:
>
>>> I am wondering what the best way to read a text file, line by line, when
>>> the file size is much larger than available memory.
>>
>> Use mmap. Scan through the bytes to find line ranges, and create strings
>> from there. Make sure it's deallocated when you're done with it, then move
>> on to the next chunk.
>
> Everything I've seen in this thread so far skimps on one important detail:
> If you're just looking at the raw data, how do you know how to interpret it?
> If it's a known fixed character width ecoding or if you don't care about
> representing it as readable text (as in the case of Hex Fiend), then no
> problem, but what if that's not the case?
It hasn't been addressed because it's not really relevant to the
question at hand. Yes, you definitely need to either know or be able
to discover the text encoding of the text files you're dealing with.
But aside from both being about text files, that question is unrelated
to the question of how to process a large text file line-by-line.
Mike
_______________________________________________
Cocoa-dev mailing list (email@hidden)
Please do not post admin requests or moderator comments to the list.
Contact the moderators at cocoa-dev-admins(at)lists.apple.com
Help/Unsubscribe/Update your Subscription:
This email sent to email@hidden