[Q] large flat files usage
[Q] large flat files usage
- Subject: [Q] large flat files usage
- From: Phillip Morelock <email@hidden>
- Date: Wed, 19 Jun 2002 13:13:58 -0700
Hello List,
In the days of wayback, I worked with flat files for CGI websites. Since
dynamic content was very light in those days, flat files were fine, even
though they were slow. They just never got big enough to be a problem.
Now I'm in a situation where I have some enormous flat files to work with
(we're talking 5, 10 megs, etc.). I am looking for some (either unix-based
or even cocoa-based) ways for dealing with that kind of data on the "back
end" of a local application, so to speak. (not a webserver).
Basically, I want what appears to be the holy grail of flat-file usage:
1) minimize in-memory footprint -- so holding a 10 meg flat file all in
memory is not really cool.
2) make both read and write speeds as unnoticeable as possible
3) be able to display records in, say, an NSTableView that would possibly
only have the "visible" rows calculated/resident (to conserve resources), or
some similar scheme,
4) to have those visible records editable and have the changes be reliably
and quickly written out
I admit that I'm at the end of my rope experience-wise as far as dealing
with this, so I would love some man pages or articles to get started with.
I could potentially create some sort of indexing byte-offset scheme, as the
files are sorted (mostly ascii-betically, I think). This would allow me to
use fopen, etc. calls to get to a certain offset in the file without using
too much memory, correct?
Of course I am hoping there's some "dyno-MITE" Cocoa way of doing this, but
a little messy C library stuff is just fine with me. Maybe I should just
wrap a perl program (not really doing much for my memory requirements,
tho...).
As you can see, I'm in need of some general direction. I've tried googling
and poking around man fopen, etc., but "working with large flat files" etc.
searches on google get you to a lot of commercial product "marketeching"
pages.
Thanks in advance
Phillip
_______________________________________________
cocoa-dev mailing list | email@hidden
Help/Unsubscribe/Archives:
http://www.lists.apple.com/mailman/listinfo/cocoa-dev
Do not post admin requests to the list. They will be ignored.