Re: Long text file manipulation...
Re: Long text file manipulation...
- Subject: Re: Long text file manipulation...
- From: Christopher Nebel <email@hidden>
- Date: Wed, 4 Apr 2001 12:08:46 -0700
On Wednesday, April 4, 2001, at 05:54 AM, bRaD Weston wrote:
I'm working on a project in which I need to manipulate large text
files, one line at a time. I've been looking for some sort of OSAX that
can do this because I cannot afford the memory and time requirements to
read in lots of big (2.5 MB) text files and then save them out, but
have been unsuccessful.
Essentially, I just want to read the first line of the file and delete
it from the file and add another line to the bottom of the file.
While adding something to the end of a file is fast and easy (just say
'write "stuff" to file "blah" starting at eof'), deleting something from
the beginning or middle is an inherently expensive operation that
requires someone to read everything in the file after the deletion point
and write it back out again. Given the tools currently available in
AppleScript, "someone" means your script. You don't have to read the
entire file at once, so you can avoid a 2.5 MB memory hit, but you're
pretty much stuck for it as far as time goes.
Something like this will work fairly efficiently using only standard
additions:
set source to open for access file "foo"
set destination to open for access file "bar" with write permission
set eof destination to 0 -- empty out the destination file.
read source until return -- discard first line
try
repeat
read source for 16384 -- read 16K worth of what remains.
write the result to destination
end
end
close access source
write "trailer" to destination
close access destination
You can twiddle the constant 16384 to alter the time vs. maximum memory
profile. (Higher numbers should be faster, but will consume more
memory.) If you want to mangle the file in place instead of generating
an altered copy, the logic is significantly more difficult.
--Chris Nebel
AppleScript Engineering