Re: Parsing Large Text Files
Re: Parsing Large Text Files
- Subject: Re: Parsing Large Text Files
- From: "Nigel Garvey" <email@hidden>
- Date: Sat, 3 May 2008 11:57:58 +0100
Bruce Robertson wrote on Fri, 02 May 2008 08:21:36 -0700:
>>[From me]
>> Perl's obviously the way to go in your case, but I can't resist tinkering
>> with the AS performance. :) This uses File Read/Write instead of the
>> shell script, 'set' instead of 'copy', referenced list variables, and a
>> more efficient way of inserting returns into the reversed text:
>Yes, that't nice, gets down to about 22MB/minute.
>
>The perl script processes at about 800MB/minute by my rough test.
I managed to knock another 25% off the running time of mine, but it's
still no match for the perl. And somewhere between my 6MB test file size
and the 80MB target, Script Editor crashes.
I couldn't get the perl script to work yesterday: the output files were
always empty. (No wonder it was fast!) The problem turned out this
morning to be the line endings in the pl file. They _have_ to be line feeds.
[OT] While optimising the AS version on a Jaguar machine, I used this
handler to get round the old 4000-element extraction limit:
-- This uses a delimiter set elsewhere.
on getTextItems(theText)
set ti to {}
set n to (count theText's text items)
repeat with i from 1 to n by 3900
set j to i + 3899
if (j > n) then set j to n
set ti to ti & text items i thru j of theText
end repeat
return ti
end getTextItems
On my Tiger machine, of course, this isn't necessary. But it turns out
(with 30004 text items, at least) that it's about 2.5 times as fast on
that machine as a single 'text items of ...' extraction! :\
NG
_______________________________________________
Do not post admin requests to the list. They will be ignored.
AppleScript-Users mailing list (email@hidden)
Help/Unsubscribe/Update your Subscription:
Archives: http://lists.apple.com/archives/applescript-users
This email sent to email@hidden