Re: How to parse CSV
Re: How to parse CSV
- Subject: Re: How to parse CSV
- From: Skeeve <email@hidden>
- Date: Wed, 23 Jan 2008 17:17:41 +0100
Benedikt Quirmbach wrote:
The real csv files have up to 35000 lines and up to 10 items every line.
Here the handler works endless. As Skeeve said (he testet with 10000
lines) there is a stack overflow.
I changed my program a bit. First version was a perl script emitting
AppleScript code for the LoL. Now, as a proof of concept, I simply
replace all the CRs and LFs by a \n and let AppleScript split the
paragraphs and text items and let it create the LoL.
The test data is created with
1st Scenario
perl -e 'for $i (1..400) { print join ("$i|",('A'..'J')),"$i\n" }'>FDF.psv
2nd Scenario
perl -e 'for $i (1..4000) { print join ("$i|",('A'..'J')),"$i\n" }'>FDF.psv
The AppleScript is
-- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
on run
set started to current date
set {oAStid, AppleScript's text item delimiters} to {AppleScript's
text item delimiters, "|"}
set LoL to {}
repeat with rec in every paragraph of convert(choose file)
copy text items of rec to the end of LoL
end repeat
set AppleScript's text item delimiters to oAStid
set stopped to current date
return "Seconds elapsed: " & (stopped - started)
end run
to convert(a_file)
return do shell script "perl -pe 'tr/\\012\\015//d; $_.=$/' " &
quoted form of POSIX path of a_file
end convert
-- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
1st Scenario: Seconds elapsed 5 => 80 lines / Second
2nd Scenario: Seconds elapssed 4345 => 2.3 lines / Second
_______________________________________________
Do not post admin requests to the list. They will be ignored.
AppleScript-Users mailing list (email@hidden)
Help/Unsubscribe/Update your Subscription:
Archives: http://lists.apple.com/archives/applescript-users
This email sent to email@hidden