Re: Importing flat files into FMP
Re: Importing flat files into FMP
- Subject: Re: Importing flat files into FMP
- From: "Arthur J. Knapp" <email@hidden>
- Date: Tue, 26 Nov 2002 12:10:54 -0500
>
Date: Mon, 25 Nov 2002 14:16:27 +0000
>
Subject: Importing flat files into FMP
>
From: Eric Schult <email@hidden>
>
I'm trying to import flat text files into FileMaker Pro, and so far it's
>
VERY slow going. I favor use of Tanaka's OSAX over Open for Access to read
>
the files. My experience over the years has been that it's faster and the
>
syntax is easier.
Your experience has served you well, but I'm afraid that it is your use
of so many scripting addition calls per record that is really slowing you
down.
>
But I've never tried having a script handle this kind of volume of importing
>
before. I have 250,000+ flat files to read, translate (using ACME Script
>
Widgets) and import into FMP. The flat files are all exported from an
>
archive of news stories (Stauffer Library System), which is soon to go
>
defunct. This archival system is proprietary, and one can only export
>
individual records as flat files - otherwise I'd export it in chunks as
>
tab-delimited text, and that would import into FMP a whole lot faster.
Just so we're clear, that's 1 record per file, right?
>
My flat file format looks something like this:
>
"DATE: Thu 03-Oct-2002
>
PUBLICATION: JT
>
CATEGORY: SP
>
AUTHOR: WIRE
>
LOCATION: D3
>
>
>
FULL TEXT:
>
>
Giant step
>
By PAUL NEWBERRY... "
>
set fileContents to (readFromFile currentFile) -- tanaka's
>
set dateLine to (pickUpFromData fileContents startOf "DATE: " ,
>
endOf return) as string
>
set dateLine to trim "DATE: " off dateLine from front side -- ACME
>
set dateLine to trim return off dateLine from back side
Yeah, I hate to say it, (since I have so much respect for ACME
Script Widgets), but a little text item delimiters alchemy can really
speed this up. Consider the following:
on ChompSingleLineField(s, sField)
(*
* returns { 1st field's data, rest of string }
*)
set o to text item delimiters
set text item delimiters to sField
set s to s's text item 2
set text item delimiters to o
set sData to s's paragraph 1
set sRest to s's text from paragraph 2 to character -1
return {sData, sRest}
end ChompSingleLineField
ChompSingleLineField(theData, "DATE: ")
set {theDate, theData} to result
ChompSingleLineField(theData, "PUBLICATION: ")
set {thePub, theData} to result
ChompSingleLineField(theData, "AUTHOR: ")
set {theCat, theData} to result
ChompSingleLineField(theData, "CATEGORY: ")
set {theAuth, theData} to result
ChompSingleLineField(theData, "LOCATION: ")
set {theLoc, theData} to result
on TrimAfter(str, sub)
(*
* returns s's text from after sub
*)
set o to text item delimiters
set text item delimiters to sub
set str to str's text from text item 2 to character -1
set text item delimiters to o
return str
end TrimAfter
set theBody to TrimAfter(theData, "FULL TEXT:")
In fact, you can avoid the constant saving and restoring of the tids by
doing all of this in one swoop:
set o to text item delimiters
set str to read customFlatFile
set text item delimiters to "DATE: "
set str to str's text item 2
set theDate to str's paragraph 1
set str to str's text from paragraph 2 to character -1
set text item delimiters to "PUBLICATION: "
set str to str's text item 2
set thePub to str's paragraph 1
set str to str's text from paragraph 2 to character -1
set text item delimiters to "CATEGORY: "
set str to str's text item 2
set theCat to str's paragraph 1
set str to str's text from paragraph 2 to character -1
set text item delimiters to "AUTHOR: "
set str to str's text item 2
set theAuthor to str's paragraph 1
set str to str's text from paragraph 2 to character -1
set text item delimiters to "LOCATION: "
set str to str's text item 2
set theLoc to str's paragraph 1
set str to str's text from paragraph 2 to character -1
set text item delimiters to "FULL TEXT:"
set theBody to str's text item 2
set text item delimiters to o
{theDate, thePub, theCat, theAuthor, theLoc, theBody}
Having done something like this for each file, you may then want to
consider appending these peices of data to a new tab-delimited or
comma-seperated file and manually importing this big file into FileMaker
Pro, as this may be faster than sending an AppleEvent for creating each
record.
>
I'd appreciate any help that might be out there!
Wouldn't we all? ;-)
{ Arthur J. Knapp, of <
http://www.STELLARViSIONs.com>
a r t h u r @ s t e l l a r v i s i o n s . c o m
}
_______________________________________________
applescript-users mailing list | email@hidden
Help/Unsubscribe/Archives:
http://www.lists.apple.com/mailman/listinfo/applescript-users
Do not post admin requests to the list. They will be ignored.