Re: Expanding Import
Re: Expanding Import
- Subject: Re: Expanding Import
- From: Chuck Hill <email@hidden>
- Date: Tue, 7 Mar 2006 14:20:24 -0800
Hi Scott,
On Mar 7, 2006, at 1:49 PM, Scott Winn wrote:
Hello List,
I am working on a fixed-width record importer for my database. I
have to interface with a legacy app, so there is no getting out of it.
My WO importer app has significant performance issues. The CPU is
very busy, memory usage is high, and the database has hardly any
load on it. I have increased my JVM size, I ec.dispose() and get a
new EC periodically, and I run garbage collection after every
imported file. I don't get out of memory errors anymore, but after
importing a few months worth of data the app runs so slow that it
might as well be stopped.
As I read the files, I get codes for an Entity like NCStation, for
example. Then I check for a match on stationCode in the database
using a fetchSpec, create it if it doesn't exist, and add it to the
appropriate relationship to a Certificate in this case. To see
what is going on, I called ec.updatedObjects() just before an
ec.saveChanges() and I got something like this. . .
values = {
stationCode = "199";
stationDescription = <com.webobjects.foundation.NSKeyValueCoding
$Null>;
Certificates = (
--> "<NCCertificate b94a3f _EOIntegralKeyGlobalID[Certificate
(java.lang.Integer)1]>",
--> "<NCCertificate b9ae95 _EOIntegralKeyGlobalID[Certificate
(java.lang.Integer)2]>",
--> "<NCCertificate 74c043 _EOIntegralKeyGlobalID[Certificate
(java.lang.Integer)3]>",
--> "<NCCertificate faa3c5 _EOIntegralKeyGlobalID[Certificate
(java.lang.Integer)4]>",
--> "<NCCertificate 15ebf0 _EOIntegralKeyGlobalID[Certificate
(java.lang.Integer)5]>",
"<NCCertificate 75deb1 <EOTemporaryGlobalID: 0 0 -64 -88 42 -3 0
0 -60 50 22 0 0 0 1 9 -73 45 -46 -91 86 -26 -122 58>>");
Company = "<NCCompany fc9484 _EOIntegralKeyGlobalID[Company
(java.lang.Integer)1]>";
stationName = <com.webobjects.foundation.NSKeyValueCoding$Null>;
};
this = "<NCStation 9096c6 _EOIntegralKeyGlobalID[Station
(java.lang.Integer)1]>";
The lines indicated with --> seem to just be along for the ride. I
don't need to make any changes to them. I don't want to make any
changes to the entity itself. All I want to do is add more objects
to the relationship. After two or three imported files, my ec.
updatedObjects() call is showing thousands of entries for the "to-
many" relationships. Obviously, it just gets worse the more I import.
So my questions are these. . .
1) Is my diagnosis on the right track?
I think so.
2) What is the right way to approach something like this? Raw
Rows? Or would that make any difference?
It might, but you would have to form the relationships yourself.
And the big question. . .
3) How do I check the database for an Entity and relate an object
to it when there are potentially a million other objects related to
it?
I think, in a round about way, you have identified the problem. I
will hazard a guess that in your model you have a Certificate ->>
Stations. So as you add a certificate to a station, this reverse
relationship grows ever larger. The solution, is to not include this
relationship in your model or to not make it a class property. In
most cases like this, you really don't need this relationship or
seldom need it. In this case you can mimic it by fetching the
related objects when needed.
Chuck
--
Coming in 2006 - an introduction to web applications using WebObjects
and Xcode http://www.global-village.net/wointro
Practical WebObjects - for developers who want to increase their
overall knowledge of WebObjects or who are trying to solve specific
problems. http://www.global-village.net/products/practical_webobjects
_______________________________________________
Do not post admin requests to the list. They will be ignored.
Webobjects-dev mailing list (email@hidden)
Help/Unsubscribe/Update your Subscription:
This email sent to email@hidden