Re: Avoiding duplicate records
Re: Avoiding duplicate records
- Subject: Re: Avoiding duplicate records
- From: Mike Schrag <email@hidden>
- Date: Tue, 15 Jan 2008 10:19:22 -0500
1) Fo a fetch request to get the contacts with the emails of the
100 contacts batch (ie, blablabla where email = email1 or email =
email2 or email = email3 ...).
2) Remove duplicates in memory using a fast method, like putting
the stuff in NSSets or whatever.
3) Try to save again. Of course, it may still fail (concurrency
sucks) but the probability is much lower.
This is all thought with the assumption that the UNIQUE-related
exception is thrown when the first offending object is inserted, so
I won't get all the information I need in one single exception,
which I'm not 100% sure it's true yet.
Depending on how your unique constraint is configured, it may throw
when the first conflicting insert happens or at the end of the commit
(this is that deferrable initially deferred, thing, which I've
honestly never tried on a unique constraint, but presumably it works
the same).
The only thing I would consider is how frequent conflicts will be. If
conflicts will be frequent, it may be cheaper to fetch dupes first to
weed them out (so you're not constantly failing out 100-insert blocks).
I think if I were in your position I would just benchmark:
1) committing one at a time -- this is logically the easiest, but it
may be the overhead for this is way high ... but WO doesn't do
batching inserts ANYWAY, so who knows
2) fetching 100, comparing, deduping, then inserting and committing
3) inserting 100, committing, catch exception (fetch 100, comparing,
deduping, inserting, rinse and repeat)
You might also just benchmark the fetching and the inserting
independently so you know the relative cost of 100 of each for your
average data.
ms
_______________________________________________
Do not post admin requests to the list. They will be ignored.
Webobjects-dev mailing list (email@hidden)
Help/Unsubscribe/Update your Subscription:
This email sent to email@hidden