Re: looong saveChanges in a background task
Re: looong saveChanges in a background task
- Subject: Re: looong saveChanges in a background task
- From: Mark Wardle <email@hidden>
- Date: Tue, 24 Feb 2015 11:40:52 +0000
I do this a lot when importing SNOMED-CT data (which is huge) but am afraid I import in batches and saveChanges and use a new editingContext for each batch.
Mark
> On 23 Feb 2015, at 18:43, OC <email@hidden> wrote:
>
> Hello there,
>
> this time, my problem -- at least I hope so -- is going to be simple and straightforward.
>
> My application allows users to import CSV; since it is a lengthy task and sice we need it to stay responsive for others, the import runs in a background task (an ERXLongResponseTask) with a lowered thread priority.
>
> Nevertheless, there still is a problem: if there's a number of imported objects, the threadEC.saveChanges() at the end can take a small eternity. And, given EOF is single-thread, it blocks other threads' access to the DB. Which, alas, rather kills the responsiveness for other users.
>
> One solution would be to use more EOF stacks, but I am told setting ERXObjectStoreCoordinatorPool.maxCoordinators>1 leaks memory as a sieve if one uses BLOBs, and we use them pretty often.
>
> Another solution would be saving changes after importing each 100-odd items, but that brings problems with potential fail of some later save and a need to clean up the previous ones.
>
> Is there another solution to the problem which I have missed? Or do I have to cope with the partial-import-saving?
>
> Thanks a lot,
> OC
>
>
> _______________________________________________
> Do not post admin requests to the list. They will be ignored.
> Webobjects-dev mailing list (email@hidden)
> Help/Unsubscribe/Update your Subscription:
>
> This email sent to email@hidden
_______________________________________________
Do not post admin requests to the list. They will be ignored.
Webobjects-dev mailing list (email@hidden)
Help/Unsubscribe/Update your Subscription:
This email sent to email@hidden