Re: looong saveChanges in a background task
Re: looong saveChanges in a background task
- Subject: Re: looong saveChanges in a background task
- From: Samuel Pelletier <email@hidden>
- Date: Tue, 24 Feb 2015 15:15:16 -0500
OC,
If you have a large number of changes (insert, update and delete), the commit can take a long time in the database server, especially if your indexes does not fit in memory. You may help the database by using a more isolated connection setting and make sure there is plenty of ram and remove non required indexes.
Make sure to understand exactly where the long time is spend.
If you need the load to be atomic, I hope you can schedule it to a time when the system is not or barely used. If the data loaded can be changed by your app, you done not have much choice, access to the data need to be locked during the load to ensure data integrity.
Samuel
> Le 2015-02-23 à 13:43, OC <email@hidden> a écrit :
>
> Hello there,
>
> this time, my problem -- at least I hope so -- is going to be simple and straightforward.
>
> My application allows users to import CSV; since it is a lengthy task and sice we need it to stay responsive for others, the import runs in a background task (an ERXLongResponseTask) with a lowered thread priority.
>
> Nevertheless, there still is a problem: if there's a number of imported objects, the threadEC.saveChanges() at the end can take a small eternity. And, given EOF is single-thread, it blocks other threads' access to the DB. Which, alas, rather kills the responsiveness for other users.
>
> One solution would be to use more EOF stacks, but I am told setting ERXObjectStoreCoordinatorPool.maxCoordinators>1 leaks memory as a sieve if one uses BLOBs, and we use them pretty often.
>
> Another solution would be saving changes after importing each 100-odd items, but that brings problems with potential fail of some later save and a need to clean up the previous ones.
>
> Is there another solution to the problem which I have missed? Or do I have to cope with the partial-import-saving?
>
> Thanks a lot,
> OC
>
>
> _______________________________________________
> Do not post admin requests to the list. They will be ignored.
> Webobjects-dev mailing list (email@hidden)
> Help/Unsubscribe/Update your Subscription:
>
> This email sent to email@hidden
_______________________________________________
Do not post admin requests to the list. They will be ignored.
Webobjects-dev mailing list (email@hidden)
Help/Unsubscribe/Update your Subscription:
This email sent to email@hidden