Re: WOWorkerThread deadlocks
Re: WOWorkerThread deadlocks
- Subject: Re: WOWorkerThread deadlocks
- From: Maik Musall <email@hidden>
- Date: Thu, 13 Sep 2012 08:52:43 +0200
Hi Alex, Hi Chuck,
Am 13.09.2012 um 02:28 schrieb Chuck Hill <email@hidden>:
>>> Never out of memory. The app is allowed to grow up to 24 GByte, stays in the 1-4 GByte range in normal use and occasionally grows up to 12 GByte with the most advanced statistics that tend to suck in the whole database.
>>>
>>> That's also the reason though that I can't use separate EOF stacks for the statistics, because as soon as there were more than one of them, I'd have multiple 10 GByte-ish snapshot caches. The server has 48 GByte and I don't really want to hit that limit... and with separate stacks, it also would be difficult to keep the stats reflect current changes in the other stacks.
>>
>>
>> I am not sure about the background threads (depends on how long OSC locks are held), but using ECs sharing the same EOF stack with regular requests is likely to cause problems like you are seeing.
>>
>> Do you mean that the application would be unresponsive while the lock was held in the background thread, or that simply doing it that way will lead to unrecoverable deadlocks?
>> If you do massive fetches in the background, that will block other requests as the only OSC is locked.
>
> Correct.
>
>
>> That said, I think (and correct me if I'm wrong) if you lock the ec but do not fetch anything with this ec, other ecs can still access the db.
>
> Also correct. The lock contention is only when fetching or saving. It can also happen if your code (or Wonder code that you are using) locks something in EOControl or EOAccess.
I'm very familiar with that stuff, and my users know how it feels to wait for that lock :-)
>> Anyway, the best practice is to use a dedicated OSC to do background work.
>> Maik, you should use a dedicated OSC for your stats, and try, if possible to clean memory, for example :
>>
>> ec.lock();
>> try {
>> // huge loop to compute stats
>> for (i = 0; i < 1000000; i++) {
>> // doing stuff with ec...
>> // cycling the ec
>> if (i % 100 == 0) {
>> ec.unlock();
>> ec.dispose();
>> ec = newEditingContextForMyWork();
>> ec.lock();
>> }
>> }
>> } finally {
>> ec.unlock();
>> }
>
> If practical (I recall that it is not in Maik's case) that can be a good way of limiting memory usage.
Right, not practical for me. I even rely on those statistics to fill the snapshot cache with data that other users will need in a minute anyway to speed up overall response times. Those statistics are not strictly background processes, they are user interaction that happens to be implemented in a worker thread while the user is displayed a long response page.
What I've done to improve concurrent response times while those stats fetch their 300000 EOs: I fetch them in batches of a few 1000 and release the lock in between. This is the method I can call on my manual-locking editing context between batches:
public void shortLockRelease() {
unlock();
try {
Thread.sleep( 50 );
} catch( InterruptedException e ) {
e.printStackTrace();
} finally {
lock();
}
}
This effectively gives other threads the opportunity to sneak in a few transactions before the stats worker resumes grabbing the OSC's resources, and is enough to keep response times within a reasonable limit. Users feel it when stats are running, but they don't have to really wait any more. I've even tuned those 50 ms. Less than that and don't get the desired effect. More than that and you needlessly increase the stats execution time.
Maik
_______________________________________________
Do not post admin requests to the list. They will be ignored.
Webobjects-dev mailing list (email@hidden)
Help/Unsubscribe/Update your Subscription:
This email sent to email@hidden