Re: EC locking (headless and threads)
Re: EC locking (headless and threads)
- Subject: Re: EC locking (headless and threads)
- From: Alexis Tual <email@hidden>
- Date: Mon, 13 Feb 2012 13:17:48 +1100
Hi Kieran and others,
related to this topic, I have a deadlock situation which I can't explain.
I have 2 background threads doing EOF stuff, the first one is doing a lot of fetches and can take several minutes. The second one is doing periodical short work (a fetch every 30sec).
For these two threads I use manual locking : locking at the begining of the processing and unlocking at the end. I don't share ec's or eo's between threads. However they use the same OSC.
Here's the stack trace, any help is welcomed !
Found one Java-level deadlock:
=============================
"Timer-1":
waiting for ownable synchronizer 7de742608, (a java.util.concurrent.locks.ReentrantLock$NonfairSync),
which is held by "QuartzScheduler_Worker-1"
"QuartzScheduler_Worker-1":
waiting for ownable synchronizer 7de40c8e8, (a java.util.concurrent.locks.ReentrantReadWriteLock$NonfairSync),
which is held by "Timer-1"
Java stack information for the threads listed above:
===================================================
"Timer-1":
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <7de742608> (a java.util.concurrent.locks.ReentrantLock$NonfairSync)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:156)
at java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:811)
at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireQueued(AbstractQueuedSynchronizer.java:842)
at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquire(AbstractQueuedSynchronizer.java:1178)
at java.util.concurrent.locks.ReentrantLock$NonfairSync.lock(ReentrantLock.java:186)
at java.util.concurrent.locks.ReentrantLock.lock(ReentrantLock.java:262)
at com.webobjects.eocontrol.EOObjectStoreCoordinator.lock(EOObjectStoreCoordinator.java:420)
at com.webobjects.eocontrol.EOEditingContext.lockObjectStore(EOEditingContext.java:4666)
at er.extensions.eof.ERXEC.lockObjectStore(ERXEC.java:724)
at com.webobjects.eocontrol.EOEditingContext.objectsWithFetchSpecification(EOEditingContext.java:4067)
at er.extensions.eof.ERXEC.objectsWithFetchSpecification(ERXEC.java:1215)
at com.webobjects.eocontrol.EOEditingContext.objectsWithFetchSpecification(EOEditingContext.java:4444)
at er.extensions.eof.ERXFetchSpecification.fetchObjects(ERXFetchSpecification.java:125)
at org.cocktail.fwkcktlevenement.FwkCktlEvenementPrincipal$DefaultDelegate.fetchEvents(FwkCktlEvenementPrincipal.java:321)
at org.cocktail.fwkcktlevenement.FwkCktlEvenementPrincipal$SchedulingEvenementTask.rescheduleAllEvents(FwkCktlEvenementPrincipal.java:374)
at org.cocktail.fwkcktlevenement.FwkCktlEvenementPrincipal$SchedulingEvenementTask.run(FwkCktlEvenementPrincipal.java:364)
at java.util.TimerThread.mainLoop(Timer.java:512)
at java.util.TimerThread.run(Timer.java:462)
"QuartzScheduler_Worker-1":
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <7de40c8e8> (a java.util.concurrent.locks.ReentrantReadWriteLock$NonfairSync)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:156)
at java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:811)
at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireQueued(AbstractQueuedSynchronizer.java:842)
at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquire(AbstractQueuedSynchronizer.java:1178)
at java.util.concurrent.locks.ReentrantReadWriteLock$WriteLock.lock(ReentrantReadWriteLock.java:807)
at com.webobjects.eocontrol.EOSharedEditingContext.lock(EOSharedEditingContext.java:736)
at com.webobjects.eocontrol.EOEditingContext.lockObjectStore(EOEditingContext.java:4664)
at er.extensions.eof.ERXEC.lockObjectStore(ERXEC.java:724)
at com.webobjects.eocontrol.EOEditingContext.objectsWithFetchSpecification(EOEditingContext.java:4067)
at er.extensions.eof.ERXEC.objectsWithFetchSpecification(ERXEC.java:1215)
at com.webobjects.eocontrol.EOEditingContext.objectsWithFetchSpecification(EOEditingContext.java:4444)
at er.extensions.eof.ERXEOGlobalIDUtilities.fetchObjectsWithGlobalIDs(ERXEOGlobalIDUtilities.java:290)
at er.extensions.eof.ERXEOGlobalIDUtilities.fetchObjectsWithGlobalIDs(ERXEOGlobalIDUtilities.java:229)
at er.extensions.eof.ERXEOGlobalIDUtilities.fetchObjectWithGlobalID(ERXEOGlobalIDUtilities.java:209)
at org.cocktail.rgrhum.serveur.metier.job.JobValidationReport.executeJob(JobValidationReport.java:67)
at org.cocktail.fwkcktlevenement.serveur.quartz.job.impl.JobEvenementImpl.execute(JobEvenementImpl.java:116)
at org.quartz.core.JobRunShell.run(JobRunShell.java:216)
at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:549)
Found 1 deadlock.
2012/2/13 Kieran Kelleher
<email@hidden>
Just to follow up..... you can use the task OSC pool as follows:
osc = ERXTaskObjectStoreCoordinatorPool.objectStoreCoordinator()
You can configure how many OSC's in the pool using this property which defaults to 1 (which is still better than using the default OSC for background EOF tasks):
er.extensions.concurrency.ERXTaskObjectStoreCoordinatorPool.maxCoordinators = 1
As a convenience, there is an abstract "task" class,ERXAbstractTask, you can subclass which has a newEditingContext() method that incorporates the OSC pool for you and adjusts EC timestamp to ensure fresh EOs in background tasks. IIRC, this is all explained in the WOWODc presentation too.
On Feb 11, 2012, at 6:00 PM, Kieran Kelleher wrote:
> Don't worry about the OSC unless you manipulating the OSC directly.
>
> BTW intensive background EOF activity can impact regular user EOF performance. One approach is to use a dedicated OSC pool for background tasks. Such a pool exists in Wonder. IIRC it is used in the WOWODC example app.
>
> Regards, Kieran.
> (Sent from my iPhone)
>
>
> On Feb 11, 2012, at 7:56 AM, Giles Palmer <
email@hidden> wrote:
>
>> Hi
>>
>> Just to extend this... when (if at all) should we be locking the EOObjectStoreCoordinator as well, in the context of background threads?
>>
>> I lock and unlock my ec in a try, catch, finally, is that enough or do I also need to worry about the OSC?
>>
>> Thanks
>>
>>
>> Giles
>>
>>
>>> Also if you use ERXExecutorService to execute any Plain Old Java Callable (or Runnable), your editing contexts will be auto unlocked by safety-net unlocker at the end of execution if you haven't done so ....... however it is highly recommend that you follow the ec lock/try/finally/unlock pattern in in your Callable.call() or Runnable.run() methods anyway which will be better for long running tasks, recycling ec's if needed, etc. it doesn't hurt.
>>>
>>> EOEditingContext ec = ERXEC.newEditingContext();
>>> ec.lock();
>>> try {
>>> ........
>>> } finally {
>>> ec.unlock();
>>> }
>>>
>>> And yeah, do what Ramsey said .... save yourself time figuring out concurrency management of EC's, etc. by watching the WOWODC presentation from 2011. There is a bunch of stuff related to this in Wonder to make life in the background easy and totally painless for you and that WOWODC session explains it's usage.
>>>
>>>
>>> On Feb 10, 2012, at 11:34 AM, Ramsey Gurley wrote:
>>>
>>>> If you use ERXRunnable, then you still get autolocking. Just don't try to pass an existing EC or EOs to a background thread. Pass EOs by global id and create the EC on the thread.
>>>>
>>>> See also Kieran's most recent WOWODC presentation on ERXExecutor stuff and background thread processing.
>>>>
>>>> Ramsey
>>>>
>>>> On Feb 10, 2012, at 9:30 AM, Michael Gargano wrote:
>>>>
>>>>> Hi everyone,
>>>>>
>>>>> I just want to get clarification on something before I get myself into trouble later. If I have a headless WO app (or potential just a spawned worker thread), and I'm using ERXEC, I need to manually lock and unlock the context, correct? I'm assuming that ERXEC does the autolock/autounlock in the RR loop, which I won't have in this situation.
>>>>>
>>>>> Thanks.
>>>>> -Mike
>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> Do not post admin requests to the list. They will be ignored.
>>>>> Webobjects-dev mailing list (
email@hidden)
>>>>> Help/Unsubscribe/Update your Subscription:
>>>>>
>>>>> This email sent to
email@hidden
>>>>
>>>>
>>>> _______________________________________________
>>>> Do not post admin requests to the list. They will be ignored.
>>>> Webobjects-dev mailing list (
email@hidden)
>>>> Help/Unsubscribe/Update your Subscription:
>>>>
>>>> This email sent to
email@hidden
>>>
>>>
>>> _______________________________________________
>>> Do not post admin requests to the list. They will be ignored.
>>> Webobjects-dev mailing list (
email@hidden)
>>> Help/Unsubscribe/Update your Subscription:
>>>
>>> This email sent to
email@hidden
>>
_______________________________________________
Do not post admin requests to the list. They will be ignored.
Webobjects-dev mailing list (
email@hidden)
Help/Unsubscribe/Update your Subscription:
This email sent to
email@hidden
_______________________________________________
Do not post admin requests to the list. They will be ignored.
Webobjects-dev mailing list (email@hidden)
Help/Unsubscribe/Update your Subscription:
This email sent to email@hidden