I did try ec.rootObjectStore.invalidate... with correct locking and also got deadlocks. I was able to get some Thread dumps. In almost all cases, the SharedEC was involved. Sample at the bottom of the message.
Even when the sharedEC was disabled, I got IllegalStateExceptions. I posted a message concerning that a few days ago regarding that:
Somehow it appears impossible to me to go the "invalidate global IDs" route because I'm getting IllegalStateExceptions during saveChanges (see bottom of post). I made some reproduction code which reproduces the problem quite reliably (also see bottom of post).
I tried tons of alternatives for the locking (ec.lockObjectectStore() instead ec.rootObjectStore().lock(), getting down to the database context and locking it as well, ...) but the problem remains. Also tried EODatabase.forgetSnapshotForGlobalID with the same result.
Before I finally give up on the subject, could someone with a deeper understanding of the locking stuff (Chuck?) take a look at it. I think it should work this way. If I'm not missing something, I currently see no way in getting any kind of change notification framework work reliably with the "invalidate global IDs" approach. And the "snapshot" approach is already known for causing deadlocks.
Locally I'm testing on Intel/Tiger/WO 5.3.1, but have seen the same Exceptions on PPC/Panther Server/WO 5.2.3.
Any help would be greatly appreciated.
Timo
PS: As Apple is going out of the tools business for WO and wants to concentrate more on the framework, I *really* hope they take a deep look into EOF concurrency and cross-VM cache synchronization. I guess that will remain a wish but would make life soooooo much easier...
PPS:
The meat of the code is:
a) in some thread read and write data concurrently (with correct locking)
b) simulate incoming change notifications in another tread
EOEditingContext ec = new EOEditingContext();
try {
ec.lock();
ec.rootObjectStore().lock();
ec.rootObjectStore().invalidateAllObjects(); // just for testing, real code would only invalidate certain GIDs
} finally {
ec.rootObjectStore().unlock();
ec.unlock();
ec.dispose();
}
Result is e.g.:
java.lang.IllegalStateException:
rowDiffsForAttributes:
snapshot in com.webobjects.eoaccess.EODatabaseOperation {
_dbSnapshot = {};
_entity = "TestEntity";
_newRow = {id = 45; testValue = "Thread-2/2200"; lastUpdate = 2006-09-13 15:12:40 Etc/GMT; };
_object = "{values = {lastUpdate = 2006-09-13 15:12:40 Etc/GMT; testValue = "Thread-2/2200"; };
this = "<timo.testet.TestEntity 5b8e6c _EOIntegralKeyGlobalID[TestEntity (java.lang.Integer)45]>"; }";
_globalID = _EOIntegralKeyGlobalID[TestEntity (java.lang.Integer)45];
_databaseOperator = "EODatabaseUpdateOperator";
}
does not contain value for attribute named id with snapshot key: id
Sample of deadlock with sharedEC involved while invalidating objects on rootObjectStore:
"ActiveMQ Session Task" daemon prio=7 tid=0x005d7bb0 nid=0x30ad8e00 in Object.wait() [f181e000..f1820b20]
at java.lang.Object.wait(Native Method)
- waiting on <0x999cb18> (a com.webobjects.foundation.NSMultiReaderLock$ConditionLock)
at java.lang.Object.wait(Object.java:429)
at com.webobjects.foundation.NSMultiReaderLock$ConditionLock.await(NSMultiReaderLock.java:505)
- locked <0x999cb18> (a com.webobjects.foundation.NSMultiReaderLock$ConditionLock)
at com.webobjects.foundation.NSMultiReaderLock._lockForWriting(NSMultiReaderLock.java:204)
at com.webobjects.foundation.NSMultiReaderLock.lockForWriting(NSMultiReaderLock.java:165)
at com.webobjects.eocontrol.EOSharedEditingContext.lock(EOSharedEditingContext.java:700)
at com.webobjects.eocontrol.EOEditingContext.lockObjectStore(EOEditingContext.java:4673)
at com.webobjects.eocontrol.EOEditingContext.refaultObject(EOEditingContext.java:3980)
at er.extensions.ERXEC.refaultObject(ERXEC.java:753)
at com.webobjects.eocontrol.EOEditingContext._refaultObjectWithGlobalID(EOEditingContext.java:3232)
at com.webobjects.eocontrol.EOEditingContext._newChangesFromInvalidatingObjectsWithGlobalIDs(EOEditingContext.java:3461)
at com.webobjects.eocontrol.EOEditingContext._processObjectStoreChanges(EOEditingContext.java:3497)
at sun.reflect.GeneratedMethodAccessor68.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:324)
at com.webobjects.foundation.NSSelector.invoke(NSSelector.java:354)
at com.webobjects.foundation.NSSelector._safeInvokeSelector(NSSelector.java:108)
at com.webobjects.eocontrol.EOEditingContext._processNotificationQueue(EOEditingContext.java:4750)
at com.webobjects.eocontrol.EOEditingContext.lock(EOEditingContext.java:4627)
at er.extensions.ERXEC.lock(ERXEC.java:259)
at com.webobjects.eocontrol.EOEditingContext.tryLock(EOEditingContext.java:4639)
at com.webobjects.eocontrol.EOEditingContext._sendOrEnqueueNotification(EOEditingContext.java:4714)
at com.webobjects.eocontrol.EOEditingContext._objectsChangedInStore(EOEditingContext.java:3537)
at er.extensions.ERXEC._objectsChangedInStore(ERXEC.java:919)
at sun.reflect.GeneratedMethodAccessor88.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:324)
at com.webobjects.foundation.NSSelector._safeInvokeMethod(NSSelector.java:120)
at com.webobjects.foundation.NSNotificationCenter$_Entry.invokeMethod(NSNotificationCenter.java:598)
at com.webobjects.foundation.NSNotificationCenter.postNotification(NSNotificationCenter.java:542)
at com.webobjects.foundation.NSNotificationCenter.postNotification(NSNotificationCenter.java:572)
at com.webobjects.eocontrol.EOObjectStoreCoordinator._objectsChangedInSubStore(EOObjectStoreCoordinator.java:744)
at sun.reflect.GeneratedMethodAccessor94.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:324)
at com.webobjects.foundation.NSSelector._safeInvokeMethod(NSSelector.java:120)
at com.webobjects.foundation.NSNotificationCenter$_Entry.invokeMethod(NSNotificationCenter.java:598)
at com.webobjects.foundation.NSNotificationCenter.postNotification(NSNotificationCenter.java:542)
at com.webobjects.foundation.NSNotificationCenter.postNotification(NSNotificationCenter.java:572)
at com.webobjects.eoaccess.EODatabaseContext._snapshotsChangedInDatabase(EODatabaseContext.java:3812)
at sun.reflect.GeneratedMethodAccessor93.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:324)
at com.webobjects.foundation.NSSelector._safeInvokeMethod(NSSelector.java:120)
at com.webobjects.foundation.NSNotificationCenter$_Entry.invokeMethod(NSNotificationCenter.java:598)
at com.webobjects.foundation.NSNotificationCenter.postNotification(NSNotificationCenter.java:542)
at com.webobjects.foundation.NSNotificationCenter.postNotification(NSNotificationCenter.java:572)
at com.webobjects.eoaccess.EODatabase.forgetSnapshotsForGlobalIDs(EODatabase.java:951)
at com.webobjects.eoaccess.EODatabaseContext.forgetSnapshotsForGlobalIDs(EODatabaseContext.java:2397)
at com.webobjects.eoaccess.EODatabaseContext.invalidateObjectsWithGlobalIDs(EODatabaseContext.java:3781)
at com.webobjects.eocontrol.EOObjectStoreCoordinator.invalidateObjectsWithGlobalIDs(EOObjectStoreCoordinator.java:718)
at er.changenotification.ERCNSubscriber._processUpdates(ERCNSubscriber.java:166)
at er.changenotification.ERCNSubscriber.onMessage(ERCNSubscriber.java:103)
at org.apache.activemq.ActiveMQMessageConsumer.dispatch(ActiveMQMessageConsumer.java:795)
at org.apache.activemq.ActiveMQSessionExecutor.dispatch(ActiveMQSessionExecutor.java:96)
at org.apache.activemq.ActiveMQSessionExecutor.iterate(ActiveMQSessionExecutor.java:149)
at org.apache.activemq.thread.PooledTaskRunner.runTask(PooledTaskRunner.java:110)
at org.apache.activemq.thread.PooledTaskRunner.access$100(PooledTaskRunner.java:25)
at org.apache.activemq.thread.PooledTaskRunner$1.run(PooledTaskRunner.java:43)
at edu.emory.mathcs.backport.java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:650)
at edu.emory.mathcs.backport.java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:675)
at java.lang.Thread.run(Thread.java:552)