Re: Avoiding expensive database roundtrips for enterprise objects
Re: Avoiding expensive database roundtrips for enterprise objects
- Subject: Re: Avoiding expensive database roundtrips for enterprise objects
- From: Chuck Hill <email@hidden>
- Date: Tue, 31 May 2011 10:34:52 -0700
Hi Paul,
These are only refaulting the objects, not really invalidating them:
>> at com.xyrality.bk.actions.BKDirectAction.editingContextShouldInvalidateObject(BKDirectAction.java:62)
>> ...
>> at com.webobjects.eocontrol.EOEditingContext._refaultObjectWithGlobalID(EOEditingContext.java:3255)
>> at com.webobjects.eocontrol.EOEditingContext._newChangesFromInvalidatingObjectsWithGlobalIDs(EOEditingContext.java:3485)
>> at com.webobjects.eocontrol.EOEditingContext._processObjectStoreChanges(EOEditingContext.java:3522)
They reused that delegate method. This should turn the EO into a fault, which means that it will get fresh data from the snapshot cache the next time it is referenced. If it is going back to the database, then either the fetch timestamp on the EC has been moved up or (somehow) the snapshot is not in the cache. That might be happening if this EO is the ONLY reference to it and it gets re-faulted.
Chuck
On May 31, 2011, at 7:09 AM, Paul Dunkler wrote:
> Hi Chuck,
>
> sorry for the late answer... Mail just wanted to hide this message from me :)
> Here are the two traces. Hope this will help
>
>> [2011-5-31 16:3:57 CAT] <WorkerThread14> java.lang.RuntimeException
>> at com.xyrality.bk.actions.BKDirectAction.editingContextShouldInvalidateObject(BKDirectAction.java:62)
>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>> at java.lang.reflect.Method.invoke(Method.java:597)
>> at com.webobjects.foundation.NSSelector._safeInvokeMethod(NSSelector.java:122)
>> at com.webobjects.foundation._NSDelegate._perform(_NSDelegate.java:223)
>> at com.webobjects.foundation._NSDelegate.booleanPerform(_NSDelegate.java:187)
>> at com.webobjects.eocontrol.EOEditingContext._refaultObjectWithGlobalID(EOEditingContext.java:3255)
>> at com.webobjects.eocontrol.EOEditingContext._newChangesFromInvalidatingObjectsWithGlobalIDs(EOEditingContext.java:3485)
>> at com.webobjects.eocontrol.EOEditingContext._processObjectStoreChanges(EOEditingContext.java:3522)
>> at er.extensions.eof.ERXEC._processObjectStoreChanges(ERXEC.java:1440)
>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>> at java.lang.reflect.Method.invoke(Method.java:597)
>> at com.webobjects.foundation.NSSelector.invoke(NSSelector.java:358)
>> at com.webobjects.foundation.NSSelector._safeInvokeSelector(NSSelector.java:110)
>> at com.webobjects.eocontrol.EOEditingContext._sendOrEnqueueNotification(EOEditingContext.java:4715)
>> at com.webobjects.eocontrol.EOEditingContext._objectsChangedInStore(EOEditingContext.java:3562)
>> at er.extensions.eof.ERXEC._objectsChangedInStore(ERXEC.java:1380)
>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>> at java.lang.reflect.Method.invoke(Method.java:597)
>> at com.webobjects.foundation.NSSelector._safeInvokeMethod(NSSelector.java:122)
>> at com.webobjects.foundation.NSNotificationCenter$_Entry.invokeMethod(NSNotificationCenter.java:588)
>> at com.webobjects.foundation.NSNotificationCenter.postNotification(NSNotificationCenter.java:532)
>> at com.webobjects.foundation.NSNotificationCenter.postNotification(NSNotificationCenter.java:562)
>> at com.webobjects.eocontrol.EOObjectStoreCoordinator._objectsChangedInSubStore(EOObjectStoreCoordinator.java:693)
>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>> at java.lang.reflect.Method.invoke(Method.java:597)
>> at com.webobjects.foundation.NSSelector._safeInvokeMethod(NSSelector.java:122)
>> at com.webobjects.foundation.NSNotificationCenter$_Entry.invokeMethod(NSNotificationCenter.java:588)
>> at com.webobjects.foundation.NSNotificationCenter.postNotification(NSNotificationCenter.java:532)
>> at com.webobjects.foundation.NSNotificationCenter.postNotification(NSNotificationCenter.java:562)
>> at com.webobjects.eoaccess.EODatabaseContext._snapshotsChangedInDatabase(EODatabaseContext.java:3664)
>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>> at java.lang.reflect.Method.invoke(Method.java:597)
>> at com.webobjects.foundation.NSSelector._safeInvokeMethod(NSSelector.java:122)
>> at com.webobjects.foundation.NSNotificationCenter$_Entry.invokeMethod(NSNotificationCenter.java:588)
>> at com.webobjects.foundation.NSNotificationCenter.postNotification(NSNotificationCenter.java:532)
>> at com.webobjects.foundation.NSNotificationCenter.postNotification(NSNotificationCenter.java:562)
>> at com.webobjects.eoaccess.EODatabaseContext.commitChanges(EODatabaseContext.java:6422)
>> at com.webobjects.eocontrol.EOObjectStoreCoordinator.saveChangesInEditingContext(EOObjectStoreCoordinator.java:386)
>> at com.webobjects.eocontrol.EOEditingContext.saveChanges(EOEditingContext.java:3192)
>> at er.extensions.eof.ERXEC._saveChanges(ERXEC.java:1085)
>> at er.extensions.eof.ERXEC.saveChanges(ERXEC.java:1007)
>> at com.xyrality.bk.actions.TestAction.testEoFaultingAction(TestAction.java:59)
>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>> at java.lang.reflect.Method.invoke(Method.java:597)
>> at com.webobjects.appserver.WODirectAction.performActionNamed(WODirectAction.java:144)
>> at er.extensions.appserver.ERXDirectAction.performActionNamed(ERXDirectAction.java:401)
>> at com.xyrality.bk.actions.BKDirectAction.performActionNamed(BKDirectAction.java:86)
>> at com.webobjects.appserver._private.WOActionRequestHandler._handleRequest(WOActionRequestHandler.java:259)
>> at com.webobjects.appserver._private.WOActionRequestHandler.handleRequest(WOActionRequestHandler.java:158)
>> at er.extensions.appserver.ERXDirectActionRequestHandler.handleRequest(ERXDirectActionRequestHandler.java:124)
>> at com.webobjects.appserver.WOApplication.dispatchRequest(WOApplication.java:1687)
>> at er.extensions.appserver.ERXApplication.dispatchRequestImmediately(ERXApplication.java:1978)
>> at er.extensions.appserver.ERXApplication.dispatchRequest(ERXApplication.java:1943)
>> at com.xyrality.bk.Application.dispatchRequest(Application.java:63)
>> at com.webobjects.appserver._private.WOWorkerThread.runOnce(WOWorkerThread.java:144)
>> at com.webobjects.appserver._private.WOWorkerThread.run(WOWorkerThread.java:226)
>> at java.lang.Thread.run(Thread.java:680)
>>
>> [2011-5-31 16:3:57 CAT] <WorkerThread14> java.lang.RuntimeException
>> at com.xyrality.bk.actions.BKDirectAction.editingContextShouldInvalidateObject(BKDirectAction.java:62)
>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>> at java.lang.reflect.Method.invoke(Method.java:597)
>> at com.webobjects.foundation.NSSelector._safeInvokeMethod(NSSelector.java:122)
>> at com.webobjects.foundation._NSDelegate._perform(_NSDelegate.java:223)
>> at com.webobjects.foundation._NSDelegate.booleanPerform(_NSDelegate.java:187)
>> at com.webobjects.eocontrol.EOEditingContext._refaultObjectWithGlobalID(EOEditingContext.java:3255)
>> at com.webobjects.eocontrol.EOEditingContext._newChangesFromInvalidatingObjectsWithGlobalIDs(EOEditingContext.java:3485)
>> at com.webobjects.eocontrol.EOEditingContext._processObjectStoreChanges(EOEditingContext.java:3522)
>> at er.extensions.eof.ERXEC._processObjectStoreChanges(ERXEC.java:1440)
>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>> at java.lang.reflect.Method.invoke(Method.java:597)
>> at com.webobjects.foundation.NSSelector.invoke(NSSelector.java:358)
>> at com.webobjects.foundation.NSSelector._safeInvokeSelector(NSSelector.java:110)
>> at com.webobjects.eocontrol.EOEditingContext._sendOrEnqueueNotification(EOEditingContext.java:4715)
>> at com.webobjects.eocontrol.EOEditingContext._objectsChangedInStore(EOEditingContext.java:3562)
>> at er.extensions.eof.ERXEC._objectsChangedInStore(ERXEC.java:1380)
>> at er.extensions.eof.ERXEC.processQueuedNotifications(ERXEC.java:1454)
>> at er.extensions.eof.ERXEC.saveChanges(ERXEC.java:1031)
>> at com.xyrality.bk.actions.TestAction.testEoFaultingAction(TestAction.java:59)
>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>> at java.lang.reflect.Method.invoke(Method.java:597)
>> at com.webobjects.appserver.WODirectAction.performActionNamed(WODirectAction.java:144)
>> at er.extensions.appserver.ERXDirectAction.performActionNamed(ERXDirectAction.java:401)
>> at com.xyrality.bk.actions.BKDirectAction.performActionNamed(BKDirectAction.java:86)
>> at com.webobjects.appserver._private.WOActionRequestHandler._handleRequest(WOActionRequestHandler.java:259)
>> at com.webobjects.appserver._private.WOActionRequestHandler.handleRequest(WOActionRequestHandler.java:158)
>> at er.extensions.appserver.ERXDirectActionRequestHandler.handleRequest(ERXDirectActionRequestHandler.java:124)
>> at com.webobjects.appserver.WOApplication.dispatchRequest(WOApplication.java:1687)
>> at er.extensions.appserver.ERXApplication.dispatchRequestImmediately(ERXApplication.java:1978)
>> at er.extensions.appserver.ERXApplication.dispatchRequest(ERXApplication.java:1943)
>> at com.xyrality.bk.Application.dispatchRequest(Application.java:63)
>> at com.webobjects.appserver._private.WOWorkerThread.runOnce(WOWorkerThread.java:144)
>> at com.webobjects.appserver._private.WOWorkerThread.run(WOWorkerThread.java:226)
>> at java.lang.Thread.run(Thread.java:680)
>
>
>
>
> Am 27.05.2011 um 18:08 schrieb Chuck Hill:
>
>> Hi Paul,
>>
>> John said:
>>>>>> I would presume that this is happening in response to the EOObjectsChangedInStoreNotification. You can implement an EOEditingContext delegate and override editingContextShouldInvalidateObject to see where it is happening and prevent it if you want.
>>
>> I assume that this is what you have done to fix the problem. In editingContextShouldInvalidateObject, add this line to log out a stack trace of where the invalidate originated in code:
>>
>> NSLog.out.appendln(new RuntimeException());
>>
>>
>> Chuck
>>
>>
>> On May 27, 2011, at 1:19 AM, Paul Dunkler wrote:
>>
>>> Am 26.05.2011 um 20:28 schrieb Chuck Hill:
>>>
>>>>
>>>> On May 26, 2011, at 9:03 AM, Paul Dunkler wrote:
>>>>
>>>>> nice - thank you for the hint. I tried it and i was successfull. But in addition to this fix, it would be nice to know why the default behaviour of the editingContext is to refault all objects on saveChanges()... maybe anyone can explain?
>>>>
>>>> That is not the default behavior. I don't see this happening in my apps. Can you post the stack traces?
>>>
>>>
>>> Hi Chuck,
>>>
>>> what dou you exactly want to have from me? I´m not really sure what you mean with stack traces in this case - no error is going to be produced - so there is not any stacktrace output produced. Or is there some magic parameter / log setting which let me see stack traces for direct action calls?
>>>
>>>
>>>>
>>>>
>>>> Chuck
>>>>
>>>>
>>>>> Am 26.05.2011 um 16:40 schrieb John Huss:
>>>>>
>>>>>> I would presume that this is happening in response to the EOObjectsChangedInStoreNotification. You can implement an EOEditingContext delegate and override editingContextShouldInvalidateObject to see where it is happening and prevent it if you want.
>>>>>>
>>>>>> John
>>>>>>
>>>>>> On Thu, May 26, 2011 at 9:16 AM, Paul Dunkler <email@hidden> wrote:
>>>>>> Hey guys,
>>>>>>
>>>>>> we are currently developing a large webobjects (plus wonder of course) driven backend application. Every time a request comes in, we fetch a big set of data for the customer related to this request.
>>>>>> In the following actions we add/edit/delete some of the data originally fetched from the database. At the end of each request, we do saveChanges() on the editing context which holds our Customer Object with all it's relationships.
>>>>>> After that all, we put a list of all the current data in the request's response for delivering fresh informations to the client.
>>>>>>
>>>>>> The problem we get here is the following:
>>>>>> At the point of the request, where the fresh data is packed for presenting it in the response, every access onto some of the attributes or relationships from our Customer object lead us to a roundtrip in the database. That is very expensive and it means, that our application is performing the same Queries at the start of the request and at the end of the request again...
>>>>>>
>>>>>> If my text was too complicated or incomprehensible, here is a short-termed explanation of what i wanted to describe:
>>>>>> 1. Incoming request
>>>>>> 2. Batch fetching of the complete Customer-Object with all it's relationships we want to access in the following code
>>>>>> 3. saveChanges() (somewhere in the process)
>>>>>> 4. Generating the response
>>>>>> -> Every call to a getter method (attributes) of the eo cause a roundtrip to the database.
>>>>>>
>>>>>> We currently don't really know why this is happening. If we fetch an object, change it's data, than save it - and then call a method on the same object, why it has to do a new roundtrip to the database?
>>>>>>
>>>>>>
>>>>>> I just tried it out in a short DirectAction to check the behaviour:
>>>>>>> // WITH EDITING AN OBJECT
>>>>>>> Thing aThing = Thing.fetchThing(this.editingContext, Thing.NAME.eq("foo")); // <-- db-roundtrip
>>>>>>> System.out.println("first try: " + aThing.fooRelationshipArray());
>>>>>>> aThing.setBar(11);
>>>>>>> this.editingContext.saveChanges();
>>>>>>> System.out.println("second try: " + aThing.fooRelationshipArray()); // <-- db-roundtrip
>>>>>>> this.responseDictionary.setObjectForKey(aThing.bar(), "points");
>>>>>>>
>>>>>>> // WITHOUT EDITING AN OBJECT - WILL NOT DO A NEW ROUNDTRIP TO THE DATABASE
>>>>>>> Thing aSecondThing = Thing.fetchThing(this.editingContext(), Thing.PRIMARY_KEY_IDENTIFIER, 47); // <-- db-roundtrip
>>>>>>> System.out.println(aSecondThing.fooRelationshipArray());
>>>>>>> System.out.println("third try: " + aSecondThing); // <-- no db-roundtrip
>>>>>>> System.out.println("fourth try: " + aSecondThing); // <-- no db-roundtrip
>>>>>>
>>>>>>
>>>>>>
>>>>>> It would be very nice if you got any ideas to avoid this behaviour. Or some nice thinkings about doing it right :)
>>>>>> I think this is a problem which can be solved - we browsed Webobjects Books / Tutorials / Mailing List History, but had no luck at all....
>>>>>>
>>>>>>
>>>>>> --
>>>>>> Mit freundlichen Grüßen / Kind regards
>>>>>>
>>>>>> Paul Dunkler
>>>>>> _______________________________________________
>>>>>> Do not post admin requests to the list. They will be ignored.
>>>>>> Webobjects-dev mailing list (email@hidden)
>>>>>> Help/Unsubscribe/Update your Subscription:
>>>>>>
>>>>>> This email sent to email@hidden
>>>>>>
>>>>>
>>>>> Mit freundlichen Grüßen
>>>>>
>>>>> Paul Dunkler
>>>>>
>>>>>
>>>>> <xyrality_logo_medium.png>
>>>>>
>>>>> -----------------------------------------------------
>>>>> XYRALITY GmbH • Lerchenstraße 28a • 22767 Hamburg
>>>>> Paul Dunkler • Softwareentwickler
>>>>> Mail: email@hidden
>>>>> Tel: +49 (0) 40 23 51 78 97
>>>>> Mobil: +49 (0) 151 11624143
>>>>> Fax: +49 (0) 40 23 51 78 98
>>>>> Web: http://www.xyrality.com/
>>>>> Registergericht: Hamburg HRB 115332
>>>>> Geschäftsführer: Sven Ossenbrüggen & Alexander Spohr
>>>>> -----------------------------------------------------
>>>>>
>>>>> _______________________________________________
>>>>> Do not post admin requests to the list. They will be ignored.
>>>>> Webobjects-dev mailing list (email@hidden)
>>>>> Help/Unsubscribe/Update your Subscription:
>>>>>
>>>>> This email sent to email@hidden
>>>>
>>>> --
>>>> Chuck Hill Senior Consultant / VP Development
>>>>
>>>> Come to WOWODC this July for unparalleled WO learning opportunities and real peer to peer problem solving! Network, socialize, and enjoy a great cosmopolitan city. See you there! http://www.wocommunity.org/wowodc11/
>>>>
>>>
>>> Mit freundlichen Grüßen
>>>
>>> Paul Dunkler
>>>
>>>
>>> <xyrality_logo_medium.png>
>>>
>>> -----------------------------------------------------
>>> XYRALITY GmbH • Lerchenstraße 28a • 22767 Hamburg
>>> Paul Dunkler • Softwareentwickler
>>> Mail: email@hidden
>>> Tel: +49 (0) 40 23 51 78 97
>>> Mobil: +49 (0) 151 11624143
>>> Fax: +49 (0) 40 23 51 78 98
>>> Web: http://www.xyrality.com/
>>> Registergericht: Hamburg HRB 115332
>>> Geschäftsführer: Sven Ossenbrüggen & Alexander Spohr
>>> -----------------------------------------------------
>>>
>>
>> --
>> Chuck Hill Senior Consultant / VP Development
>>
>> Come to WOWODC this July for unparalleled WO learning opportunities and real peer to peer problem solving! Network, socialize, and enjoy a great cosmopolitan city. See you there! http://www.wocommunity.org/wowodc11/
>>
>
> Mit freundlichen Grüßen
>
> Paul Dunkler
>
>
> <xyrality_logo_medium.png>
>
> -----------------------------------------------------
> XYRALITY GmbH • Lerchenstraße 28a • 22767 Hamburg
> Paul Dunkler • Softwareentwickler
> Mail: email@hidden
> Tel: +49 (0) 40 23 51 78 97
> Mobil: +49 (0) 151 11624143
> Fax: +49 (0) 40 23 51 78 98
> Web: http://www.xyrality.com/
> Registergericht: Hamburg HRB 115332
> Geschäftsführer: Sven Ossenbrüggen & Alexander Spohr
> -----------------------------------------------------
>
--
Chuck Hill Senior Consultant / VP Development
Come to WOWODC this July for unparalleled WO learning opportunities and real peer to peer problem solving! Network, socialize, and enjoy a great cosmopolitan city. See you there! http://www.wocommunity.org/wowodc11/
Attachment:
smime.p7s
Description: S/MIME cryptographic signature
_______________________________________________
Do not post admin requests to the list. They will be ignored.
Webobjects-dev mailing list (email@hidden)
Help/Unsubscribe/Update your Subscription:
This email sent to email@hidden