Re: Immortal… ehm, frozen instances.
Re: Immortal… ehm, frozen instances.
- Subject: Re: Immortal… ehm, frozen instances.
- From: Altera WO Team <email@hidden>
- Date: Thu, 21 Mar 2013 20:30:54 +0100
Chuck, I might owe you a lot of beers… Going to remove all the terminate() and see what happens!
Thanks,
Matteo
On 21/mar/2013, at 19:19, Chuck Hill <email@hidden> wrote:
>
> On 2013-03-21, at 10:54 AM, Altera WO Team wrote:
>
>>
>> On 21/mar/2013, at 18:46, Chuck Hill <email@hidden> wrote:
>>
>>>
>>> On 2013-03-21, at 6:47 AM, Altera WO Team wrote:
>>>
>>>> I tried anyway and I can confirm. I'm not creating needless sessions.
>>>>
>>>> The app locks up anyway though…
>>>
>>> Did adding the finally blocks in your terminate() method help? Is anything in your code calling session.terminate() directly? A logout link?
>>
>> the finally blocks didn't help.
>>
>> Something is calling session.terminate() directly though, happens when a something bad happens in a direct action.
>>
>> if (existingSession() != null)
>> sess().terminate();
>>
>> It's called only if we can't parse the parameters and should never happen. I'll put some logging in those cases.
>>
>> Why should this be a problem?
>
> I thought it was fixed in 5.4.3, but I could be wrong. In some past version if that same URL can in in rapid succession, you would get deadlocks like you are seeing. The solution at the time was to not call terminate directly and instead set the session timeout to a couple seconds. That allowed both requests to clear before terminating the session.
>
>
> Chuck
>
>
>>>> I lost hope in finding the real reason, for now I would be happy to find a dirty workaround. I need those apps to bounce anyway.
>>>>
>>>> I already tried using ERTimeToKill and it's ineffective.
>>>>
>>>> The lockups are quite rare: I have 10 instances running, bouncing every 6 hours and it happens maybe 2 times a week, but if an instance locks up the instances running become 9 and in the long run I'll have everything locked up.
>>>>
>>>> Any suggestions?
>>>
>>> I don't recall how ERTimeToKill works, but with those stuck threads you are going to need to do
>>>
>>> Runtime.getRuntime().exit(1);
>>>
>>> and if THAT does not kill it, this should:
>>>
>>> Runtime.getRuntime().halt(1);
>>>
>>>
>>> Chuck
>>>>
>>>> On 20/mar/2013, at 01:16, Chuck Hill <email@hidden> wrote:
>>>>
>>>>> I think the problem is not that it is creating needless sessions, but that a checked-out session is not getting checked back in.
>>>>>
>>>>> Chuck
>>>>>
>>>>>
>>>>> On 2013-03-19, at 5:08 PM, Simon wrote:
>>>>>
>>>>>> if we have this kind of issue the first thing we do is log out session creation and check the stack traces. stick something like this in your session constructor then check the output:
>>>>>>
>>>>>> public Session() {
>>>>>> super();
>>>>>> StringWriter stringWriter = new StringWriter();
>>>>>>
>>>>>> PrintWriter printWriter = new PrintWriter(stringWriter);
>>>>>>
>>>>>> (new Throwable()).printStackTrace(printWriter);
>>>>>>
>>>>>> String trace = stringWriter.toString(); <---- log this somewhere that you can get easy access to
>>>>>>
>>>>>> }
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> On 19 March 2013 23:15, Altera WO Team <email@hidden> wrote:
>>>>>> Wow, good hint!
>>>>>>
>>>>>> In theory I'm not touching a Session but sometimes, when something in an EO changes I trigger e-mail sending.
>>>>>> I use ERMailDeliveryHTML using a component instantiated with a brand new wocontext using ERXWOContext.newContext()
>>>>>> the component is not referencing a Session in any part.
>>>>>>
>>>>>> Could it be the cause?
>>>>>>
>>>>>> Matteo
>>>>>>
>>>>>>
>>>>>> On 19/mar/2013, at 21:24, Simon <email@hidden> wrote:
>>>>>>
>>>>>>> ...or you are touching the session object outside the RR loop. the classic gotcha is rendering a component in a thread that touches the session in some way e.g. delivering an email in a thread that uses a component to render it's content... been caught by that one soooo many times!
>>>>>>>
>>>>>>>
>>>>>>> On 19 March 2013 17:51, Chuck Hill <email@hidden> wrote:
>>>>>>> Hi Matteo,
>>>>>>>
>>>>>>> You have one or more Zombie (aka Immortal) Sessions, as shown by stack traces like this:
>>>>>>> "WorkerThread11" prio=10 tid=0x0000000041848800 nid=0x1010 in Object.wait() [0x00007f16f7cfa000]
>>>>>>> java.lang.Thread.State: WAITING (on object monitor)
>>>>>>> at java.lang.Object.wait(Native Method)
>>>>>>> - waiting on <0x00000000d120b328> (a com.webobjects.appserver.WOSessionStore$TimeoutEntry)
>>>>>>> at java.lang.Object.wait(Object.java:485)
>>>>>>> at com.webobjects.appserver.WOSessionStore.checkOutSessionWithID(WOSessionStore.java:191)
>>>>>>> - locked <0x00000000d120b328> (a com.webobjects.appserver.WOSessionStore$TimeoutEntry)
>>>>>>> at com.webobjects.appserver.WOApplication.restoreSessionWithID(WOApplication.java:1913)
>>>>>>> at er.extensions.appserver.ERXApplication.restoreSessionWithID(ERXApplication.java:2403)
>>>>>>> at er.extensions.appserver.ERXWOContext.existingSession(ERXWOContext.java:57)
>>>>>>> at er.extensions.appserver.ERXWOContext.hasSession(ERXWOContext.java:69)
>>>>>>> at com.webobjects.appserver.WOAction.existingSession(WOAction.java:190)
>>>>>>> at com.tla.calendar.DirectAction.goToAction(DirectAction.java:454)
>>>>>>>
>>>>>>>
>>>>>>> This likely has one of two causes:
>>>>>>> 1. The application is getting OutOfMemory errors, which can leave the session store in an insane state
>>>>>>> 2. The app is throwing an exception from sleep() in Session. If you overrride sleep() it should use a try...finally block
>>>>>>>
>>>>>>> public void sleep() {
>>>>>>> try {
>>>>>>> // Your code here!
>>>>>>> } finally {
>>>>>>> super.sleep();
>>>>>>> }
>>>>>>> }
>>>>>>>
>>>>>>>
>>>>>>> Chuck
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On 2013-03-19, at 9:38 AM, Altera WO Team wrote:
>>>>>>>
>>>>>>>> Hi all,
>>>>>>>>
>>>>>>>> I'm having a strange issue on a WO installation on EC2 (oracle jvm).
>>>>>>>> Same strange application which had immortal sessions…
>>>>>>>>
>>>>>>>> Sometimes (quite rarely) a bounced application (put in refuse new sessions) never quits and it's not accessible from JavaMonitor.
>>>>>>>> If I look at the logs i see:
>>>>>>>>
>>>>>>>> Mar 19 12:38:52 B2C[2002] (ERXNSLogLog4jBridge.java:44) WARN NSLog - <com.tla.calendar.Application>: refusing new clients and below min active session threshold
>>>>>>>> Mar 19 12:38:52 B2C[2002] (ERXNSLogLog4jBridge.java:44) WARN NSLog - <com.tla.calendar.Application>: about to terminate...
>>>>>>>>
>>>>>>>> The only thing left is to kill the instance… Which is not nice.
>>>>>>>>
>>>>>>>> I'm not overriding the terminate() method in Application.
>>>>>>>>
>>>>>>>> I am attaching a stack trace if it helps.
>>>>>>>> <B2Cjstack.txt>
>>>>>>>>
>>>>>>>> Thanks,
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> Matteo Centro
>>>>>>>> _______________________________________________
>>>>>>>> Do not post admin requests to the list. They will be ignored.
>>>>>>>> Webobjects-dev mailing list (email@hidden)
>>>>>>>> Help/Unsubscribe/Update your Subscription:
>>>>>>>>
>>>>>>>> This email sent to email@hidden
>>>>>>>
>>>>>>> --
>>>>>>> Chuck Hill
>>>>>>> Executive Managing Partner, VP Development and Technical Services
>>>>>>>
>>>>>>> Practical WebObjects - for developers who want to increase their overall knowledge of WebObjects or who are trying to solve specific problems.
>>>>>>> http://www.global-village.net/gvc/practical_webobjects
>>>>>>>
>>>>>>> Global Village Consulting ranks 13th in 2012 in BIV's Top 100 Fastest Growing Companies in B.C!
>>>>>>> Global Village Consulting ranks 76th in 24th annual PROFIT 200 ranking of Canada’s Fastest-Growing Companies by PROFIT Magazine!
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> _______________________________________________
>>>>>>> Do not post admin requests to the list. They will be ignored.
>>>>>>> Webobjects-dev mailing list (email@hidden)
>>>>>>> Help/Unsubscribe/Update your Subscription:
>>>>>>>
>>>>>>> This email sent to email@hidden
>>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>> --
>>>>> Chuck Hill
>>>>> Executive Managing Partner, VP Development and Technical Services
>>>>>
>>>>> Practical WebObjects - for developers who want to increase their overall knowledge of WebObjects or who are trying to solve specific problems.
>>>>> http://www.global-village.net/gvc/practical_webobjects
>>>>>
>>>>> Global Village Consulting ranks 13th in 2012 in BIV's Top 100 Fastest Growing Companies in B.C!
>>>>> Global Village Consulting ranks 76th in 24th annual PROFIT 200 ranking of Canada’s Fastest-Growing Companies by PROFIT Magazine!
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>
>>>
>>> --
>>> Chuck Hill
>>> Executive Managing Partner, VP Development and Technical Services
>>>
>>> Practical WebObjects - for developers who want to increase their overall knowledge of WebObjects or who are trying to solve specific problems.
>>> http://www.global-village.net/gvc/practical_webobjects
>>>
>>> Global Village Consulting ranks 13th in 2012 in BIV's Top 100 Fastest Growing Companies in B.C!
>>> Global Village Consulting ranks 76th in 24th annual PROFIT 200 ranking of Canada’s Fastest-Growing Companies by PROFIT Magazine!
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>
>
> --
> Chuck Hill
> Executive Managing Partner, VP Development and Technical Services
>
> Practical WebObjects - for developers who want to increase their overall knowledge of WebObjects or who are trying to solve specific problems.
> http://www.global-village.net/gvc/practical_webobjects
>
> Global Village Consulting ranks 13th in 2012 in BIV's Top 100 Fastest Growing Companies in B.C!
> Global Village Consulting ranks 76th in 24th annual PROFIT 200 ranking of Canada’s Fastest-Growing Companies by PROFIT Magazine!
>
>
>
>
>
>
>
>
>
>
>
_______________________________________________
Do not post admin requests to the list. They will be ignored.
Webobjects-dev mailing list (email@hidden)
Help/Unsubscribe/Update your Subscription:
This email sent to email@hidden
| References: | |
| | >Immortal… ehm, frozen instances. (From: Altera WO Team <email@hidden>) |
| | >Re: Immortal… ehm, frozen instances. (From: Chuck Hill <email@hidden>) |
| | >Re: Immortal… ehm, frozen instances. (From: Simon <email@hidden>) |
| | >Re: Immortal… ehm, frozen instances. (From: Altera WO Team <email@hidden>) |
| | >Re: Immortal… ehm, frozen instances. (From: Simon <email@hidden>) |
| | >Re: Immortal… ehm, frozen instances. (From: Chuck Hill <email@hidden>) |
| | >Re: Immortal… ehm, frozen instances. (From: Altera WO Team <email@hidden>) |
| | >Re: Immortal… ehm, frozen instances. (From: Chuck Hill <email@hidden>) |
| | >Re: Immortal… ehm, frozen instances. (From: Altera WO Team <email@hidden>) |
| | >Re: Immortal… ehm, frozen instances. (From: Chuck Hill <email@hidden>) |