Re: Deployment with Embedded Frameworks and Duplicate libraries
Re: Deployment with Embedded Frameworks and Duplicate libraries
- Subject: Re: Deployment with Embedded Frameworks and Duplicate libraries
- From: Chuck Hill <email@hidden>
- Date: Wed, 5 Aug 2009 12:10:38 -0700
On Aug 5, 2009, at 7:08 AM, Mr. Frank Cobia wrote:
Thanks for all the help. I am making progress.
Also, I have to turn direct connect off to be able to do the
refuse new sessions, so is there another way to figure out which
app is hung so that it can be killed?
Look at the stack trace, like below. Also look for growing
numbers of worker threads. If you have more than 16, you
probably have a problem.
I have checked and this seems to be my last problem. Every once in a
while I get an instance that spawns 256 worker threads.
First off, I'd avoid that. It would be a remarkable application that
could handle more than 16 or so concurrent requests. Take a look here
http://developer.apple.com/documentation/WebObjects/WOAppProperties/Articles/ApplicationProperties.html#/
/apple_ref/doc/uid/TP40005337-SW1
Specifically WOListenQueueSize and WOWorkerThreadCountMax (I don't see
the latter one documented).
The best way to set these is using JavaMonitor to configure the
application. I'd try a listen queue size of 5 and a max worker thread
count of 10 to start. Having lower numbers will make the app fail
faster when it deadlocks and this can result in a better user
experience.
The statistics show that this does not even require a lot of
transactions. I did a stack trace and there were a few funny looking
things,
Maybe show us all of the ones that look like they are doing something.
but the one that stood out the most was the following appeared a lot:
Thread t@80131: (state = BLOCKED)
- java.lang.Class.forName0(java.lang.String, boolean,
java.lang.ClassLoader) @bci=0 (Interpreted frame)
My best guess is that you have a class with a static initializer that
is either (a) deadlocking or (b) taking a loooong time - waiting for a
network timeout perhaps?
- java.lang.Class.forName(java.lang.String) @bci=5, line=164
(Interpreted frame)
-
com
.webobjects
.foundation._NSUtilities._searchForClassInPackages(java.lang.String,
com.webobjects.foundation.NSArray, boolean, boolean) @bci=92,
line=417 (Interpreted frame)
-
com
.webobjects
.foundation._NSUtilities._classWithPartialName(java.lang.String,
boolean) @bci=154, line=375 (Interpreted frame)
-
com
.webobjects.foundation._NSUtilities.classWithName(java.lang.String)
@bci=26, line=334 (Interpreted frame)
-
com
.webobjects
.appserver
._private.WOBundle.lookForClassInAllBundles(java.lang.String)
@bci=5, line=92 (Interpreted frame)
-
com
.webobjects
.appserver
._private
.WOActionRequestHandler._actionClassForName(java.lang.String)
@bci=1, line=70 (Interpreted frame)
-
com
.webobjects
.appserver
._private
.WOActionRequestHandler
.getRequestActionClassAndNameForPath
(com.webobjects.foundation.NSArray) @bci=114, line=389 (Interpreted
frame)
That looks like it is trying to load a WODirectAction subclass. Start
searching your app there. Logging out the request can help, except
that matching "Thread t@80131" to a worker thread is going to be
hard. kill -QUIT produces an easier to read trace.
-
com
.webobjects
.appserver
._private
.WOActionRequestHandler
._handleRequest(com.webobjects.appserver.WORequest) @bci=88,
line=226 (Interpreted frame)
-
com
.webobjects
.appserver
._private
.WOActionRequestHandler
.handleRequest(com.webobjects.appserver.WORequest) @bci=79, line=161
(Interpreted frame)
-
er
.extensions
.appserver
.ERXDirectActionRequestHandler
.handleRequest(com.webobjects.appserver.WORequest) @bci=222,
line=123 (Interpreted frame)
-
com
.webobjects
.appserver
.WOApplication.dispatchRequest(com.webobjects.appserver.WORequest)
@bci=32, line=1687 (Interpreted frame)
-
er
.extensions
.appserver
.ERXApplication
.dispatchRequestImmediately(com.webobjects.appserver.WORequest)
@bci=74, line=1791 (Interpreted frame)
-
er
.extensions
.appserver
.ERXApplication.dispatchRequest(com.webobjects.appserver.WORequest)
@bci=13, line=1756 (Interpreted frame)
- com.webobjects.appserver._private.WOWorkerThread.runOnce()
@bci=473, line=144 (Interpreted frame)
- com.webobjects.appserver._private.WOWorkerThread.run() @bci=129,
line=226 (Interpreted frame)
- java.lang.Thread.run() @bci=11, line=613 (Interpreted frame)
So it appears that a lot of threads are getting locked up in
Class.forName(). I want to print out some debug information to try
to figure out which class is being instantiated, but I did not want
to fill up the logs too much. I have already run out of disk space
because of too much logging. So my question is, is there a call
somewhere where I can find the number of worker threads or when the
number of worker threads is increased to 32? That way I can turn on
the logging when it really matters.
This information is not exposed via API or notifications.
Chuck
--
Chuck Hill Senior Consultant / VP Development
Practical WebObjects - for developers who want to increase their
overall knowledge of WebObjects or who are trying to solve specific
problems.
http://www.global-village.net/products/practical_webobjects
_______________________________________________
Do not post admin requests to the list. They will be ignored.
Webobjects-dev mailing list (email@hidden)
Help/Unsubscribe/Update your Subscription:
This email sent to email@hidden