Re: Deployed app performance degrades over time
Re: Deployed app performance degrades over time
- Subject: Re: Deployed app performance degrades over time
- From: Chuck Hill <email@hidden>
- Date: Sat, 27 Sep 2008 19:24:10 -0700
On Sep 27, 2008, at 3:46 PM, William Hatch wrote:
WO 5.4.3, mysql, tomcat 6, Linux
We have a utility app that basically crunches numbers and aggregates
various data points. It was originally conceived as a stand alone
java app, but we moved into a wo based servlet because we already
had a pretty solid ORM framework used by the front end apps, and for
easier monitoring and control, etc. When we first deploy it, if it
hasn't been running in a while, the performance is great; it can
process thousands of rows a second, and we see a very high qps count
on the db. But after a while, it will slow down to the point where
it's taking multiple seconds to process a single row, and the db qps
count is extremely low, as if it's not getting hit at all. More
bizarre, if we stop and restart tomcat and then redeploy the app, it
will degrade even faster than originally. Tomcat stats look good as
far as memory, and over time, we've done optimizations to the point
where we're barely using EOF; most calls are now rawRows.... and
then we'll instance the EO, do something with it, then discard it.
I've read in the wiki's that when running an EOF app outside of a
typical R-R based web app, there's some things you need to do
occasionally to help EOF clean up after itself, such as this call:
NSDelayedCallbackCenter.defaultCenter().eventEnded();
We're also combing over all the places where we instantiate an
editing context and making sure we don't have these piling up in
memory. We do have a few caching mechanisms in place to make
subsequent look ups in the process less db intensive, but those get
cleared out after after batch of processed rows. I'm curious if
we're looking in the right places and am looking for feedback as to
how best monitor, diagnose and manage performance for an app in this
kind of context where you don't have any requests or responses being
generated. TIA
Have you run something like Shark on it? The symptoms sounds like
memory starvation where the JVM starts to spend more time running GC
than executing your code. Though that completely fails to explain why
a restart degrades even faster.
Chuck
--
Chuck Hill Senior Consultant / VP Development
Practical WebObjects - for developers who want to increase their
overall knowledge of WebObjects or who are trying to solve specific
problems.
http://www.global-village.net/products/practical_webobjects
_______________________________________________
Do not post admin requests to the list. They will be ignored.
Webobjects-dev mailing list (email@hidden)
Help/Unsubscribe/Update your Subscription:
This email sent to email@hidden