Apple

On Apr 22, 2011, at 4:27 AM, Rafal Szczepanski wrote:

Hi Chuck,

Thank you for your email. As I already told Kieran I couldn't answer before - sorry for that. Please see my comments inline.

On Thu, Mar 24, 2011 at 8:22 PM, Chuck Hill <email@hidden> wrote:

Hi Rafal,

On Mar 24, 2011, at 7:50 AM, Rafal Szczepanski wrote:

I would like to ask you about the optimal number of WorkerThreads (application settings: WOWorkerThreadCountMin & WOWorkerThreadCount).

That depends. :-)
If someone says "that depends" I always start to fear :-)

Deployment tuning and optimization depends on so many variables that it is more an art than a science.

I would say that a min of 4 is reasonable and that max should not be high enough to allow the instance to get backlogged with requests. How many instances are you running?
Well. We are running multiple applications on one server. The application that is causing the problems consists of 4 instances.

That is reasonable.

Are they spread over multiple servers?
Not at this moment.

If you have multiple app servers, distributing the instances over multiple servers is usually the best setup.

`
Is concurrent request handling enabled?
Yes. We needed to enable it because we are calling third party web services and slow response was causing the whole application to freeze.

Having it enabled is good. Calling slow third party webservices in the R-R loop is probably not the best plan. If you are doing that, you may need to increases the listen queue size and max worker threads. I would view that as a work-around, not a solution.

How much heap space are you allocating to each instance?

Currently the instances are running with 1.5GB each.

That should be enough. :-)

Are the app servers swapping?
Nope. The sum of the memory assigned is less than the server's memory.

One of the my applications is serving sometimes more than 2000 pages per hour.
The current setting is 16 WorkerThreads but I noticed that from time to time this number is not enough and the server is creating more WorkerThreads which is freezing for a while the application.

I think the more likely interpretation of this is that your app froze (was slow to process requests), so it created more worker threads to record the unprocessed requests waiting in line. WorkerThreads are created in response to slowness, creating them does not cause slowness.

Well you're right - I have taken a look on one of the instances using VisualVM. Usually there was one WorkerThread that was causing other WorkerThreads to wait, hence when new requests were coming WebObjects was creating additional ones.

Slow database access (lots of SQL statements, or slow queries) will cause this problem.

Do you know if the number of WorkerThreads affects application’s performance? If not then perhaps I should start with 256 right from the beginning….

It does not affect performance directly, controls how many requests can be queued up in an application before wotaskd considers it too full to send more requests to. Making more workerthreads just means that the app will queue up more pending requests. This is usually a Bad Thing (tm) as it will take so long to process them and return a response that the user will click stop or click the link again, thus using another worker thread and making things worse. Find out what is making the app slow and fix that.

It seems that the problem was created by one page that was running a slow query. As I have written before other WorkerThreads (which I assume wanted to query the db) were waiting until the first query is finished. I have modified the query so it runs faster and the situation improved significantly.

You could also create a different EOF stack which would give you a different database connection to run this slower query in. That will not make it faster, but will eliminate the impact on the other requests.

I’ve got also another question do you know some real life examples of applications built in WebObjects that are serving huge number of requests/heavy load?

The iTunes music store is the classic example, as well as the Apple Store. There are lots of others around that process a heavy load. You just need to optimize for your specific case.

That's obviously a very promising example but I wonder how much of original WebObjects is used there... :-)

That may be a good point. But they also have rather unique optimization requirements. What do you consider a huge number of requests/heavy load? GVC.SiteMaker at the University of Michigan gets over 1 million unique visits a month and it is only partially optimized. There is a guitar tabs site that was recently mentioned here that gets more than that.

Chuck

--
Chuck Hill Senior Consultant / VP Development

Come to WOWODC this July for unparalleled WO learning opportunities and real peer to peer problem solving! Network, socialize, and enjoy a great cosmopolitan city. See you there! http://www.wocommunity.org/wowodc11/

References:
	>Re: Optimal number of WorkerThreads (From: Rafal Szczepanski <email@hidden>)