Re: app seems to trigger runaway apache threads (was Troubleshooting help)
Re: app seems to trigger runaway apache threads (was Troubleshooting help)
- Subject: Re: app seems to trigger runaway apache threads (was Troubleshooting help)
- From: Chuck Hill <email@hidden>
- Date: Wed, 24 Aug 2011 11:46:58 -0700
That does not sound like what I have observed.
On 2011-08-24, at 11:42 AM, Tim Worman wrote:
> On Aug 22, 2011, at 12:53 PM, Chuck Hill wrote:
>
>> On 2011-08-22, at 12:48 PM, Tim Worman wrote:
>>> On Aug 22, 2011, at 11:51 AM, Tim Worman wrote:
>>> On Aug 22, 2011, at 11:38 AM, Chuck Hill wrote:
>>>>> On 2011-08-21, at 12:43 PM, Tim Worman wrote:
>>>>>> On Aug 21, 2011, at 11:52 AM, Chuck Hill wrote:
>>>>>>> On 2011-08-20, at 4:02 PM, Tim Worman wrote:
>>>>>>>
>>>>>>>> Back in January I started this discussion on this same topic:
>>>>>>>>
>>>>>>>> http://lists.apple.com/archives/webobjects-dev/2011/Jan/msg00224.html
>>>>>>>>
>>>>>>>>>> I have an app that, during the course of normal usage, is starting httpd processes on the server that instantly hit 100% CPU usage of one core. This can happen multiple times during times when the app is under heavier load. After some time I can have many httpd processes where TOP reports each using 100% of a core. When I try to log into the app and poke around to try and reproduce the issue, I am unable.
>>>>>>>>
>>>>>>>> This is an update to my original post hoping to see if there are anymore thoughts on origin. More recently, I have been able to reproduce the issue in my own usage of the app - something I wasn't able to do before. It seems to be easier to generate the issue now that there are more ajax requests. The methods executed by these requests are not intensive or long responses and should return a result in seconds. Some symptoms:
>>>>>>>>
>>>>>>>> - When the actions are executed, busy indicators properly spin while the browser awaits a response from the server. When the issue occurs, the response never comes.
>>>>>>>> - while continuing to await a response there is concurrently an httpd process that pegs he processor at 100%
>>>>>>>> - if I kill the process on the server, the browser immediately updates properly as if the request had run properly
>>>>>>>>
>>>>>>>> It's almost as if apache is somehow receiving an ill-formed request and chokes on it. The problem is, there are no errors in the console or anything strange in any apache logs. Has anyone ever seen behavior like this or have any ideas as to how I could analyze it further?
>>>>>>>
>>>>>>> I've seen something like this. It appeared that the woadaptor (i.e. mod_webobjects) did not believe that it had received all of the response from the application. The app had nothing more to send and so the woadaptor just hung there waiting for data that would never come. I did not track down why this happened, but it did seem to be load related. My suspicion was that there is a concurrency bug in the woadaptor.
>>>>>>
>>>>>> I'm really at a loss about what to do about it. It's only gotten worse as I've included more ajax actions in my app - and, of course, I don't experience this behavior in development. I just deployed a major update to my app - pretty much unaware that a small problem was going to become a big problem with the new version.
>>>>>>
>>>>>> In one example, a have a calendar where clicking on a day simply calls an AjaxUpdate marking that date as selected to the calendar. The result also has to update the entire page though because other things on the page need to change in those circumstances. This alone can cause the issue - but not always. And it happens even when I'm the only logged in user - so the load isn't high.
>>>>>>
>>>>>> As one solution, I've considered rolling a custom apache instead of using Apple's. But since the server also runs shibboleth, the setup isn't exactly simple. But I'm really not sure how to ascertain if the problem is the woadaptor or how I can settle it.
>>>>>
>>>>>
>>>>> You could try the CGI adaptor and see if that makes any difference. If it IS a bug in mod_webobjects... that could be hard to find and fix.
>>>>
>>>> Yeah, I'd say so. I don't even know C. :-) I'm gonna pursue some other things first - like a possible hardware issue, or other software. I'm going to try a deployment on another server and see if that has any effect - also without shibboleth. Is there any chance that an issue with the id's of page elements could cause an issue like this - say if they're dynamic and the response doesn't find a matching id?
>>>
>>> One other thing - if this was a bug in mod_webobjects, wouldn't a whole lot of projects being experiencing this same issue? Obviously, there are wo projects out there taking on a lot more load than mine. I have to think there is something about this specific deployment OR something about my specific requests that's causing my little hell here. Obviously, in wo, we don't construct our own requests in a fine-grained way so that is why I'm wondering about some of the elements on my pages.
>>
>>
>> If it is a bug in the adaptor it has to be something very specific, maybe a size or the value of some bytes at the start or end or the phase of the moon or... What platform do you deploy on?
>
> What's the possibility that this could be related?
>
> http://www.h-online.com/open/news/item/Tool-causes-Apache-web-server-to-freeze-1330105.html
>
>> Chuck
--
Chuck Hill Senior Consultant / VP Development
Practical WebObjects - for developers who want to increase their overall knowledge of WebObjects or who are trying to solve specific problems.
http://www.global-village.net/products/practical_webobjects
_______________________________________________
Do not post admin requests to the list. They will be ignored.
Webobjects-dev mailing list (email@hidden)
Help/Unsubscribe/Update your Subscription:
This email sent to email@hidden