Re: Weird JavaMonitor wotaskd behavior on Snow Leopard
Re: Weird JavaMonitor wotaskd behavior on Snow Leopard
- Subject: Re: Weird JavaMonitor wotaskd behavior on Snow Leopard
- From: Pascal Robert <email@hidden>
- Date: Wed, 28 Jul 2010 21:14:28 -0400
Do you have multiple IPs on this server? Check also in /var/log/system.log for DNS warnings.
> I'm not using the default Javamonitor port. I'm using 57913.
>
>
> db1p:~ admin$ ps auxwww | grep 57913
> _appserver 76 0.0 0.5 3024004 113680 ?? Ss 4:19PM 0:07.48 /usr/bin/java -XX:NewSize=2m -Xmx64m -Xms32m -DWORootDirectory=/System -DWOLocalRootDirectory= -DWOUserDirectory=/ -DWOEnvClassPath= -DWOApplicationClass=Application -DWOPlatform=MacOS -Dcom.webobjects.pid=76 -Dcom.sun.management.jmxremote=false -classpath WOBootstrap.jar com.webobjects._bootstrap.WOBootstrap -WOPort 57913
> admin 2943 0.0 0.0 2435036 540 s000 R+ 5:55PM 0:00.00 grep 57913
>
>
> db1p:~ admin$ ps auxwww | grep 2001
> admin 2953 0.0 0.0 2435036 540 s000 R+ 5:55PM 0:00.00 grep 2001
> _appserver 2828 0.0 0.4 2960344 91916 ?? S 5:52PM 0:03.86 /usr/bin/java -XX:NewSize=2m -Xmx64m -Xms32m -DWORootDirectory=/System -DWOLocalRootDirectory= -DWOUserDirectory=/System/Library/WebObjects/JavaApplications/wotaskd.woa -DWOEnvClassPath= -DWOApplicationClass=com.test.mytest.Application -DWOPlatform=MacOS -Dcom.webobjects.pid=2828 -classpath WOBootstrap.jar com.webobjects._bootstrap.WOBootstrap -WOPort 2001 -WOCachingEnabled YES -WODebuggingEnabled YES -WOOutputPath /Library/WebObjects/Logs/testapp-1 -WOAutoOpenInBrowser NO -WOAutoOpenClientApplication NO -WOLifebeatInterval 30 -WOLifebeatEnabled YES -WOLifebeatDestinationPort 1085 -WOAdaptor WODefaultAdaptor -WOWorkerThreadCount 8 -WOListenQueueSize 128 -WOWorkerThreadCountMin 16 -WOWorkerThreadCountMax 256 -NSProjectSearchPath () -WOSessionTimeOut 3600 -WOApplicationName testapp -WOMonitorEnabled YES -WONoPause YES
>
>
> db1p:~ admin$ ps auxwww | grep 1085
> _appserver 2828 0.0 0.4 2960344 91916 ?? S 5:52PM 0:03.86 /usr/bin/java -XX:NewSize=2m -Xmx64m -Xms32m -DWORootDirectory=/System -DWOLocalRootDirectory= -DWOUserDirectory=/System/Library/WebObjects/JavaApplications/wotaskd.woa -DWOEnvClassPath= -DWOApplicationClass=com.test.mytest.Application -DWOPlatform=MacOS -Dcom.webobjects.pid=2828 -classpath WOBootstrap.jar com.webobjects._bootstrap.WOBootstrap -WOPort 2001 -WOCachingEnabled YES -WODebuggingEnabled YES -WOOutputPath /Library/WebObjects/Logs/testapp-1 -WOAutoOpenInBrowser NO -WOAutoOpenClientApplication NO -WOLifebeatInterval 30 -WOLifebeatEnabled YES -WOLifebeatDestinationPort 1085 -WOAdaptor WODefaultAdaptor -WOWorkerThreadCount 8 -WOListenQueueSize 128 -WOWorkerThreadCountMin 16 -WOWorkerThreadCountMax 256 -NSProjectSearchPath () -WOSessionTimeOut 3600 -WOApplicationName testapp -WOMonitorEnabled YES -WONoPause YES
> _appserver 75 0.0 0.4 3001996 89128 ?? Ss 4:19PM 0:05.05 /usr/bin/java -XX:NewSize=2m -Xmx64m -Xms32m -DWORootDirectory=/System -DWOLocalRootDirectory= -DWOUserDirectory=/ -DWOEnvClassPath= -DWOApplicationClass=Application -DWOPlatform=MacOS -Dcom.webobjects.pid=75 -Dcom.sun.management.jmxremote=false -classpath WOBootstrap.jar com.webobjects._bootstrap.WOBootstrap -WOPort 1085
> admin 2960 0.0 0.0 2435036 540 s000 R+ 5:55PM 0:00.00 grep 1085
>
>
>
>
>
> On Jul 28, 2010, at 5:51 PM, Pascal Robert wrote:
>
>>> java 2532 _appserver 80u IPv6 0xffffff80350ce760 0t0 TCP [::xx.xxx.xxx.71]:49639->[::xx.xxx.xxx.71]:webobjects (ESTABLISHED)
>>
>> So that line shows that wotaskd and the instance are talking to each other, so I guess the problem lies in JavaMonitor.
>>
>> Do :
>>
>> ps auxwww | grep 56789
>> ps auxwww | grep 2001
>> ps auxwww | grep 1085
>>
>> And send the results.
>>
>>>
>>> These are new server without any OS X installed before. Brand new out of the box.
>>>
>>>
>>>
>>> On Jul 28, 2010, at 5:20 PM, Chuck Hill wrote:
>>>
>>>>
>>>> On Jul 28, 2010, at 5:10 PM, Tusker wrote:
>>>>
>>>>> I'm using the DNS domain which resolves to the IP of the machine. I just got the DNS setup today in reverse as well. I was using the .local address before.
>>>>
>>>> This is a fun puzzle! See below.
>>>>
>>>>
>>>>
>>>>> On Jul 28, 2010, at 4:43 PM, Chuck Hill wrote:
>>>>>
>>>>>> What are you using for the host name? It needs to DNS resolve to primary IP of this machine. You can't use .local addresses.
>>>>>>
>>>>>> On Jul 28, 2010, at 4:29 PM, Tusker wrote:
>>>>>>
>>>>>>> No luck yet.
>>>>>>>
>>>>>>> I added WOHost in the additional args for the app in JavaMonitor and added it in launchd. I couldn't even get wotaskd to start with this argument. (see error below)
>>>>>>>
>>>>>>> Instead, I left wotaskd as is and changed the application host in JavaMonitor.
>>>>>>>
>>>>>>> The WOAdaptor is up and reporting the application running.
>>>>>>> Javamonitor us up and running
>>>>>>> Wotaskd is reporting all the configuration changes and I see the applications there as well.
>>>>>>>
>>>>>>> I can get to the following url:
>>>>>>>
>>>>>>> http://myappdomain:2001/test.woa/wa/
>>>>
>>>> That goes directly to the application, cutting Apache, the woadaptor, and wotaskd out of the picture.
>>>>
>>>>
>>>>>>> but I can't get to
>>>>>>>
>>>>>>> http://mywebserverdomain/test.woa/wa
>>>>>>>
>>>>>>> I restart the server and try starting the application. This time, I get an error in the SpawnofWotaskd.log which makes no sense. It actually starts a application but JavaMonitor does not see it.
>>>>
>>>> That suggests that all the pieces are not using the same host name.
>>>>
>>>>
>>>>>>> JavaMonitor reports a death. But I can get to http://myappdomain:2001/test.woa/wa/
>>>>>>>
>>>>>>>
>>>>>>> Here are the Logs when I start application using JavaMonitor after a server restart.
>>>>>>>
>>>>>>> --------------------------------------------------------
>>>>>>> Jul 28 16:21:55 contactme[N/A] (ERXNSLogLog4jBridge.java:43) WARN NSLog - Unable to establish a connection to port 2001 on this host. Perhaps this port is already in use by another WebObjects application instance.
>>>>>>> Jul 28 16:21:55 contactme[N/A] (ERXNSLogLog4jBridge.java:43) WARN NSLog - <WOApplication>: Cannot be initialized.java.lang.reflect.InvocationTargetException
>>>>>>> Jul 28 16:21:55 contactme[N/A] (ERXNSLogLog4jBridge.java:43) WARN NSLog - A fatal exception occurred: <WOApplication>: Cannot be initialized.
>>>>>>> -------------------------------------------------------
>>>>
>>>> So it is already running. Try
>>>> sudo lsof -i tcp:2001
>>>> to see the running process id. You can also try
>>>> ps auxwww | grep 2001
>>>>
>>>>
>>>>>>> wotaskd error when adding WOHost:
>>>>>>>
>>>>>>> [2010-7-28 13:39:31 PDT] <main> WebObjects version = 5.4.3
>>>>>>> [2010-7-28 13:39:31 PDT] <main> Unable to establish a connection to port 1085 on this host. Perhaps this port is already in use by another WebObjects application instance.
>>>>>>> [2010-7-28 13:39:31 PDT] <main> A fatal exception occurred: <WOApplication>: Cannot be initialized.
>>>>>>> [2010-7-28 13:39:31 PDT] <main> com.webobjects.foundation.NSForwardException [java.lang.reflect.InvocationTargetException]
>>>> ...
>>>>>>> Caused by: java.net.BindException: Can't assign requested address
>>>>>>
>>>>
>>>> So either wotaskd is already running or some other process is using that port.
>>>>
>>>> Try seeing what the pid is with this:
>>>> sudo lsof -i tcp:1085
>>>>
>>>> Check in /Library/LaunchDaemons and /System/Library/LaunchDaemons to ensure that wotaskd is not getting started twice.
>>>>
>>>> New and clean means that no previous OS X version was installed on these machines?
>>>>
>>>> Chuck
>>>>
>>>>
>>>>>>>
>>>>>>> On Jul 28, 2010, at 11:54 AM, Simon wrote:
>>>>>>>
>>>>>>>> i recently had the misfortune of having to set up a snow leopard deployment and ran into similar problems. remember there are 3 golden URL's to diagnosing deployment issues:
>>>>>>>>
>>>>>>>> http://yourdomain/cgi-bin/WebObjects/WOAdaptorInfo
>>>>>>>> http://yourdomain:1085 // assuming you are running wotaskd on standard port
>>>>>>>> http://yourdomain:56789 // assuming you are running javamonitor on standard port
>>>>>>>>
>>>>>>>> if we ever have deployment issues these are my first 3 ports of call. in particular, make sure the first two are both reporting your running instances. with our snow leopard issues we could see running instance in the wotaskd config, but the WOAdaptorInfo page was reporting nothing ... bingo.
>>>>>>>>
>>>>>>>> simon
>>>>>>>>
>>>>>>>> On 28 July 2010 19:05, Chuck Hill <email@hidden> wrote:
>>>>>>>> Use the hostname of the app server in JavaMonitor as the host and also set this in the Properties file with WOHost=.... for the app and wotaskd. Or set WOHost in the additional args for the app in Java Monitor and in the launchd config for wotaskd.
>>>>>>>>
>>>>>>>> Did that help?
>>>>>>>>
>>>>>>>>
>>>>>>>> Chuck
>>>>>>>>
>>>>>>>> On Jul 28, 2010, at 10:54 AM, Tusker wrote:
>>>>>>>>
>>>>>>>>> Hi,
>>>>>>>>>
>>>>>>>>> I'm seeing some very bizarre things with javamonitor and wotaskd. I have new clean Snow Leopard servers. I have a 3 machine setup. 1 Websever, 1 Application server (Javamonitor, wotaskd) and 1 database server. I followed the following instructions (http://wiki.objectstyle.org/confluence/display/WO/Programming__WebObjects-Web+Applications-Deployment-Mac+OS+X+Server).
>>>>>>>>>
>>>>>>>>> I can initially start up wotaskd and javamonitor. I setup host and my test application using Javamonitor. I can start up my test application fine. Everything works fine, but as soon as I restart the server, I can't start the application that I just setup. Both the wotaskd and javamonitor process start up and I can see them in the Activity Monitor. I modified SpawnOfWotaskd.sh to capture the logs when I hit go. Initially, I see these logs. When I restart the server, these stop as well.
>>>>>>>>>
>>>>>>>>> I tried adding -_DeploymentDebugging true to the plist but it will not output the debugging level logs.
>>>>>>>>>
>>>>>>>>> Any clues?
>>>>>>>>>
>>>>>>>>> Thanks,
>>>>>>>>> M
>>>>>>>>
>>>>>>>>
>>>>>>>> --
>>>>>>>> Chuck Hill Senior Consultant / VP Development
>>>>>>>>
>>>>>>>> Practical WebObjects - for developers who want to increase their overall knowledge of WebObjects or who are trying to solve specific problems.
>>>>>>>> http://www.global-village.net/products/practical_webobjects
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> _______________________________________________
>>>>>>>> Do not post admin requests to the list. They will be ignored.
>>>>>>>> Webobjects-dev mailing list (email@hidden)
>>>>>>>> Help/Unsubscribe/Update your Subscription:
>>>>>>>>
>>>>>>>> This email sent to email@hidden
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>> --
>>>>>> Chuck Hill Senior Consultant / VP Development
>>>>>>
>>>>>> Practical WebObjects - for developers who want to increase their overall knowledge of WebObjects or who are trying to solve specific problems.
>>>>>> http://www.global-village.net/products/practical_webobjects
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>
>>>> --
>>>> Chuck Hill Senior Consultant / VP Development
>>>>
>>>> Practical WebObjects - for developers who want to increase their overall knowledge of WebObjects or who are trying to solve specific problems.
>>>> http://www.global-village.net/products/practical_webobjects
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>
>>> _______________________________________________
>>> Do not post admin requests to the list. They will be ignored.
>>> Webobjects-dev mailing list (email@hidden)
>>> Help/Unsubscribe/Update your Subscription:
>>>
>>> This email sent to email@hidden
>>
>
_______________________________________________
Do not post admin requests to the list. They will be ignored.
Webobjects-dev mailing list (email@hidden)
Help/Unsubscribe/Update your Subscription:
This email sent to email@hidden