Re: Server not longer reachable because of 211GB logfile
Re: Server not longer reachable because of 211GB logfile
- Subject: Re: Server not longer reachable because of 211GB logfile
- From: Pascal Robert <email@hidden>
- Date: Mon, 18 Jun 2012 10:07:06 -0400
Le 2012-06-17 à 08:59, ute Hoffmann a écrit :
> Hallo,
> I have a Mac Os X server running a number of WebObjects Applications for two years now.
> In the morning the server was no longer reachable via ssh or via Apache.
>
> The reason was a logfile of one Application instance which had the size of 211GB.
>
> The instances are all scheduled nightly thus the file had piled up to that size from sometime early morning
> to Midday. As the webobjects logs are pure textfiles something must have gone completely wrong in this instance
> today to get that amount of output. Unfortunately this file had to be deleted at once (it blocked the disk) without a chance to
> have a look on it.
>
> Another instance had this in the logfiles (some of this):
> 22192168 [WorkerThread64] WARN NSLog - <WOWorkerThread id=64 socket=null> IOException occurred while accepting server socket: java.net.SocketException: Too many open files in system
> 22192168 [WorkerThread65] WARN NSLog - <WOWorkerThread id=65 socket=null> IOException occurred while accepting server socket: java.net.SocketException: Too many open files in system
We had this problem years ago (read: 9 years ago). The problem in our case was that property files were opened, but never closed, so we reached the limit (which was quite low on 10.2, I think it was 512 files). Adding code to close the file fixed the problem.
You can use 'lsof' to see which files are in use.
sudo lsof -M | grep "REG"
>
> Has someone seen it and could this be the cause of my problem (I can image it filling a logfile when repeated often enough to a enourmous size)?
>
> If this could be the cause, does someone know how to detect this bevore I get to a filesize of the logfile
> which kills the server? Or what is likely to be the reason of this. The App has been running stable for years, no large logfiles...
>
> By the way, is there a way to detect the size of a textfile (logfile) and have a process send me a message that something goes wrong,
> when the file reaches a certain size?
Nagios!
> Thanks a lot
>
> Ute
>
>
> _______________________________________________
> Do not post admin requests to the list. They will be ignored.
> Webobjects-dev mailing list (email@hidden)
> Help/Unsubscribe/Update your Subscription:
>
> This email sent to email@hidden
_______________________________________________
Do not post admin requests to the list. They will be ignored.
Webobjects-dev mailing list (email@hidden)
Help/Unsubscribe/Update your Subscription:
This email sent to email@hidden