I'm using Marc Liyanage's binary distribution of Apache on MacOS X
10.3.8. It has been working fine forever, hosting about a dozen sites
on a G4/350.
Within the past few weeks, customers have complained that it was
slow, and Hyperspin would say it was down now and then. When I'd
investigate, the load average would be way up (like 8+, whereas it is
rarely above 1 normally) with dozens of httpd processes running. I'd
do "apachectl stop; apachectl start" and things would go back to
normal for a while.
Further investigation with top shows that lookupd is soaking the
processor, using as much as 90% of the CPU. SIGHUPping it or -
flushcache doesn't help; the only thing that brings things back is to
restart Apache.
Recently, I left top running, and kept an eye on it until I saw idle
time go to 0%, then immediately did a tail on access log. It was
always msnbot. First, I made a robots.txt that blocked msnbot from
that website, but it immediately caused the problem in another
website. I blocked the msnbot IP in my router, and it happened again
on a different IP. I've blocked Microsoft's entire Class B, and the
problem has gone from every ten minutes to every twelve hours or so,
with other bots and crawlers showing up when I check. (Latest culprit
was yahoo.)
Any ideas on why seemingly innocuous bots should be soaking my
server? Is Microsoft secretly conspiring against MacOS X hosted
servers? :-)
I swear, I made no changes to Apache configuration or the Mac. I'm my
own DNS primary, and the only coincidental thing that has happened is
that I've asked my ISP to add a few new domains to their DNS
secondary that I added to my primary MONTHS ago.
Any ideas? What should I try next?
_______________________________________________
Do not post admin requests to the list. They will be ignored.
Web-dev mailing list (email@hidden)
Help/Unsubscribe/Update your Subscription:
http://lists.apple.com/mailman/options/web-dev/email@hidden
