From: Max Roeleveld - Qualion Internet Services <email@hidden>
Date: Tue, 14 Jun 2005 00:38:14 +0200
Delivered-to: email@hidden
Delivered-to: email@hidden
Op 13-jun-05 om 22:17 heeft Jan Steinman het volgende geschreven:
Any ideas on why seemingly innocuous bots should be soaking my server?
Is Microsoft secretly conspiring against MacOS X hosted servers? :-)
I believe that some bots are less-than-nice when crawling sites, and I
believe MSN is one of them.
Apart from that, what kind of sites are you hosting? If it's an
intensive beast in terms of script and/or database use, it's easier to
make the server choke on it, of course. Having dozens of httpd
processes running isn't a bad thing in itself, it just means that it's
handling a bunch of requests -- the Apache config dictates how many
"spare" processes should be kept, etcetera. Depending on how much
memory the G4 has, it shouldn't be an issue if there are some processes
kept around.
I swear, I made no changes to Apache configuration or the Mac. I'm my
own DNS primary, and the only coincidental thing that has happened is
that I've asked my ISP to add a few new domains to their DNS secondary
that I added to my primary MONTHS ago.
I don't think that increased web traffic will bog down your server with
DNS requests for the domains you do DNS for, so I think you can rule
this one out. How about the other way around: is Apache doing DNS
lookups? That could slow things down (although it wouldn't matter for a
lot of requests from one IP, I suppose Apache will use some sort of DNS
cache...). When top reaced 0% idle, was named anywhere near the busiest
processes (top -u)?
Of course, blocking all search engine bots is not a long term solution.
Apart from the fact that blacklisting never works forever, some of your
clients might actually want to be _found_ on the web =].
For now, I'd think towards Apache (maybe combined with MySQL if you
host database-driven sites).
How many child processes is Apache instructed to keep right now? Is
that figure realistic? What happens if you increase it (memory
permitting)? Having to start a lot of processes (which would happen on
a server facing a sudden increase in requests with not enough child
processes at hand to handle them) is a sure way to slow things down.
Does the slowness persist (if you do nothing), or does it fade away
after the bots did their thing? During the slowdown, what are the most
CPU-hungry processes (top -u)? Does Apache complain about anything (in
the error log)? How about memory? How much of it is there? How much of
it is free? How about VM?
(Yes, I know, it's bad manners to answer a question with more questions
=] )
Good luck answering all of the above... =]
Max
_______________________________________________
Do not post admin requests to the list. They will be ignored.
Web-dev mailing list (email@hidden)
Help/Unsubscribe/Update your Subscription:
http://lists.apple.com/mailman/options/web-dev/email@hidden
