Re: D2W and Direct Actions
Re: D2W and Direct Actions
- Subject: Re: D2W and Direct Actions
- From: Guido Neitzer <email@hidden>
- Date: Sun, 3 Jun 2007 10:38:37 -0600
On 03.06.2007, at 02:29, Denis Frolov wrote:
We don't do any special handling of search spiders requests. The
point is that you should be anyway ready for peaks in user
generated traffic which can lead to even more sessions created.
We were - but you wouldn't think of that, when you hope for about 10k
requests per day that you get 3000+ sessions in a couple of minutes
and when it is one of your first WO project with public access.
After that I rewrote the application to be session free in most of
the parts and create only sessions where it needed to do (e.g. long
response pages) - that was done in about two weeks. We had never any
issue again and the application handled up to nearly a 100k page
views on some days (when spider traffic was heavy) - on an old dual
G4 887MHz! Yeah, I know. I'd love to get a couple of Xserves for
these projects ... ;-) this one is on an Intel Xserve now and just
screams with performance ...
So, the proper solution is probably minimizing session memory
footprint, adding more instances and lowering session timeout.
Yep, even at that time I was aware of these things but you can't
really lower the session timeout to under 10 Minutes - only if you're
using DAs and create a session if you need one but don't loose the
page. But the application wasn't written that way. I had to rewrite
everything from component actions to DAs and then - why creating a
session if you don't need one?
Another idea is using one common session for all the spiders.
That is an interesting idea. I don't need it anymore as we are going
"session free as long as we can" which works really well even on
really low level hardware (What my own projects normally have as I
use abandoned old hardware for them. My little CMS runs on a
blue&white G3-400 with two SCSI drives, 512MB RAM - and it is usable
but memory is a real issue for me.). The other issue (I guess, not
sure because it's years ago now) was when spiders came back with lots
of URLs with sessions in it, came to the session timeout page,
created a new session and so on ...
You would also probably want to add a "Disallow: /wo/" line to
robots.txt if you use url shortening approach suggested in Mike's
email.
I don't need that anymore. But I have that setting in my robots.txt
and ban robots which do ignore that file. And as 99% of the pages are
session free I normally just don't worry about that anymore.
cug
_______________________________________________
Do not post admin requests to the list. They will be ignored.
Webobjects-dev mailing list (email@hidden)
Help/Unsubscribe/Update your Subscription:
This email sent to email@hidden