I would definitely start with looking at SpamAssassin and ClamAV as
the cause of your slowdowns. We had the same issue, and resolved it
by throwing a Barracuda in front of the mail server to do the spam/
virus filtering.
There's no magic to Barracuda. It's just easy to buy a toaster if you
want toast rather than grilling it yourself.
Moving these functions, and probably your whole MTA in general, to
another machine will be a huge gain.
And using a better MTA that can reject the actual connection rather
than accept the mail and have to process it with SA or ClamAV
altogether, like using exim and it's strong ACLs to drop spam before
you even accept it and then scan it further, should solve the
problem, even if on the same box.
-dhan
------------------------------------------------------------------------
Dan Shoop
Computer Scientist
iWiring / U.S. Technical Services