Re: [Wonder-disc] EOF & Multithreading
Re: [Wonder-disc] EOF & Multithreading
- Subject: Re: [Wonder-disc] EOF & Multithreading
- From: Miguel Arroz <email@hidden>
- Date: Fri, 11 Jul 2008 16:15:06 +0100
Hi!
I suppose you could use multiple EOF stacks, but I don't know if it
will perform better or not. It depends on what you call "waiting for
data".
If "waiting for data" is waiting for the data to be read from the
hard drive to memory, then you will make it even worse, because you
will cause more seek events, reducing the efective speed.
If "waiting for data" is waiting for Java to convert the raw data
to objects, then yes, you may gain a significative speed boost *if*
your machine is multi-core or multi-CPU, because you will be able to
do that conversion in parallel, using one core/CPU to each EOF stack.
Also, think is you can transfer your work for the DB itself. Like
if you are counting rows with certain characteristics, etc. If you are
working with so much data in a read-only fashion, it might be
interesting to push much of the work to the DB itself, to avoid all
the expensive overhead of fetching the data to WO, converting it to
objects, manage the memory usage, etc.
Yours
Miguel Arroz
On 2008/07/11, at 16:04, Randy Wigginton wrote:
I am working on an analysis application that reads tens of millions of
rows in order to analyze website behavior. Because the application is
so read-intensive, it currently spends 90% of its time waiting for
data. To address this, I am considering moving the DB reading into
one or more separate threads in order to more fully utilize the DB and
application servers. A search did not turn up anything that I found
useful.
Has anyone on this list done this previously? Are there any kind of
guidelines, best practices, etc?
Thanks for any guidance!
-------------------------------------------------------------------------
Sponsored by: SourceForge.net Community Choice Awards: VOTE NOW!
Studies have shown that voting for your favorite open source project,
along with a healthy diet, reduces your potential for chronic lameness
and boredom. Vote Now at http://www.sourceforge.net/community/cca08
_______________________________________________
Wonder-disc mailing list
email@hidden
https://lists.sourceforge.net/lists/listinfo/wonder-disc
http://www.survs.com
Attachment:
smime.p7s
Description: S/MIME cryptographic signature
_______________________________________________
Do not post admin requests to the list. They will be ignored.
Webobjects-dev mailing list (email@hidden)
Help/Unsubscribe/Update your Subscription:
This email sent to email@hidden