> OUCH! Use mmap!
I will look into mmap.
> You should be doing both of these steps in one pass thru the data.
It is not, but will be done in one pass.
> Sequentially or randomly? (probably doesn't matter; I'd recommend mmap ether
way).
currently it is sequentially but later it will be random. mmap, will do.
> Where does SHARK say your system is spending it's time?
I have profiled the app but not with SHARK. I will use SHARK.
> I'm assuming that you mean different performance results not computational
results? ...
performance results.
As I read this I'm sort of getting the impression that I should use SHARK.
I think I'll use SHARK.
Thank you very much,
W.
-----Original Message-----
From: George Warner [mailto:email@hidden]
Sent: Wednesday, March 30, 2005 1:35 PM
To: email@hidden
Cc: Edwards, Waverly
Subject: re: performance issues
On Tue, 29 Mar 2005 18:42:57 -0500, "Edwards, Waverly"
<email@hidden> wrote:
> Would someone provide me with some tips for getting the most
> performance out of my operating systems / virtual memory.
>
> I have a 378 megabyte file that I
>
> 1. read the entire file into a handle
OUCH! Use mmap!
> 2. change from little endian to big endian (*A)
> 3. convert all the doubles to floats
You should be doing both of these steps in one pass thru the data.
> 4. display all the floating point data using quartz
This is good. Best optimization here would be to get the data into a format
that Quartz can digest easiest.
> A. not a simple conversion. I have to seek out where
> in the file where the little endian info is located.
Sequentially or randomly? (probably doesn't matter; I'd recommend mmap ether
way).
> I have an 800 mhz g3 ibook with 640 megs of ram running 10.2.3
> that I do all my testing with. When I run my program in a tight
> loop, monitoring cpu and memory usage I never get more than
> 50% memory usage and 60% cpu utilization.
Where does SHARK say your system is spending it's time?
> My first issue is that I'm stress testing the program but
> the OS isn't giving me as much cpu or memory as I would think.
My first guess is that you're thrashing your caches. SHARK would tell you
for sure.
> My second issue is that I have no other programs running but
> running my program against the same file over and over again
> I can get very different results.
I'm assuming that you mean different performance results not computational
results? If you're getting different computational results then you have a
completely different problem.
> If I perform just steps 1 & 2 my times have been as low as 48
> seconds and as high as 90 seconds. Adding step 3 my times from
> 93 seconds to 147 seconds. The times are off the top of
> my head but I think they are pretty accurate.
You should probably combine steps 2 and 3. Doing them as separate steps
destroys all locality of reference. (And by all means use AltiVec! I know
you said you're on a G3 but this in itself would recommend that it's time
for an upgrade to a nice tibook. ;-)
> The same program under the same environment produces results that
> are not very close. To a degree I attribute my performance issue
> to memory thrashing however if that were the only reason then I
> would expect approximately the same times between runs.
Another factor here is your "Processor Performance" setting in your "Energy
Saver" preference panel. If you're power cycling then this can adversely
effect the consistence of your results.
> Now taken
> a step further the only reason I can come up with for getting really
> low times is that there is some "sweet spot" or magically touched
> area that occurs in virtual memory space that once created the OS
> more easily refers to. I came to that conclusion because at
> times I would do one stress test immediately after another and those
> runs would be would have lower elapsed times. Waiting a few minutes
> would yield lower run times, but not always. That's my best guess.
This definitely points a finger at vm cache thrashing. It also enforces what
I said earlier about locality of reference.
> My questions are
>
> 1. how do I get as much cpu time as possible. Making it a user configurable
> setting. use more, use less, let OS handle that.
Optimize your code and put it on it's own thread. (you didn't mention if
your code is threaded.)
> 2 how do I get as much (real) memory that I can. User configurable
> use more, use less, let OS handle that.
The vm system should handle this for you. There are ways to optimize memory
usage. I'd start by using mmap instead of handles.
> 3. How do I keep my virtual memory usage as low a possible (keeping
> performance high)
Try not to touch any place in the data more often than necessary. You may
find that deferring the endian and double to float conversions until you're
actually using the data may give you a speed up. Or not. (At the very least
you should combing these into a single pass thru the data).
> 4 Can I create a sweet spot to make processing faster.
Don't know. Ask SHARK!
> OK, here is the BIG kicker, step 4 (first list). The entire process including
> displaying the data in quartz takes 88 minutes. It *is* scaled to fit on the
> screen and yes it is a stress test however I can not escape the fact that
> drawing is much too slow in quartz. The drawing portion little more than:
> [...]
> taking the time to convert all the double to floats so I could use
> cgcontextaddlines reduce my time by 4 whole minutes. Not substantial at all.
I suspect that most of what you gain by using cgcontextaddlines you're
losing in the double to float conversion step.
> I don't know anything more that can be done to speed drawing up and what make
> it worse is that if I needed to redraw the whole thing, that's another 88
> minutes under the best conditions.
I'd certainly start by coming up with a way of culling the dataset to limit
runtimes to something more manageable (like perhaps five minutes) and start
optimizing that. As you find (and fix) performance bottlenecks you can
continue adding more of the original data to keep the overall runtime around
5 minutes. This keeps your testing cycles manageable.
Oh, and use SHARK! (did I say that before? ;-)
--
Enjoy,
George Warner,
Schizophrenic Optimization Scientist
Apple Developer Technical Support (DTS)
_______________________________________________
Do not post admin requests to the list. They will be ignored.
Carbon-dev mailing list (email@hidden)
Help/Unsubscribe/Update your Subscription:
http://lists.apple.com/mailman/options/carbon-dev/email@hidden
This email sent to email@hidden
