> How do you perform these steps? Is step 1 one loop, step 2 another and ...
Step 1 is one function
Step 2 is another
Steps 3&4 are combined but easily separated.
The entire file is read in at once because the file is not easily segmented
and because in a real situation I might not have that as an option.
> I don't know if this will actually speed things up substantially, but ...
>From all the post that I've read and the extra reading I've done on optimizing
the file system, I think it may, though I have to see if that is an option.
<<
So your saying that your culling out lines that will be too small to be
differentiated? Or you are drawing all the lines in some particular
view (including lines that may overlap and produce very little
distinguishable output)?
>>
drawing includes lines that may overlap and produce very little distinguishable output
in this particular file but not all of them.
Much good info is provide by all.
I'm still absorbing it all.
Thanks,
W.
-----Original Message-----
From: David Duncan [mailto:email@hidden]
Sent: Tuesday, March 29, 2005 7:18 PM
To: Edwards, Waverly
Cc: email@hidden
Subject: Re: performance issues
On Mar 29, 2005, at 06:42 PM, Edwards, Waverly wrote:
> Would someone provide me with some tips for getting the most
> performance out of my operating systems / virtual memory.
>
> I have a 378 megabyte file that I
>
> 1. read the entire file into a handle
> 2. change from little endian to big endian (*A)
> 3. convert all the doubles to floats
> 4. display all the floating point data using quartz
How do you perform these steps? Is step 1 one loop, step 2 another and
so on, or is this one big loop where all these processes occur
sequentially? Or something else entirely? Have you run Shark on this to
see where the hotspots are?
Personally, if I were to implement something like this, I wouldn't read
the entire file in, ever, unless I absolutely needed any sample at any
time with very small latencies. For something specifically like this, I
would read in a few pages work (say 16k) of the file, do the little to
big, double to float and submit on this data. Then read another 16k
into the same buffer and so on. If you wanted to get fancy, you might
even put thee file read on a separate thread and ping pong two buffers
in order to overlap file read with your other calculations.
I don't know if this will actually speed things up substantially, but
at the very least you will use significantly less memory doing this.
> Now taken a step further the only reason I can come up with for
> getting really low times is that there is some "sweet spot" or
> magically touched area that occurs in virtual memory space that once
> created the OS more easily refers to. I came to that conclusion
> because at times I would do one stress test immediately after another
> and those runs would be would have lower elapsed times. Waiting a few
> minutes would yield lower run times, but not always. That's my best
> guess.
You are probably seeing the system file caching at work. Thus if you
run again immediately you get the file you read out of memory instead
of off disk, whereas later the data may have been replaced.
> My questions are
>
> 1. how do I get as much cpu time as possible.
In general, the OS will give you as much CPU time as possible. This is
not necessarily 100% of the CPU, so don't expect it to be. There are
ways to increase your priority to get CPU time, but but most people
don't really need it (even when they think they do).
> 2 how do I get as much (real) memory that I can.
You can take as much real memory as you can use. But the OS will not
give you all the real memory that is available without you locking
those pages (BAD idea unless you know what you are doing and why).
> 3. How do I keep my virtual memory usage as low a possible (keeping
> performance high)
By using less memory, or by locking pages in memory (again BAD idea).
> 4 Can I create a sweet spot to make processing faster.
I'm not certain what you mean by this really. I think if you do other
things to optimize your application this "sweet spot" you are thinking
of will appear naturally...
> OK, here is the BIG kicker, step 4 (first list). The entire process
> including displaying
> the data in quartz takes 88 minutes. It *is* scaled to fit on the
> screen and yes
> it is a stress test however I can not escape the fact that drawing is
> much too
> slow in quartz. The drawing portion little more than:
So your saying that your culling out lines that will be too small to be
differentiated? Or you are drawing all the lines in some particular
view (including lines that may overlap and produce very little
distinguishable output)?
> taking the time to convert all the double to floats so I could use
> cgcontextaddlines
> reduce my time by 4 whole minutes. Not substantial at all.
Well, it was an improvement :). I personally wouldn't consider a 5%
speedup to be insubstantial, but given the overall time things are
taken I can see why you are frustrated.
--
Reality is what, when you stop believing in it, doesn't go away.
Failure is not an option. It is a privilege reserved for those who try.
David Duncan
_______________________________________________
Do not post admin requests to the list. They will be ignored.
Carbon-dev mailing list (email@hidden)
Help/Unsubscribe/Update your Subscription:
http://lists.apple.com/mailman/options/carbon-dev/email@hidden
This email sent to email@hidden
