Fwd: Optimal read buffer size?
Fwd: Optimal read buffer size?
- Subject: Fwd: Optimal read buffer size?
- From: Peter Paulus <email@hidden>
- Date: Fri, 20 May 2005 10:19:11 +0200
Did a reply direct instead of sending to list.
Begin forwarded message:
From: Peter Paulus <email@hidden>
Date: Thu May 19, 2005 09:32:38 Europe/Amsterdam
To: Chaz McGarvey <email@hidden>
Subject: Re: Optimal read buffer size?
Hello Chaz,
I've used an approach that could be called an elastic buffer.
I allocate a fixed size ring buffer. Each element in the ring buffer
holds a buffer with a variable amount of data. The size of the
elementbuffer depends on the amount of data that the network stack has
available for me. The elastic buffer is variable in both width (size
of each elementbuffer) as height (the number of elementbuffers).
This structure is exposed to the client software with an iterator,
this could be a byte-iterator or a block-iterator.
Any iterator should merge to data in consecutive element buffers into
1 buffer on request, hiding the underlaying structure of the elastic
buffer. elementbuffers that have been read completely can be released.
Of course there could be variations to the theme. You could have a
double ended queue (deque) instead of a fixed size ring buffer. You
could also eliminate the 'network stack to buffer copy' by
preallocating fixed size elementbuffers and passing these buffers to
the network stack to fill. Although you gain perfomance, you'll loose
'elasticity'. You iterator could than pass the entire buffer over to
the client software, eliminating an other copy.
This has worked well for the HTTP-stack that I've written. No
complaints about buffer overflow or performance.
An other scenario could be that you create a 'Listener' for a
DataAvaibleEvent reporting the amount of data available at the network
stack. The listener could then allocate an appropriate buffer and
fetch this amount from the network stack by passing the buffer in.
With kind regards,
Peter Paulus
On Wednesday, May 18, 2005, at 20:06 Europe/Amsterdam, Chaz McGarvey
wrote:
Hello:
I find that when I write network code, I never really know how big I
should make the read buffer. Some protocols have only short packets
so I usually have a pretty small buffer, or only read in one packet
at a time, but if I'm reading in a very big packet like a mult-
megabyte resource download that I'm writing to disk, is there a sweet
spot that requires the fewest read calls and doesn't waste (much)
memory?
Thanks,
chaz
_______________________________________________
Do not post admin requests to the list. They will be ignored.
Macnetworkprog mailing list (email@hidden)
Help/Unsubscribe/Update your Subscription:
email@hidden
This email sent to email@hidden
*
*
_______________________________________________
Do not post admin requests to the list. They will be ignored.
Macnetworkprog mailing list (email@hidden)
Help/Unsubscribe/Update your Subscription:
This email sent to email@hidden