Buffer Size Limit - Rule Of Thumb

10 May 2007 11:46 PM

Lorin

Is there a rule of thumb to determine the largest size a buffer should be?
e.g 10% of available RAM.
Does VB6 have a limit (say 2G max).
I am reading a large file and looking a individual bytes within the buffer.
Currently I use a multiple of the bytes per sector (around 16Mbyte buffer).
Wondering if there is any advantage to going larger?

11 May 2007 12:00 AM

Karl E. Peterson

Lorin <Lo***@discussions.microsoft.com> wrote:

> Is there a rule of thumb to determine the largest size a buffer should be?
> e.g 10% of available RAM.
> Does VB6 have a limit (say 2G max).

Windows limits each process to 2Gb of address space. (With one notable exception
that you should never expect on machines not under your direct control.)

> I am reading a large file and looking a individual bytes within the buffer.
> Currently I use a multiple of the bytes per sector (around 16Mbyte buffer).
> Wondering if there is any advantage to going larger?

Probably not, hard to say. Kinda doubtful.

--
..NET: It's About Trust!
http://vfred.mvps.org

11 May 2007 6:35 AM

J French

On Thu, 10 May 2007 16:46:04 -0700, =?Utf-8?B?TG9yaW4=?=
<Lo***@discussions.microsoft.com> wrote:

>Is there a rule of thumb to determine the largest size a buffer should be?
>e.g 10% of available RAM.
>Does VB6 have a limit (say 2G max).
>I am reading a large file and looking a individual bytes within the buffer.
>Currently I use a multiple of the bytes per sector (around 16Mbyte buffer).
>Wondering if there is any advantage to going larger?

I've found that the advantages of large buffers tend to tail off
pretty rapidly - around the 100kb level

I don't think I've ever used as much as 1mb

With lines of say 100 bytes a 100kb buffer reduces 1000 disk accesses
to 1 disk access
- 10 disk accesses per megabyte is barely noticable

The problem with large buffers is that they may cause swap file
thrashing

11 May 2007 7:12 AM

Mike Williams

"Lorin" <Lo***@discussions.microsoft.com> wrote in message
news:017C7C6D-8DD9-4796-9005-DAA022564A84@microsoft.com...

> Is there a rule of thumb to determine the largest size a buffer
> should be? e.g 10% of available RAM. Does VB6 have a
> limit (say 2G max). I am reading a large file and looking a
> individual bytes within the buffer. Currently I use a multiple
> of the bytes per sector (around 16Mbyte buffer).
> Wondering if there is any advantage to going larger?

I performed a few tests, mostly out of idle curiosity, a couple of weeks
ago. I can't remember the exact result but I don't think I saw any
significant benefit in going much above a couple of Mbytes. Going too high
can often prove counter productive.

Mike