Re: 64-bit problem with VM statistics
Re: 64-bit problem with VM statistics
- Subject: Re: 64-bit problem with VM statistics
- From: William Kucharski <email@hidden>
- Date: Mon, 6 Jun 2005 02:05:00 -0600
On Jun 5, 2005, at 9:26 AM, Andrew Gallatin wrote:
This is not really true. Many PCI devices these days are capable of
addressing 64-bits of physical memory (eg, "DAC"). The DART enables
address limited devices to address memory anywhere, at the price of
artificially crippling DAC capable devices because of an oversight
on IBM or Apple's part.
Though due to the PCI bus' 32-bit address width, DAC needs two cycles
to specify the 64-bit address to be referenced, so depending upon
locality it's not necessarily any faster for a DAC device to address
memory than it is to do it via the DART or an IOMMU. One can easily
craft scenarios in which each method of access is faster.
This is important to OS-bypass HPC network devices because it allows
a large amount of memory to be pinned and available for network
communication. Whereas on G5s (and IBM js20s), only 2GB of memory can
be pinned at any one time. Allowing for the needs of other devices,
this realistically means that only 1.5GB can be pinned at any one
time.
This is one of the handicaps the G5 platform faces in the HPC
cluster market.
I wouldn't consider the G5's performance in HPC clusters such as
Virginia Tech's
to exactly be "handicapped." Every system has trade-offs between
theoretical
throughput and actual implementation, and while the DART can
admittedly be a
non-optimal solution, system software can work around the limitations
in most
cases, often by the use of "kernel cages" to partition memory for use by
particular devices.
Just my $.02 on the subject; now back to your regularly scheduled
programming. :-)
William Kucharski
email@hidden
_______________________________________________
Do not post admin requests to the list. They will be ignored.
Darwin-kernel mailing list (email@hidden)
Help/Unsubscribe/Update your Subscription:
This email sent to email@hidden