Re: 64-bit problem with VM statistics
Re: 64-bit problem with VM statistics
- Subject: Re: 64-bit problem with VM statistics
- From: Andrew Gallatin <email@hidden>
- Date: Sun, 5 Jun 2005 11:26:41 -0400 (EDT)
William Kucharski writes:
>
> If you think this is fun, PCI devices can only address a 32-bit
> memory space, so when using PCI devices on a 64-bit architecture,
> memory address translation must be done if you want to allow those
> devices to access the entire memory space. This is what the DART is
> all about, as described here:
>
> <http://developer.apple.com/documentation/Darwin/Conceptual/
> KernelProgramming/vm/chapter_7_section_8.html>
>
This is not really true. Many PCI devices these days are capable of
addressing 64-bits of physical memory (eg, "DAC"). The DART enables
address limited devices to address memory anywhere, at the price of
artificially crippling DAC capable devices because of an oversight
on IBM or Apple's part.
Nearly 10 years ago, DEC Alpha PCI chipsets also had an IOMMU which
made 64-bits of physical memory available to address limited devices.
However, DEC wisely included the "monster window" as an escape hatch
to allow 64-bit capable devices to bypass the IOMMU. The driver for a
64-bit capable device simply did the virt->phys translation, and set a
very high bit to indicate to the IOMMU that it should pass the
physical address unmolested. This high bit was larger than the
biggest possible physical address, 1UL << 40 on older alphas and 1UL
<< 49 on newer alphas.
So on alphas (as well as more modern 64-bit platforms, such as AMD64,
EM64T, and ia64), 64-bit devices are capable of addressing any
physical address without going through an IOMMU. This is important to
OS-bypass HPC network devices because it allows a large amount of
memory to be pinned and available for network communication. Whereas
on G5s (and IBM js20s), only 2GB of memory can be pinned at any one
time. Allowing for the needs of other devices, this realistically
means that only 1.5GB can be pinned at any one time. This is one of
the handicaps the G5 platform faces in the HPC cluster market.
Drew
_______________________________________________
Do not post admin requests to the list. They will be ignored.
Darwin-kernel mailing list (email@hidden)
Help/Unsubscribe/Update your Subscription:
This email sent to email@hidden