Mailing Lists: Apple Mailing Lists
Image of Mac OS face in stamp
 
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: Zero-copy PXI-X driver for G5 Mac, process access to 7 GB memory

Andrew & Larry,
While it is not easy or straight forward to handle DMAs on
large amounts of memory, it is possible. I have enclosed a skeleton
that show basically how to do it. Here's a brief overview of how
this works.

<IOKit/IOMapper.h> has some API that allow IOMapper (DART)
allocations to be decoupled from IOMemoryDescriptors and the 2 GB
limit. IOMapperIOVMAlloc allocates space in the DART.
IOMapperInsertPPNPages gives the previous allocation a mapping. The
mapping can be changed quickly and easily.

Early in the life of the system the driver will use
IOMapperIOVMAlloc to allocate a number of DMA windows for later use.
Window sizes that are a power of 2 minus one page are good; eg 256 MB
- 4KB.
A client program will allocate a large amount of memory.
Each non virtually contiguous chunk will be wired by the client.
vm_wire can be used, but mlock is easier. This will make the memory
resident and wire it. Then the memory chunks will be registered with
the driver.
The driver will add each chunk of memory to a control
structure so that it can be referenced later. The driver will also
use a regular IOMemoryDescriptor to map the chunk of memory 1 MB at a
time. It will then loop over the IOMD to get the list of real
physical pages using getPhysicalSegment64.
Later the client will make a request that will start DMA to
one of the memory chunks. The driver will look up the chunk in the
control structures. It will then use one or more of the DMA windows
and set their mappings with IOMapperInsertPPNPages and the previously
collected physical page list. Then the driver can initiate the DMA
activity. Obviously, the windows and mappings can not be reused
until the DMA is complete. If there are multiple clients, the driver
will need to arbitrate the available windows among the clients.
The performance of this technique should be pretty good. The
expensive operation, the memory wire, will have been done once and up
front. The only thing done on the data path is
IOMapperInsertPPNPages which just loops the page list and fills a
table, so it should be pretty fast.

Josh




At 5:15 PM -0500 11/12/03, Andrew Gallatin wrote:
>Miller, Larry writes:
> >
> > 2. Is is possible to write a PCI-X driver that lets our PCI-X
>card transfer
> > data to these blocks of memory directly without a copy. (i.e. a zero-copy
> > driver).
>
>No. You can only do transfers to/from up to 2GB at any one time.
>
>All IOKIt "physical" addresses are 32-bit. The G5 uses an IOMMU
>(called the DART) which translates real 64-bit physical addresses to
>these 32-bit PCI addresses. So you are
>
>a) Not able to get the real physical address via IOKit. The IOKit
>routines will give you translated "dart" addresses.
>
>b) If you had a real 64-bit address, the DART would not know what to
>do with it. Ie. your transfer would fail.
>
>So your application would at least need to call into the kernel to
>prepare() (pin) and release() (unpin) smaller chunks of memory.
>
>I'm in the same boat -- I'd love for a way to give a real 64-bit
>address to my device, and a way to make the Dart just pass my transfer
>through, and not attempt to translate it. Most other IOMMUs have this
>feature.
>
>Drew
>_______________________________________________
>darwin-drivers mailing list | email@hidden
>Help/Unsubscribe/Archives:
>http://www.lists.apple.com/mailman/listinfo/darwin-drivers
>Do not post admin requests to the list. They will be ignored.

[demime 0.98b removed an attachment of type application/mac-binhex40 which had a name of BigDMA.h]

[demime 0.98b removed an attachment of type application/mac-binhex40 which had a name of BigDMA.cpp]
_______________________________________________
darwin-drivers mailing list | email@hidden
Help/Unsubscribe/Archives: http://www.lists.apple.com/mailman/listinfo/darwin-drivers
Do not post admin requests to the list. They will be ignored.
References: 
 >Zero-copy PXI-X driver for G5 Mac, process access to 7 GB memory (From: "Miller, Larry" <email@hidden>)
 >Re: Zero-copy PXI-X driver for G5 Mac, process access to 7 GB memory (From: Andrew Gallatin <email@hidden>)



Visit the Apple Store online or at retail locations.
1-800-MY-APPLE

Contact Apple | Terms of Use | Privacy Policy

Copyright © 2007 Apple Inc. All rights reserved.