May 2006 - Darwin-kernel - lists.apple.com

Re: Write/Read Bulk communication process in KEXT. How?
by Michael Smith 25 May '06

25 May '06

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com On May 24, 2006, at 9:02 PM, Chinki wrote: Step4. Communicating with the KEY [ MAIN PART OF OUR DISCUSSION] // assign for IN pipe IOMemoryDescriptor *memDescIN = NULL; I assume you mean kIODirectionOut? // check for validation of memDescOut // Fill myOutStructure You are missing the ->prepare() and ->complete() calls. // Similarly for IN structure I assume you mean kIODirectionIn? // check for validation of memDescIn // Fill myOutStructure You are missing the ->prepare() and ->complete() calls. = Mike _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… a) I get the structure (got from user level) and Allocate the memory using IOMemoryDescriptor in my communication module (kext) i.e, // assign for OUT pipe IOMemoryDescriptor *memDescOut = NULL; // Now create and initialize an IOMemoryDescriptor for memory FOR WRITE memDescOut = IOMemoryDescriptor::withAddress( (void*) &myOutStructure, sizeof(MY_OUTSTRUCTURE), kDirectionOUT) // Do write operation with the allocated outpipe status = (pdx->fOUTPipe)->Write( memDescOut, (UInt32)1000, (UInt32) 2000, size, NULL); // Now create and initialize an IOMemoryDescriptor for memory FOR READ memDescIN = IOMemoryDescriptor::withAddress( (void*)&myINStructure, sizeof(MY_OUTSTRUCTURE), kDirectionOUT) // Do write operation with the allocated outpipe status = (pdx->fINPipe)->Read( memDescIN , (UInt32)1000, (UInt32) 2000, size, NULL, &gotsize); I also note that you're not checking the result of the IOMemoryDescriptor::withAddress() calls. It's imperative that you do this, and handle the case where they return NULL in a sensible fashion. This email sent to site_archiver(a)lists.apple.com

1 0

Re: Write/Read Bulk communication process in KEXT. How?
by Chinki 25 May '06

25 May '06

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com Domainkey-signature: a=rsa-sha1; q=dns; c=nofws; s=beta; d=gmail.com; h=received:message-id:date:from:to:subject:cc:in-reply-to:mime-version:content-type:references; b=mgZnbZ65d5ESJVoyw5CHAuksbI9i1yE86+5yWTqHdKgbnaRXg/stce5KUWMYglDCgic/Wk1qJkVK+X9CpK9dG+lbw9DPvH2rTq7GF6tEZTkuAENkr5QBnrrKbh8S/TjPI5Jn7DjbVtgRu/6Z9iTHk2yyLqk2t76P0+SLkblky/A= _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… This email sent to site_archiver(a)lists.apple.com

1 0

Re: Write/Read Bulk communication process in KEXT. How?
by Andrew Gallatin 24 May '06

24 May '06

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com Terry Lambert writes: > Have you file a bug report requesting this? I could swear I did, but then I could not find it. I just filed a new one as 4561295. Drew _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… This email sent to site_archiver(a)lists.apple.com

1 0

Re: File I/O in the kernel.
by Dean Reece 24 May '06

24 May '06

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… On May 24, 2006, at 12:32, Andrew Gallatin wrote: Michael Smith writes: On May 24, 2006, at 11:52 AM, Andrew Gallatin wrote: Terry Lambert writes: If you need to do file I/O to load firmware (as an example), either put the firmware in a discardable section (vm_deallocate () it after it's loaded in the card), or have a user space daemon communicate the firmware to your driver via a series of ioctl() calls to push the data *temporarily* into the kernel, so it doesn't take up real or virtual memory in your driver. This comes up over, and over, and over again here. Is there any chance Apple can provide a centralized way to do this in the OS, Yes. It's called a user-space process. OK, what you mean is that it is called N user space process, a different one for each driver, all re-inventing the same wheel with different bugs. Right - this request has come up several times, and I think it is perfectly reasonable. Adding a KPI that allows a kext to read a file from within it's bundle for the purposes of downloading firmware is something we have considered, and will probably do at some point. For anybody that cares, this enhancement request is being tracked internally by Radar# 4487388 "Need KPI to read a file out of a kext bundle (firmware download)" This is not the same problem as general file I/O (which I don't think kexts should be doing), nor is it the problem the original poster was asking about... - Dean This email sent to site_archiver(a)lists.apple.com

1 0

Re: Write/Read Bulk communication process in KEXT. How?
by Terry Lambert 24 May '06

24 May '06

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com On May 24, 2006, at 11:52 AM, Andrew Gallatin wrote: This comes up over, and over, and over again here. Is there any chance Apple can provide a centralized way to do this in the OS, so we don't all have to use our own home grown and (potentially) buggy or wasteful mechanisms? At least 3 other OSes provide a way for drivers to load firmware easily: Windows' NdisOpenFile / NdisMapFile: Reads files from in the kernel. Linux: request_firmware(): Signals a centralized user-space daemon (udev) to copy firmware image file into the kernel. FreeBSD: firmware(9) Makes it easy to wrap a firmware image in a loadable module, and allows a driver to easily load and unload the module on demand via firmware_get() and firmware_put(). These are something that it'd probably be useful to explore. Have you file a bug report requesting this? -- Terry Drew _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… Terry Lambert writes: If you need to do file I/O to load firmware (as an example), either put the firmware in a discardable section (vm_deallocate() it after it's loaded in the card), or have a user space daemon communicate the firmware to your driver via a series of ioctl() calls to push the data *temporarily* into the kernel, so it doesn't take up real or virtual memory in your driver. The normal way people avoid having to load the firmware from their driver is with a ROM. 8-). The normal way people support updating firmware is to replace that ROM with flash. For vendors which only put RAM on their hardware, and don't include either ROM or flash, the normal way you deal with this on MacOS X is to have a helper application in user space, and push the data down into the kernel from user space to the driver into a space temporarily allocated by the driver, and then discard the memory once it's loaded into the card. The proper way to do this is the Windows kernel is actually to mark the firmware section as "INITIALIZED_DATA" and "MEM_DISCARDABLE| MEM_READ" in the Portable Executable file (actually, extended COFF). Then when it's loaded in with PELoader, it's used during the init process, and then it's thrown away (exactly what would happen if you #pragma'ed it into it's own section in your KEXT to put it on a page boundary, and then called vm_deallocate() on it when you are done with it. Normally, Microsoft recommends against doing file I/O from their kernel as well. Realize that external bug reports get a lot more attention than ones Apple engineers file as "this would be cool", since external bug reports tend to be customer problems in search of a solution, rather than engineering solutions in search of a customer problem. This email sent to site_archiver(a)lists.apple.com

1 0

Re: File I/O in the kernel.
by Andrew Gallatin 24 May '06

24 May '06

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com Michael Smith writes: > > On May 24, 2006, at 11:52 AM, Andrew Gallatin wrote: > > > Terry Lambert writes: > >> If you need to do file I/O to load firmware (as an example), either > >> put the firmware in a discardable section (vm_deallocate() it after > >> it's loaded in the card), or have a user space daemon communicate the > >> firmware to your driver via a series of ioctl() calls to push the > >> data > >> *temporarily* into the kernel, so it doesn't take up real or virtual > >> memory in your driver. > > > > This comes up over, and over, and over again here. Is there any > > chance Apple can provide a centralized way to do this in the OS, > > Yes. It's called a user-space process. OK, what you mean is that it is called N user space process, a different one for each driver, all re-inventing the same wheel with different bugs. Drew _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… This email sent to site_archiver(a)lists.apple.com

1 0

Re: Write/Read Bulk communication process in KEXT. How?
by Michael Smith 24 May '06

24 May '06

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com On May 24, 2006, at 11:52 AM, Andrew Gallatin wrote: This comes up over, and over, and over again here. Is there any chance Apple can provide a centralized way to do this in the OS Yes. It's called a user-space process. = Mike _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… Terry Lambert writes: If you need to do file I/O to load firmware (as an example), either put the firmware in a discardable section (vm_deallocate() it after it's loaded in the card), or have a user space daemon communicate the firmware to your driver via a series of ioctl() calls to push the data *temporarily* into the kernel, so it doesn't take up real or virtual memory in your driver. This email sent to site_archiver(a)lists.apple.com

1 0

Re: Write/Read Bulk communication process in KEXT. How?
by Michael Smith 24 May '06

24 May '06

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com On May 24, 2006, at 11:34 AM, Terry Lambert wrote: = Mike _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… The general consensus is DO NOT DO FILE I/O FROM THE KERNEL. As far as I can tell, the OP is not trying to do file I/O, but rather USB transfers to/from a Bulk endpoint. This email sent to site_archiver(a)lists.apple.com

1 0

Re: Write/Read Bulk communication process in KEXT. How?
by Andrew Gallatin 24 May '06

24 May '06

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com Terry Lambert writes: > If you need to do file I/O to load firmware (as an example), either > put the firmware in a discardable section (vm_deallocate() it after > it's loaded in the card), or have a user space daemon communicate the > firmware to your driver via a series of ioctl() calls to push the data > *temporarily* into the kernel, so it doesn't take up real or virtual > memory in your driver. This comes up over, and over, and over again here. Is there any chance Apple can provide a centralized way to do this in the OS, so we don't all have to use our own home grown and (potentially) buggy or wasteful mechanisms? At least 3 other OSes provide a way for drivers to load firmware easily: Windows' NdisOpenFile / NdisMapFile: Reads files from in the kernel. Linux: request_firmware(): Signals a centralized user-space daemon (udev) to copy firmware image file into the kernel. FreeBSD: firmware(9) Makes it easy to wrap a firmware image in a loadable module, and allows a driver to easily load and unload the module on demand via firmware_get() and firmware_put(). Drew _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… This email sent to site_archiver(a)lists.apple.com

1 0

Re: Write/Read Bulk communication process in KEXT. How?
by Terry Lambert 24 May '06

24 May '06

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com On May 23, 2006, at 8:39 PM, Chinki wrote: Hi Garth, Thanks for your mail. The general consensus is DO NOT DO FILE I/O FROM THE KERNEL. The consequence of this is twofold: Etc.. After reading the documenation, i could get just the rough idea : But I am not sure if the above steps are correct. So... What problem are you trying to solve? -- Terry Thanks in advance, Chinki On 5/23/06, Garth Cummings <gcummings(a)apple.com> wrote: On May 23, 2006, at 6:53 AM, Chinki wrote: > Hello all, > > Greetings!!! > > Upon searching the archives, I found out that although this > question has been posted but there has been NO Answer for the same > till date. > > > I need to bulk write/read through my IO Driver (kext). Can any one > explain how to use IOMemoryDescriptor ? Any sample example that > illustrates the very simple communication. It sounds like you might not be aware of the existence of the Apple Developer reference library. I highly recommend you start at <http:// developer.apple.com/referencelibrary> and use the search engine on those pages to find what you're looking for. The reference library includes developer documentation, technical notes, sample code, and technical Q&As. This information is also on your Mac OS X machine at /Developer/ADC Reference Library if you have the development tools installed. You might also want to review the mailing list guidelines at <http:// lists.apple.com>. Note particularly the statement "While Apple employees monitor the lists, Apple doesn't guarantee that questions will be answered. The mailing lists are not a replacement for formal support or bug reporting procedures." If you need help beyond what the documentation and sample code provides, you can also make use of Apple Developer Technical Support. You can read more about this program at the URL in my email signature. Best wishes, --gc ____________________________________________________________________ Garth Cummings gcummings(a)apple.com Sr. Software Engineer Apple Developer Technical Support <http://developer.apple.com/technicalsupport> _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/tlambert%40apple.com _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… I had already searched the APPLE website and old archives but there is no sample provided. Believe me, but its truth that there are several queries for the same question. But no straight solution. Can ANY BODY FROM YOUR TEAM post a simple straight method how to use IOMemoryDescriptor to write / read data.\ The bulk write/read at USER-LEVEL framework is QUITE EASY. But at kernel level it is mentioned Write / Read take a pointer to IOMemoryDescriptor and its no where described how to exactly fill this structure. What needs to filled in this IOMemoryDescriptor ?What are the steps ? At Write function , Apple documentation tells nothing about the same. There are a number of companies that believe they can interpose block I/O drivers and substitute data on the way in and out of the device driver and/or monitor the contents of data transactions, and they come to this list and ask how they can write different data to a block device behind the kernel's back. They are mistaken, since it's not possible to do safely. Doing this ignores the fact that the kernel does caching, and they have no mechanism for cache invalidation in layers above themselves that they are interposing. Specifically, it ignores data and buffer caching that occurs at upper layers, or even in user space, and the fact that there's no invalidation mechanism to indicate to upper level consumers that their cached contents are stale and must be reloaded. For example, say you had an interposing layer whose intent was to encrypt directory entry names on the way to disk, and decrypt them on the way back out during lookup/readdir operations. The problem with such a layer is that by returning an unencrypted name to the upper layers of code, it will end up having the unencrypted name cached in the directory name cache. (1) If someone can get a handle to a file or directory in the volume, they can use an fcntl() to obtain an unencrypted path to the file. (2) Subsequent forward lookups for the unencrypted name will get hits in the directory name cache on their unencrypted values, and not actually hit the FS - which means they would not actually hit your encruption layer, and therefore the lookup would be successful on the unencrypted path, even if you didn't have the encryption keys (i.e. the name cache happens at a layer higher than your implementation, and the name cache, when you get a hit, does not hit the FS which means it doesn't hit your interposing block driver). In general, it's a *VERY* bad idea to do file I/O from the kernel. Instead, you should pass your information back to a user space daemon, and have it do the work for you, instead. If you need to do file I/O to load firmware (as an example), either put the firmware in a discardable section (vm_deallocate() it after it's loaded in the card), or have a user space daemon communicate the firmware to your driver via a series of ioctl() calls to push the data *temporarily* into the kernel, so it doesn't take up real or virtual memory in your driver. 1. Use prepare() to make sure that physical memory is availabefor i/ o transfer 2. Use writebytes/readbytes 3. Give the direction, kDirectionOut or kDirectionIn 4. Do i/o 5. complete() unwire the memory So i would request again to atleast give the exact path incase you have some simple example at your end. And if its not availalbe, then plz atleast paste a simple steps for the same, Again, in general, the question you are asking will be met with the question "what are you actually trying to do?". Knowing the problem you are trying to solve would enable us to provide you with pointers to alternate approaches to solving it - approaches that don't require the use of file I/O from the kernel. This email sent to tlambert(a)apple.com This email sent to site_archiver(a)lists.apple.com

1 0