April 2003 - Darwin-kernel - lists.apple.com

Re: File Access with VFS
by Michael Cashwell 12 Apr '03

12 Apr '03

Greetings all, I feel the need to chime in here. I worked on a Darwin / MacOS X driver for Lucent -> Agere Orinoco cards. (There was little company support for the project, it was never finished, and I no longer work there.) But I also wrote the last major update to the MacOS 8/9 version (March of 2002) and one of the features was this sort of firmware download. I cannot speak to the architecture of the cards in question but in the Orinoco case there was a shift to downloading the firmware into RAM and not messing with the FLASH. (The code runs out of RAM anyway for performance reasons.) If your card is similar then do keep in mind that the power to PC Card / Cardbus slots is cut off when the host system sleeps. And even if it weren't, the user can pull the card out (and swap it with another) while asleep! So assuming the OpenFirmware case is out of the running (as it sounds) and the firmware runs out of card RAM this focus on "boot time" is somewhat misplaced. While the kext indeed needs to load the firmware during system boot, the trigger for this (IOKit matching) is essentially identical to what needs to happen when the card is hot-inserted or merely present when waking from sleep. All three cases should be covered by a unified technique which of necessity needs to be mostly independent of system boot and StartupItems. (That may be one way to get the firmware into the kernel space without reading a file within the kernel but you'd have to hold it there to deal with the other cases.) That said, everything Godfrey said below makes sense to me. Just don't get overly focused on system boot time since that's not really the issue. Best of luck! Michael Cashwell On Friday, April 11, 2003, at 02:50 PM, Godfrey van der Linden wrote: Actually Scott, This is good news. There are 3 major phases in a boot. 1> The firmware. For a driver to work here it MUST have an open firmware driver, i.e. Forth The firmware is actually 2 phases as well. First the rom loads the OS relocator and loader, i.e. the OS Booter. The OSBooter then loads the kernel and all drivers (kexts) that claim to be required i.e. all drivers that have the appropriate OSBundleRequired settings. 2> The boot to single user. This phase of the boot starts the OS proper up, i.e. VM, threading and Unix Init, and brings enough of the system up to mount the root device, usually the disk but if we have a firmware enabled network driver then we can 'netboot'. This part of the boot is what I was loosely calling the boot and where we are confusing nomenclature. 3> The remainder of the boot to multi-user, aka login window. This phase does many jobs but briefly it loads all drivers that aren't required for the root device or single user. Then it mounts the non-root filesystems and starts the networking up. So stage three is still a boot loaded driver but it isn't the same as being required for rooting, sorry for the confusion. Stage three is where your driver is should load. Fortunately IOKit is designed for exactly this phase. Load you driver normally as a 'late' loadable kext. Then setup a StartupItem, see /System/Library/StartupItems for an example on how do to this. I recommend using the IOService::setProperties subclass to get the load the firmware. Your driver will overload this function, and your startup script will discover the driver in the registry and send down a firmware by using the IORegistryEntrySetCFProperties() function. The data will be read into your helper APPs memory, or mapped if you wish to use mmap(). Then you wrap the address in a CFData object. In the kernel you grab the corresponding OSData object and create an IOMemoryDescriptor for the current_task() in your setProperty. There is an example of a simple implementation of IOService::setProperties in the IOSerialFamily/IOSerialBSDClient.cpp. Yes this example is a User Client but remember that the setProperties call can be passed to ANY object in the registry. I know I have been a little handwavy in my descriptions here. This technique will work but you will have to do some research. If you have any questions don't hesitate. Godfrey At 11:25 -0700 03-4-11, Scott Lance wrote: The device is a 802.11g cardbus adapter. No I don't think that this driver is netbootable, as for the firmware, I didn't write it, it was obtained from the company who made the chip (as will be all the updates). In anycase I would like to be able to turn on my laptop with the card in and have it load during boot time (as for it being required, yes it is, and well lets just say I don't make those decisions) ;) _______________________________________________ darwin-kernel mailing list | darwin-kernel(a)lists.apple.com Help/Unsubscribe/Archives: http://www.lists.apple.com/mailman/listinfo/darwin-kernel Do not post admin requests to the list. They will be ignored. _______________________________________________ darwin-kernel mailing list | darwin-kernel(a)lists.apple.com Help/Unsubscribe/Archives: http://www.lists.apple.com/mailman/listinfo/darwin-kernel Do not post admin requests to the list. They will be ignored.

1 0

Re: File Access with VFS
by Godfrey van der Linden 11 Apr '03

11 Apr '03

Actually Scott, This is good news. There are 3 major phases in a boot. 1> The firmware. For a driver to work here it MUST have an open firmware driver, i.e. Forth The firmware is actually 2 phases as well. First the rom loads the OS relocator and loader, i.e. the OS Booter. The OSBooter then loads the kernel and all drivers (kexts) that claim to be required i.e. all drivers that have the appropriate OSBundleRequired settings. 2> The boot to single user. This phase of the boot starts the OS proper up, i.e. VM, threading and Unix Init, and brings enough of the system up to mount the root device, usually the disk but if we have a firmware enabled network driver then we can 'netboot'. This part of the boot is what I was loosely calling the boot and where we are confusing nomenclature. 3> The remainder of the boot to multi-user, aka login window. This phase does many jobs but briefly it loads all drivers that aren't required for the root device or single user. Then it mounts the non-root filesystems and starts the networking up. So stage three is still a boot loaded driver but it isn't the same as being required for rooting, sorry for the confusion. Stage three is where your driver is should load. Fortunately IOKit is designed for exactly this phase. Load you driver normally as a 'late' loadable kext. Then setup a StartupItem, see /System/Library/StartupItems for an example on how do to this. I recommend using the IOService::setProperties subclass to get the load the firmware. Your driver will overload this function, and your startup script will discover the driver in the registry and send down a firmware by using the IORegistryEntrySetCFProperties() function. The data will be read into your helper APPs memory, or mapped if you wish to use mmap(). Then you wrap the address in a CFData object. In the kernel you grab the corresponding OSData object and create an IOMemoryDescriptor for the current_task() in your setProperty. There is an example of a simple implementation of IOService::setProperties in the IOSerialFamily/IOSerialBSDClient.cpp. Yes this example is a User Client but remember that the setProperties call can be passed to ANY object in the registry. I know I have been a little handwavy in my descriptions here. This technique will work but you will have to do some research. If you have any questions don't hesitate. Godfrey At 11:25 -0700 03-4-11, Scott Lance wrote: The device is a 802.11g cardbus adapter. No I don't think that this driver is netbootable, as for the firmware, I didn't write it, it was obtained from the company who made the chip (as will be all the updates). In anycase I would like to be able to turn on my laptop with the card in and have it load during boot time (as for it being required, yes it is, and well lets just say I don't make those decisions) ;) _______________________________________________ darwin-kernel mailing list | darwin-kernel(a)lists.apple.com Help/Unsubscribe/Archives: http://www.lists.apple.com/mailman/listinfo/darwin-kernel Do not post admin requests to the list. They will be ignored.

1 0

Re: File Access with VFS
by Scott Lance 11 Apr '03

11 Apr '03

The device is a 802.11g cardbus adapter. No I don't think that this driver is netbootable, as for the firmware, I didn't write it, it was obtained from the company who made the chip (as will be all the updates). In anycase I would like to be able to turn on my laptop with the card in and have it load during boot time (as for it being required, yes it is, and well lets just say I don't make those decisions) ;) Scott Lance ----- Original Message ----- From: "Godfrey van der Linden" <gvdl(a)apple.com> To: "Scott Lance" <scottdlance(a)hotmail.com> Cc: "Darwin Kern Dev" <darwin-kernel(a)lists.apple.com> Sent: Friday, April 11, 2003 10:46 AM Subject: Re: File Access with VFS > G'day Scott, > > Does your device change its matching parameters with the firmware > download? What sort of device is it, i.e. PCI, firewire etc. > > In what way can we boot of your device. Can we net boot from your > network driver? Do you have the appropriate firmware to make it > accessible to OpenFirmware? > > If you don't and you don't intend to write it in the future then your > best bet is not to be a boot driver at all. Then you can rely on a > single setProperties from your StartupItem script. If you are not a > driver REQUIRED for booting this is the easiest thing to do. > > Godfrey van der Linden > > At 10:14 -0700 03-4-11, Scott Lance wrote: > >What I am currently doing is copying the firmware into big firmware arrays > >in a C file, then loading all the firmware information when I initialize, I > >guess it would make the driver a lot smaller in size if I created a 2nd > >module, but for right now (in this development iteration) it is a lot easier > >(and a lot less messy) to do it the way I currently am. Also, I don't > >really have to worry about releasing any other module, but in answer to your > >question Quinn and DTS seem to indicate that it is possible in Darwin. > >----- Original Message ----- > >From: "Andrew Gallatin" <gallatin(a)cs.duke.edu> > >To: "Scott Lance" <scottdlance(a)hotmail.com> > >Cc: "Darwin Kern Dev" <darwin-kernel(a)lists.apple.com> > >Sent: Friday, April 11, 2003 10:04 AM > >Subject: Re: File Access with VFS > > > > > >> > >> In addition to statically compiling and having a user-client, would it > >> be possible to use a second kernel module? > >> > >> Eg, we have the "real" device driver, and we have a module which > >> consists of nothing but firmware arrays. When the device is probed, > >> both modules are loaded (the driver moduke would "somehow" depend on > >> the firmware module). The driver then figures out what device(s) is > >> present, and copies the appropriate firmware array to malloced memory > >> in the driver. It then "somehow" releases the firmware module, and > >> that module is unloaded, saving memory. If a similar device was added > >> to the system, the firmware module would "somehow" get re-loaded, and > >> the new instance of the device would copy the firmware, as above.. > >> > >> As you can see by the "somehows", I'm a little fuzzy on the details. > > > Is a scheme like this possible in Darwin? _______________________________________________ darwin-kernel mailing list | darwin-kernel(a)lists.apple.com Help/Unsubscribe/Archives: http://www.lists.apple.com/mailman/listinfo/darwin-kernel Do not post admin requests to the list. They will be ignored.

1 0

Re: File Access with VFS
by Godfrey van der Linden 11 Apr '03

11 Apr '03

G'day Scott, Does your device change its matching parameters with the firmware download? What sort of device is it, i.e. PCI, firewire etc. In what way can we boot of your device. Can we net boot from your network driver? Do you have the appropriate firmware to make it accessible to OpenFirmware? If you don't and you don't intend to write it in the future then your best bet is not to be a boot driver at all. Then you can rely on a single setProperties from your StartupItem script. If you are not a driver REQUIRED for booting this is the easiest thing to do. Godfrey van der Linden At 10:14 -0700 03-4-11, Scott Lance wrote: What I am currently doing is copying the firmware into big firmware arrays in a C file, then loading all the firmware information when I initialize, I guess it would make the driver a lot smaller in size if I created a 2nd module, but for right now (in this development iteration) it is a lot easier (and a lot less messy) to do it the way I currently am. Also, I don't really have to worry about releasing any other module, but in answer to your question Quinn and DTS seem to indicate that it is possible in Darwin. ----- Original Message ----- From: "Andrew Gallatin" <gallatin(a)cs.duke.edu> To: "Scott Lance" <scottdlance(a)hotmail.com> Cc: "Darwin Kern Dev" <darwin-kernel(a)lists.apple.com> Sent: Friday, April 11, 2003 10:04 AM Subject: Re: File Access with VFS In addition to statically compiling and having a user-client, would it be possible to use a second kernel module? Eg, we have the "real" device driver, and we have a module which consists of nothing but firmware arrays. When the device is probed, both modules are loaded (the driver moduke would "somehow" depend on the firmware module). The driver then figures out what device(s) is present, and copies the appropriate firmware array to malloced memory in the driver. It then "somehow" releases the firmware module, and that module is unloaded, saving memory. If a similar device was added to the system, the firmware module would "somehow" get re-loaded, and the new instance of the device would copy the firmware, as above.. As you can see by the "somehows", I'm a little fuzzy on the details. > Is a scheme like this possible in Darwin? _______________________________________________ darwin-kernel mailing list | darwin-kernel(a)lists.apple.com Help/Unsubscribe/Archives: http://www.lists.apple.com/mailman/listinfo/darwin-kernel Do not post admin requests to the list. They will be ignored.

1 0

Re: File Access with VFS
by Scott Lance 11 Apr '03

11 Apr '03

What I am currently doing is copying the firmware into big firmware arrays in a C file, then loading all the firmware information when I initialize, I guess it would make the driver a lot smaller in size if I created a 2nd module, but for right now (in this development iteration) it is a lot easier (and a lot less messy) to do it the way I currently am. Also, I don't really have to worry about releasing any other module, but in answer to your question Quinn and DTS seem to indicate that it is possible in Darwin. ----- Original Message ----- From: "Andrew Gallatin" <gallatin(a)cs.duke.edu> To: "Scott Lance" <scottdlance(a)hotmail.com> Cc: "Darwin Kern Dev" <darwin-kernel(a)lists.apple.com> Sent: Friday, April 11, 2003 10:04 AM Subject: Re: File Access with VFS > > In addition to statically compiling and having a user-client, would it > be possible to use a second kernel module? > > Eg, we have the "real" device driver, and we have a module which > consists of nothing but firmware arrays. When the device is probed, > both modules are loaded (the driver moduke would "somehow" depend on > the firmware module). The driver then figures out what device(s) is > present, and copies the appropriate firmware array to malloced memory > in the driver. It then "somehow" releases the firmware module, and > that module is unloaded, saving memory. If a similar device was added > to the system, the firmware module would "somehow" get re-loaded, and > the new instance of the device would copy the firmware, as above.. > > As you can see by the "somehows", I'm a little fuzzy on the details. > Is a scheme like this possible in Darwin? > > Drew _______________________________________________ darwin-kernel mailing list | darwin-kernel(a)lists.apple.com Help/Unsubscribe/Archives: http://www.lists.apple.com/mailman/listinfo/darwin-kernel Do not post admin requests to the list. They will be ignored.

1 0

Re: File Access with VFS
by Andrew Gallatin 11 Apr '03

11 Apr '03

In addition to statically compiling and having a user-client, would it be possible to use a second kernel module? Eg, we have the "real" device driver, and we have a module which consists of nothing but firmware arrays. When the device is probed, both modules are loaded (the driver moduke would "somehow" depend on the firmware module). The driver then figures out what device(s) is present, and copies the appropriate firmware array to malloced memory in the driver. It then "somehow" releases the firmware module, and that module is unloaded, saving memory. If a similar device was added to the system, the firmware module would "somehow" get re-loaded, and the new instance of the device would copy the firmware, as above.. As you can see by the "somehows", I'm a little fuzzy on the details. Is a scheme like this possible in Darwin? Drew _______________________________________________ darwin-kernel mailing list | darwin-kernel(a)lists.apple.com Help/Unsubscribe/Archives: http://www.lists.apple.com/mailman/listinfo/darwin-kernel Do not post admin requests to the list. They will be ignored.

1 0

Re: File Access with VFS
by Scott Lance 11 Apr '03

11 Apr '03

I have been working on writing a user client to send the firmware to the kernel, but here is the $42 dollar question: Will I be able to use my user client when the driver kext is loading at boot time? The spec. indicates that the firmware must be read into the device at the device's initialization time (boot time), so is it going to be possible to use a user client? As for the 2nd option, that's close to what I am already doing, and I guess it would be possible to write a little util. to read out the firmware from the file and put it into a C file. I guess this is the only option if I cannot use the user client. In any case, I appreciate the time and effort you have put into helping me with this problem. Thanks, Scott Lance ----- Original Message ----- From: "Quinn" <eskimo1(a)apple.com> To: "Darwin Kern Dev" <darwin-kernel(a)lists.apple.com> Sent: Friday, April 11, 2003 9:02 AM Subject: Re: File Access with VFS > At 13:08 +0100 11/4/03, Quinn wrote: > >Ah. You're not the first developer to hit this. This has been > >answered by DTS before. I'm not 100% sure of the current > >recommendation. I'll check up on this and get back to the list. > > Here's the current advice from the DTS I/O Kit experts on this topic. > > >He has two options (beside the skanky try-to-read-a-file-in-the-kernel): > > > >1. Write a userland process that downloads the firmware. This means > >that there must be some property in the I/O Registry that differs > >between the "raw" device state (no firmware) and the > >firmware-uploaded state. The example > >/Developer/Examples/IOKit/usb/Ezloader\ Example shows how to do this > >for certain USB devices. > > > >If the bus family for his device doesn't provide user client access, > >he can use the I/O Registry as a communications pipe if the firmware > >isn't too large. The DTS sample VendorSpecificType00 shows how to do > >this. Of course, if he has a custom kernel driver, he can write a > >user client for it to provide a way to send firmware to the kernel. > > > >2. If he wants to do this from the kernel, he can write a utility > >that reads his firmware file and generates a C source file with that > >firmware encoded as a character array. He then would build a subkext > >with a ::start routine that loads the firmware into his device and > >returns NULL. This subkext would match to his device in its > >uninitialized state. The fact the driver fails to start will cause > >it to be unloaded from the kernel after about a minute. > > S+E > -- > Quinn "The Eskimo!" <http://www.apple.com/developer/> > Apple Developer Technical Support * Networking, Communications, Hardware > _______________________________________________ > darwin-kernel mailing list | darwin-kernel(a)lists.apple.com > Help/Unsubscribe/Archives: http://www.lists.apple.com/mailman/listinfo/darwin-kernel > Do not post admin requests to the list. They will be ignored. _______________________________________________ darwin-kernel mailing list | darwin-kernel(a)lists.apple.com Help/Unsubscribe/Archives: http://www.lists.apple.com/mailman/listinfo/darwin-kernel Do not post admin requests to the list. They will be ignored.

1 0

Re: File Access with VFS
by Quinn 11 Apr '03

11 Apr '03

At 13:08 +0100 11/4/03, Quinn wrote: Ah. You're not the first developer to hit this. This has been answered by DTS before. I'm not 100% sure of the current recommendation. I'll check up on this and get back to the list. Here's the current advice from the DTS I/O Kit experts on this topic. He has two options (beside the skanky try-to-read-a-file-in-the-kernel): 1. Write a userland process that downloads the firmware. This means that there must be some property in the I/O Registry that differs between the "raw" device state (no firmware) and the firmware-uploaded state. The example /Developer/Examples/IOKit/usb/Ezloader\ Example shows how to do this for certain USB devices. If the bus family for his device doesn't provide user client access, he can use the I/O Registry as a communications pipe if the firmware isn't too large. The DTS sample VendorSpecificType00 shows how to do this. Of course, if he has a custom kernel driver, he can write a user client for it to provide a way to send firmware to the kernel. 2. If he wants to do this from the kernel, he can write a utility that reads his firmware file and generates a C source file with that firmware encoded as a character array. He then would build a subkext with a ::start routine that loads the firmware into his device and returns NULL. This subkext would match to his device in its uninitialized state. The fact the driver fails to start will cause it to be unloaded from the kernel after about a minute. S+E -- Quinn "The Eskimo!" <http://www.apple.com/developer/> Apple Developer Technical Support * Networking, Communications, Hardware _______________________________________________ darwin-kernel mailing list | darwin-kernel(a)lists.apple.com Help/Unsubscribe/Archives: http://www.lists.apple.com/mailman/listinfo/darwin-kernel Do not post admin requests to the list. They will be ignored.

1 0

Re: File Access with VFS
by Quinn 11 Apr '03

11 Apr '03

At 8:18 -0700 10/4/03, Scott Lance wrote: The big picture is simply this: I am working on writing a driver for 802.11g support on Mac OS X, after reading the chip spec I found that I needed to take a firmware file and load it into the wireless card at initialization time. Ah. You're not the first developer to hit this. This has been answered by DTS before. I'm not 100% sure of the current recommendation. I'll check up on this and get back to the list. S+E -- Quinn "The Eskimo!" <http://www.apple.com/developer/> Apple Developer Technical Support * Networking, Communications, Hardware _______________________________________________ darwin-kernel mailing list | darwin-kernel(a)lists.apple.com Help/Unsubscribe/Archives: http://www.lists.apple.com/mailman/listinfo/darwin-kernel Do not post admin requests to the list. They will be ignored.

1 0

Re: missing system headers
by Brian Bergstrand 10 Apr '03

10 Apr '03

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Wednesday, April 9, 2003, at 09:32 PM, Wally Crooze wrote: > G'day all, > > I recently downloaded the msdosfs to learn how to implement a kext > (and also created a kext for nullfs). I attempted to build the msdosfs > and received an error for missing header <miscfs/specfs/specdev.h> and > <sys/signalvar.h>. I did a search in /System/Library/Frameworks and > couldn't find them. They are in /usr/include... but I'm surprised they > don't exist in the frameworks. > specdev.h is not part of the kernel API, so it is not included in the Kernel framework. You will need to grab this from Darwin CVS. As for signalvar.h, Apple screwed up in the beta's of Jaguar (May 2002) and removed 5-10 important system headers from the Kernel Framework. A year and three Dev Tool release later, and this has still not been fixed. So you will need to grab signalvar.h (and a few others) from Darwin CVS and manually copy them to the Kernel framework. > P.S. Does this mean that I would expect all headers for > filesystem/kernel development to be found in the Frameworks, rather > than the /usr/include? Please explain. > No, never use userland includes for KEXT development. Grab the missing headers from xnu in CVS. > Also, I receive a linker error for -ldisk when building mount_msdos. > You need to get the disk lib project from CVS and build it. > Has anybody attempted to make ufs a KEXT with mount command included, > like msdos, that actually builds? I'm sure this has been done many > times by people... > Actually, it hasn't been done a lot. There are a few network FS extensions from third parties, and PGP has done it. Those along with my implementation of Ext2 are about it. You may want to look at Ext2 FS to get an idea of what third parties have to go through to build a Filesystem driver; MSDOSFS is from Apple so you don't quite get the whole picture. <http://sourceforge.net/projects/ext2fsx> Look at <http://cvs.sourceforge.net/cgi-bin/viewcvs.cgi/ext2fsx/Build.rtf> for build dependencies. > I'm not trying to be lazy but rather re-use the experiences and the > product of others, instead of reinventing the brick over-again. > > Thanks. I appreciate any assistance and hopefully can return such in > the future. > HTH. Brian Bergstrand <http://www.classicalguitar.net/brian/> PGP Key ID: 0xB6C7B6A2 It is easy to convert a bug to a feature; document it. - The Microsoft Employee Handbook, p.215, section B, article VII -----BEGIN PGP SIGNATURE----- Version: PGP 8.0.2 iQA/AwUBPpXihXnR2Fu2x7aiEQIy0QCg+X/7xsOhwkDQwDUqmYzfYBysuuUAoMd4 JSWlSrJSAQ5f0hfRim5ffmkK =UlPk -----END PGP SIGNATURE----- _______________________________________________ darwin-kernel mailing list | darwin-kernel(a)lists.apple.com Help/Unsubscribe/Archives: http://www.lists.apple.com/mailman/listinfo/darwin-kernel Do not post admin requests to the list. They will be ignored.

1 0