November 2006 - Darwin-kernel

Re: How to build a new ISO 9660 FS and test it
by Terry Lambert 15 Nov '06

15 Nov '06

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com On Nov 14, 2006, at 6:13 AM, Thomas Tempelmann wrote: Undefined symbols: _kld_load_basefile_from_memory _kld_set_link_options Looks like it was updated just a week ago. -- Terry _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… Hi, I've been writing ISO 9660 file system drivers for Mac OS 9 and for Toast, and I'd now like to bring OS X's old ISO 9660 driver up to date, with the hope that Apple will incorporate the improvements into OS X. As I've never done anything with Darwin before, I have a few beginner's problem. Please guide me a little: 1. All I need is to build the XNU sources, right? That gets me a new kernel which I can then boot from. But I wonder if, for testing, there is a better way, i.e. if I can exchange the file system alone at runtime without the need to rebuild the entire kernel? I've heard that it's possible to load file systems dynamically, I just wonder how hard it would be to turn that ISO file system from the kernel into a loadable one? Or is the one in the kernel not even the one OS X uses - is there another one that's loaded later? 2. I have some trouble building the kernel. The included README is not helpful. I found some tutorials, all several years old, on the web, which suggest that I need some extra files (cctools, relpath and some libs), which I all installed as well. Finally, I found out that I need to switch to gcc3.3 to compile it. So, with that, using the latest "xnu-792.12.6", I could make all sources, it appears, but it stops with these linker errors: At this, I am stuck. Any suggestions? You are missing libkld.a, which is where those symbols are exported. It's a static library (this is a kernel...), and you have to have it before you can build a kernel. Here's recent instructions on building a kernel: <http://developer.apple.com/documentation/Darwin/Conceptual/KernelProgrammin… > This should get you to the point you can build a kernel of your own that should be capable of booting. 3. BTW, one site, macosforge.org, announced already a 8.8.1 Darwin release, but it's not available from the apple darwin download pages yet (they only list 8.8). I tried to sign up with that site, even received a login, but my login is not accepted. Is that normal? Does it take a while? 4. Lastly, if I should succeed, what's the chances that Apple incorporates the improvements? How would I go about that? I've already filed a bug report about what I plan to fix, but I know of others who files related bug reports years ago and nothing ever changed with the iso 9660 file system. Normally, if you file a an enhancement request with bug reporter that includes source code that fixes some issue, it's _seriously_ looked at by the engineers who would otherwise have to do the work themselves. Include in the bug report contact information in case there are questions that need to be answered. Obviously we can't promise anything, but code contributions _are_ looked at closely and with great interest. This email sent to site_archiver(a)lists.apple.com

1 0

OT: Re: UDF file system driver
by Andrei E. Warkentin 14 Nov '06

14 Nov '06

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com Domainkey-signature: a=rsa-sha1; q=dns; c=nofws; s=beta; d=gmail.com; h=received:in-reply-to:references:mime-version:content-type:message-id:cc:content-transfer-encoding:from:subject:date:to:x-mailer; b=mUpGPLyz+7neax4YzwroOpBFOQoxckNjEVw3OEkv5cxd/goaZzguB085/649vc2lC04aAfxFRz9q98NJhBWMnRjRpvJhx3Hwj9V2lj6tb4/daeInpxHod1+eKa2SAZ07VvR82L6dtclvMyjD5EZ6UkRmeTIZtZMzpGNEhORpH/I= On 14.11.2006, at 15:06, Thomas Tempelmann wrote: Hi Quinn! Does anybody know if the Apple UDF driver is open source and if so which package it is in? _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… At 16:01 +0000 14/11/06, Jeremy Pereira wrote: It is not open source. Darn, because it could need some improvements as well, and I'd know how to, having written the UDF code for Toast and another UDF driver for Mac OS 8... (BTW, I had recently applied for a job at Apple, but they wouldn't even consider me, being from outside the US without a HB-1 visa. Now I'll consider New Zealand... :) That's awful. So basically either you get here yourself or you don't work for them at all? :( Thomas _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/ andrey.warkentin%40gmail.com This email sent to andrey.warkentin(a)gmail.com This email sent to site_archiver(a)lists.apple.com

1 0

Re: How to build a new ISO 9660 FS and test it
by Thomas Tempelmann 14 Nov '06

14 Nov '06

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com >> I know of others who files related bug reports years ago and >> nothing ever changed with the iso 9660 file system. > >They even dropped multisession support after 10.1.5. They did?? Oh my... To give an idea what I planned to do: 1. Support ISO Level 3. This is important to be able to read DirectCD discs (on Mac OS 9, it was possible to use my Joliet driver or the Adaptec UDF driver, which also I did write, but on OS X neither the ISO nor the UDF driver handle those discs) 2. Support for files > 4GB in size by using 64 bits for file sizes. This builds on top of the Level 3 support: It would allow to read files from a ISO formatted large disc (e.g. a DVD) with files > 4GB on them. This is achieved by using the fragmentation feature of Level 3 (also known as multi-extent). Windows XP supports this, and Toast might soon as well. It's a shame that OS X can't handle such files. 3. And while I'm at it, why not add support for old-style Apple Extensions in ISO 9660 volumes? 4. Lastly, fix Joliet related problems as indicated by William Kucharski (he's have to be more specific on it, though) BTW, thanks for all the other hints from all of who replied. Thomas _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… This email sent to site_archiver(a)lists.apple.com

1 0

Re: How to build a new ISO 9660 FS and test it
by Quinn 14 Nov '06

14 Nov '06

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com <http://developer.apple.com/samplecode/MFSLives/index.html> S+E -- Quinn "The Eskimo!" <http://www.apple.com/developer/> Apple Developer Relations, Developer Technical Support, Core OS/Hardware _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… At 19:55 +0100 14/11/06, Markus Hitter wrote: Most/all of the drivers moved out into dedicated kexts, see /System/Library/Extensions/..., ISO 9660 is still statically linked into the kernel. However, the actual matching of file systems to devices is handled by DiskArb, and it should be possible to use that to take over any ISO disks as they mount. Look at the current ISO 9660 file system bundle plist file ("/System/Library/Filesystems/cd9660.fs/Contents/Info.plist"). You'll see an FSProbeOrder on 3000. If you use a lower number, you should be called before the built-in file system. If you're wondering how all this fits together, take a look at MFSLives. This is an easy to understand and tweak example of a local file system. This email sent to site_archiver(a)lists.apple.com

1 0

Re: UDF file system driver
by Thomas Tempelmann 14 Nov '06

14 Nov '06

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com Hi Quinn! >At 16:01 +0000 14/11/06, Jeremy Pereira wrote: >>Does anybody know if the Apple UDF driver is open source and if so >>which package it is in? > >It is not open source. Darn, because it could need some improvements as well, and I'd know how to, having written the UDF code for Toast and another UDF driver for Mac OS 8... (BTW, I had recently applied for a job at Apple, but they wouldn't even consider me, being from outside the US without a HB-1 visa. Now I'll consider New Zealand... :) Thomas _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… This email sent to site_archiver(a)lists.apple.com

1 0

Re: How to build a new ISO 9660 FS and test it
by William Kucharski 14 Nov '06

14 Nov '06

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com On Tuesday, November 14, 2006, at 11:42AM, "Jonas Maebe" <jonas.maebe(a)elis.ugent.be> wrote: > >... one of which was written by Thomas Tempelmann, to whom you were >replying :) (Thwack, as Bill slaps his forehead...) Ironically, his is the very extension that I have long wished worked on Mac OS X. :D Teaches me to reply without googling the subject first. ;-) William Kucharski kucharsk(a)mac.com _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… This email sent to site_archiver(a)lists.apple.com

1 0

Re: IOAudioDevice Subclass KEXT not calling initHardware
by Jeff Moore 14 Nov '06

14 Nov '06

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com On Nov 13, 2006, at 7:54 PM, Jason Williams wrote: So here's my stuff I got thus far.... from the ESDAudioDriver.h file #define ESDAudioDriver org_tulg_driver_ESDAudioDriver class ESDAudioDriver : public IOAudioDevice ... continue with class def.... and from the Info.plist <key>IOKitPersonalities</key> <dict> <key>ESDAudioDriver</key> <dict> <key>CFBundleIdentifier</key> <string>org.tulg.driver.ESDAudioDriver</string> <key>IOClass</key> <string>org_tulg_driver_ESDAudioDriver</string> <key>IOMatchCategory</key> <string>ESDAudioDriver</string> <key>IOProviderClass</key> <string>IOResources</string> <key>IOReourceMatch</key> <string>IOKit</string> </dict> </dict> perhaps a second pair of eyes may see something I'm missing? I can send the full source if anyone would like to see. On 11/13/06, Jeff Moore <jcm10(a)apple.com> wrote: First off, the PhantomAudioDriver sample code was obsoleted a while back with the introduction of the AudioReflectorDriver in the CoreAudio SDK. You can find it on an OS X system in /Developer/ Examples/CoreAudio/HAL/AudioReflectorDriver. Also, if you have audio specific questions, as opposed to general kernel development questions, you should check out the CoreAudio-API mailing list on lists.apple.com. All that said, your current problem sounds to me like your driver isn't matching anything and so none of your kext's classes are being instantiated. Your matching information should be in the "IOKitPersonalities" dictionary of your kext's info.plist. Chances are that you lost that info in the shuffle from the original project to your freshly built project. As such, you should check over all the data in the info.plist file just to be sure it got transferred over correctly. On Nov 13, 2006, at 6:28 PM, Jason Williams wrote: > Now I posted earlier this month about building the example project > for a dummy IOAudioDevice. I got it to build, so I've gone on to > the next step which is to build my own modified version. I > basically copied most of the code from PhantomAudioDriver, which is > what builds and loads, and put in some simple stuff for an example > AudioEngine, and built it. I've gotten it as far as building > perfectly, so it seems, and loading it into the kernel with > kextload. Ran kextload with -t and all looks well. kextstat even > shows the driver loaded. > > Problem is, I put some debug log lines in initHardware in the > IOAudioDevice subclass, and nothing is coming out. Nor does my > device show up in the audio output pane in system prefs. The > difference is that I started with a clean Xcode Kernel EXT project > for the new version so I could understand the settings that went > into building a KEXT. According to the docs, when the kext is > loaded, the start() function on the super class calls initHardware > on the subclass. But it seems to not be doing so. Any thoughts/ > Ideas/Suggestions on where to look? I'm guessing it's a setting or > a something I am just missing, but I made sure I had everything set > right like it is on the PhantomAudioDriver, but to no avail. -- Jeff Moore Core Audio Apple _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… I don't see anything obviously wrong with your plist. Presumably all those fields match their counterparts in the other parts of the build. One thing you might want to do is to change the value for the key, IOResourceMatch, from IOKit to IOBSD. This will make the matching of your driver happen a bit later in the boot process so that all the dependencies are there. Probably won't make a difference in what you are doing, but it does produce a better citizen of a driver. After you kextload your driver, you should check the IORegistry with ioreg or IORegistryExplorer.app and see if any of your objects got instantiated. You might also want to check out this helpful document about debugging matching and loading problems: http://developer.apple.com/documentation/DeviceDrivers/Conceptual/WritingDe… Thanks jeff, I'll check out the list and the CoreAudio example too. I'm sorta working my way up through the Kernel Audio examples anyway. I just need to figure out where I am going wrong on this build... The plist file looks pretty much the same as the other with my minor modifications. This email sent to site_archiver(a)lists.apple.com

1 0

Re: How to build a new ISO 9660 FS and test it
by Ronnie Misra 14 Nov '06

14 Nov '06

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com On Nov 14, 2006, at 6:13 AM, Thomas Tempelmann wrote: # RELEASE = [... cd9660 ...] and remove cd9660. Ronnie _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… Hi, I've been writing ISO 9660 file system drivers for Mac OS 9 and for Toast, and I'd now like to bring OS X's old ISO 9660 driver up to date, with the hope that Apple will incorporate the improvements into OS X. As I've never done anything with Darwin before, I have a few beginner's problem. Please guide me a little: 1. All I need is to build the XNU sources, right? That gets me a new kernel which I can then boot from. But I wonder if, for testing, there is a better way, i.e. if I can exchange the file system alone at runtime without the need to rebuild the entire kernel? I've heard that it's possible to load file systems dynamically, I just wonder how hard it would be to turn that ISO file system from the kernel into a loadable one? Or is the one in the kernel not even the one OS X uses - is there another one that's loaded later? You should be able to turn off the ISO filesystem in the kernel by turning off the "cd9660" configuration option in MASTER.i386 and/or MASTER.ppc in the bsd/conf directory. Find the line that looks something like You can then try to turn cd9660 into a kext, but that might be difficult since it probably depends on having access to internal kernel functions that aren't exposed through KPIs... 4. Lastly, if I should succeed, what's the chances that Apple incorporates the improvements? How would I go about that? I've already filed a bug report about what I plan to fix, but I know of others who files related bug reports years ago and nothing ever changed with the iso 9660 file system. As others mentioned on this list, you should file a bug on <http:// bugreport.apple.com> and attach your patches. Also, mention it on the list and hopefully the Apple engineers on the list will try to champion your cause. :) This email sent to site_archiver(a)lists.apple.com

1 0

Re: How to build a new ISO 9660 FS and test it
by Markus Hitter 14 Nov '06

14 Nov '06

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com Am 14.11.2006 um 15:13 schrieb Thomas Tempelmann: I'd now like to bring OS X's old ISO 9660 driver up to date, with the hope that Apple will incorporate the improvements into OS X. There are several locations for drivers: - OpenFirmware: only used on ppc and by BootX, so dumped early. - The kernel it's self. Undefined symbols: _kld_load_basefile_from_memory _kld_set_link_options I tried to sign up with that site, even received a login, but my login is not accepted. Is that normal? 4. Lastly, if I should succeed, what's the chances that Apple incorporates the improvements? They even dropped multisession support after 10.1.5. Markus - - - - - - - - - - - - - - - - - - - Dipl. Ing. Markus Hitter http://www.jump-ing.de/ _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… Even if Apple doesn't, you'd be able to replace their driver on the fly by providing a version with a higher number. Ryan Rempel did this for XPostFacto and IIRC, a speech of him describing his work is floating around on the 'net. 1. All I need is to build the XNU sources, right? Most/all of the drivers moved out into dedicated kexts, see /System/ Library/Extensions/..., so there's likely no need to change the kernel at all. With some luck you can change the driver by unmounting the volume and unloading the .kext, then doing the reverse for the new driver. Being successful here obviously needs some patience, googling and Technical Note reading, as Apple's Darwin lists are full of straightforward and confused questions, vague and valueable hints ;-) 2. I have some trouble building the kernel. If you still want to build a kernel, download the sources, unpack them, type "make". You'll need a few additional tools as you found out already. There's a way with even less typing, using darwinbuild, described at Mac OS Forge, OpenDarwin's successor. If you still experience trouble, please note wether you attempt to build for ppc or i386, as both used different sources after the launch of Intel Macs and until recently and differ in some technologies, e.g. all this TPM stuff, as well. Or is the [ISO driver] in the kernel not even the one OS X uses - is there another one that's loaded later? - The bootloader, alias BootX. This part is dumped as soon as the kernel loads, AFAIK. - EFI: initially similar to OpenFirmware on i386, but I've read rumours about high performance driver sighted there, effectively replacing OS drivers. Only unconfirmed rumours, though. - Kernel Extensions (kexts). Find all the .kext files on your system, examine their name for broad hints; their Info.plists for more detailed hints. Many drivers were moved out of the kernel and some of them into kexts between public beta and Mac OS X 10.0. Maybe this ist the reason why Mac OS Forge announced Darwin 8.8.1 rarther than 8.8? My subscription went flawlessly; pinging one of the administrators there might help. As the current driver works sufficiently in Apple's opinion, I wouldn't expect them to invest efforts here. How would I go about that? Make the thing public, similar to DarwinPorts, Unsanities Haxies, etc. IMHO the best one can do. I know of others who files related bug reports years ago and nothing ever changed with the iso 9660 file system. This email sent to site_archiver(a)lists.apple.com

1 0

Re: How to build a new ISO 9660 FS and test it
by Jonas Maebe 14 Nov '06

14 Nov '06

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com On 14 Nov 2006, at 19:30, William Kucharski wrote: The official channel for submitting patches is bugreporter. Jonas _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… I'm not aware of an actual contributor form, so it may be that the open source channel is one way from Apple, but it certainly couldn't hurt. As long as you're in the ISO9660 code, perhaps you could improve support for the Joliet extensions; the Mac OS X code is a bit lacking there, such that CDs burned from PCs are often missing large swaths of files when inserted into a Mac running OS X; the same CDs often show all their files to Macs running Mac OS 9 natively, as there were many free and commercial ISO9660 extensions that did fully support Joliet... ... one of which was written by Thomas Tempelmann, to whom you were replying :) This email sent to site_archiver(a)lists.apple.com

1 0