April 2003 - Darwin-kernel - lists.apple.com

Re: File Access with VFS
by Scott Lance 10 Apr '03

10 Apr '03

The big picture is simply this: I am working on writing a driver for 802.11g support on Mac OS X, after reading the chip spec I found that I needed to take a firmware file and load it into the wireless card at initialization time. This must happen EVERY TIME the card is initialized (which btw seems very weird to me, I usually thought that firmware resided on the card and the only time firmware need to be loaded from outside the card is when a firmware update was applied), in any case I copied the firmware into a hardcoded byte array and wrote it onto the card. The problem with this is when a new firmware update comes out, I am going to have to go through a lengthy conversion process for the raw firmware file and then copy it into the code to run the card. What I want to do is be able to open up a firmware file when I start the driver do my conversions and load it onto the chip so all I will really have to do when the firmware update comes out is change a file name (either in the code or the filename of the firmware). In looking into this I came across the BSD functions, and yesterday looking for more answers I came across a message similar to what you just sent (basically saying that accessing these functions in a I/O Kit driver is just not a good idea). Later when I tried to access the vn_open function, my driver would not link (again something you mentioned as a problem). So my conundrum now is: what can I now do? What would be involved using the file I/O to pass this information from user space to my I/O Kit driver (and is this possible considering that I will be loading this driver at the OS boot time)? > ----- Original Message ----- > From: "Quinn" <eskimo1(a)apple.com> > To: "Darwin Kern Dev" <darwin-kernel(a)lists.apple.com> > Sent: Thursday, April 10, 2003 5:06 AM > Subject: RE: File Access with VFS > > > > At 10:37 -0700 9/4/03, Scott Lance wrote: > >I am working in the context of a I/O Kit Driver. I am getting access to > >these file operations through the Kernel.Framework (to include vnode.h, > >namei.h, uio.h, ect..), which from what I can tell is running from the > > BSD > >portions of the kernel (I'm not really sure how correct I am about that). > > Before you start you should read the Mac OS X section of DTS Technote > 2028 "Threading Architectures". > > <http://developer.apple.com/technotes/tn/tn2028.html> > > This has a short (but information dense) description of funnels. > > By default, I/O Kit drivers do not operate under either of the kernel > funnels. If your I/O Kit driver has to interoperate with BSD, such > as reading a file, it must take a funnel before entering the BSD. > For network operations, you take the network funnel, and for all > other BSD operations you take the kernel funnel. > > This makes your I/O Kit driver dependent on I/O Kit and BSD. BSD > provides much less of a binary compatibility guarantee than I/O Kit. > You are exposing yourself to an increased compatibility liability > when doing this. Moreover, it's vital that you express this > dependency in your KEXT's OSBundleLibraries property, so that if we > change BSD in an incompatible way your driver will refuse to load > (rather than load and panic). > > ><http://www.opensource.apple.com/projects/documentation/howto/html/kext_tut o > rials/kext_dependencies.html> > > Once you get past the above issues, the BSD functions to do file I/O > from in-kernel code are pretty standard. Relevant functions include: > > o NDINIT -- To initialise a struct nameidata from a path. > o vn_open -- To open the file; this gives you back a vnode. > o various vnode manipulation routines, for example, vn_lock, VOP_UNLOCK > o vn_close > o VOP_READ/VOP_WRITE -- To read and write data. > > This is not really an API. Your driver is basically calling a bunch > of BSD kernel functions that are exposed so that other parts of the > BSD kernel can call them. While it's likely that the overall > technique will continue to work for the foreseeable future, it's also > likely that changes will occur that break binary compatibility. > Don't say you haven't been warning (-: > > Finally, there are other, bigger, problems associated with accessing > the file system from within an I/O Kit driver. The kernel has a > locking hierarchy that it uses to prevent deadlocks. Calling the top > of the kernel (the BSD file system entry points) from the bottom (I/O > Kit) inverts that locking hierarchy, and can thus lead to nasty > deadlocks. There is no general way to avoid this. > > Because of these problems, Apple's advice is not to access the file > system from within an I/O Kit driver. If there is no alternative > (see below), you should limit the amount of file I/O you do. The > general rule is to move as much file I/O as possible into user space. > > There are a number of alternative designs that avoid the need for > file I/O from I/O Kit. If you can explain a little of the big > picture, I may be able to suggest other, more compatible, solutions. > > S+E > -- > Quinn "The Eskimo!" <http://www.apple.com/developer/> > Apple Developer Technical Support * Networking, Communications, Hardware > _______________________________________________ > darwin-kernel mailing list | darwin-kernel(a)lists.apple.com > Help/Unsubscribe/Archives: > http://www.lists.apple.com/mailman/listinfo/darwin-kernel > Do not post admin requests to the list. They will be ignored. _______________________________________________ darwin-kernel mailing list | darwin-kernel(a)lists.apple.com Help/Unsubscribe/Archives: http://www.lists.apple.com/mailman/listinfo/darwin-kernel Do not post admin requests to the list. They will be ignored.

1 0

Re: soreserve max_value?
by Justin Walker 10 Apr '03

10 Apr '03

On Thursday, Apr 10, 2003, at 03:47 US/Pacific, Stiphane Sudre wrote: I'm making some tests to see what the max_value can be for the rcv_cc for soreserve and it seems that 256 KB (I may be way too optimistic) is too big and 128 KB is OK. There seems to be a limit from what I can see in the source code of sbreserve but finding this limit is not that easy following the source. Another point is that there's a sysctl for the net.inet.raw.recvspace which is said to be 8192. Does someone know the MAX limit for the recv buff size of soreserve? You're right; the source is not easy to follow. There are two other 'recvspace' values, one for TCP, one for that other protocol: net.inet.tcp.recvspace: 32768 net.inet.udp.recvspace: 42080 These act as defaults, and you can change them via setsockopt(). I have this vestigial recollection that the max is limited, but I can't find anyplace in the source where this is done, and the sysctl doesn't complain if I set it to 1,000,000. Regards, Justin -- /~\ The ASCII Justin C. Walker, Curmudgeon-at-Large \ / Ribbon Campaign X Help cure HTML Email / \ _______________________________________________ darwin-kernel mailing list | darwin-kernel(a)lists.apple.com Help/Unsubscribe/Archives: http://www.lists.apple.com/mailman/listinfo/darwin-kernel Do not post admin requests to the list. They will be ignored.

1 0

RE: File Access with VFS
by Quinn 10 Apr '03

10 Apr '03

At 10:37 -0700 9/4/03, Scott Lance wrote: I am working in the context of a I/O Kit Driver. I am getting access to these file operations through the Kernel.Framework (to include vnode.h, namei.h, uio.h, ect..), which from what I can tell is running from the BSD portions of the kernel (I'm not really sure how correct I am about that). Before you start you should read the Mac OS X section of DTS Technote 2028 "Threading Architectures". <http://developer.apple.com/technotes/tn/tn2028.html> This has a short (but information dense) description of funnels. By default, I/O Kit drivers do not operate under either of the kernel funnels. If your I/O Kit driver has to interoperate with BSD, such as reading a file, it must take a funnel before entering the BSD. For network operations, you take the network funnel, and for all other BSD operations you take the kernel funnel. This makes your I/O Kit driver dependent on I/O Kit and BSD. BSD provides much less of a binary compatibility guarantee than I/O Kit. You are exposing yourself to an increased compatibility liability when doing this. Moreover, it's vital that you express this dependency in your KEXT's OSBundleLibraries property, so that if we change BSD in an incompatible way your driver will refuse to load (rather than load and panic). <http://www.opensource.apple.com/projects/documentation/howto/html/kext_tuto…> Once you get past the above issues, the BSD functions to do file I/O from in-kernel code are pretty standard. Relevant functions include: o NDINIT -- To initialise a struct nameidata from a path. o vn_open -- To open the file; this gives you back a vnode. o various vnode manipulation routines, for example, vn_lock, VOP_UNLOCK o vn_close o VOP_READ/VOP_WRITE -- To read and write data. This is not really an API. Your driver is basically calling a bunch of BSD kernel functions that are exposed so that other parts of the BSD kernel can call them. While it's likely that the overall technique will continue to work for the foreseeable future, it's also likely that changes will occur that break binary compatibility. Don't say you haven't been warning (-: Finally, there are other, bigger, problems associated with accessing the file system from within an I/O Kit driver. The kernel has a locking hierarchy that it uses to prevent deadlocks. Calling the top of the kernel (the BSD file system entry points) from the bottom (I/O Kit) inverts that locking hierarchy, and can thus lead to nasty deadlocks. There is no general way to avoid this. Because of these problems, Apple's advice is not to access the file system from within an I/O Kit driver. If there is no alternative (see below), you should limit the amount of file I/O you do. The general rule is to move as much file I/O as possible into user space. There are a number of alternative designs that avoid the need for file I/O from I/O Kit. If you can explain a little of the big picture, I may be able to suggest other, more compatible, solutions. S+E -- Quinn "The Eskimo!" <http://www.apple.com/developer/> Apple Developer Technical Support * Networking, Communications, Hardware _______________________________________________ darwin-kernel mailing list | darwin-kernel(a)lists.apple.com Help/Unsubscribe/Archives: http://www.lists.apple.com/mailman/listinfo/darwin-kernel Do not post admin requests to the list. They will be ignored.

1 0

soreserve max_value?
by Stéphane Sudre 10 Apr '03

10 Apr '03

I'm making some tests to see what the max_value can be for the rcv_cc for soreserve and it seems that 256 KB (I may be way too optimistic) is too big and 128 KB is OK. There seems to be a limit from what I can see in the source code of sbreserve but finding this limit is not that easy following the source. Another point is that there's a sysctl for the net.inet.raw.recvspace which is said to be 8192. Does someone know the MAX limit for the recv buff size of soreserve? _______________________________________________ darwin-kernel mailing list | darwin-kernel(a)lists.apple.com Help/Unsubscribe/Archives: http://www.lists.apple.com/mailman/listinfo/darwin-kernel Do not post admin requests to the list. They will be ignored.

1 0

Re: missing system headers
by Jim Riggs 10 Apr '03

10 Apr '03

Also, I receive a linker error for -ldisk when building mount_msdos. Has anybody attempted to make ufs a KEXT with mount command included, like msdos, that actually builds? I'm sure this has been done many times by people... For mount commands, you will need to build the disk library (as you can tell from your link error) which can be found in diskdev_cmds (diskdev_cmds/disklib, IIRC). Build that and then try building the mount command. It's been awhile, so I can't remember, but you may need to build some things in Libc also.... - Jim _______________________________________________ darwin-kernel mailing list | darwin-kernel(a)lists.apple.com Help/Unsubscribe/Archives: http://www.lists.apple.com/mailman/listinfo/darwin-kernel Do not post admin requests to the list. They will be ignored.

1 0

missing system headers
by Wally Crooze 10 Apr '03

10 Apr '03

G'day all, I recently downloaded the msdosfs to learn how to implement a kext (and also created a kext for nullfs). I attempted to build the msdosfs and received an error for missing header <miscfs/specfs/specdev.h> and <sys/signalvar.h>. I did a search in /System/Library/Frameworks and couldn't find them. They are in /usr/include... but I'm surprised they don't exist in the frameworks. P.S. Does this mean that I would expect all headers for filesystem/kernel development to be found in the Frameworks, rather than the /usr/include? Please explain. Also, I receive a linker error for -ldisk when building mount_msdos. Has anybody attempted to make ufs a KEXT with mount command included, like msdos, that actually builds? I'm sure this has been done many times by people... I'm not trying to be lazy but rather re-use the experiences and the product of others, instead of reinventing the brick over-again. Thanks. I appreciate any assistance and hopefully can return such in the future. -- Wally Crooze +-------------------------------------------------------+ | ,-_|\ Wally Crooze <wallycrooze(a)pobox.com> | | / \ | | \_,-._/ | | v | +-------------------------------------------------------+ _______________________________________________ darwin-kernel mailing list | darwin-kernel(a)lists.apple.com Help/Unsubscribe/Archives: http://www.lists.apple.com/mailman/listinfo/darwin-kernel Do not post admin requests to the list. They will be ignored.

1 0

RE: File Access with VFS
by Scott Lance 09 Apr '03

09 Apr '03

I am working in the context of a I/O Kit Driver. I am getting access to these file operations through the Kernel.Framework (to include vnode.h, namei.h, uio.h, ect..), which from what I can tell is running from the BSD portions of the kernel (I'm not really sure how correct I am about that). ----- Original Message ----- From: "Quinn" <eskimo1(a)apple.com> To: <darwin-kernel(a)lists.apple.com> Sent: Wednesday, April 09, 2003 9:44 AM Subject: Re: File Access with VFS > At 7:36 -0700 9/4/03, Scott Lance wrote: > >I am trying to open a file for manipulation in Kernel mode. After several > >hours of searching, limited documentation seems to suggest that I would be > >able to use a VNode to accomplish my objective. I have tried to find > >documentation on how this works and what I must do to be able to get a handle > >to a file and open it up for manipulation. Does anyone have any suggestions > >on where I can look for information on how vnodes and the VFS work? Or > >perhaps someone could enlighten me on what I must do to be able to open a file > >for use??? Or might there be a different way to open a file in Kernel mode > >that I've missed all together? > > There isn't a single "kernel mode" on Mac OS X. The answer differs > depending on whether you're running within BSD portions of the kernel > (indeed, what part of BSD), within an I/O Kit driver, and so on. > What context is your code executing in? > > S+E > -- > Quinn "The Eskimo!" <http://www.apple.com/developer/> > Apple Developer Technical Support * Networking, Communications, Hardware > _______________________________________________ > darwin-kernel mailing list | darwin-kernel(a)lists.apple.com > Help/Unsubscribe/Archives: http://www.lists.apple.com/mailman/listinfo/darwin-kernel > Do not post admin requests to the list. They will be ignored. _______________________________________________ darwin-kernel mailing list | darwin-kernel(a)lists.apple.com Help/Unsubscribe/Archives: http://www.lists.apple.com/mailman/listinfo/darwin-kernel Do not post admin requests to the list. They will be ignored.

1 0

Re: NKE socket communication and soreserve
by Stéphane Sudre 09 Apr '03

09 Apr '03

On mercredi, avr 9, 2003, at 18:57 Europe/Paris, Justin Walker wrote: On Wednesday, Apr 9, 2003, at 07:28 US/Pacific, Stiphane Sudre wrote: Does someone know what the purpose of soreserve is when it is used in the TCPLogger NKE communication sample code? For questions like these, the answer is UTSL! In xnu/bsd/kern/uipc_socket2.c, there is this comment, near the code for soreserve:[...] and On mercredi, avr 9, 2003, at 18:25 Europe/Paris, Joshua Graessley wrote: soreserve is used to set the maximum amount of data the socket will store in the socket buffer. Sockets have two buffers, a send buffer and a receive buffer. There is a s[...] Thanks for both answers, it definitely clarifies what values should be used there. This also explains the sbappend call in TCPLogger. I will file a bug report on the NKE doc for the typo and bad proto. _______________________________________________ darwin-kernel mailing list | darwin-kernel(a)lists.apple.com Help/Unsubscribe/Archives: http://www.lists.apple.com/mailman/listinfo/darwin-kernel Do not post admin requests to the list. They will be ignored.

1 0

Re: NKE socket communication and soreserve
by Justin Walker 09 Apr '03

09 Apr '03

On Wednesday, Apr 9, 2003, at 07:28 US/Pacific, Stiphane Sudre wrote: Does someone know what the purpose of soreserve is when it is used in the TCPLogger NKE communication sample code? For questions like these, the answer is UTSL! In xnu/bsd/kern/uipc_socket2.c, there is this comment, near the code for soreserve: * Before using a new socket structure it is first necessary to reserve * buffer space to the socket, by calling sbreserve(). This should commit * some of the available buffer space in the system buffer pool for the * socket (currently, it does nothing but enforce limits). The space * should be released by calling sbrelease() when the socket is destroyed. so the purpose is to check limits and reserve space (by calling sbreserve on each of the sockbuts') (:-}). And what is the meaning of the value used in it (the last 2)? As the comment says, each is a SWAG at the amount of space assumed to be needed in each sockbuf. Apropos of the 'limit' statement in the comment, note that the sockbuf limits don't involve allocation, and that the queue space they measure is a bit funky. The meaning of sockbuf limits is discussed in Stevens's TCP/IP Illustrated, V.2, and has been discussed on the darwin-development list a while back. There seems to be a bug in the NKE Documentation as the documentation is stating that the soreserve prototype is: soreserve(struct socket *, struct sockbut *); (yes, there's a typo too in the doc) when in the real world, it's not the case. Oops... The url of the buggy doc being: developer.apple.com/techpubs/macosx/Darwin/ IOKit/DeviceDrivers/Network/NKE.pdf Strangely enough, there's no man page for soreserve. How odd. I'm sure I had one when I got here. :-}. Cheers, Justin -- Justin C. Walker, Curmudgeon-At-Large * Institute for General Semantics | Some people have a mental | horizon of radius zero, and | call it their point of view. | -- David Hilbert *--------------------------------------*-------------------------------* _______________________________________________ darwin-kernel mailing list | darwin-kernel(a)lists.apple.com Help/Unsubscribe/Archives: http://www.lists.apple.com/mailman/listinfo/darwin-kernel Do not post admin requests to the list. They will be ignored.

1 0

Re: File Access with VFS
by Quinn 09 Apr '03

09 Apr '03

At 7:36 -0700 9/4/03, Scott Lance wrote: I am trying to open a file for manipulation in Kernel mode. After several hours of searching, limited documentation seems to suggest that I would be able to use a VNode to accomplish my objective. I have tried to find documentation on how this works and what I must do to be able to get a handle to a file and open it up for manipulation. Does anyone have any suggestions on where I can look for information on how vnodes and the VFS work? Or perhaps someone could enlighten me on what I must do to be able to open a file for use??? Or might there be a different way to open a file in Kernel mode that I've missed all together? There isn't a single "kernel mode" on Mac OS X. The answer differs depending on whether you're running within BSD portions of the kernel (indeed, what part of BSD), within an I/O Kit driver, and so on. What context is your code executing in? S+E -- Quinn "The Eskimo!" <http://www.apple.com/developer/> Apple Developer Technical Support * Networking, Communications, Hardware _______________________________________________ darwin-kernel mailing list | darwin-kernel(a)lists.apple.com Help/Unsubscribe/Archives: http://www.lists.apple.com/mailman/listinfo/darwin-kernel Do not post admin requests to the list. They will be ignored.

1 0