July 2004 - Darwin-kernel - lists.apple.com

Re: Info regarding module and kernel building
by Justin Walker 13 Jul '04

13 Jul '04

On Jul 13, 2004, at 2:40, Manjunath wrote: Is there any concept of module in darwin kernel similar to linux? Also can i configure the darwin kernel and then build the same? If i can configure the kernel how do i do it? To add to Quinn's reply, the Darwin kernel is pretty much complete as it comes from the factory. There are no source-level modules that you can enable/disable during the kernel config/build cycle. The only reason to build your own is to fix a bug or make a change for your own purposes. Note that there may be #defines in the source to enable or disable some piece of code, but those situations are completely untested, and may not even build; if they do build, they may not run; and if they run, they may chew up your disk and spit the remains out in your lap. :-}. Regards, Justin -- Justin C. Walker, Curmudgeon-At-Large * Institute for General Semantics | If you're not confused, | You're not paying attention *--------------------------------------*-------------------------------* _______________________________________________ darwin-kernel mailing list | darwin-kernel(a)lists.apple.com Help/Unsubscribe/Archives: http://www.lists.apple.com/mailman/listinfo/darwin-kernel Do not post admin requests to the list. They will be ignored.

1 0

Re: Finder Get Info not allowing permission change with CXFS
by Quinn 13 Jul '04

13 Jul '04

At 16:57 +1000 13/7/04, Nick Blievers wrote: Hmmm, I tried setting the vnode and getattrlist to return VT_HFS rather than VT_OTHER... but finder still reported 'Unknown'. What does 'Format' get set from? It's quite complex but the basic idea is that Finder call FSGetVolumeInfo and tests the filesystemID field of the returned FSVolumeInfo structure to determine whether the detailed privileges should be shown. FSGetVolumeInfo is implemented by the Core Services File Manager. It derives the filesystemID in two different ways, depending on whether the volume is volfs-based or Posix-based, but the basic algorithm is the same: it uses a bunch of special cases based on the value returned in the f_fstypename field of the statfs structure returned by statfs Clearly this is unsatisfactory. I don't think that you'll be able to work around this in your file system without claiming that you're HFS, which is not a good idea. Thanks for the Radar number. I've commented the bug appropriately. S+E -- Quinn "The Eskimo!" <http://www.apple.com/developer/> Apple Developer Technical Support * Networking, Communications, Hardware _______________________________________________ darwin-kernel mailing list | darwin-kernel(a)lists.apple.com Help/Unsubscribe/Archives: http://www.lists.apple.com/mailman/listinfo/darwin-kernel Do not post admin requests to the list. They will be ignored.

1 0

Re: Info regarding module and kernel building
by Quinn 13 Jul '04

13 Jul '04

At 15:10 +0530 13/7/04, Manjunath wrote: Is there any concept of module in darwin kernel similar to linux? The Darwin kernel supports loadable kernel extensions, KEXTs. There's a bunch of documentation about this on the Apple developer web site. <http://developer.apple.com/documentation/Darwin/Kernel-date.html#//apple_re…> Also can i configure the darwin kernel and then build the same? If i can configure the kernel how do i do it? I don't really understand your question. You can build the Darwin kernel from source if you like, but you don't need to do this in order to build and load a kernel extension. Similarly, you don't need to specifically configure the kernel in order to load a KEXT. Some classes of KEXTs are loaded on demand when the appropriate hardware is found (I/O Kit) and others can be loaded programmatically at any time. If you're looking for more pointers about implementing a kernel extension, you'll have to provide us more information about what you're trying to do. There are three basic classes of KEXTs on Mac OS X, I/O Kit drivers, VFS plug-ins, and Network Kernel Extensions (NKEs). Your first step is to decide what class of KEXT you want to be. S+E -- Quinn "The Eskimo!" <http://www.apple.com/developer/> Apple Developer Technical Support * Networking, Communications, Hardware _______________________________________________ darwin-kernel mailing list | darwin-kernel(a)lists.apple.com Help/Unsubscribe/Archives: http://www.lists.apple.com/mailman/listinfo/darwin-kernel Do not post admin requests to the list. They will be ignored.

1 0

Info regarding module and kernel building
by Manjunath 13 Jul '04

13 Jul '04

Is there any concept of module in darwin kernel similar to linux? Also can i configure the darwin kernel and then build the same? If i can configure the kernel how do i do it? Manjunath _______________________________________________ darwin-kernel mailing list | darwin-kernel(a)lists.apple.com Help/Unsubscribe/Archives: http://www.lists.apple.com/mailman/listinfo/darwin-kernel Do not post admin requests to the list. They will be ignored.

1 0

Re: Finder Get Info not allowing permission change with CXFS
by Nick Blievers 13 Jul '04

13 Jul '04

Hmmm, I tried setting the vnode and getattrlist to return VT_HFS rather than VT_OTHER... but finder still reported 'Unknown'. What does 'Format' get set from? Thanks Nick Using 10.3.3 the Finder 'Get Info' file/dir option (on a CXFS volume) only shows a read only summary of permissions. eg it says under Ownership & Permissions: "You can read and write" It does not show any combo boxes or the 'details' drop down section for changing owner and group info. This works with 10.2.8, using the same code. So, any ideas what has changed in Finder? And what we can do about it? What does the "Format" item in the Info window show? It seems that Finder has a special case for certain volume formats (UFS, NFS, AppleShare, HFS Plus, HFS Plus Journaled) and only shows the detailed privileged view for those formats. I'd encourage you to file a bug to request a generic interface for this. <http://developer.apple.com/bugreporter/> As to why this worked on 10.2.x, I don't really know. However, looking at the source it seems that there was a major rework of the dialog between 10.2.x and 10.3. S+E _______________________________________________ darwin-kernel mailing list | darwin-kernel(a)lists.apple.com Help/Unsubscribe/Archives: http://www.lists.apple.com/mailman/listinfo/darwin-kernel Do not post admin requests to the list. They will be ignored.

1 0

Re: Finder Get Info not allowing permission change with CXFS
by Nick Blievers 13 Jul '04

13 Jul '04

Thanks Quinn I've raised 3725785 about Finder, and raised 3521889 a while ago about CXFS using VT_OTHER as its vnode tag. Format says 'Unknown', I would guess due to us returning VT_OTHER. Thanks for your help, Nick At 17:00 +1000 23/6/04, Nick Blievers wrote: Using 10.3.3 the Finder 'Get Info' file/dir option (on a CXFS volume) only shows a read only summary of permissions. eg it says under Ownership & Permissions: "You can read and write" It does not show any combo boxes or the 'details' drop down section for changing owner and group info. This works with 10.2.8, using the same code. So, any ideas what has changed in Finder? And what we can do about it? What does the "Format" item in the Info window show? It seems that Finder has a special case for certain volume formats (UFS, NFS, AppleShare, HFS Plus, HFS Plus Journaled) and only shows the detailed privileged view for those formats. I'd encourage you to file a bug to request a generic interface for this. <http://developer.apple.com/bugreporter/> As to why this worked on 10.2.x, I don't really know. However, looking at the source it seems that there was a major rework of the dialog between 10.2.x and 10.3. S+E _______________________________________________ darwin-kernel mailing list | darwin-kernel(a)lists.apple.com Help/Unsubscribe/Archives: http://www.lists.apple.com/mailman/listinfo/darwin-kernel Do not post admin requests to the list. They will be ignored.

1 0

Re: why does nfs use it's own buffer cache?
by Mike Mackovitch 12 Jul '04

12 Jul '04

On Sat, Jul 10, 2004 at 10:11:40AM +0400, Nikita Danilov wrote: > > what are the reasons of XNU kernel using separate buffer cache (storing > struct nfsbuf's) for NFS? It seems that all buffer cache operations are > going through file system VOP_STRATEGY() at which point NFS could > initiate networking IO. > > There should be some important reason I am missing, because in the > current state of affairs there is significant code duplication (hashing, > async IO daemon, whole nfs_bio.c), NFS doesn't enjoys advantages of > vfs_cluster.c, etc. The NFS buffer cache code was added in 10.3. Before that (10.2 and earlier) NFS used the standard buffer cache. So, if you'd like to see what the code looked like when NFS did use the standard buffer cache, check out the 10.2 code. The primary reasons for NFS using its own buffer cache are multi-page buffer support and better unstable write support: * The standard buffer cache code doesn't handle multi-page buffers well. struct buf has only a single bit (B_WASDIRTY) to reflect the clean/dirty status of the buffer/page in the VM. In order to increase NFS I/O performance in 10.3, we needed to be able to increase the sizes of the NFS I/O requests and also the buffers. With only one bit to reflect the clean/dirty status of the buffer in the VM, you can't (correctly) handle multi-page buffers being mmap()ed. * The standard buffer cache and cluster code don't understand NFSv3's notion of unstable writes. The NFS client can ask the NFS server to respond to a write immediately without waiting for the data to make it to stable storage. Later the NFS client can make one "commit" request to force all that data to stable storage (it isn't there already). During the time between the completion of the unstable write and the commit, the buffer on the NFS client needs to be marked as "needs commit" which essentially means that this data was written, so we should only need to send a commit later, but if the server goes down we may also need to write that data to the server again. Before 10.3, NFS performance suffered whenever the bcleanbuf thread would try to clean up a bunch of "dirty" buffers. bcleanbuf would pick out a single buffer to try to clean and issue a write for it. The NFS code would see that the buffer only needed to be committed so it would simply send a commit request for that buffer. Whenever the system would run low on buffers, this would cause the NFS code to send a single commit request for each buffer instead of allowing the code to coalesce the "needs commit" buffers into one commit request. Hope that helps. --macko _______________________________________________ darwin-kernel mailing list | darwin-kernel(a)lists.apple.com Help/Unsubscribe/Archives: http://www.lists.apple.com/mailman/listinfo/darwin-kernel Do not post admin requests to the list. They will be ignored.

1 0

Re: Re-routing System Calls
by Quinn 12 Jul '04

12 Jul '04

At 23:03 -0400 24/6/04, Marek Kozubal wrote: I would believe apple would suggest a layered file system before they suggested doing anything with the syscall table. Thats even more likely to break than the layered file systems at the moment. The Apple folks nearly had a heart attack when someone was doing this with exit(). Indeed. Patching the system call table is not something that Apple supports. Apple is in the process of moving non-I/O Kit kernel developers to new, sustainable programming interfaces, known as KPIs. The current plan is that Apple will provide KPIs for all supported areas of kernel development. For example, there will be KPIs for leaf file systems development and another for NKE development. The reason for this effort is that Apple needs to innovate within the kernel, introducing 64-bit user address spaces, finer grained locking, ACLs, and all of the other goodies that are part of Tiger <http://www.apple.com/macosx/tiger/unix.html>. Without a well-defined set of KPIs it is impossible for us to make significant changes in the kernel without breaking non-I/O Kit KEXTs. OTOH, a KPI isolates KEXT developers from the internals of the kernel, allowing Apple to make significant changes to the kernel while maintaining KEXT binary compatibility. It is unlikely that Apple will ever produce a KPI for patching the system call table. There are at least two technical reasons for this: o KPIs must be sustainable (that is, the presence of a KPI must not adversely affect the evolution of the kernel) and we're don't want to constrain the evolution of the system call table at this time. o A system call table KPI would allow unregulated patching of arbitrary system functionality, which is not a direction that we want to take Mac OS X. The ghost of traditional Mac OS looms large! It's likely that you'll still be able to hack your way into the system call dispatch table on a future Mac OS X kernel. However, it's also likely that doing this will "rev lock" you to the kernel. That is, you'll need to update your product for every release (major or minor) of the kernel. We've designed this back door for use during development, or where your target market is tiny (for example, a university research group developing a new network protocol). Apple hopes that developers of shrink wrap software will move to the KPIs as soon as possible. Those of you who went to WWDC received a seed build of the next major Mac OS X release, Tiger. That seed includes information about the KPIs that Apple is currently planning. You can also download supplemental information from the <http://connect.apple.com>. Look for the disk image associated with Session 103 "Kernel Programming Interfaces". I don't know when this information will be made available to the general developer population. One final note: the KPI effort will not affect developers of pure I/O Kit KEXTs, that is, KEXTs that just use I/O Kit (and libkern). I/O Kit is already a sustainable programming interface and we don't intend to mess with it. S+E -- Quinn "The Eskimo!" <http://www.apple.com/developer/> Apple Developer Technical Support * Networking, Communications, Hardware _______________________________________________ darwin-kernel mailing list | darwin-kernel(a)lists.apple.com Help/Unsubscribe/Archives: http://www.lists.apple.com/mailman/listinfo/darwin-kernel Do not post admin requests to the list. They will be ignored.

1 0

Re: IOConnect Functions
by Quinn 12 Jul '04

12 Jul '04

At 9:54 -0700 28/6/04, Krishna Monian wrote: When I define a function that is going to be called as a result of an IOConnectMethodScalarIStructureO are there any restrictions on the parameters the function has to take? Yes. Your input parameters must be a sequence of scalars and your output parameters must a contiguous block of memory. That's the meaning of "ScalarI" (scalar in) and "StructureO" (structure out) respectively. Of course, there are other IOConnectMethod routines for the other permutations. I want to pass in a char** from user-space(memory already allocated) to kernel space, and copy some data into it. It's generally best if you embed your data in a parameter block and pass back the data using a "StructureO" routine. That's the approach that will yield the best long term compatibility. However, you can, if necessary, pass a pointer in from user space as a scalar parameter, create a memory descriptor from that pointer (make sure you use a routine that takes a "withTask" parameter, so you can specify the address space for the memory), and transfer data to that. >From your other posts it seems like you're working deep in the kernel. If so, perhaps using I/O Kit for this is the wrong approach. There may be a more appropriate mechanism for you to transfer data to user space (sysctl, ioctl, a driver, copyin/out, and so on). S+E -- Quinn "The Eskimo!" <http://www.apple.com/developer/> Apple Developer Technical Support * Networking, Communications, Hardware _______________________________________________ darwin-kernel mailing list | darwin-kernel(a)lists.apple.com Help/Unsubscribe/Archives: http://www.lists.apple.com/mailman/listinfo/darwin-kernel Do not post admin requests to the list. They will be ignored.

1 0

Re: STL in Kexts
by Quinn 12 Jul '04

12 Jul '04

At 7:25 -0700 30/6/04, Krishna Monian wrote: Templates (my own templated classes) seem to work fine with kexts. However when I try to use the STL I get the following errors in XCode. /usr/lib/gcc/darwin/3.3/libstdc++.a(new_opv.o) incompatible, file contains unsupported type of section (__TEXT,__picsymbolstub1) in load command 0(must specify "-dy [...] I am guessing it is saying I need to add the -dynamic flag. However, that doesn't seem to help. This is because you're trying to link a user space library (libstdc++.a) into a kernel program. That doesn't work. Is there any equivalent of the STL "list" or "slist" in libkern? No. As Dean said: At 18:04 -0700 29/6/04, Dean Reece wrote: Sorry, but templates and STL are not supported in the kernel. You may be able to make this work by compiling your own version of STL (with the same options you use to compile your KEXT) but it's not something that we support. libkern does containing various container classes (OSArray, OSDictionary), some of which might provide enough functionality for your needs. However, these classes are not in any way like STL. S+E -- Quinn "The Eskimo!" <http://www.apple.com/developer/> Apple Developer Technical Support * Networking, Communications, Hardware _______________________________________________ darwin-kernel mailing list | darwin-kernel(a)lists.apple.com Help/Unsubscribe/Archives: http://www.lists.apple.com/mailman/listinfo/darwin-kernel Do not post admin requests to the list. They will be ignored.

1 0