June 2004 - Darwin-kernel - lists.apple.com

RE: Loading kerenl extensions
by Matthew Jaffa 24 Jun '04

24 Jun '04

I ran into this problem when I didn't make the kernel extension with root as the owner and wheel as the group. These owners must be propagated through the whole extensions. I usually get info on the .kext and change the owners then sudo cp it to where I want it to go, Just make sure that the contents inside the .kext package are also root/wheel Matt From: Krishna Monian <k_tutorials(a)yahoo.com> To: darwin-kernel(a)lists.apple.com Subject: Loading kerenl extensions Date: Thu, 24 Jun 2004 07:40:42 -0700 (PDT) Hi All, I have currently written a Kernel extension on my development machine which has Mac OS X 10.3.4 (Build 7H63) running on it. It seems to run fine on this machine. However when I try to load it on another machine which has Mac OS X 10.3.3 (Build 7F44) I get the following message. kextload: extension xxx/HelloKernel.kext is not authentic. Could someone tell me the reason for this. Is it due to the version difference or due to some other reason? Thanks Krishna __________________________________ Do you Yahoo!? New and Improved Yahoo! Mail - Send 10MB messages! http://promotions.yahoo.com/new_mail _______________________________________________ darwin-kernel mailing list | darwin-kernel(a)lists.apple.com Help/Unsubscribe/Archives: http://www.lists.apple.com/mailman/listinfo/darwin-kernel Do not post admin requests to the list. They will be ignored. _______________________________________________ darwin-kernel mailing list | darwin-kernel(a)lists.apple.com Help/Unsubscribe/Archives: http://www.lists.apple.com/mailman/listinfo/darwin-kernel Do not post admin requests to the list. They will be ignored.

1 0

Loading kerenl extensions
by Krishna Monian 24 Jun '04

24 Jun '04

Hi All, I have currently written a Kernel extension on my development machine which has Mac OS X 10.3.4 (Build 7H63) running on it. It seems to run fine on this machine. However when I try to load it on another machine which has Mac OS X 10.3.3 (Build 7F44) I get the following message. kextload: extension xxx/HelloKernel.kext is not authentic. Could someone tell me the reason for this. Is it due to the version difference or due to some other reason? Thanks Krishna __________________________________ Do you Yahoo!? New and Improved Yahoo! Mail - Send 10MB messages! http://promotions.yahoo.com/new_mail _______________________________________________ darwin-kernel mailing list | darwin-kernel(a)lists.apple.com Help/Unsubscribe/Archives: http://www.lists.apple.com/mailman/listinfo/darwin-kernel Do not post admin requests to the list. They will be ignored.

1 0

Getting the full path for a file
by Krishna Monian 23 Jun '04

23 Jun '04

Hi All, I was looking at the MoreFiles samples on the Apple site and the FullPath code in particular. I am trying to modify the fs_usage system call so that it reports the full path. However I am not sure how exactly I should go about using the getFullPath function. I would really appreciate it if someone could guide me on this. Thanks Krishna __________________________________ Do you Yahoo!? New and Improved Yahoo! Mail - Send 10MB messages! http://promotions.yahoo.com/new_mail _______________________________________________ darwin-kernel mailing list | darwin-kernel(a)lists.apple.com Help/Unsubscribe/Archives: http://www.lists.apple.com/mailman/listinfo/darwin-kernel Do not post admin requests to the list. They will be ignored.

1 0

Re: Kernel Panics in my NKE
by Nikita Danilov 23 Jun '04

23 Jun '04

Matt Jaffa writes: > Hi, > > I was wondering if there is a limit within a NKE of how much data can > be allocated. > > For instance my NKE is working great until a packet with a length of > 65000+ come in and when this line > is reached unsigned char buffer[sizeofpacket] it crashes cause it if you have something like int foo(...) { unsigned char buffer[sizeofpacket]; ... } in your code, then buffer is allocated on the _stack_. Kernel stack is of small fixed size and is not auto-expandable (as it is in the user-space). You should never ever allocate large data structures on stack in the kernel. Use dynamic memory allocation facilities instead. > is too big I guess, usually packets are only 2 - 1000 so this has no > problem, when I remove that line of code my NKE never crashes. > > So what is the limit of what I can do? I mean the size of the unsigned > char buffer[size]. > > Matt Nikita. _______________________________________________ darwin-kernel mailing list | darwin-kernel(a)lists.apple.com Help/Unsubscribe/Archives: http://www.lists.apple.com/mailman/listinfo/darwin-kernel Do not post admin requests to the list. They will be ignored.

1 0

Re: File Monitoring under Mac OS X
by Sam Hart 23 Jun '04

23 Jun '04

Krishna wrote: It is my understanding that the FreeBSD file system, which is the one used in the Darwin kernel of Mac OS is stackable. Is writing a loadable module on top of this a better option? in reference to stacking file systems in Darwin, you might want to read this : <http://developer.apple.com/qa/qa2001/qa1242.html> to quote : "Stacking File Systems -- Apple does not support the development of stacking VFS plug-ins on Mac OS X. We've taken this position because, in our opinion, it's not possible to create a stacking VFS plug-in that: 1. works reliably, 2. does not severely impinge on system performance, and 3. has any hope of binary (or even source-level) compatibility with future systems." AFAIK that still holds However I am led to believe that it has been done, but am not sure if it met Apple's criteria quoted above. S a m ------------------------------------------------------- Drawing on my fine command of language, I said nothing. _______________________________________________ darwin-kernel mailing list | darwin-kernel(a)lists.apple.com Help/Unsubscribe/Archives: http://www.lists.apple.com/mailman/listinfo/darwin-kernel Do not post admin requests to the list. They will be ignored.

1 0

Finder Get Info not allowing permission change with CXFS
by Nick Blievers 23 Jun '04

23 Jun '04

Hi, Using 10.3.3 the Finder 'Get Info' file/dir option (on a CXFS volume) only shows a read only summary of permissions. eg it says under Ownership & Permissions: "You can read and write" It does not show any combo boxes or the 'details' drop down section for changing owner and group info. This works with 10.2.8, using the same code. So, any ideas what has changed in Finder? And what we can do about it? Thanks Nick _______________________________________________ darwin-kernel mailing list | darwin-kernel(a)lists.apple.com Help/Unsubscribe/Archives: http://www.lists.apple.com/mailman/listinfo/darwin-kernel Do not post admin requests to the list. They will be ignored.

1 0

Re: Calling all Filesystem guru's: Help!
by Brian Bergstrand 23 Jun '04

23 Jun '04

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Well, thanks to Sam, I've found this bug and it is in the kernel: Reported as bug 3707337: "Crash/Data loss when FS block size is < 4KB because of bug in cluster_io()." Summary: I found a bug in cluster_io() when writing sparse files to my filesystem (Ext2 for OS X: http://sourceforge.net/projects/ext2fsx/) My own bug report can be found at : http://sourceforge.net/tracker/index.php? func=detail&aid=965119&group_id=64713&atid=508406 Basically, cluster_io() will overflow the var it uses to track the size of the IO when a sparse file is read or written to a filesystem with a block size < PAGE_SIZE (4096 in all current Darwin releases). This causes continuous calls to VOP_CMAP() and it can also cause data to be lost during a write. I suspect this same bug would happen with Apple UFS if it allowed < 4KB block sizes, but it currently doesn't. Steps to reproduce: Write any sparse file to file system. Details: I made an internal copy of cluster_io(), and put a bp on VOP_CLOSE() and traced into my cluster_io() copy. On a 1KB block filesystem, I wrote 17 bytes at offset 0, and then the same 17 bytes at offset 2048. This created a 2065 byte file and left a 1 block hole in the file: i_size = 2065 i_db = {547, 0, 548, 0, 0, 0, 0, 0, 0, 0, 0, 0}, i_ib = {0, 0, 0}, cluster_push(vp=0x29ae570) cluster_try_push (vp=0x29ae570, EOF=2065, can_delay=0, push_all=1) cluster_push_x (vp=0x29ae570, EOF=2065, first=0, last=1, can_delay=0) cluster_io (vp=0x29ae570, upl=0x29a4d80, upl_offset=0, f_offset=0, non_rounded_size=2065, devblocksize=512, flags=38, real_bp=0x0, iostate=0x0) // flags = CL_COMMIT | CL_AGE | CL_ASYNC 388 size = (non_rounded_size + (devblocksize - 1)) & ~(devblocksize - - 1); (gdb) p size $36 = 2560 In the while loop on line 443 (vfs_cluster.c - xnu-517.7.7) The first time through the loop, VOP_CMAP() replies with: (gdb) p blkno $40 = 1094 (gdb) p io_size $41 = 1024 And we get to the end of the loop with: (gdb) p upl_offset $55 = 1024 (gdb) p f_offset $56 = 1024 (gdb) p size $57 = 1536 This is all fine. Now the second time through the loop, we are on the sparse hole and VOP_CMAP() returns: (gdb) p blkno $61 = -1 (gdb) p io_size $62 = 1024 So now the following is true, and we enter that condition block: if ( (!(flags & CL_READ) && (long)blkno == -1) || io_size == 0) { .... 482 upl_offset += PAGE_SIZE_64; (gdb) 483 f_offset += PAGE_SIZE_64; (gdb) 484 size -= PAGE_SIZE_64; (gdb) 485 continue; } AndB this is where the trouble happens: (gdb) p upl_offset $66 = 5120 (gdb) p f_offset $67 = 5120 (gdb) p size $68 = 4294964736 So not only does the size overflow, but the block after the hole is skipped! If io_size was used instead, then everything would be correct: upl_offset += io_size == 2048 f_offset += io_size == 2048 size -= io_size == 512 In fact, this is what is done in the READ case: 510 if ((flags & CL_READ) && (long)blkno == -1) { ... 575 upl_offset += io_size; 576 f_offset += io_size; 577 size -= io_size; } Isolation: I've also reproduced this bug on 10.3.3, and 10.2.8. On Jun 20, 2004, at 1:19 PM, Sam Vaughan wrote: > Hi Brian, > > My first guess would be that you're underflowing the size variable in > the while loop in cluster_io(). It's fairly easy to do, trust me ;-) > If you're returning a blkno of -1 for a hole and the size has > underflowed, you end up in a loop with size working its way down from > 2^32 one page at a time and f_offset working its way up just as you > described. > > I can't seem to get web access working in this hotel room right now, > so I can't follow the links you posted. Send me an email with some > more details attached if you like. > > Sam > >> Okay, I'm at my wits end on a bug in the ext2 filesystem for OS X >> <http://sourceforge.net/projects/ext2fsx>: >> >> When writing a sparse file (even one with just one hole), the fs can >> hang or panic (with a Disk Image) the system. >> >> I've tracked this down to cluster_io() repeatedly calling VOP_CMAP >> with >> higher and higher offsets (way past the file size) that eventually >> cause our logical block address to overflow (this is much more likely >> on <4KB block FS), thus making a negative offset which VOP_BMAP treats >> as an indirect block address and I assume causes a weird page to be >> read and passed to the storage driver by cluster_io/VOP_STRATEGY. >> >> Anyway, I can't figure why this happens. I've gone through everything >> I >> can think of: >> >> <http://sourceforge.net/tracker/index.php? >> func=detail&aid=965119&group_id=64713&atid=508406> > Brian Bergstrand <http://www.bergstrand.org/brian/>, AIM: triryche206 PGP Key: <http://www.bergstrand.org/brian/misc/public_key.txt> A society that will trade a little liberty for a little order will lose both, and deserve neither. - Thomas Jefferson As of 09:32:00 PM, iTunes is playing "Cry Freedom" from "Crash" by "Dave Matthews Band" -----BEGIN PGP SIGNATURE----- Version: PGP 8.1 iQA/AwUBQNjeeXnR2Fu2x7aiEQLY1wCfcwsp/UajJX4rz3JBNf2RN/bLNjEAn1fp 9cqSvNFN3ANDAHbxg3DPL3hp =urtE -----END PGP SIGNATURE----- _______________________________________________ darwin-kernel mailing list | darwin-kernel(a)lists.apple.com Help/Unsubscribe/Archives: http://www.lists.apple.com/mailman/listinfo/darwin-kernel Do not post admin requests to the list. They will be ignored.

1 0

Re: File Monitoring under Mac OS X
by Krishna Monian 22 Jun '04

22 Jun '04

fs_usage doesn't seem to give me the absolute path and it also truncates the path beyond a certain length as it has says in the man pages. It is my understanding that the FreeBSD file system, which is the one used in the Darwin kernel of Mac OS is stackable. Is writing a loadable module on top of this a better option? Thanks Krishna Monian --- Jonas Maebe <jonas(a)zeus.ugent.be> wrote: > On 21 jun 2004, at 22:03, Krishna Monian wrote: > > >> What is "the complete path"? In world with hard > >> links, one file can > >> have countless "complete paths". > > > > What I mean is the actual path for a given file > and > > not just a relative path. > > > > /Users/admin/.... and so on > > And what I mean is that a file may have more than > one actual path and > location, and that it wouldn't surprise me if inside > the kernel there > is no notion of "the path" to a file, only some > location on disk > (possibly together with whatever the user specified, > but as you noted > that does not have to be a complete path) > > > Jonas > _______________________________________________ > darwin-kernel mailing list | > darwin-kernel(a)lists.apple.com > Help/Unsubscribe/Archives: > http://www.lists.apple.com/mailman/listinfo/darwin-kernel > Do not post admin requests to the list. They will be > ignored. > > __________________________________ Do you Yahoo!? New and Improved Yahoo! Mail - 100MB free storage! http://promotions.yahoo.com/new_mail _______________________________________________ darwin-kernel mailing list | darwin-kernel(a)lists.apple.com Help/Unsubscribe/Archives: http://www.lists.apple.com/mailman/listinfo/darwin-kernel Do not post admin requests to the list. They will be ignored.

1 0

boundary crossing latencies
by Andrew Gallatin 22 Jun '04

22 Jun '04

I'm working on commercial OS-bypass networking software shared among Linux, FreeBSD, MacOSX and others (Solaris, windows, etc). Due to the shared nature of the code, we've been using ioctls, since they work everywhere. The latency to complete an ioctl to our driver is becoming more important. Unfortunately, MacOSX seems to be rather bad at this. I performed micro-benchmarks to quantify this. (see table below). In addition to a null (getppid()) syscall, I timed a simple ioctl which does nothing more than copy a few ints into and out of the driver. I found that MacOSX 10.3.3's ioctl takes nearly 4x as long as the same code under linux ppc64 on the same exact G5. And it takes 8x as long as the same ioctl on an AMD64. Given that ioctls are so slow, I hoped the Mach IPC based IOKit UserClient method of boundary crossing would be faster. So I downloaded the "SimpleUserClient" demo, removed the IOLogs and printfs, and instrumented it by timing how long it takes to call "IOConnectMethodScalarIStructureO" 100,000 times (this does something roughly equivalent to the ioctl I'm timing above -- copies a few ints in and out). It looks like IOKit User Clients are even worse than ioctls. A user client call takes 7x as long as an ioctl under linux ppc64 on the same G5. And 15x as long as an ioctl on the AMD64 competition. Is Apple planning to improve either ioctl or IOKit user-client latencies? Thanks, Drew Platform null (us/call) ioctl(us/call) user client(us/call) -------- ----------- -------- ------------------- G5/linux2.6.5 0.2112 0.6679 N/A G5/OSX 10.3.3 1.4568 2.5152 4.7201 P4/linux2.4.23 0.4179 0.6207 N/A P4/linux2.6.6 0.3833 0.6502 N/A P4/FreeBSD 5.2 0.4827 1.1024 N/A AMD/linux2.4.21 0.0982 0.3131 N/A "G5" is a dual 2.0GHz G5 "P4" is a dual 3.06GHz Xeon "AMD" is a dual opteron 244 running 64-bit linux. _______________________________________________ darwin-kernel mailing list | darwin-kernel(a)lists.apple.com Help/Unsubscribe/Archives: http://www.lists.apple.com/mailman/listinfo/darwin-kernel Do not post admin requests to the list. They will be ignored.

1 0

Re: File Monitoring under Mac OS X
by Krishna Monian 22 Jun '04

22 Jun '04

Has anyone looked at the source for fs_usage. I am a little confused in ceratin areas and some help would be really appreciated. Thanks --- Jonas Maebe <jonas(a)zeus.ugent.be> wrote: > On 21 jun 2004, at 22:03, Krishna Monian wrote: > > >> What is "the complete path"? In world with hard > >> links, one file can > >> have countless "complete paths". > > > > What I mean is the actual path for a given file > and > > not just a relative path. > > > > /Users/admin/.... and so on > > And what I mean is that a file may have more than > one actual path and > location, and that it wouldn't surprise me if inside > the kernel there > is no notion of "the path" to a file, only some > location on disk > (possibly together with whatever the user specified, > but as you noted > that does not have to be a complete path) > > > Jonas > _______________________________________________ > darwin-kernel mailing list | > darwin-kernel(a)lists.apple.com > Help/Unsubscribe/Archives: > http://www.lists.apple.com/mailman/listinfo/darwin-kernel > Do not post admin requests to the list. They will be > ignored. > > __________________________________ Do you Yahoo!? Read only the mail you want - Yahoo! Mail SpamGuard. http://promotions.yahoo.com/new_mail _______________________________________________ darwin-kernel mailing list | darwin-kernel(a)lists.apple.com Help/Unsubscribe/Archives: http://www.lists.apple.com/mailman/listinfo/darwin-kernel Do not post admin requests to the list. They will be ignored.

1 0