August 2005 - Darwin-kernel

Checksums on modified inbound packets?
by Frank Thomas 04 Aug '05

04 Aug '05

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com Domainkey-signature: a=rsa-sha1; q=dns; c=nofws; s=beta; d=gmail.com; h=received:message-id:date:from:reply-to:to:subject:mime-version:content-type:content-transfer-encoding:content-disposition; b=DbK9iGb6odEjbEAjGesiJyPtLsS0IoRDlVp0LhvmBlqKegX5Wu5oGJPZJfu/y+I5/8RKR9JIebPcf9vlw/Z0h4HbmCo95cow9MUPebeOsdAWlq2yjw61qs74jowQcY9I1e1QVHjQ6X70Ow0I80jfxSIPEudQZNMB30YKor35CBY= Well I rebuilt my NKE to use the EJUSTRETURN/ipf_inject_output model for inspecting outbound traffic. This seems to be working great, so apparently my prior attempts were doing something wrong. Anyways, that bit is operating beautifully, but I've gotten into some trouble on the inbound side. Basically I'm flagging "interesting" TCP connections as they are detected on the outbound side, and then waiting for return traffic. When this traffic arrives it is slightly adjusted. To do this, my inbound filter hook function does some simple steps: - Allocate a buffer and copy the original packet data into it via mbuf_copydata() - Detect the TCP payload section of the packet and adjust it - Duplicate the original mbuf passed to the hook via mbuf_dup - Use mbuf_copyback to copy over the adjusted data buffer into the duplicated mbuf - Send the newly built packet to ipf_inject_input - Discard the original packet by returning EJUSTRETURN This is working excellently on vanilla Ethernet connections. The adjusted data shows up in the final client application and everything is great and very stable (automated tests have run over 10,000 adjusted packets across the running NKE without problems). Unfortunately, something goes wrong on both my Airport-enabled iBook and the AIrport Extreme connection on my Mini. On these connections, the modified packet never reaches the client application up in userland. ipf_inject_input returns 0 and no errors occur during the construction process. I've seen on other platforms (specifically Linux) that modifying a TCP packet in this way can cause the TCP stack to drop the packet because the checksumming becomes invalid. Even though just the payload (and not any of the IP/TCP headers) is modified, I suspect that may be happening somehow here as well - perhaps Airport's hardware checksumming is working differently from the Ethernet NIC? I vaguely remember reading somewhere on the list about the loopback interface setting a flag that effectively bypassed the checksum validation in the TCP stack. I've tried a couple of things like mbuf_inbound_modified() but that seems to cause the packet to get dropped in all connections. Is there a way to directly compute the checksums or something so that this modified packet will pass through the stack properly? Or am I barking up entirely the wrong tree with this modification method? Thanks in advance, Frank _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… This email sent to site_archiver(a)lists.apple.com

1 0

Re: When using mbuf_dup, I get a PANIC
by Josh Graessley 02 Aug '05

02 Aug '05

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com The bug number is 4117818. -josh On Aug 2, 2005, at 7:55 AM, Markus Hanauska wrote: Hello Everyone, Straight to the point: Why does this part of a KEXT crash? /* IP Filter */ errno_t pre(ipf_input_func)( void *cookie, mbuf_t *data, int offset, u_int8_t protocol ) { char *tag; size_t len; if (pre(connectedProcesses) == 0) return KERN_SUCCESS; if (protocol == IPPROTO_ESP) { if ( mbuf_tag_allocate( *data, pre(mbufTag), 1, strlen(STR(ESP_MARKER)) + 1, MBUF_DONTWAIT, (void **)&tag ) == 0) { bcopy(STR(ESP_MARKER), tag, strlen(STR(ESP_MARKER)) + 1); } } else if ( mbuf_tag_find( *data, pre(mbufTag), 1, &len, (void **) &tag ) == 0) { if (len == strlen(STR(ESP_MARKER)) + 1 && memcmp(tag, STR(ESP_MARKER), len) == 0) { size_t mbufSize, space = -1; mbuf_tag_free(*data, pre(mbufTag), 1); mbufSize = (mbuf_flags(*data) & MBUF_PKTHDR) ? mbuf_pkthdr_len(*data) : mbuf_len(*data); if ( ctl_getenqueuespace( pre(userSpaceSock), 0, &space ) == 0 && space >= mbufSize ) { mbuf_t dupe; if (mbuf_dup(*data, MBUF_DONTWAIT, &dupe) == 0) { if (ctl_enqueuembuf ( pre(userSpaceSock), 0, dupe, 0) != 0 ) { printf("Failed to send\n"); mbuf_freem(dupe); } } else printf("Failed to dupe\n"); } else printf("Not enough space: %d >= %d\n", space, mbufSize); } else printf("Marker does not match\n"); } return KERN_SUCCESS; } I know where it crashes, it crashes in /xnu-792.2.4/bsd/kern/uipc_mbuf.c:1267 where a manual panic is created: "m_freem_list: m_nextpkt of m_next != NULL" mbuf_t dupe2 = dupe; while (dupe2) { if (mbuf_nextpkt(dupe2)) mbuf_setnextpkt(dupe2, NULL); dupe2 = mbuf_next(dupe2); } BTW, I'll did the same for *data in another test and then for both. Oh, BTW pre(...) is a makro, it expands the name to a more unique one: #define pre3(x,y) x ## y #define pre2(x,y) pre3(x,y) #define pre(x) pre2(FUNCTION_PREFIX, x) #define STR2(x) #x #define STR(x) STR2(x) Any help appreciated, thank you very much! -- Best Regards, Markus Hanauska This email sent to jgraessley(a)apple.com _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… This email sent to site_archiver(a)lists.apple.com There is a known issue with mbuf_dup and mbuf tags. mbuf_dup copies the tag pointer. The tags are stored in a chain of mbufs. When you free either the original or the dup'd packet, the mbufs for the tags are freed. When you free the remaining mbuf, the machine panics because the mbufs used for the tags are already free. One option would be to use mbuf_copym instead of mbuf_dup. It should be quicker too because it doesn't need to duplicate the data. Another option would be to not add the tags until after you have done the dupe. That's a bad solution because you never know if some other third party kext has added tags already. Fine, but I did the following as well (just from top of my head, excute stupid coding errors, of course the version I tried had no such errors): So there was definately no m_nextpkt set along the chain. Modifying the code to send the original mbuf (*data) and then not freeing it (EJUSTRETURN as ctl_getenqueuespace consumes the package on success) works just fine. So I don't see how this could be my mistake. I tried not to equeue the package at all, just dupe and then mbuf_freem and I run into a panic, too. Haven't checked so far if it's the same one. and FUNCTION_PREFIX is a long, unique reverted domain name (no string). This does not relate to the problem. STR converts defines to strings: And I also know that the above function is anything but high- performance, however it was just quick and dirty as a proof of concept, that did not work so far, cause it panics. _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/jgraessley% 40apple.com smime.p7s

1 0

Re: SCSITask User Client Driver issue in MAC X.3
by Shawn Erickson 02 Aug '05

02 Aug '05

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com On Aug 1, 2005, at 10:21 PM, RajaGovarthanan Velusamy wrote: Hi, I am using an SCSITask User Client to send the scsi commands to my device in MAC X.1 and MAC X.2. Now the same kernel code is not working in MAC X.3. When we try to load the driver ten the message "Invalid file found" is diaplyed. Can you outline better how you are loading your "driver"? -Shawn _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… As an FYI it is "Mac" not "MAC". The later implies to many Media Access Controller. Also the OS versions are really Mac OS X 10.1, 10.2, etc. If you cannot understand what file is missing try using statfs at the same time you attempt load your driver. You should be able to spot what file it attempted to open, at least I think you can. This email sent to site_archiver(a)lists.apple.com

1 0

When using mbuf_dup, I get a PANIC
by Markus Hanauska 02 Aug '05

02 Aug '05

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com Hello Everyone, Straight to the point: Why does this part of a KEXT crash? /* IP Filter */ errno_t pre(ipf_input_func)( void *cookie, mbuf_t *data, int offset, u_int8_t protocol ) { char *tag; size_t len; if (pre(connectedProcesses) == 0) return KERN_SUCCESS; if (protocol == IPPROTO_ESP) { if ( mbuf_tag_allocate( *data, pre(mbufTag), 1, strlen(STR(ESP_MARKER)) + 1, MBUF_DONTWAIT, (void **)&tag ) == 0) { bcopy(STR(ESP_MARKER), tag, strlen(STR(ESP_MARKER)) + 1); } } else if ( mbuf_tag_find( *data, pre(mbufTag), 1, &len, (void **) &tag ) == 0) { if (len == strlen(STR(ESP_MARKER)) + 1 && memcmp(tag, STR(ESP_MARKER), len) == 0) { size_t mbufSize, space = -1; mbuf_tag_free(*data, pre(mbufTag), 1); mbufSize = (mbuf_flags(*data) & MBUF_PKTHDR) ? mbuf_pkthdr_len(*data) : mbuf_len(*data); if ( ctl_getenqueuespace( pre(userSpaceSock), 0, &space ) == 0 && space >= mbufSize ) { mbuf_t dupe; if (mbuf_dup(*data, MBUF_DONTWAIT, &dupe) == 0) { if (ctl_enqueuembuf ( pre(userSpaceSock), 0, dupe, 0) != 0 ) { printf("Failed to send\n"); mbuf_freem(dupe); } } else printf("Failed to dupe\n"); } else printf("Not enough space: %d >= %d\n", space, mbufSize); } else printf("Marker does not match\n"); } return KERN_SUCCESS; } I know where it crashes, it crashes in /xnu-792.2.4/bsd/kern/uipc_mbuf.c:1267 where a manual panic is created: "m_freem_list: m_nextpkt of m_next != NULL" mbuf_t dupe2 = dupe; while (dupe2) { if (mbuf_nextpkt(dupe2)) mbuf_setnextpkt(dupe2, NULL); dupe2 = mbuf_next(dupe2); } BTW, I'll did the same for *data in another test and then for both. Oh, BTW pre(...) is a makro, it expands the name to a more unique one: #define pre3(x,y) x ## y #define pre2(x,y) pre3(x,y) #define pre(x) pre2(FUNCTION_PREFIX, x) #define STR2(x) #x #define STR(x) STR2(x) Any help appreciated, thank you very much! -- Best Regards, Markus Hanauska _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… Fine, but I did the following as well (just from top of my head, excute stupid coding errors, of course the version I tried had no such errors): So there was definately no m_nextpkt set along the chain. Modifying the code to send the original mbuf (*data) and then not freeing it (EJUSTRETURN as ctl_getenqueuespace consumes the package on success) works just fine. So I don't see how this could be my mistake. I tried not to equeue the package at all, just dupe and then mbuf_freem and I run into a panic, too. Haven't checked so far if it's the same one. and FUNCTION_PREFIX is a long, unique reverted domain name (no string). This does not relate to the problem. STR converts defines to strings: And I also know that the above function is anything but high- performance, however it was just quick and dirty as a proof of concept, that did not work so far, cause it panics. This email sent to site_archiver(a)lists.apple.com

1 0

SCSITask User Client Driver issue in MAC X.3
by RajaGovarthanan Velusamy 02 Aug '05

02 Aug '05

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com Domainkey-signature: a=rsa-sha1; q=dns; c=nofws; s=beta; d=gmail.com; h=received:message-id:date:from:reply-to:to:subject:mime-version:content-type:content-transfer-encoding:content-disposition; b=jeGRY2DTmX4ip+DDqU+YLQEZRLA2bKIEdqIjpMkDGzEo0t2PyCb1QwFodKg03kiQu1O8QC8pllisjrMFV6fMfuOC/s0TxkP7VPpr9j4OAiHVJetKuuFyfA/oD4R9/eIe0VudOrA44MgyYdG+donBLrMZC24L219Z55Ybc312Sto= Hi, I am using an SCSITask User Client to send the scsi commands to my device in MAC X.1 and MAC X.2. Now the same kernel code is not working in MAC X.3. When we try to load the driver ten the message "Invalid file found" is diaplyed. Can somebody help me to solve this issue. To know somethings more related to my issue I tried downloading the tarball from Apple openSorce. http://www.opensource.apple.com/darwinsource/tarballs/apsl/IOUSBMassStorage… AM I doing some thing wrong. Any responsible person I can contact to get these breaked links fixed. Regards, Raja. _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… This email sent to site_archiver(a)lists.apple.com

1 0

Michael Gong/Toronto/IBM is out of the office.
by Michael Gong 01 Aug '05

01 Aug '05

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com I will be out of the office starting 08/01/2005 and will not return until 08/12/2005. I will respond to your message when I return.For issues, please contact with my manager Wang Chen. _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… This email sent to site_archiver(a)lists.apple.com

1 0

Re: Next VFS KPI ?
by Quinn 01 Aug '05

01 Aug '05

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com S+E -- Quinn "The Eskimo!" <http://www.apple.com/developer/> Apple Developer Technical Support * Networking, Communications, Hardware _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… At 2:11 -0500 1/8/05, Brian Bergstrand wrote: The app I'm testing is TextEdit, and using fs_usuage, I can see exchangedata is returning 45 (ENOTSUP) when trying to save a modified file. SMBFS works fine though. So I'm guessing something is wrong in our vol caps. Last I checked, TextEdit doesn't actually check whether exchangedata is supported. It just calls it, and if that fails it falls back to its alternative algorithm. Looking at TextEdit with fs_usage on Tiger, that's exactly what I see: exchangedata returns 45 (ENOTSUP) and then TextEdit continues on its merry way. So, if TextEdit's saving is failing, it's not because of your volume capabilities, it's because of some part of its alternative algorithm is failing. This email sent to site_archiver(a)lists.apple.com

1 0

Re: How to get the Darwin kernel source code from cvs?
by Shawn Erickson 01 Aug '05

01 Aug '05

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com On Aug 1, 2005, at 12:17 AM, jackie xie wrote: Could you tell me how to get the Darwin kernel source code from cvs? I try to usgin following command, but it cannot work: (Could I use the Apple ID to do that?) Apple doesn't provide source access via CVS these days. -Shawn [1] <http://www.opensource.apple.com/darwinsource/> [2] <http://www.opendarwin.org/> _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… You can download tar balls or browse source via web from Apple's site [1] or head over to OpenDarwin.org [2] if you want cvs access. This email sent to site_archiver(a)lists.apple.com

1 0

How to get the Darwin kernel source code from cvs?
by jackie xie 01 Aug '05

01 Aug '05

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com Domainkey-signature: a=rsa-sha1; q=dns; c=nofws; s=beta; d=gmail.com; h=received:message-id:date:from:reply-to:to:subject:cc:in-reply-to:mime-version:content-type:references; b=JfZ5rKWUe2qbV4y3HDcMraNBb/oSjTP3p+7qT7qB5gePSYArRQkv3gYcix//fO6O7TXLcbbjECfLiVE5D6VBaUZGQgeUlDF9aAVa7c25n99EqRfztpqAN1VE9Fxc6u9jzLPHwSV7WCMYE/qTUiGH1K3rlomyJmEK9zHYMS6IuE8= _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… This email sent to site_archiver(a)lists.apple.com somebody@mac.com:secret

1 0

Next VFS KPI ?
by Brian Bergstrand 01 Aug '05

01 Aug '05

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Anyway, anyone have an idea as to what may be going on? Here's our vol cap settings: TIA. Brian Bergstrand <http://www.bergstrand.org/brian/> PGP Key ID: 0xB6C7B6A2 -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.1 (Darwin) iD8DBQFC7csOedHYW7bHtqIRArWQAKDd7g5Zo/MwMq7XwVgSia/X59TRfACfQQ1a 8JY2pz0Ydjx5+ZsFm+LXUw0= =w516 -----END PGP SIGNATURE----- _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… Well, thanks to Quinn and Mike I solved the Finder missing files problem. Turned out that some assumptions we were making in readdir broke with the new Finder version and our readdir was returning EINVAL. Fixing this opened a whole can of worms that required me to rewrite our EOF detection completely. Now for a new problem. We do not support exchangedata(). This wasn't a problem on Panther, as the system worked around it. And it appears Tiger still does this, but not for our FS. SMB and NFS don't support exchangedata either, but they still work with those apps that use it. The app I'm testing is TextEdit, and using fs_usuage, I can see exchangedata is returning 45 (ENOTSUP) when trying to save a modified file. SMBFS works fine though. So I'm guessing something is wrong in our vol caps. VOL_CAP_INT_EXCHANGEDATA is not set in capabilities, and initially it wasn't set in valid either, but I saw that both NFS and SMBFS set it in valid, so I tried that -- still the same problem. So, I tried to use DYLD_IMAGE_SUFFIX=_debug on TextEdit, but it just would not load the debug versions. Next, I went looking for the exchangedata source on Darwin, but apparently libSystem does not contain actual source code and I just could not figure out where to look for it (tried libc too) -- maybe libSystem is not open source. void volcaps(mount_t mp, vol_capabilities_attr_t *capsp) { // Supported caps capsp->capabilities[VOL_CAPABILITIES_FORMAT] = VOL_CAP_FMT_CASE_PRESERVING /* Not true for DOS NS */ | VOL_CAP_FMT_FAST_STATFS /* NCIOM caches stat info */ #ifdef OSX_KPI | VOL_CAP_FMT_2TB_FILESIZE // We support at least 2TB files. #endif ; #ifdef OSX_KPI capsp->capabilities[VOL_CAPABILITIES_INTERFACES] = 0; // attrlist is handled by the kernel #else capsp->capabilities[VOL_CAPABILITIES_INTERFACES] = VOL_CAP_INT_ATTRLIST; #endif capsp->capabilities[VOL_CAPABILITIES_RESERVED1] = 0; capsp->capabilities[VOL_CAPABILITIES_RESERVED2] = 0; /* Capabilities we know we the meaning of (whether they're supported or not) */ capsp->valid[VOL_CAPABILITIES_FORMAT] = VOL_CAP_FMT_PERSISTENTOBJECTIDS | VOL_CAP_FMT_SYMBOLICLINKS | VOL_CAP_FMT_HARDLINKS | VOL_CAP_FMT_JOURNAL | VOL_CAP_FMT_JOURNAL_ACTIVE | VOL_CAP_FMT_NO_ROOT_TIMES | VOL_CAP_FMT_SPARSE_FILES | VOL_CAP_FMT_ZERO_RUNS | VOL_CAP_FMT_FAST_STATFS #ifdef OSX_KPI | VOL_CAP_FMT_2TB_FILESIZE #endif ; capsp->valid[VOL_CAPABILITIES_INTERFACES] = VOL_CAP_INT_SEARCHFS | VOL_CAP_INT_ATTRLIST | VOL_CAP_INT_NFSEXPORT | VOL_CAP_INT_READDIRATTR | VOL_CAP_INT_EXCHANGEDATA | VOL_CAP_INT_COPYFILE | VOL_CAP_INT_ALLOCATE | VOL_CAP_INT_VOL_RENAME | VOL_CAP_INT_ADVLOCK | VOL_CAP_INT_FLOCK; capsp->valid[VOL_CAPABILITIES_RESERVED1] = 0; capsp->valid[VOL_CAPABILITIES_RESERVED2] = 0; } This email sent to site_archiver(a)lists.apple.com

1 0