site_archiver@lists.apple.com Delivered-To: darwin-kernel@lists.apple.com Here's what's happening: -josh On Aug 4, 2005, at 3:19 PM, Frank Thomas wrote: Well I rebuilt my NKE to use the EJUSTRETURN/ipf_inject_output model for inspecting outbound traffic. This seems to be working great, so apparently my prior attempts were doing something wrong. Anyways, that bit is operating beautifully, but I've gotten into some trouble on the inbound side. Basically I'm flagging "interesting" TCP connections as they are detected on the outbound side, and then waiting for return traffic. When this traffic arrives it is slightly adjusted. To do this, my inbound filter hook function does some simple steps: - Allocate a buffer and copy the original packet data into it via mbuf_copydata() - Detect the TCP payload section of the packet and adjust it - Duplicate the original mbuf passed to the hook via mbuf_dup - Use mbuf_copyback to copy over the adjusted data buffer into the duplicated mbuf - Send the newly built packet to ipf_inject_input - Discard the original packet by returning EJUSTRETURN This is working excellently on vanilla Ethernet connections. The adjusted data shows up in the final client application and everything is great and very stable (automated tests have run over 10,000 adjusted packets across the running NKE without problems). Unfortunately, something goes wrong on both my Airport-enabled iBook and the AIrport Extreme connection on my Mini. On these connections, the modified packet never reaches the client application up in userland. ipf_inject_input returns 0 and no errors occur during the construction process. I've seen on other platforms (specifically Linux) that modifying a TCP packet in this way can cause the TCP stack to drop the packet because the checksumming becomes invalid. Even though just the payload (and not any of the IP/TCP headers) is modified, I suspect that may be happening somehow here as well - perhaps Airport's hardware checksumming is working differently from the Ethernet NIC? I vaguely remember reading somewhere on the list about the loopback interface setting a flag that effectively bypassed the checksum validation in the TCP stack. I've tried a couple of things like mbuf_inbound_modified() but that seems to cause the packet to get dropped in all connections. Is there a way to directly compute the checksums or something so that this modified packet will pass through the stack properly? Or am I barking up entirely the wrong tree with this modification method? Thanks in advance, _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel@lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.a... This email sent to site_archiver@lists.apple.com On the inbound path, you modify the payload. This makes the TCP checksum invalid. The hardware has already done a checksum. After your filter does it's thing, the tcp portion of the stack verifies the checksum. Since the checksum was already doneSince the value calculated by hardware was done before you modified the packet, it matches the checksum that you didn't update. When you are using an interface that does not do hardware checksums, the checksum is done entirely in software, after you modified the packet. Since you didn't update the tcp checksum when you modified the payload, the checksum is no longer valid. The packet will be discarded. You can see the number of packets discarded due to an invalid checksum by using netstat -s. mbuf_inbound_modified will clear the flags that indicate the hardware has performed a checksum. This will force the checksum in software. It may be possible to set the flags indicating that the checksum was calculated in hardware and it is valid. Unless you verify the checksum before setting these flags, this would bypass any checksum verification and allow corrupt data through. It is very important that this not be done. If you're going to modify the packet, you should modify the checksum appropriately. If you choose to recalculate the checksum entirely, you should verify the original checksum is correct before making any modifications to the packet. Once the checksum has been updated, call mbuf_inbound_modified to flush the invalid hardware checksum flags. Frank smime.p7s