December 2006 - Darwin-kernel

Re: mbuf_outbound_finalize bug?
by Bhavesh Davda 22 Dec '06

22 Dec '06

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com Domainkey-signature: a=rsa-sha1; q=dns; c=nofws; s=beta; d=gmail.com; h=received:message-id:date:from:to:subject:cc:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references; b=Y0+trIKMW/5G1zuLzxbD9onbTkAMCchrjnfMO2EUa4mf7Iie1ejOI6rP3ZLYnA6hJHJfQC/z/NXYElFPZKUYu+aE3PCX5eGG7jLDNwtBSElMwk2NCq5K4PfhiQFK67yM7D+vw+FkOyxiTpKJyALYzyLt5j5hz6QLRYZvqodjOEU= So I did this from my iff_output_func callback: mbuf_get_csum_requested(*data, &csumFlags, NULL); if (csumFlags) { ipHdr = (struct iphdr *)((uint8 *)mbuf_data(*data) + ETHER_HDR_LEN); if (ipHdr->tot_len > mbuf_pkthdr_len(*data)) { /* ip_len must be in network-byte-order. Byte-swap it. */ ipHdr->tot_len = ntohs(ipHdr->tot_len); swapIpLen = TRUE; } mbuf_outbound_finalize(*data, PF_INET, ETHER_HDR_LEN); /* byte-swap ip_len back. */ if (swapIpLen) { ipHdr->tot_len = htons(ipHdr->tot_len); } I *think* this is what your suggestion was. This makes the kernel panic! I attached to the kernel and examined the mbuf, and it looked totally messed up. Several pointers were pointing to la-la land, the data in m->m_hdr.mh_data was munged from its original values, etc. The same was the case (messed up mbuf) when I set a breakpoint after mbuf_outbound_finalize() and examined it, on a non-panic'ed kernel. What am I, as poor KEXT author, doing wrong? The i386 version of in_cksum_skip() is too hard for a simple minded person like me to figure out :) Thanks! - Bhavesh On Dec 20, 2006, at 8:37 PM, Bhavesh Davda wrote: > Thanks, Adi, > > So how do I do the workaround (byte-swap ip->ip_len in iff_output_func > before calling mbuf_outbound_finalize, and then byte-swap it back), in > a "future proof" way, so I don't have to reverse this workaround when > Apple fixes the bug? > In theory you won't have to do anything; the stack should detect if the length has been reversed by the filter and internally do the byte-swap (from network to host) only if necessary prior to calculating the transport checksum. Likewise, it will do the host to network byte-swap of that value (internally) for the IP header checksum calculation. In any case, the length in the IP header of the packet will be restored to what the filter has set it prior to calling the routine, upon return. That way the packet won't end up incorrect should the filter reverses the length back to network byte-order prior to sending it down. With the fix in place, the only redundant work done by your filter would then be the IP header checksum calculation though it shouldn't cost too much when compared to the cost of checksumming the payload. (I.e. without the fix, this step is necessary. With the fix, it's not, but it won't hurt as long as your code does the right computation.) Adi > Or the fact that my NKE has to work on both older and newer kernels? > > Thanks! > > - Bhavesh > > > On 12/20/06, Adi Masputra <adi(a)apple.com> wrote: >> >> On Dec 20, 2006, at 6:17 PM, Bhavesh Davda wrote: >> >> > When calling mbuf_outbound_finalize() to compute the incomplete >> > TCP/UDP checksums on the mbuf from my iff_output_func (interface >> > filter output function), I kept getting messages from the kernel >> > "cksum: out of data". >> > >> > Looking at the xnu-792.13.8 source, it looks like the underlying >> > in_cksum_skip, called from in_delayed_cksum_offset, expects the >> ip_len >> > field in the IP header to be in host-byte-order, while in >> reality the >> > mbuf handed to my iff_output_func had the ip_len field in the IP >> > header correctly populated in network-byte-order. >> > >> > What is my iff_output_func to do in such a situation? >> >> This is a known bug and a fix is already in order. Assuming that >> you call this from an interface filter, and that only the transport >> checksum is to be recomputed, you should be able to byte-swap the >> IP length, call mbuf_outbound_finalize() and restore the IP length >> back to the network-byte order. If the driver offloads IP header >> checksumming, and if this is an IPv4 packet, then you would need >> to recompute the IP header checksum yourself prior to sending the >> packet downstream. Once the fix is made available, you will no >> longer need to do any of the above. >> >> (Also note that this issue does not exist for PPC.) >> >> Adi >> >> > >> > Thanks! >> > >> > -- >> > Bhavesh P. Davda >> > _______________________________________________ >> > Do not post admin requests to the list. They will be ignored. >> > Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) >> > Help/Unsubscribe/Update your Subscription: >> > http://lists.apple.com/mailman/options/darwin-kernel/adi% >> 40apple.com >> > >> > This email sent to adi(a)apple.com -- Bhavesh P. Davda _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… On 12/20/06, Adi Masputra <adi(a)apple.com> wrote: This email sent to site_archiver(a)lists.apple.com

1 0

Re: IOMemoryDescriptor / prepare selectively wireing only parts of it
by Michael Smith 21 Dec '06

21 Dec '06

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com User-agent: Thunderbird 1.5.0.9 (Macintosh/20061207) We have to do a DMA transfer with that buffer. Use an IOSubMemoryDescriptor. = Mike _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… devmaillists wrote: we currently trying to catch the problem that we get an arbitrary sized buffer supplied from userspace. The user land interface is fixed, so we have to work with what we get. We have to use IODMACommand to ensure 64 Bit compatibility. To get the segments from the IODMACommand we have to prepare the whole memory descriptor. It appears our pci device preferres page sized chunks of wich we can send 64 in one transfer. So we like prepare only these 64 pages for each transfer. I know references to linux source are somewhat shunned, but pci_map_page does this exactly. We want to avoid to prepare the entire memory range, because wireing down 200mb seams not to be a good idea. This email sent to site_archiver(a)lists.apple.com

1 0

IOMemoryDescriptor / prepare selectively wireing only parts of it
by devmaillists 21 Dec '06

21 Dec '06

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… This email sent to site_archiver(a)lists.apple.com

1 0

Re: FTP: 229 Entering extended passive mode. ??
by Terry Lambert 21 Dec '06

21 Dec '06

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com Hi, While trying to do an FTP to a server, my client hangs on > ls Am able to login into the server, afterwards on ls I get > 229 Entering Extended passive mode followed by a timeout and connection close by the server. If I turnoff EPSV doing > EPSV (its a toggle) , just after login, Note: This has really nothing to do with Darwin Kernel... -- Terry _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… On Dec 20, 2006, at 5:48 PM, Ravi wrote: Things work fine, is the mac default ftp client not compatible to the Extended passive mode? Or do I need to change some setting in some configuration file? It's not your client; the response you say you are getting is invalid according to RFC 2428. Check your firewall, their firewall, and whether their FTP server is advertising that it supports EPSV. If it's your firewall, then either reconfigure it or change your FTP client defaults. If their FTP server advertises that they support EPSV, then either their firewall is misconfigured (or broken), or their FTP server is misconfigured with respect to their firewall settings (though a good stateful firewall would take this into account already, so this could be considered firewall breakage as well). Some FTP software doesn't support reconfiguration of this option (and you are back to the firewall taking this into account). This email sent to site_archiver(a)lists.apple.com

1 0

Re: mbuf_outbound_finalize bug?
by Adi Masputra 21 Dec '06

21 Dec '06

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com On Dec 20, 2006, at 8:37 PM, Bhavesh Davda wrote: Thanks, Adi, So how do I do the workaround (byte-swap ip->ip_len in iff_output_func before calling mbuf_outbound_finalize, and then byte-swap it back), in a "future proof" way, so I don't have to reverse this workaround when Apple fixes the bug? In theory you won't have to do anything; the stack should detect if the length has been reversed by the filter and internally do the byte-swap (from network to host) only if necessary prior to calculating the transport checksum. Likewise, it will do the host to network byte-swap of that value (internally) for the IP header checksum calculation. In any case, the length in the IP header of the packet will be restored to what the filter has set it prior to calling the routine, upon return. That way the packet won't end up incorrect should the filter reverses the length back to network byte-order prior to sending it down. With the fix in place, the only redundant work done by your filter would then be the IP header checksum calculation though it shouldn't cost too much when compared to the cost of checksumming the payload. (I.e. without the fix, this step is necessary. With the fix, it's not, but it won't hurt as long as your code does the right computation.) Adi Or the fact that my NKE has to work on both older and newer kernels? Thanks! - Bhavesh On Dec 20, 2006, at 6:17 PM, Bhavesh Davda wrote: This is a known bug and a fix is already in order. Assuming that you call this from an interface filter, and that only the transport checksum is to be recomputed, you should be able to byte-swap the IP length, call mbuf_outbound_finalize() and restore the IP length back to the network-byte order. If the driver offloads IP header checksumming, and if this is an IPv4 packet, then you would need to recompute the IP header checksum yourself prior to sending the packet downstream. Once the fix is made available, you will no longer need to do any of the above. (Also note that this issue does not exist for PPC.) Adi _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… This email sent to site_archiver(a)lists.apple.com On 12/20/06, Adi Masputra <adi(a)apple.com> wrote: > When calling mbuf_outbound_finalize() to compute the incomplete > TCP/UDP checksums on the mbuf from my iff_output_func (interface > filter output function), I kept getting messages from the kernel > "cksum: out of data". > > Looking at the xnu-792.13.8 source, it looks like the underlying > in_cksum_skip, called from in_delayed_cksum_offset, expects the ip_len > field in the IP header to be in host-byte-order, while in reality the > mbuf handed to my iff_output_func had the ip_len field in the IP > header correctly populated in network-byte-order. > > What is my iff_output_func to do in such a situation? > > Thanks! > > -- > Bhavesh P. Davda > _______________________________________________ > Do not post admin requests to the list. They will be ignored. > Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) > Help/Unsubscribe/Update your Subscription: > http://lists.apple.com/mailman/options/darwin-kernel/adi% 40apple.com > > This email sent to adi(a)apple.com smime.p7s

1 0

Re: mbuf_outbound_finalize bug?
by Bhavesh Davda 21 Dec '06

21 Dec '06

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com Domainkey-signature: a=rsa-sha1; q=dns; c=nofws; s=beta; d=gmail.com; h=received:message-id:date:from:to:subject:cc:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references; b=UvtMkGPWimralzo5efjSdvSzkP6ue/KEcPTH0Goni4XZxTc6zbHAdPQz6gCS3FPcobQuf1DJiQ/bK+y9/GutC4ZDji2+vDx7lQdtY62s3zWXGjrW3fqQJY0ItHLOQdIoDtgMh4ZQljKyi0ZRlKZC+CeTXx7COiBFaBFGbScjCp4= Thanks, Adi, So how do I do the workaround (byte-swap ip->ip_len in iff_output_func before calling mbuf_outbound_finalize, and then byte-swap it back), in a "future proof" way, so I don't have to reverse this workaround when Apple fixes the bug? Or the fact that my NKE has to work on both older and newer kernels? Thanks! - Bhavesh On Dec 20, 2006, at 6:17 PM, Bhavesh Davda wrote: > When calling mbuf_outbound_finalize() to compute the incomplete > TCP/UDP checksums on the mbuf from my iff_output_func (interface > filter output function), I kept getting messages from the kernel > "cksum: out of data". > > Looking at the xnu-792.13.8 source, it looks like the underlying > in_cksum_skip, called from in_delayed_cksum_offset, expects the ip_len > field in the IP header to be in host-byte-order, while in reality the > mbuf handed to my iff_output_func had the ip_len field in the IP > header correctly populated in network-byte-order. > > What is my iff_output_func to do in such a situation? This is a known bug and a fix is already in order. Assuming that you call this from an interface filter, and that only the transport checksum is to be recomputed, you should be able to byte-swap the IP length, call mbuf_outbound_finalize() and restore the IP length back to the network-byte order. If the driver offloads IP header checksumming, and if this is an IPv4 packet, then you would need to recompute the IP header checksum yourself prior to sending the packet downstream. Once the fix is made available, you will no longer need to do any of the above. (Also note that this issue does not exist for PPC.) Adi > > Thanks! > > -- > Bhavesh P. Davda > _______________________________________________ > Do not post admin requests to the list. They will be ignored. > Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) > Help/Unsubscribe/Update your Subscription: > http://lists.apple.com/mailman/options/darwin-kernel/adi%40apple.com > > This email sent to adi(a)apple.com _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… On 12/20/06, Adi Masputra <adi(a)apple.com> wrote: This email sent to site_archiver(a)lists.apple.com

1 0

Re: FTP: 229 Entering extended passive mode. ??
by Paul Ripke 21 Dec '06

21 Dec '06

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com Mail-followup-to: Ravi <ryadav(a)appliedidentity.com>, darwin-kernel(a)lists.apple.com User-agent: Mutt/1.4.2.1i On Wed, Dec 20, 2006 at 05:48:09PM -0800, Ravi wrote: > Hi, > While trying to do an FTP to a server, my client hangs on > > ls > Am able to login into the server, afterwards on ls I get > > 229 Entering Extended passive mode > followed by a timeout and connection close by the server. > > If I turnoff EPSV doing > > EPSV (its a toggle) , just after login, > > Things work fine, is the mac default ftp client not compatible to the > Extended passive mode? > Or do I need to change some setting in some configuration file? Works fine for me on 10.4.8 - is your connection going through a firewall or NAT gateway by any chance? Even some of the leading brand firewall/NAT products break Extended Passive mode FTP. Cheers, -- Paul Ripke Unix/OpenVMS/TSM/DBA I love deadlines. I like the whooshing sound they make as they fly by. -- Douglas Adams _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… This email sent to site_archiver(a)lists.apple.com

1 0

Re: mbuf_outbound_finalize bug?
by Adi Masputra 21 Dec '06

21 Dec '06

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com On Dec 20, 2006, at 6:17 PM, Bhavesh Davda wrote: When calling mbuf_outbound_finalize() to compute the incomplete TCP/UDP checksums on the mbuf from my iff_output_func (interface filter output function), I kept getting messages from the kernel "cksum: out of data". Looking at the xnu-792.13.8 source, it looks like the underlying in_cksum_skip, called from in_delayed_cksum_offset, expects the ip_len field in the IP header to be in host-byte-order, while in reality the mbuf handed to my iff_output_func had the ip_len field in the IP header correctly populated in network-byte-order. This is a known bug and a fix is already in order. Assuming that you call this from an interface filter, and that only the transport checksum is to be recomputed, you should be able to byte-swap the IP length, call mbuf_outbound_finalize() and restore the IP length back to the network-byte order. If the driver offloads IP header checksumming, and if this is an IPv4 packet, then you would need to recompute the IP header checksum yourself prior to sending the packet downstream. Once the fix is made available, you will no longer need to do any of the above. (Also note that this issue does not exist for PPC.) Adi Thanks! -- Bhavesh P. Davda _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/adi%40apple.com _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… This email sent to site_archiver(a)lists.apple.com What is my iff_output_func to do in such a situation? This email sent to adi(a)apple.com smime.p7s

1 0

mbuf_outbound_finalize bug?
by Bhavesh Davda 21 Dec '06

21 Dec '06

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com Domainkey-signature: a=rsa-sha1; q=dns; c=nofws; s=beta; d=gmail.com; h=received:message-id:date:from:to:subject:mime-version:content-type:content-transfer-encoding:content-disposition; b=gPuO+2IPVvS6BgUBgAF7ViXsrX2M8NjtxLBgAzaBTIPyBhYKKwb4/nz74diIu2L4XP1QfomAJ58Cnt7g02fVyI0Efh0M3eBza+yjtvwn9kWNHEAna6F/AjiwInqJTH0bL+8u9cTi6jX+ez3qyR1uwPsyvb3xnPi8Dpck4qkKaoo= When calling mbuf_outbound_finalize() to compute the incomplete TCP/UDP checksums on the mbuf from my iff_output_func (interface filter output function), I kept getting messages from the kernel "cksum: out of data". Looking at the xnu-792.13.8 source, it looks like the underlying in_cksum_skip, called from in_delayed_cksum_offset, expects the ip_len field in the IP header to be in host-byte-order, while in reality the mbuf handed to my iff_output_func had the ip_len field in the IP header correctly populated in network-byte-order. What is my iff_output_func to do in such a situation? Thanks! -- Bhavesh P. Davda _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… This email sent to site_archiver(a)lists.apple.com

1 0

FTP: 229 Entering extended passive mode. ??
by Ravi 21 Dec '06

21 Dec '06

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com Hi, While trying to do an FTP to a server, my client hangs on > ls Am able to login into the server, afterwards on ls I get > 229 Entering Extended passive mode followed by a timeout and connection close by the server. If I turnoff EPSV doing > EPSV (its a toggle) , just after login, Thanks _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… Things work fine, is the mac default ftp client not compatible to the Extended passive mode? Or do I need to change some setting in some configuration file? This email sent to site_archiver(a)lists.apple.com

1 0