December 2006 - Darwin-kernel

Re: implementing an AudioIOFunction
by Josh Rodatus 26 Dec '06

26 Dec '06

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com Hello all, I apologize for not being clear. Once again, sorry if I wasn't clear the first time. Thanks, Josh Is there any reason you can't do your audio processing in-line? = Mike _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… As I mentioned earlier, my user-space program is a music beat visualizer. It merely needs read access to the outbound audio stream, and it gets it from my kernel extension. No audio is sent back to the kernel. To minimize latency, all the AudioIOFunction does is send the samples over a socket to my program - no preprocessing. You will find the source code and more detail in one of my previous posts in this thread. So once again my problem is this. The client program is using very little CPU, yet the system audio stutters when I even as much as activate a window. So I conclude that using a socket to transfer the samples from kernel to program isn't efficient enough, and am in search of an alternative. On Dec 25, 2006, at 5:16 PM, Michael Smith wrote: Reading between the lines, I assume you are trying to process the outbound audio stream in user-space, before pushing it back into the kernel? This will be hard to make work well; the code in the kernel is likely to assume that the output path has a fairly low and constant latency. To emulate this you are going to have to elevate the priority of your user task, and if you mess that up the UI implications are severe. This email sent to site_archiver(a)lists.apple.com

1 0

Re: implementing an AudioIOFunction
by Josh Rodatus 25 Dec '06

25 Dec '06

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com Thanks, Josh _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… Okay. While the basic algorithm itself runs smoothly so far, my AudioIOFunction in the kernel is on the verge of breaking up the audio stream. So I'm thinking maybe sockets aren't efficient enough. Can I use an IOMemoryDescriptor to directly map the IOAudioStream's sample buffer into user space, or would I have to repeatedly copy it into a specially allocated area first? Any advice or suggestions are welcome. This email sent to site_archiver(a)lists.apple.com

1 0

Re: implementing an AudioIOFunction
by Ethan Clark 25 Dec '06

25 Dec '06

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… This email sent to site_archiver(a)lists.apple.com

1 0

Re: implementing an AudioIOFunction
by Josh Rodatus 25 Dec '06

25 Dec '06

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… This email sent to site_archiver(a)lists.apple.com

1 0

Re: implementing an AudioIOFunction
by Herb Petschauer 25 Dec '06

25 Dec '06

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com Domainkey-signature: a=rsa-sha1; q=dns; c=nofws; s=beta; d=gmail.com; h=received:message-id:date:from:to:subject:cc:in-reply-to:mime-version:content-type:references; b=uKBO4Z5wsnbeAtP/tY/96nPAFtCVIAM2j1MkyMiuYoEk2pndSrncKrwQIzBNq4dxhPnp+JA1sT+PLnGuXowMYBxmA4/wIx4kz5P5Dd50IV8TuDbdY1NbXhfUdGjHgpXwJ56X9i6jVxlk9IG+Rl4pXll4QF1wA9ewerXGoK3MReg= _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… This email sent to site_archiver(a)lists.apple.com

1 0

implementing an AudioIOFunction
by Josh Rodatus 24 Dec '06

24 Dec '06

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… This email sent to site_archiver(a)lists.apple.com

1 0

Re: mbuf_outbound_finalize bug?
by Bhavesh Davda 22 Dec '06

22 Dec '06

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com Domainkey-signature: a=rsa-sha1; q=dns; c=nofws; s=beta; d=gmail.com; h=received:message-id:date:from:to:subject:cc:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references; b=aXCELIwn4RYtPTry+zqwL5NDucVpZuEwE88tYHlxrQssVt8QNe5JH9hSqFFsyJhDvVm8iQXXyLviRlTse8mPli1jPxZd2ii3j+47bYmGxG8/owaMFOomBDvTWShCKXb4Kmi29g/ykYLlTpcsX9ZjWP5vIktB8YMnSIqk28wc2Ew= Since your module is an interface filter, the field(s) in the header(s) will be in network byte-order. All you need to do is to byte-swap the length at all times assuming it is an IPv4 packet. Okay. I'll treat this fact as an invariant. What will change is the expectation in mbuf_outbound_finalize() that the IP length field should be in host-byte-order. Well, you are already requesting the stack to perform software checksum calculation on the packet which includes the transport payload; in terms of per-byte cost that is much more significant than traversing the mbuf chain to safely obtain the 2-octets IP length in the header. :) Good point. So just to be clear: even if I *always* byte-swap the IP length to make it host-byte-order in my iff_output_func before calling mbuf_outbound_finalize, it will work now (10.4.8) *and* in a future kernel where mbuf_outbound_finalize and friends have been fixed? In other words: how do I write code to figure out that the issue has been fixed, and no byte-swapping is necessary any more? Thanks! -- Bhavesh P. Davda _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… the fix is already in order. This email sent to site_archiver(a)lists.apple.com

1 0

Re: mbuf_outbound_finalize bug?
by Adi Masputra 22 Dec '06

22 Dec '06

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com I compare the result of mbuf_pkthdr_len agains the network-byte-order value of IP length on purpose, to know whether indeed the value of IP length in the packet is *not* in host-byte-order, as I thought you said mbuf_outbound_finalize() expects. Since your module is an interface filter, the field(s) in the header(s) will be in network byte-order. All you need to do is to byte-swap the length at all times assuming it is an IPv4 packet. Oh boy. This is going to get quite expensive to do on a per packet basis. I thought I could simply get a pointer into the packet payload to the IP length field in the IP header, and do a memory read/write, rather than going through so many function calls and memcpy's, albeit small (2 byte) ones. Well, you are already requesting the stack to perform software checksum calculation on the packet which includes the transport payload; in terms of per-byte cost that is much more significant than traversing the mbuf chain to safely obtain the 2-octets IP length in the header. :) Is there any other way to do it more optimally? I understand your concern about non-linear mbufs, and not assuming that the IP header is in the pkthdr mbuf. But realistically, is that ever going to be the case? There's no guarantee that the headers will be in contiguous span although for the majority of cases they would. Sorry, but as I said the fix is already in order. Adi Thanks! -- Bhavesh P. Davda _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… This email sent to site_archiver(a)lists.apple.com On Dec 22, 2006, at 9:58 AM, Bhavesh Davda wrote: smime.p7s

1 0

Re: mbuf_outbound_finalize bug?
by Bhavesh Davda 22 Dec '06

22 Dec '06

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com Domainkey-signature: a=rsa-sha1; q=dns; c=nofws; s=beta; d=gmail.com; h=received:message-id:date:from:to:subject:cc:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references; b=K9Gs8+Kj1FDKlTsMjnlcJDYRY/224FG+8XZuueF6HrD9Oo/xkbVOQGcpgNxsN8ECjIFS7vtgnaKGje0GRO9zJrkN5SeKVpG3j2s0ITTv0jVHMBofGyKa5i4k4mGfFIEnsCFZ6gWkNo7XN1o55AHXc2rWDMaT5TuRw1xzQcOHC90= Hi! Your filter shouldn't assume that the headers are in a contiguous span; instead, you should probably use mbuf_copydata() to be safe. In addition, your code above compares the result of mbuf_pkthdr_len() against the *network* byte-order value of IP length; the ntohs() part should be done before that check. I compare the result of mbuf_pkthdr_len agains the network-byte-order value of IP length on purpose, to know whether indeed the value of IP length in the packet is *not* in host-byte-order, as I thought you said mbuf_outbound_finalize() expects. if (mbuf_copydata(*data, offset, sizeof (ip_len), &ip_len) == 0) { ip_len = ntohs(ip_len); mbuf_copyback(*data, offset, sizeof (ip_len), &ip_len, MBUF_WAITOK); mbuf_outbound_finalize(*data, protocol, sizeof (struct ether_header)); mbuf_copydata(*data, offset, sizeof (ip_len), &ip_len); ip_len = htons(ip_len); mbuf_copyback(*data, offset, sizeof (ip_len), &ip_len, MBUF_WAITOK); Oh boy. This is going to get quite expensive to do on a per packet basis. I thought I could simply get a pointer into the packet payload to the IP length field in the IP header, and do a memory read/write, rather than going through so many function calls and memcpy's, albeit small (2 byte) ones. Is there any other way to do it more optimally? I understand your concern about non-linear mbufs, and not assuming that the IP header is in the pkthdr mbuf. But realistically, is that ever going to be the case? Thanks! -- Bhavesh P. Davda _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… This email sent to site_archiver(a)lists.apple.com

1 0

Re: mbuf_outbound_finalize bug?
by Adi Masputra 22 Dec '06

22 Dec '06

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com On Dec 21, 2006, at 6:55 PM, Bhavesh Davda wrote: So I did this from my iff_output_func callback: mbuf_get_csum_requested(*data, &csumFlags, NULL); if (csumFlags) { ipHdr = (struct iphdr *)((uint8 *)mbuf_data(*data) + ETHER_HDR_LEN); if (ipHdr->tot_len > mbuf_pkthdr_len(*data)) { /* ip_len must be in network-byte-order. Byte-swap it. */ ipHdr->tot_len = ntohs(ipHdr->tot_len); swapIpLen = TRUE; } mbuf_outbound_finalize(*data, PF_INET, ETHER_HDR_LEN); /* byte-swap ip_len back. */ if (swapIpLen) { ipHdr->tot_len = htons(ipHdr->tot_len); } I *think* this is what your suggestion was. This makes the kernel panic! I attached to the kernel and examined the mbuf, and it looked totally messed up. Several pointers were pointing to la-la land, the data in m->m_hdr.mh_data was munged from its original values, etc. The same was the case (messed up mbuf) when I set a breakpoint after mbuf_outbound_finalize() and examined it, on a non-panic'ed kernel. What am I, as poor KEXT author, doing wrong? The i386 version of in_cksum_skip() is too hard for a simple minded person like me to figure out :) Try this instead: static errno_t iff_output(void *cookie, ifnet_t interface, protocol_family_t protocol, mbuf_t *data) { if (protocol == AF_INET) { u_int16_t ip_len; size_t offset = sizeof (struct ether_header) + offsetof(struct ip, ip_len); return (0); } Adi _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… This email sent to site_archiver(a)lists.apple.com Your filter shouldn't assume that the headers are in a contiguous span; instead, you should probably use mbuf_copydata() to be safe. In addition, your code above compares the result of mbuf_pkthdr_len() against the *network* byte-order value of IP length; the ntohs() part should be done before that check. if (mbuf_copydata(*data, offset, sizeof (ip_len), &ip_len) == 0) { ip_len = ntohs(ip_len); mbuf_copyback(*data, offset, sizeof (ip_len), &ip_len, MBUF_WAITOK); mbuf_outbound_finalize(*data, protocol, sizeof (struct ether_header)); mbuf_copydata(*data, offset, sizeof (ip_len), &ip_len); ip_len = htons(ip_len); mbuf_copyback(*data, offset, sizeof (ip_len), &ip_len, MBUF_WAITOK); } } smime.p7s

1 0