Re: Poor performance of SOCK_RAW/PF_PACKET on MacOSX.. (libpcap is better.. sorta..)
Re: Poor performance of SOCK_RAW/PF_PACKET on MacOSX.. (libpcap is better.. sorta..)
- Subject: Re: Poor performance of SOCK_RAW/PF_PACKET on MacOSX.. (libpcap is better.. sorta..)
- From: Robert Monaghan <email@hidden>
- Date: Mon, 17 Feb 2014 10:23:29 +0100
Hi Ivan,
Very good suggestions! As far as DTrace goes, I haven’t excersized it in any serious fashion.
I have been attempting to use Instruments. But I get a lot of DTrace errors caused by GCD, which in turn causes Instruments to fail when I try to profile.
It is something I need to work through. Creating reproducible examples for Apple is really challenging in this case, too. “Simple” examples work fine. (Naturally).
As for traffic, I do have the following over the same link:
- TCP/IP socket for issuing commands and other data read back.
- UDP “ping” for ensuring the device is still there.
Now that you mention it, while I have some code in place that pauses TCP/IP activity while the download is in progress, I need to do the same for my UDP traffic.
Thanks for the suggestions!!
bob.
On Feb 17, 2014, at 10:06 AM, Ivan Ostres <email@hidden> wrote:
> Hi Robert,
>
> first smart thing to do would be to exactly identify where the problem happens. Too many interrupts from NIC (I doubt that but worth to check anyway)? Ring buffer overflow? user-kernel-user context switches? Slow mbuf handling? kernel-userland data copy? DTrace is very useful in those cases and I would really recommend it.
>
> Also a few interesting points - you say this device is connected via point-to-point link with your Mac; does that mean that traffic you're interested in is only traffic that is transmitted via this interface (no ARP, ICMP or any other traffic) or there is also other kind of traffic that needs to be processed via network stack?
>
> BR,
> Ivan
>
> On 13/02/14 15:02, Robert Monaghan wrote:
>> Hi Everyone,
>>
>> I am looking for suggestions!
>>
>> I am working on a software package that reads custom network frames from a hardware device.
>> This device works over 10GBaseT and can saturate the network pretty easily. As the data is image data, all of the data is needed.
>> Dropped packets during a transfer, means that the entire image has to be re-downloaded.
>>
>> On Linux, I have sockets based code using SOCK_RAW,PF_PACKET in combination with PF_RING (using a setsockopt command).
>> This code is easily able to capture all of the raw packets as they are sent from my hardware device.
>> Downloading a bunch of packets with this code results in about 400 to 600Mb/s.
>>
>> I am struggling to get similar performance with the Mac. (MBP/Retina, MacPro 2010 - 12 Core, 2011 iMac — I have tried several models.)
>>
>> The first attempt used sockets in a very similar fashion as the code mentioned above, but without any kind of “PF_RING” type memory buffer. I ended up abandoning this approach, as there doesn’t appear to be able to capture all of the packets reliably. I have to throttle the hardware device down to about 50Mb/s just to get all of the packets.
>>
>> I might as well use Gigabit Ethernet at this point.
>>
>> I also worked with BPF Filters in libpcap, too. This has provided significantly better performance, but requires you to process each and ever packet as they come in. There is no mechanism to batch a bunch of packets using its pcap_loop() function. Not only that, but each packet has to be copied out of the BPF buffer into your own buffer, adding additional work that has to done during the transfer. (Needless to say, this causes a lot dropped packets, too.)
>>
>> Is there a way to open a raw socket and reliably capture number of frames into a fast ring buffer? - or -
>> Is there a way to have libpcap capture directly into a user land buffer return the buffer when complete?
>>
>> Am I going to have to “roll my own” BPF based system, reading into some sort of user land buffer?
>>
>> Thanks!
>>
>> bob.
>>
>
_______________________________________________
Do not post admin requests to the list. They will be ignored.
Macnetworkprog mailing list (email@hidden)
Help/Unsubscribe/Update your Subscription:
This email sent to email@hidden