Re: Network stack/ethernet driver issues
Re: Network stack/ethernet driver issues
- Subject: Re: Network stack/ethernet driver issues
- From: Andreas Fink <email@hidden>
- Date: Wed, 9 Apr 2008 23:31:42 +0000
On 09.04.2008, at 21:09, email@hidden wrote: Greetings,
Here's my situation: I have a custom piece of data logging hardware that
generates about 50MB/sec of data and sends it out over a gigabit ethernet
link as a steady stream of UDP packets (1k packet every ~20 microseconds,
very periodic, not bursty at all) broadcasted to the local subnet. I want
to log all this data to disk for offline analysis. As a first step I
wrote a program to just listen for the UDP packets and verify that they
were all being received. This is easy to do because each packet has a 16
bit sequence ID in it. Much to my dismay, I was unable to reliably
achieve this. I am running 10.5.2 on a 2.33 gHz MacBook Pro with 2G of
memory. After making SO_RCVBUF big enough, I was still occasionally
missing a block of 100+ packets at a time. Looking in the system.log file
I realized that each dropout was correlated to a message like:
Apr 9 13:38:51 Macintosh-6 kernel[0]: AppleYukon2: 00000000,00000000
skgehw - cppSkDrvEvent - SK_DRV_RX_OVERFLOW: rcv fifo overflow
Apr 9 13:38:51 Macintosh-6 kernel[0]: AppleYukon2: 00000000,00000000 sky2
- RX ring overflow -- dropped a packet
I think your problem is related to the fact that your userspace application can't keep up with the sent data from the kernel and thus the buffer in the kernel overruns. 50MBytes/sec of data is quite a bit and because your application writes it to disk, you need disks which can write that fast (my MacBook Pro disk can only handle something like 35MB/sec) If they are not as fast, you will naturally run out of buffer space at some time. I would suggest you redo this experiment with a striped array of firewire 800 disks.
If you of course only write it to memory, its a different story. You can test this, by writing a sample application which only reads the id and drops the packet without doing anything else and see if that still doesn't keep up. You might have to think about how you read the UDP packets and if there's anything you can do there to improve that speed.
I believe your problem however is simply disk speed.
Andreas Fink
Fink Consulting GmbH Global Networks Schweiz AG BebbiCell AG
--------------------------------------------------------------- Tel: +41-61-6666330 Fax: +41-61-6666331 Mobile: +41-79-2457333 Address: Clarastrasse 3, 4058 Basel, Switzerland --------------------------------------------------------------- ICQ: 8239353 MSN: email@hidden AIM: smsrelay Skype: andreasfink Yahoo: finkconsulting SMS: +41792457333
|
_______________________________________________
Do not post admin requests to the list. They will be ignored.
Darwin-kernel mailing list (email@hidden)
Help/Unsubscribe/Update your Subscription:
This email sent to email@hidden