February 2005 - Darwin-kernel

Re: select() weirdness
by Justin Walker 26 Feb '05

26 Feb '05

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com On Feb 25, 2005, at 17:34, Steve Dekorte wrote: How many sockets are connecting simultaneously to this server? Regards, Justin -- Justin C. Walker, Curmudgeon-At-Large * Institute for General Semantics | Men are from Earth. | Women are from Earth. | Deal with it. *--------------------------------------*-------------------------------* _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… On Feb 25, 2005, at 5:18 PM, Agent M wrote: The funnel issue mentioned by Quinn could pose the issue but unlikely for 20 secs. Have you used a time counter within the program or done profiling which confirms the issue? You should definitely run Shark as Quinn mentioned to confirm that your program is blocking on select and not on something else- you may simply have a starved coroutine, if this is io you're working on. Also, count the number of calls you can make before the weird blocking and see if it consistent. You issue does sound odd, so I wouldn't rule out an OS bug, but double-check all the inputs into select- I usually forget that select changes most of its pointer arguments on return. Also check for possible interfering signals- select could be interrupted (EINTR) and you must catch that. Some further poking around revealed that select() was not the culprit (though the fact that gdb always broke on select() made it suspicious) the problem was that some socket connects take a mysteriously long time or timeout altogether. This occurs whether I'm hitting an apache server or a mini server I've written for testing. The solution seems to be setting the Socket connect timeout lower and just dealing with the failed connections. I'd really like to know why a connect would ever fail in this situation though. Is > 6,000 connections per second too much to ask for? Possibly. It depends on how much memory you have; how many ephemeral ports you have for connections; and some random variables I've forgotten. To debug this, it might be of interest to know: - is there a low-level error, or is "connect()" itself really hanging? - what is happening on the wire when the stall occurs (tcpdump). - what does "netstat" say when you are in this condition? If you have a lot of connections in close-wait or closed state, that might be the answer (although I'd be hard-pressed to explain the hang if that's the case). If there is really a hang, I'd look elsewhere for an answer (e.g., off-host or in a library somewhere). This email sent to site_archiver(a)lists.apple.com

1 0

Re: KEV_DL_IF_DETACHED/KEV_DL_IF_ATTACHED: how to?
by Justin Walker 26 Feb '05

26 Feb '05

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com On Feb 25, 2005, at 8:36, Stephane Sudre wrote: Regards, Justin -- Justin C. Walker, Curmudgeon-At-Large * Institute for General Semantics | When LuteFisk is outlawed | Only outlaws will have | LuteFisk *--------------------------------------*-------------------------------* _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… What kind of operation/action can trigger a KEV_DL_IF_ATTACHED or KEV_DL_IF_DETACHED kernel event? If you haven't tried it yet, consider getting a copy of the xnu kernel source and using 'cscope'. This will let you track down the answer to such questions (as long as the answers lie in the BSD/Mach part of the kernel; cscope does not grok C++ [and it's not alone]). Ejecting/inserting a WiFi PCMCIA card in an old PowerBook for instance? It probably depends on exactly what the device handler and the corresponding IONetwork family support does. I believe that, for the Ethernet family, insertion/ejection of the card (or a cable) will cause such events, but the code tells the real story. Note that the IONetworkFamily code is a separate module, and is packaged separately in the darwin repository. This email sent to site_archiver(a)lists.apple.com

1 0

Re: select() weirdness
by Steve Dekorte 26 Feb '05

26 Feb '05

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com How many sockets are connecting simultaneously to this server? -- Steve _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… On Feb 25, 2005, at 5:18 PM, Agent M wrote: The funnel issue mentioned by Quinn could pose the issue but unlikely for 20 secs. Have you used a time counter within the program or done profiling which confirms the issue? You should definitely run Shark as Quinn mentioned to confirm that your program is blocking on select and not on something else- you may simply have a starved coroutine, if this is io you're working on. Also, count the number of calls you can make before the weird blocking and see if it consistent. You issue does sound odd, so I wouldn't rule out an OS bug, but double-check all the inputs into select- I usually forget that select changes most of its pointer arguments on return. Also check for possible interfering signals- select could be interrupted (EINTR) and you must catch that. Some further poking around revealed that select() was not the culprit (though the fact that gdb always broke on select() made it suspicious) the problem was that some socket connects take a mysteriously long time or timeout altogether. This occurs whether I'm hitting an apache server or a mini server I've written for testing. The solution seems to be setting the Socket connect timeout lower and just dealing with the failed connections. I'd really like to know why a connect would ever fail in this situation though. Is > 6,000 connections per second too much to ask for? This email sent to site_archiver(a)lists.apple.com

1 0

Re: select() weirdness
by Quinn 26 Feb '05

26 Feb '05

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com I hope (-: S+E -- Quinn "The Eskimo!" <http://www.apple.com/developer/> Apple Developer Technical Support * Networking, Communications, Hardware _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… At 13:56 -0800 25/2/05, Steve Dekorte wrote: Here's the results of grabbing a sample while select() was hanging for several seconds in Shark: http://www.dekorte.com/Library/select.jp2 If I understand this properly, you've selected just your process (ioServer) in the "Process" box at the bottom of the Shark display. If so, that's not going to help you much because your process is probably blocked waiting for another process to yield the funnel. If you switch to viewing all processes, you'll be able to see what the rest of the system was doing while you were waiting. btw It might be helpful to disable your second CPU for the duration of this test. This email sent to site_archiver(a)lists.apple.com

1 0

Re: select() weirdness
by Steve Dekorte 25 Feb '05

25 Feb '05

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com Thanks for the response. Notes: Any suggestions? -- Steve _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… On Feb 25, 2005, at 7:48 AM, Quinn wrote: At 7:19 -0800 25/2/05, Steve Dekorte wrote: I'm calling select() with a timeout of 0 and it usually returns immediately, but every few thousand calls, it will hang for a few seconds or more (up to about a 20 seconds!). The read and write fdset sizes are always 4. Does anyone know what could be causing this? select is implemented by the BSD subsystem. Concurrency control in the BSD subsystem (at least prior to Tiger) is controlled by two funnels (giant mutexen), one for networking and another for the rest of the kernel. It could be that your call to select is being held up behind another thread doing something expensive while holding one of the BSD funnels. My guess is that you could debug this using Shark (run Shark's sampling profiler, then stop it programmatically when you notice a long latency select; failing that, the "chudKDebugTrace.h" looks mighty interesting), but I don't have time to research the details today. Here's the results of grabbing a sample while select() was hanging for several seconds in Shark: http://www.dekorte.com/Library/select.jp2 System: Darwin Kernel Version 7.8.0 Power Macintosh powerpc (dual 2.5Ghz G5) Select is only being given sockets waiting to connect only. As soon as they connect, I write one line and then close the socket. This is a single threaded application and I shut down all desktop apps (other than Terminal and Shark) for this test. This email sent to site_archiver(a)lists.apple.com

1 0

Re: Multiple IOMedia objects creation in Filter Scheme Driver
by Garth Cummings 25 Feb '05

25 Feb '05

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com Hi Imran, On Feb 24, 2005, at 10:05 PM, Imran wrote: Hi Garth, How did you put filesystems on those IOMedia objects? Thanks in Advance Imran --gc Hi Imran, On Feb 23, 2005, at 12:33 AM, Imran wrote: Hi Garth, Please find the output of 'ioreg -lw O' after loading my filter scheme driver for Snap1 and Snap2 partitions with ContentHint as "MACOSX_MyContent". <snip> This is behaving correctly. If I strip out the properties it'll be easier to see. | | | | | +-o CNSi R2500:A Media <class IOMedia, ...> | | | | | | | | | | | +-o IOMediaBSDClient <class IOMediaBSDClient, ...> | | | | | +-o IOApplePartitionScheme <class IOApplePartitionScheme, ...> | | | | | | | | | | | +-o Apple@1 <class IOMedia, ...> | | | | | | | | | | | | | +-o IOMediaBSDClient <class IOMediaBSDClient, ...> | | | | | +-o Snap1@2 <class IOMedia, ...> | | | | | | | | | | | | | +-o IOMediaBSDClient <class IOMediaBSDClient, ...> | | | | | | | | | | | | | +-o com_MySoftwareCompany_driver_MyFilterScheme <class com_MySoftwareCompany_driver_MyFilterScheme, ...> | | | | | | | | | | | | | +-o MACOSX_MyContent 1@1 <class IOMedia, ...> | | | | | | | | | | | | | +-o IOMediaBSDClient <class IOMediaBSDClient, ...> | | | | | | | | | | | +-o Snap2@3 <class IOMedia, ...> | | | | | | | | | | | +-o IOMediaBSDClient <class IOMediaBSDClient, ...> | | | | | | | | | | | +-o com_MySoftwareCompany_driver_MyFilterScheme <class com_MySoftwareCompany_driver_MyFilterScheme, ...> | | | | | | | | | | | +-o MACOSX_MyContent 1@1 <class IOMedia, ...> | | | | | | | | | | | +-o IOMediaBSDClient <class IOMediaBSDClient, ...> | | | | | Both the Snap1 and Snap2 partitions have the same chain of objects attached: IOMedia (Snap*) > IOMediaBSDClient > com_MySoftwareCompany_driver_MyFilterScheme > IOMedia (MACOSX_MyContent) > IOMediaBSDClient. The child media objects you published are at /dev/disk2s2s1 and /dev/disk2s3s1 respectively (the "BSD Name" property in the IOMedia objects). Is this clearer now? __________________________________________________________________ Garth Cummings Apple Developer Technical Support gcummings(a)apple.com _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… This email sent to site_archiver(a)lists.apple.com Thanks a lot Garth for your help. But the problem I am facing is not getting solved. Diskutil shows the IOMedia objects as disk2s2s1 disk2s3s1 respectively, but this IOMedia objects are not getting mounted after making file system on it. It may be that we are missing something you can point out What does "sudo fsck_hfs -n -f /dev/rdisk2s2s1" and "sudo fsck_hfs -n -f /dev/rdisk2s3s1" tell you? -----Original Message----- From: darwin-kernel-bounces+sayyad.asif=spsoftindia.com(a)lists.apple.com [mailto:darwin-kernel- bounces+sayyad.asif=spsoftindia.com(a)lists.apple.com] On Behalf Of Garth Cummings Sent: Friday, February 25, 2005 12:23 AM To: Imran Cc: darwinKernel Apple List Subject: Re: Multiple IOMedia objects creation in Filter Scheme Driver --gc http://developer.apple.com/technicalsupport smime.p7s

1 0

getfsent return only nfs volumes entries ??
by Hervé Kergourlay 25 Feb '05

25 Feb '05

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com User-agent: Mozilla Thunderbird 1.0 (Windows/20041206) I've a problem with the getfsent() api the api returns only my nfs mount point the local disks are ignored, getfsent return NULL after the nfs mount point shrek:/etc root# mount /dev/disk0s5 on / (local, journaled) devfs on /dev (local) fdesc on /dev (union) <volfs> on /.vol /dev/disk0s2 on /Volumes/Data UFS (asynchronous, local) /dev/disk0s3 on /Volumes/Data (local, journaled) /dev/disk0s7 on /Volumes/Tiger (local, journaled) automount -nsl [359] on /Network (automounted) automount -fstab [366] on /automount/Servers (automounted) automount -static [366] on /automount/static (automounted) /dev/disk0s9 on /Volumes/Data HFS (local) /dev/disk0s11 on /Volumes/Desaster (local) canardo:/usr/people on /private/automount/usr/people (automounted) is it a bug ? is there any other "posix" solution ? best regards hervé _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… I want to check the "dev" field of all theses disks as the "mount" command shows it here This email sent to site_archiver(a)lists.apple.com

1 0

KEV_DL_IF_DETACHED/KEV_DL_IF_ATTACHED: how to?
by Stephane Sudre 25 Feb '05

25 Feb '05

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com Ejecting/inserting a WiFi PCMCIA card in an old PowerBook for instance? _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… What kind of operation/action can trigger a KEV_DL_IF_ATTACHED or KEV_DL_IF_DETACHED kernel event? This email sent to site_archiver(a)lists.apple.com

1 0

Re: select() weirdness
by Quinn 25 Feb '05

25 Feb '05

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com S+E -- Quinn "The Eskimo!" <http://www.apple.com/developer/> Apple Developer Technical Support * Networking, Communications, Hardware _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… At 7:19 -0800 25/2/05, Steve Dekorte wrote: I'm calling select() with a timeout of 0 and it usually returns immediately, but every few thousand calls, it will hang for a few seconds or more (up to about a 20 seconds!). The read and write fdset sizes are always 4. Does anyone know what could be causing this? select is implemented by the BSD subsystem. Concurrency control in the BSD subsystem (at least prior to Tiger) is controlled by two funnels (giant mutexen), one for networking and another for the rest of the kernel. It could be that your call to select is being held up behind another thread doing something expensive while holding one of the BSD funnels. My guess is that you could debug this using Shark (run Shark's sampling profiler, then stop it programmatically when you notice a long latency select; failing that, the "chudKDebugTrace.h" looks mighty interesting), but I don't have time to research the details today. This email sent to site_archiver(a)lists.apple.com

1 0

select() weirdness
by Steve Dekorte 25 Feb '05

25 Feb '05

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com Does anyone know what could be causing this? -- Steve _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… I'm calling select() with a timeout of 0 and it usually returns immediately, but every few thousand calls, it will hang for a few seconds or more (up to about a 20 seconds!). The read and write fdset sizes are always 4. This email sent to site_archiver(a)lists.apple.com

1 0