site_archiver@lists.apple.com Delivered-To: darwin-dev@lists.apple.com On Aug 21, 2009, at 12:58 AM, Dave Keck wrote: davekeck 4880 alex 4u KQUEUE count=0, state=0 Woops, I excluded this line in that code: CFFileDescriptorEnableCallBacks(fileDescriptor, kCFFileDescriptorReadCallBack); Also, why isn't the CFFileDescriptor source posted (http://www.opensource.apple.com/source/CF/CF-476.19/)? Cheers, M _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev@lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.appl... I shaved down the code from the previous email from code which did indeed add the CFFileDescriptor to the runloop and leaked. However, I did not remove the source from the runloop manually like you do above and the Apple example code ( http://developer.apple.com/documentation/CoreFoundation/Reference/CFFileDesc...) doesn't remove the runloop source either. Unfortunately, the code you laid out above does indeed sometimes leak a single kqueue fd (OS 10.5.8): Add that after the CFRunLoopAddSource() line. For me, this always prevents the leaks (also on 10.5.8.) Does that do it for you? Thanks- that works. I found the main difference in the usage that leaks and the one that doesn't is the removal of the source from the runloop (the Apple sample code doesn't use it either). I assumed that the CFFileDescriptorInvalidate function takes care of the removal from the runloop, but apparently not. Unfortunately, in my application, it would be difficult to track the runloops that the CFFileDescriptor is scheduled on. Tracking when runloops come and go via NSThread death notifications is annoying and error-prone. At least it is not leaking threads. Speaking of which, why does each file descriptor need to be watched by a new thread? Not even CFSocketManager is that dumb. In any case, this is looking so bad that I will likely need to implement my own CFFileDescriptor replacement. I'd imagine creating one thread per monitored CFFD would be significantly easier to implement than creating one centralized thread for every monitored CFFD. Also, a thread blocked in kevent() waiting for an event is going to use very little resources and no (?) CPU time, so I don't see the 1-1 nature being an issue. I have a thread pool handling socket communications via kqueue. Sometimes these communications need to be processed on NSRunLoops (some threads use a custom kqueue-based runloop, some use NSRunLoop), so I setup CFFileDescriptors to use the existing kqueue descriptors (CF kqueue wait then on my kqueue). By adding CFFileDescriptor, the new thread count is 2 x old thread-count (since each kqueue-based runloop can now be wrapped in NSRunLoop), and fd usage is 2 x kqueue- based runloop count right off the bat. This is after I add each CFFileDescriptor source to mutliple CFRunLoops (assigning a source to mulitple runloops seems to work, but I wonder if the CFFileDescriptor tracks the CFRunLoops on which it is scheduled). I needed to identify this issue because the leak eventually exhausted the limit on file descriptors- now that the leak is fixed, the app hovers around 50 fds. Before you go off creating your own version of CFFD, I'd suggest actually using CFFD in a realistic scenario and checking for these leaks. I've been using CFFD for awhile, and it's never posed any issue (granted I haven't been actively searching for these leaks). Are you opening so many files that these leaks would pose an issue? I agree that it's unsettling to see these leaks in such a simple scenario, but implementing your own version of CFFD using a centralized thread etc. is also not an easy task, and that code will just become obsolete once the CFFD leaks are addressed. And as far as your hack... well, I'd consider that a case of the cure being worse than the symptom. =) Again, are you opening so many files that it warrants such scandalous tactics? Anyway, I'm going to radar this, and I hope you'll do the same. Thanks for your replies, Dave. The evil hack is obviously just a stop- gap measure until I figure out a real plan. I still may need to write my own CFFD replacement to cut down on thread and fd usage. This email sent to site_archiver@lists.apple.com