Re: CFFileDescriptor fd leak?
Re: CFFileDescriptor fd leak?
- Subject: Re: CFFileDescriptor fd leak?
- From: "A.M." <email@hidden>
- Date: Fri, 21 Aug 2009 14:13:26 -0400
On Aug 21, 2009, at 12:58 AM, Dave Keck wrote:
I shaved down the code from the previous email from code which did
indeed
add the CFFileDescriptor to the runloop and leaked. However, I did
not
remove the source from the runloop manually like you do above and
the Apple
example code (
http://developer.apple.com/documentation/CoreFoundation/Reference/CFFileDescriptorRef/Reference/reference.html)
doesn't remove the runloop source either. Unfortunately, the code
you laid
out above does indeed sometimes leak a single kqueue fd (OS 10.5.8):
davekeck 4880 alex 4u KQUEUE count=0,
state=0
Woops, I excluded this line in that code:
CFFileDescriptorEnableCallBacks(fileDescriptor,
kCFFileDescriptorReadCallBack);
Add that after the CFRunLoopAddSource() line. For me, this always
prevents
the leaks (also on 10.5.8.) Does that do it for you?
Thanks- that works. I found the main difference in the usage that
leaks and the one that doesn't is the removal of the source from the
runloop (the Apple sample code doesn't use it either). I assumed that
the CFFileDescriptorInvalidate function takes care of the removal from
the runloop, but apparently not.
Unfortunately, in my application, it would be difficult to track the
runloops that the CFFileDescriptor is scheduled on. Tracking when
runloops come and go via NSThread death notifications is annoying and
error-prone.
At least it is not leaking threads. Speaking of which, why does each
file
descriptor need to be watched by a new thread? Not even
CFSocketManager is
that dumb. In any case, this is looking so bad that I will likely
need to
implement my own CFFileDescriptor replacement.
I'd imagine creating one thread per monitored CFFD would be
significantly
easier to implement than creating one centralized thread for every
monitored
CFFD. Also, a thread blocked in kevent() waiting for an event is
going to
use very little resources and no (?) CPU time, so I don't see the
1-1 nature
being an issue.
I have a thread pool handling socket communications via kqueue.
Sometimes these communications need to be processed on NSRunLoops
(some threads use a custom kqueue-based runloop, some use NSRunLoop),
so I setup CFFileDescriptors to use the existing kqueue descriptors
(CF kqueue wait then on my kqueue). By adding CFFileDescriptor, the
new thread count is 2 x old thread-count (since each kqueue-based
runloop can now be wrapped in NSRunLoop), and fd usage is 2 x kqueue-
based runloop count right off the bat. This is after I add each
CFFileDescriptor source to mutliple CFRunLoops (assigning a source to
mulitple runloops seems to work, but I wonder if the CFFileDescriptor
tracks the CFRunLoops on which it is scheduled). I needed to identify
this issue because the leak eventually exhausted the limit on file
descriptors- now that the leak is fixed, the app hovers around 50 fds.
Before you go off creating your own version of CFFD, I'd suggest
actually
using CFFD in a realistic scenario and checking for these leaks.
I've been
using CFFD for awhile, and it's never posed any issue (granted I
haven't
been actively searching for these leaks). Are you opening so many
files that
these leaks would pose an issue? I agree that it's unsettling to see
these
leaks in such a simple scenario, but implementing your own version
of CFFD
using a centralized thread etc. is also not an easy task, and that
code will
just become obsolete once the CFFD leaks are addressed.
And as far as your hack... well, I'd consider that a case of the
cure being
worse than the symptom. =) Again, are you opening so many files that
it
warrants such scandalous tactics?
Anyway, I'm going to radar this, and I hope you'll do the same.
Thanks for your replies, Dave. The evil hack is obviously just a stop-
gap measure until I figure out a real plan. I still may need to write
my own CFFD replacement to cut down on thread and fd usage.
Also, why isn't the CFFileDescriptor source posted (http://www.opensource.apple.com/source/CF/CF-476.19/)?
Cheers,
M
_______________________________________________
Do not post admin requests to the list. They will be ignored.
Darwin-dev mailing list (email@hidden)
Help/Unsubscribe/Update your Subscription:
This email sent to email@hidden