Re: Problem with garbage collection in server application
Re: Problem with garbage collection in server application
- Subject: Re: Problem with garbage collection in server application
- From: Rick Hoge <email@hidden>
- Date: Sun, 23 Jan 2011 08:06:35 -0500
Hi Quincy,
Thanks so much for the detailed reply.
On 2011-01-23, at 12:28 AM, Quincey Morris wrote:
>> [...]
>
> You *cannot* rely on GC to satisfy your timing requirements for releasing resources used by your application. You can't even rely rely on GC to release memory at all, in any deterministic sense -- its function (conceptually) is to make memory available when it's asked for, and *not* simply to release unused memory. It's certainly true that *this* garbage collector always releases unused memory, eventually, but the timing is unpredictable.
I am indeed learning this the hard way - it does work very well for some of the apps we use, but the present project might be an edge case.
> If your application is crashing due to running out of memory, looking at what happens at 'finalize' time is probably not going to get you anywhere useful. I think you have to start by diagnosing the problem more exactly, into one of these categories:
>
> 1. Too much "live" (i.e. uncollectible) data at once. This breaks down into three subcategories:
>
> 1a. Objects that you create, and keep strong references to, use too much memory in total.
It's clear that, ultimately, too much memory is retained. However I'm pretty sure that all the strong references I create are cleared at the appropriate times. However the scenarios you raise below are very relevant.
> 1b. Your code depends on a confusion between the end of the lifetimes of your objects and the collection times of the memory they use. You shouldn't be designing with collection times in mind.
Guilty as charged. However I would like to think that some of the really useless memory, possibly referenced only by internal pointers I don't control, should be cleared before the system grinds to a near halt and the app ultimately crashes. I am suspecting problem 1c might be the reason.
> 1c. Objects that are created by the frameworks internally, but which somehow support objects that you create (think of a cache, for example), have lifetimes not under your control, and remain live after your own objects' lifetimes are over.
It's clear when breaking on retain in the non-GC version that there are a lot of KVO copies of my objects and other UI things that must be creating strong references to the large dataset objects. All of these UI objects vanish from the window at the appropriate time, but it's quite likely that they don't actually get collected until much later. It's very possible that these UI objects, with their strong references to my dataset object, are keeping the dataset objects alive. A solution might be to bind the UI elements to a lightweight proxy of my object to avoid any strong references between the UI and the big datasets. This might only be making more complexity for nothing though...
> (That's the scenario where I ran into a problem similar to yours -- Core Data has an internal cache that can't be controlled outside of the frameworks, and it can get very large in very strange ways. I spent months of my life trying to solve the resultant memory issues.)
I can't think of anything beyond the UI objects that would be doing this. I'm not using Core Data but will have to keep this type of thing in mind.
> 2. You outrun the collector. Certain patterns of data usage can interact badly with the GC heuristics. Submitting a bug report is a good idea, because Apple likes to find the edge cases. That likely won't help with finding an immediate solution, of course.
I'll have to put together a toy program to model this. Basically I create big (200MB) objects every few seconds, and do a few operations on them, save the results, and 'close' them. The 'close' should clear all strong references that I control.
> 3. There's bug in the collector. Not likely, but possible.
Agree the problem's probably in my code.
> If you've designed your application carefully, then problem 1a likely means you have a bug somewhere in your code. Problem 1b is an application design flaw. All the other problems (1c-3) basically suck, because you're dealing with behavior outside your control.
>
> Switching to a retain/release pattern is unlikely to help. It has the essentially the same limitations as GC with regard to resource releasing, though you can get away with Doing It Wrong more often.
I think the retain/release has potential, but it did provide a strong clue that UI objects hanging around until the end of the runloop might be to blame. I forgot to mention that I wrap the code creating the objects in my own autorelease pool, to keep the objects alive long enough for me to retain them where they will actually be needed.
> I don't want to sound too discouraging here. My suggestion is to put the collection timing (and finalize) out of your mind, and concentrate on the crashes instead. At that moment, you want to know not what's pending collection, but what's live. As others have already mentioned, you have some good tools available to you -- Instruments for memory usage, and 'info gc-roots' for tracking down strong references.
Thanks for the advice - I'm realizing it's wrong to try and cajole the collector to do anything at a specific time.
Rick
>
>
_______________________________________________
Cocoa-dev mailing list (email@hidden)
Please do not post admin requests or moderator comments to the list.
Contact the moderators at cocoa-dev-admins(at)lists.apple.com
Help/Unsubscribe/Update your Subscription:
This email sent to email@hidden