Re: Garbage collection, core data, and tight loops
Re: Garbage collection, core data, and tight loops
- Subject: Re: Garbage collection, core data, and tight loops
- From: Bill Bumgarner <email@hidden>
- Date: Tue, 30 Oct 2007 14:56:41 -0700
On Oct 30, 2007, at 2:03 PM, John R.Timmer wrote:
I'm trying to process an enormous XML file that should ultimately
fill a Core Data sqlite store with millions of records. This should
only happen once, so i'm willing to let the app go non-responsive;
i'm doing all the parsing in an extremely tight loop. My first time
through, i did things with manual memory management, but the memory
use kept going up. So, i switched to mandatory Garbage Collection,
leaving my autorelease pool statements in place, since they should
be converted to no-ops. I confirmed that a collector thread existed
when i launched the app.
It still ran out of memory:
malloc: *** auto malloc[333]: Zone::Can not allocate 0x8200000 bytes
I'm assuming the problem is an interaction between the tight loop
and either Core Data or the Garbage Collector:
The GC doesn't recognize which objects to collect while the loop is
still chugging away.
The sqlite store can't send objects out to disk and release their
memory while the loop is still chugging away.
I'm leaning towards the latter explanation.
I'm going to redesign things so that the main runloop has a chance
to act sporadically, but i was just curious as to which of these is
a problem so that i can design specifically to avoid this in the
future.
This sounds like you might have a situation where allocations are
outrunning the collector (a known issue that is being worked on).
Specifically, if you are allocating tons and tons of memory in a tight
loop, the collector can't scan and purge the garbage quickly enough
and, thus, you can cause this kind of behavior (there is more
information on this in the documentation).
However, that could easily not be the case -- I would need more
information to know.
Specifically -- are you periodically saving changes in your loop?
Are you asking Core Data to purge objects?
Does your object graph allow partial subgraphs to exist in memory
without eventually faulting everything into memory?
... do you have any kind of caches or hashes that are effectively
keeping the objects around?
b.bum
_______________________________________________
Cocoa-dev mailing list (email@hidden)
Please do not post admin requests or moderator comments to the list.
Contact the moderators at cocoa-dev-admins(at)lists.apple.com
Help/Unsubscribe/Update your Subscription:
This email sent to email@hidden