Recursion taking up far too much memory... any ideas?
Recursion taking up far too much memory... any ideas?
- Subject: Recursion taking up far too much memory... any ideas?
- From: Sam Stigler <email@hidden>
- Date: Mon, 4 Jun 2007 03:37:12 +1000
Hi,
I'm writing a program to automatically, recursively, fetch every
single Amazon BrowseNode from BrowseNodes.com (I subscribed to their
web service) into a Core Data persistent store that I can use in
another program. What it does is go through every single category,
then every single category within each of those categories, etc., and
builds a Core Data-based tree out of them.
The problem I'm having is that the first time I ran the program, (1)
it took forever (but Shark says this is due mainly to Cocoa API calls
that I can't get rid of unfortunately), but more importantly, (2) it
took up a whole lot of memory -- up to around 50 MB by the time it
was far less than halfway through. The heavy lifting is being done
in a separate thread with its own autorelease pool; it looks to me
like the problem is that the autorelease pools aren't being released
until the recursions (if you can call a void method a recursion) are
finished.
Can anyone confirm if I'm right about this; and give any ideas on
how to correct the problem (whatever the problem is)? Can it be
corrected? I'm currently spinning off all of the recursive calls
(that is, all but the first one) into the second thread, but I tried
spinning those all off into their own thread and while it took up
less memory, I got hundreds of threads and ended up having to deal
with racetrack conditions -- not ideal. I've posted my relevant code
to http://homepage.mac.com/sstigler1985/recursionProblem.html .
Some additional notes: Firstly, I've tried searching the archives
for this issue and found something on it, but the message thread was
very hard to follow, and it's unclear if the issue was actually
solved in the end. I'm hoping someone could (please?) clear it up
for me. Second, when I mentioned "racetrack conditions" for me using
all threads instead of just going halfway, I meant that I'm getting
seemingly-random mutateErrors appending a certain string in the
middle of the addInfoAboutNode method, somewhere in the middle of the
run of the program. It's my hypothesis that this is due to racetrack
conditions with at least 170-odd threads competing for the variables,
but I've put one NSRecursiveLock over each of the methods (see http://
homepage.mac.com/sstigler1985/recursionProblem_allThreads.html) and
that didn't make a difference. Has anyone seen this problem before?
The weird thing is that in the racetrack conditions, I'm getting the
exception at the same line each time:
[currentStatus appendString:nodeTitle]; .
This is my first major use of recursion (I'm finally getting my head
around it), so it's very possible that I've missed something; but
having said that, I think the general algorithm is sound. Please let
me know, though, if there's something glaringly obvious that I've
left out.
Thanks for any help you can give me,
Sam
_______________________________________________
Cocoa-dev mailing list (email@hidden)
Do not post admin requests or moderator comments to the list.
Contact the moderators at cocoa-dev-admins(at)lists.apple.com
Help/Unsubscribe/Update your Subscription:
This email sent to email@hidden