Re: Memory Problems
Re: Memory Problems
- Subject: Re: Memory Problems
- From: "Adam R. Maxwell" <email@hidden>
- Date: Fri, 05 Feb 2010 19:49:59 -0800
On Feb 5, 2010, at 5:49 PM, Greg Robertson wrote:
> I am trying to parse a large CSV (5MB) file using some code I found here:
> http://www.macresearch.org/cocoa-scientists-part-xxvi-parsing-csv-data
>
> but I am getting what I think is an out of memory error part way through, the error is:
>
> DataImport(1324,0xa09f3500) malloc: *** mmap(size=16777216) failed (error code=12)
> *** error: can't allocate region
>
> I'm fairly certain I do not have a memory leak it is just that the NSString is very large. Is there anyway to increase the memory allocated or clean up the code to reduce its memory usage?
Did you try running it with one of the memory profiling tools to see what's happening? I wouldn't consider a 5MB csv file large, so I'd guess that you're blowing up the autorelease pool; this code looks really inefficient in that respect. You end up creating a mutable array and at least one mutable string per row, and in addition calling -[NSScanner scanCharactersFromSet:] is extremely inefficient (it inverts and autoreleases a copy of the character set). You can invert the set and use scanUpToCharactersFromSet: to work around that bug. Additionally, using immutable character sets should be faster.
As a bandaid, you could try creating an autorelease pool in the outer while loop and drain it on each pass (once per row if I read the code correctly). Alternately, you could use alloc/init/release instead of the convenience methods, or even rewrite using CFStringInlineBuffer if you have lots of time to kill :).
hth,
Adam
_______________________________________________
Cocoa-dev mailing list (email@hidden)
Please do not post admin requests or moderator comments to the list.
Contact the moderators at cocoa-dev-admins(at)lists.apple.com
Help/Unsubscribe/Update your Subscription:
This email sent to email@hidden