• Open Menu Close Menu
  • Apple
  • Shopping Bag
  • Apple
  • Mac
  • iPad
  • iPhone
  • Watch
  • TV
  • Music
  • Support
  • Search apple.com
  • Shopping Bag

Lists

Open Menu Close Menu
  • Terms and Conditions
  • Lists hosted on this site
  • Email the Postmaster
  • Tips for posting to public mailing lists
Memory Problems
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Memory Problems


  • Subject: Memory Problems
  • From: Greg Robertson <email@hidden>
  • Date: Fri, 5 Feb 2010 20:49:24 -0500

I am trying to parse a large CSV (5MB) file using some code I found here:
http://www.macresearch.org/cocoa-scientists-part-xxvi-parsing-csv-data

but I am getting what I think is an out of memory error part way through, the error is:

DataImport(1324,0xa09f3500) malloc: *** mmap(size=16777216) failed (error code=12)
*** error: can't allocate region

I'm fairly certain I do not have a memory leak it is just that the NSString is very large. Is there anyway to increase the memory allocated or clean up the code to reduce its memory usage?

Thanks

Greg


Here is the code:

#import <Cocoa/Cocoa.h>
#import <Foundation/Foundation.h>

@interface NSString (ParsingExtensions)

-(NSArray *)csvRows;

@end

---------

#import "NSStringCategory.h"

@implementation NSString (ParsingExtensions)

-(NSArray *)csvRows
{
	NSMutableArray *rows = [NSMutableArray array];

    // Get newline character set
    NSMutableCharacterSet *newlineCharacterSet = (id)[NSMutableCharacterSet whitespaceAndNewlineCharacterSet];
    [newlineCharacterSet formIntersectionWithCharacterSet:[[NSCharacterSet whitespaceCharacterSet] invertedSet]];

    // Characters that are important to the parser
    NSMutableCharacterSet *importantCharactersSet = (id)[NSMutableCharacterSet characterSetWithCharactersInString:@",\""];
    [importantCharactersSet formUnionWithCharacterSet:newlineCharacterSet];

    // Create scanner, and scan string
    NSScanner *scanner = [NSScanner scannerWithString:self];
    [scanner setCharactersToBeSkipped:nil];
    while ( ![scanner isAtEnd] ) {
        BOOL insideQuotes = NO;
        BOOL finishedRow = NO;
        NSMutableArray *columns = [NSMutableArray arrayWithCapacity:12];
        NSMutableString *currentColumn = [NSMutableString string];
        while ( !finishedRow ) {
            NSString *tempString;
            if ( [scanner scanUpToCharactersFromSet:importantCharactersSet intoString:&tempString] ) {
                [currentColumn appendString:tempString];
            }

            if ( [scanner isAtEnd] ) {
                if ( ![currentColumn isEqualToString:@""] ) [columns addObject:currentColumn];
                finishedRow = YES;
            }
            else if ( [scanner scanCharactersFromSet:newlineCharacterSet intoString:&tempString] ) {
                if ( insideQuotes ) {
                    // Add line break to column text
                    [currentColumn appendString:tempString];
                }
                else {
                    // End of row
                    if ( ![currentColumn isEqualToString:@""] ) [columns addObject:currentColumn];
                    finishedRow = YES;
                }
            }
            else if ( [scanner scanString:@"\"" intoString:NULL] ) {
                if ( insideQuotes && [scanner scanString:@"\"" intoString:NULL] ) {
                    // Replace double quotes with a single quote in the column string.
                    [currentColumn appendString:@"\""];
                }
                else {
                    // Start or end of a quoted string.
                    insideQuotes = !insideQuotes;
                }
            }
            else if ( [scanner scanString:@"," intoString:NULL] ) {
                if ( insideQuotes ) {
                    [currentColumn appendString:@","];
                }
                else {
                    // This is a column separating comma
                    [columns addObject:currentColumn];
                    currentColumn = [NSMutableString string];
                    [scanner scanCharactersFromSet:[NSCharacterSet whitespaceCharacterSet] intoString:NULL];
                }
            }
        }
        if ( [columns count] > 0 ) [rows addObject:columns];
    }

    return rows;
}
@end

_______________________________________________

Cocoa-dev mailing list (email@hidden)

Please do not post admin requests or moderator comments to the list.
Contact the moderators at cocoa-dev-admins(at)lists.apple.com

Help/Unsubscribe/Update your Subscription:

This email sent to email@hidden

  • Follow-Ups:
    • Re: Memory Problems
      • From: "Adam R. Maxwell" <email@hidden>
    • Re: Memory Problems
      • From: Jens Alfke <email@hidden>
  • Prev by Date: Re: Versioning / increasing build number
  • Next by Date: Re: Memory Problems
  • Previous by thread: Re: does result of LSSharedFileListInsertItemURL() need to be released on return if GC is enabled ?
  • Next by thread: Re: Memory Problems
  • Index(es):
    • Date
    • Thread