Re: Reading in dictionary from txt file: options for speed
Re: Reading in dictionary from txt file: options for speed
- Subject: Re: Reading in dictionary from txt file: options for speed
- From: Miles <email@hidden>
- Date: Thu, 16 Apr 2009 10:10:47 -0700
Marcel, NOW we're talking. This has really been such an eye-opening thread.
Now it's googling time to try to figure out how to search for a string in
there.
Thanks!
On Wed, Apr 15, 2009 at 7:01 PM, WT <email@hidden> wrote:
> Hi Marcel,
>
> that's quite impressive. On the simulator on my machine, it took 0.007
> seconds, consistently. Learned something new with your message. Thanks!
>
> Wagner
>
> On Apr 16, 2009, at 12:35 AM, Marcel Weiher wrote:
>
> I would do the following:
>>
>> 1. map the file into memory using -[NSData dataWithContentsOfMappedFile:]
>> (or mmap() if you really want to)
>> 2. Do not convert to individual objects for the words
>> 3. get the pointer to the raw bytes
>> 4. search using a little bit of plain old C (assuming you're OK with
>> encodings)
>>
>>
>> Memory mapping will be essentially instantaneous, with the I/O performed
>> on-demand when its actually needed (or you can pre-heat the data, for
>> example in a background thread). More importantly, you will be doing good
>> things for memory consumption, because the mapped memory can be released to
>> the OS without having to kill your app in low-memory situations (without
>> paging it out on Mac OS X, but the iPhone doesn't page memory out).
>>
>> I added an implementation of this approach to the testing program provided
>> by Wagner (thanks!) and it loads + counts the words in 0.084 seconds on the
>> device. That's anywhere from around 50 - 100 times faster than the other
>> methods implemented in DictTest (plist / xml / txt ). On the simulator, it
>> runs in 0.043 seconds, so around 30-40 times faster than the other methods.
>>
>> Download can be found here:
>>
>> http://www.metaobject.com/downloads/Objective-C/DictTest.tgz
>>
>> You mentioned that you were OK with search performance, so I won't go into
>> that.
>>
>> Cheers,
>>
>> Marcel
>>
> _______________________________________________
>
> Cocoa-dev mailing list (email@hidden)
>
> Please do not post admin requests or moderator comments to the list.
> Contact the moderators at cocoa-dev-admins(at)lists.apple.com
>
> Help/Unsubscribe/Update your Subscription:
>
> This email sent to email@hidden
>
_______________________________________________
Cocoa-dev mailing list (email@hidden)
Please do not post admin requests or moderator comments to the list.
Contact the moderators at cocoa-dev-admins(at)lists.apple.com
Help/Unsubscribe/Update your Subscription:
This email sent to email@hidden