Re: Application Architecture Question...
Re: Application Architecture Question...
- Subject: Re: Application Architecture Question...
- From: Wade Tregaskis <email@hidden>
- Date: Thu, 30 Oct 2003 12:23:14 +1100
I'm currently starting work on a project which utilizes the SearchKit
framework to search a potentially large index of documents. This is
not unlike Mail.app where you have a large number of messages all
indexed and searchable through the SearchKit.
What my question is, is how should I store each of the documents.
Currently when a new document is created, I add it it's contents to an
index and then store the NSData, amongst some other meta data
information in a NSMutableDictionary. What this means is that I have a
single dictionary object (persisted to file) which contains all the
messages in the system. I'm thinking that once the number of messages
starts to get large this type of single file storage mechanism will be
undesirable (loading it into memory will be a waste). So I suppose
what I'm wondering is what alternatives are there to this type of
infrastructure, and is this infrastructure as bad as I suspect?
Thanks for any input anyone out there may have. (sorry for the
newbie'ish nature of this question, while I'm not new to programming,
I am quite new to cocoa/objective-c programming)
The next step I would take is to store each record (or groups of) in
separate files, for reliability reasons - a crash while writing data
out won't corrupt your entire collection, just one record at most. You
could then use a placeholder value in the dictionary, indicating where
the record can be found on disk if it is not already in memory. Thus,
you need only load records that are needed, as they are needed. You
could still load the meta data for each entry - perhaps keep this
predetermined in an index file - and so you'll retain all your
searching and sorting capabilities.
You could then extend this system to cull unused in-memory records over
time, to minimize running memory use (if your app is any sort of
server, this is particularly important).
Wade Tregaskis
-- Sed quis custodiet ipsos custodes?
_______________________________________________
cocoa-dev mailing list | email@hidden
Help/Unsubscribe/Archives:
http://www.lists.apple.com/mailman/listinfo/cocoa-dev
Do not post admin requests to the list. They will be ignored.