Re: Design pattern for bulk data handling
Re: Design pattern for bulk data handling
- Subject: Re: Design pattern for bulk data handling
- From: Jens Alfke <email@hidden>
- Date: Sat, 08 Feb 2014 14:25:24 -0800
On Feb 8, 2014, at 10:20 AM, Ben <email@hidden> wrote:
> This is fine for most things, except that I sometimes need faster access to the underlying database - for example, when importing/exporting data. In these cases I'm after bulk data throughput without the overhead of creating/destroying many NSOperations with completion handlers since there can be in the order of millions of statements to handle.
You can read arbitrarily large amounts of data with a single statement, so that's not an issue. But in general it takes many statements to insert a lot of rows. (And for performance reasons you really want to group all of those statements in a single transaction, or you'll lose an order of magnitude of performance.)
What I'd do is provide a new operation type that does a bulk-insert or bulk-update. When run the operation performs a series of SQL statements starting with a "BEGIN" and ending with an "END".
—Jens
_______________________________________________
Cocoa-dev mailing list (email@hidden)
Please do not post admin requests or moderator comments to the list.
Contact the moderators at cocoa-dev-admins(at)lists.apple.com
Help/Unsubscribe/Update your Subscription:
This email sent to email@hidden