Re: [CodeData] Generic model migration code?
Re: [CodeData] Generic model migration code?
- Subject: Re: [CodeData] Generic model migration code?
- From: Bill Bumgarner <email@hidden>
- Date: Fri, 8 Sep 2006 09:41:28 -0700
On Sep 8, 2006, at 2:28 AM, Kaspar Fischer wrote:
I have two questions:
- Does anybody have a generic piece of code that takes ANY pair
of old model and new model with the assumption that the latter
extends the former, and converts a data store in the old
model to one from the new model?
I don't know of any generic code. It is a remarkably difficult
problem for which rather large companies have formed and do quite a
brisk bit of consulting services business migrating data from one spot
to another.
Even something as comparatively simple as a Core Data model can be
remarkably hard to migrate for reasons mentioned before.
Typically, I do one of two things:
--
For apps that use working sets that comfortably fit into memory, I
will typically use an in memory store and then take care of archiving
the data out to disk in my own custom (or standard) format.
Populating the in memory store is then just a matter of creating the
appropriately configured set of managed objects and "saving" them to
the in memory store.
By doing this, I gain the huge set of advantages of using Core Data --
undo, redo, validation, object graph management, etc.. Saving
atomically is just a matter of walking the full object graph and
persisting the data as needed. If I need to take action on updates
to the object graph as they ahppen, I can hook into the "did save" and
"did change" notifications sent by the managed object context.
I have used the above model to quite effectively deploy an in memory
store as a cache for client/server based transactions to some random
RPC (typically XML-RPC -- only 10x the bloat in on-the-wire data size
as opposed to SOAP's 100x) server.
Migration in this model is mostly a non-issue. If I modify the model,
it is a matter of modifying my store/load code to populate the new
model. Since there is no data persisted against the model, there is
no need to migrate from an old version of the model to a new version.
--
For applications with working sets too large to fit entirely into
memory, migration is a bit more complex. Honestly, I haven't had to
do this in a production context. But I have had to migrate models in
one-off situations. If I had to take it to production, I would do
something like the following.
Now, this assumes that the entire set of objects does not have to be
loaded for the purposes of migration. This is typically the case.
Given the difficultly of visualizing tens of thousands of nodes in an
object graph where each node has significant meaning, it is rare that
tens of thousands of objects would have to be in memory all at the
same time for there to be a "complete and migrateable" object graph.
That is, there is typically some cross cut of the dimensions of the
object graph that you can make such that you only have to pull in a
subset of the objects. More often than not, there is some entity
that is the "root" of any given subset of objects.
Equally as likely, there are probably a set of entities that are,
effectively, descriptors for other objects. These are often treated
as leaf nodes. For example, a database of locations may have an
entity that describes city/state/zip triplets.
Assuming the above is true, migration then becomes a matter of:
- migrate any standalone data
- migrating all leaf node or descriptor data; the city/state/zip
triplets can be migrated directly, for example.
- migrate the sub-graphs of main body of data individually. For
example, if you had a database of clients, you would likely migrate
each client individually.
- fix up any relationships that may have been broken during the
migration of the sub-graphs of the main body of data.
Doing all this would require two stores and two managed object
contexts; one initialized against the old object model and one
against the new. You fetch into the old and populate-and-save into
the new. If at all possible, do so such that any given incremental
save in the new context validates. If that isn't possible, you can
load the object model, then traverse the entities and remove
validation rules and/or set the class to be something generic
(NSManagedObject) to avoid validation issues.
If you have to "fix up" relationships after the fact, you are going to
need some kind of a means of correlating between objects outside of
data within the object graph. Not hard to do, just something to be
aware of.
Yes, it is a hard problem. The above is a fairly generic
description. A specific implementation is likely to be a lot simpler.
- In this particular situation (adding and not changing attributes)
it seems (see bottom of 2)) that the problem can be solved by
saving as XML and then reopening as a store of the new model.
Is this safe to do in the production environment?
It is safe until it breaks or is insufficient. Validation might cause
issues -- older objects may not have the new fields appropriately
populated, for example.
b.bum
_______________________________________________
Do not post admin requests to the list. They will be ignored.
Cocoa-dev mailing list (email@hidden)
Help/Unsubscribe/Update your Subscription:
This email sent to email@hidden