re: CoreData database sharing and migration
re: CoreData database sharing and migration
- Subject: re: CoreData database sharing and migration
- From: Steve Steinitz <email@hidden>
- Date: Mon, 22 Mar 2010 22:06:10 +1100
Hi Ben,
Thank you for your detailed and candid reply. I have some
comments and questions on a few points, below...
On 18/3/10, Ben Trumbull wrote:
there wasn't a good solution for multiple machines simultaneously
accessing an SQLite db file (or most other incrementally updated
binary file formats). By "good" I mean a solution that worked
reliably and didn't make other more important things work less well.
I'm curious about the reliability issues you saw. Also, by less
well do you mean slower?
NFS is just not a free Oracle.
That's a great sound bite.
But to be clear, database cost is not the issue. We're not all
cheapskates looking to save a buck on our database :) The real
issue is that Core Data works with and only with SQLite. If it
worked with Oracle I'd raise a purchase order and do a little
dance. And I wouldn't sneer down my nose at mysql.
For nearly all Mac OS X customers (sadly not you) achieving a near
100x performance boost when accessing database files on an AFP or SMB
mount (like their home directory in educational deployments) is pretty
huge.
I agree. But wouldn't those same educational institutions be
prime candidates for multiple machine access?
So we focused on making the experience that could work well work even
better. 10.6 is a significantly better network FS client as Apple
applications like Mail don't slam the servers with byte range lock
requests all the time (good for NFS), and on AFP also gets to use
local file caching.
That makes sense and sounds good.
To address both sets of problems on all network FS, we enforce a
single exclusive lock on the server for the duration the application
has the database open. Closing the database connection (or logging
out) allows another machine to take its turn.
Could my application close the database connection and re-open
it to work around the problem? How would I do that? I suppose
once I got it going I'd have to retry saves, fetches etc.
This behavior was supposed to be opt in for Core Data apps, but on 10.6.2 is not.
Opt in would be ideal -- especially dynamically configurable opt
in (for when the CEO wants to access the system from home at night).
connecting from five machines to a single database on a Synology
network drive over afp. ...latest values from the database during
idle time.
It can work technically on AFP. However, the distributed cache
coherency problem is avoided by these network FS because they don't do
any file caching on files with locks. Your server set up and
networking hardware is pretty sophisticated compared to most so the
performance is adequate.
Actually, the performance is nearly as fast as a local hard
disk. My measuring system is not perfect but I've seen sql
select times of 0.003 seconds on (well-indexed) tables with tens
of thousands of records. Sometimes I wonder if that's even possible.
As an engineer, I would wish AFP over VPN over Airport was the more
uncommon deployment scenario, but sadly not.
I can see how that would create pressure on your engineering
team. Also, I confess I'd like to set up exactly that scenario
for the CEO :)
There are mysterious but harmless optimistic locking errors
once in a while where no attributes appear have changed
Those mysterious updates are probably version numbers being bumped
because the object's participation in a to-many relationship, either
outgoing or incoming, changed.
Thanks for confirming that. My conflict logging avoids showing
related objects.
10.6.2 only one machine can connect to a Core Data store.
ADC did pass your bug report along to us,
For the record, you were my first, very helpful, contact
regarding the issue. Only later did I file a bug report.
and it is a backward binary compatibility issue, and a regression from
10.6.0. It will be fixed in a future update.
I'm relieved that you are clear about that. I could easily see
it slipping through the cracks.
You'll get the 10.5 performance characteristics, however.
Again, the 10.5 performance over gigabit ethernet is almost
unbelievably fast. I may know why. Despite your helpful
explanations I'm still not exactly clear on the relationship
between caching and locking, but I wonder if the speed we are
seeing is helped by the fact that the entire database fits into
the Synology NAS's 128meg cache?
[for status updates regarding your bug] You should follow up with ADC
ADC explained to me that its best to add status requests
directly to the bug report. Is that what you mean?
or Evangelism directly
That sounds like a great idea, but I was unable to find the
Evangelism contact information. Can you (or anyone on the list)
point me in the right direction?
In another message in this thread, you made a tantalizing statement:
Each machine can have its own database and they can share their results
with NSDistributedNotification or some other IPC/networking
protocol. You can hook into the
NSManagedObjectContextDidSaveNotification to track
when one of the peers has committed changes locally.
Let me guess how that would work: before saving, the peer would
create a notification containing the inserted, updated and
deleted objects as recorded in the MOC. The receiving machine
would attempt to make those changes on its own database. Some questions:
Would that really be feasible?
Would it be a problem that each machine would have having
different primary keys?
How would the receiving machine identify the local objects
that changed remotely?
Could relationships (indeed the object graph itself)
feasibly be maintained?
How would relationships between the remote objects be
identified? Hand-parsing?
Has anyone done it that you know of?
Is there sample code?
Ben, you've helped save our project a number of time over the
years. We appreciate your help and candor,
Steve
_______________________________________________
Cocoa-dev mailing list (email@hidden)
Please do not post admin requests or moderator comments to the list.
Contact the moderators at cocoa-dev-admins(at)lists.apple.com
Help/Unsubscribe/Update your Subscription:
This email sent to email@hidden