re: CoreData file format stability
re: CoreData file format stability
- Subject: re: CoreData file format stability
- From: Ben Trumbull <email@hidden>
- Date: Tue, 3 Jun 2008 18:34:57 -0400
1) The file format for saved files. I'd rather not make some
proprietary/closed Microsoft-ish thing - I'd like it to be possible
for other programs to read/write my file format, including
hypothetical programs that might get written for other platforms so
that my file format could possibly be readable and writable by our
Linux friends (and Windows carbon units as well).
As Kyle notes, you could use a specific interchange format instead.
Many applications do something like this since the feature
requirements for cross platform sharing are usually not 100% congruous
with those for daily use. As an example, often people use an XML
format for interchange, but that's much too slow for day to day work
on large data sets.
The OmniFocus team wrote a blog post on what they did in this area
using Core Data.
Since CoreData has a SQLite-based format, and since SQLite is
available pretty much
everywhere, this seems pretty good as long as I am able to figure out
how CoreData sets up the tables and such in its documents, except for
one concern: what if the layout of the SQLite file CoreData creates
changes?
We do not document nor support such details.
If CoreData on 10.6, 10.7, etc. all generate SQLite files
that are set up a bit differently from the way the previous version of
OS X did it, that could create a very annoying moving target for
anyone trying to access the files via a non-CoreData means.
We do reserve the right to make backward binary compatible
adjustments. Typically, this takes the form of offering new features
that are only available to new applications. For example,
transformable properties are new in 10.5. Since it wasn't possible to
write a 10.4 application that could use, or even express in the model,
transformable properties, data files from 10.5 leveraging 10.5
specific features can't be opened on 10.4. Generally, this isn't a
problem, since the application itself can't run on 10.4
10.4 developed applications running on 10.5 still create files that
can be used on 10.4
Is there any documentation anywhere promising that the format of
the CoreData save files won't
change in future versions of the OS,
On the contrary, we explicitly reserve the right to make changes,
without notice as implementation details, to any undocumented
functionality, SPI, or file formats. We explicitly promise that any
SPI, undocumented functions, methods, classes, etc may change at any
time without notice, or even be removed entirely.
or failing that, could one of the Apple employees here let me know
if this is a valid concern or not?
This is not your largest concern. Your most significant issue is
whether or not you could even deploy such an application to other
platforms. Core Data is not an Objective-C wrapper around SQLite. It
is a Cocoa application development framework focused on the M in MVC.
The fact that it also does object persistence is perhaps it's least
interesting feature.
Assuming you wrote a Core Data application on OSX, you'd need to
reimplement nearly every Core Data feature you used on Linux or
Windows. This includes integration with your controller layer (Cocoa
Bindings), undo support, KVC & KVO equivalents, an entire schema
metadata class hierarchy, sophisticated relationship modeling, and
many other features.
If you have a very simple application, you could hypothetically use
Core Data as a wrapper instead of an open source wrapper for SQLite.
It would make an excellent tool for rapid prototyping. But the
temptation or inadvertence of using a more sophisticated feature would
be very high. For example, Core Data has an entire optimistic
concurrency control layer for resolving multi-writer conflicts between
both threads & processes. Most Cocoa developers don't know what that
means, and don't care, but they get it for free anyway. There are a
variety of subtle, complex, or extremely technical (problem domain
specific) edges cases that Core Data resolves.
In this sense, Core Data is a framework in the traditional sense of
the GoF Design Patterns book.
2) Is CoreData able to edit files in place at all?
Yes. Currently the SQLite store is the only nonatomic store.
The reason I am asking is because the documents my hypothetical app
would work with
could possibly become very large, maybe even in the 300-400 MB range.
I don't consider that to be very large, but I would recommend using
10.5 for that. I'm aware of customers with a few million rows in
databases as large as 1GB.
Would CoreData be able to just open the files, let me search them, add
stuff to them, etc., like one would normally be able to do just by
using SQLite directly,
Yes.
or would it be copying the whole thing into a
store file somewhere or into RAM every time a file is opened?
No, not for the SQLite store. However, the XML and binary stores are
atomic and behave like NSDocument.
[ snip out of order reply ]
It also means that even for a Mac-only app you could end up with this
really weird situation where an app running on a later version of OS X
could end up saving a file that was unreadable by the same version of
the same app that just happens to be running on an earlier OS X
version.
No. It does not mean that. Just because the file format is opaque
doesn't mean it's going to magically cease functioning. Several
million customers rely upon applications using Core Data.
This isn't any different from NSCoder. Changing the file format
radically required creating NSKeyedArchiver and having new
applications opt in to new features. Nobody broke existing
applications either dual booting into an older OS or sharing files
with colleagues on an older OS.
Any issue preventing people from sharing files from the same binary
version of an application on different versions of OSX would be a very
serious bug.
Basically what I need to do at this stage is determine whether
CoreData is suitable for my hypothetical app, or whether I should just
roll something myself using the SQLite3 APIs directly. As always, any
help is greatly appreciated.
You haven't really articulated your feature requirements, or how
important cross platform support really is to your application. None
of the Cocoa frameworks are focused toward helping people write Linux
or Windows software. The problems you'll have with Core Data are the
same ones you'll have with Cocoa Bindings, AppKit, and even large
portions of Foundation. Uhm, right, and Objective-C itself. Yes,
it's cross platform, but most software on Linux & Windows is written
in C/C++ or other languages and the platform specific developer tools
are geared toward that reality.
And none of the Linux or Windows frameworks are designed to create
best of breed cross platform software either.
It's not clear why you're even interested in Linux as a deployment
platform. Not bashing Linux, I was programming on SunOS boxes twenty
years ago and think Linux is just swell, but it's market share for
desktop applications is a fraction of 1%. And most people with Linux
have machines that can dual boot into Windows or OSX. If you're
writing a configuration tool for enterprise software, for example,
different features have priorities that would be considered unusual
for a consumer desktop application (and vice versa)
I would encourage you to consider developing your software with the
platform tools and frameworks best suited to that platform. When in
Rome ...
Most customers don't appreciate applications that fail to appear and
behave as native applications, and Windows and Linux have their own
set of frameworks for application development. Typically, it's easier
to leverage the platform frameworks than to fight them by forcing
designs from other platforms into their API. This places a burden on
you to design your custom logic more elegantly to plug in to many
different APIs, but it is possible, and generally the best way to
write high quality applications on multiple platforms.
This advice is focused on desktop applications with user interfaces.
Obviously, daemons, tools, and scripts have completely difference
issues and are infinitely simpler to share across platforms.
I see a lot of badly written SQLite3 code. Writing software that
scales to handle 100s of MB of data is not easy. Just scaling to
efficiently handle thousands of objects is not as easy as it appears.
- Ben
_______________________________________________
Cocoa-dev mailing list (email@hidden)
Please do not post admin requests or moderator comments to the list.
Contact the moderators at cocoa-dev-admins(at)lists.apple.com
Help/Unsubscribe/Update your Subscription:
This email sent to email@hidden