Re: Binary property list efficiency issue

Larry,
 we had a similar problem.

First of all, I would take this statement to heart, as we did:
" For situations where you need to store small amounts of persistent data,
less than a few hundred kilobytes, Core Foundation provides property lists."
this is from the "Introduction to Property Lists" at
http://developer.apple.com/documentation/CoreFoundation/Conceptual/CFPropert
yLists/index.html#//apple_ref/doc/uid/10000130i.

We write very large xml structures by hand, bypassing CoreFoundation. To
save space on disk we compress them using zlib. It is built into the system,
extremely easy to use and very fast (much faster than the binary property
list routines, even for moderately-sized lists).

hth, Stuart

on 1/20/06 10:01 AM, email@hidden at
email@hidden wrote:

  

Message: 1
Date: Fri, 20 Jan 2006 02:16:39 -0500
From: Laurence Harris <email@hidden>
Subject: Re: Binary property list efficiency issue
To: John Stiles <email@hidden>
Cc: email@hidden
Message-ID: BFF5F887.36F1E%email@hidden"><BFF5F887.36F1E%email@hidden>
Content-Type: text/plain; charset="US-ASCII"

On 1/19/06 8:41 PM, John Stiles didst favor us with:

    

Have you Sharked it? Even if it's all inside Apple code, their
function names are usually descriptive enough that you can get a
picture of what's going on behind the scenes.
      

Well, we've Sharked it now. Shark shows that 44%+ of the time is
spent __plistUniquingEqual(), followed distantly with 12% in
__CFNumberGetValue(). Looking at the CF source, it looks like most of
the time involved here is spent in the inline CFTypeID comparison,
since CFEqual() is far, far behind anything of time significance in
the trace. CFNumberGetType() came in an even further third, with some
9% of the total time. Shark also reveals that __plistUniquingEqual()
is poorly optimized (not inlined and often returns early). All of this takes
place inside CFSet. In the worst-case scenario, uniquing should be turned
off completely. Even without it, the binary plist is going to be smaller
than XML.

Here are some times I got in testing. Property list has a small header which
is the same size for every list and a main array. All times are the time
spent in CFPropertyListWriteToStream:

1,538 items:
binary 1.65
XML 0.317

8,365 items:
binary 63.58
XML 0.933

31,041 items:
binary 1499.033 secs
XML 3.583 secs

The XML times go up linearly. The binary times appear to be going up
exponentially. This makes the binary format completely unusable for large
lists with its current implementation. I can't ask users to wait 25 minutes
while I save a file.

Larry

    

On Jan 19, 2006, at 5:34 PM, Laurence Harris wrote:

      

I'm converting one of my file formats over to use a property list,  and in
the interest of saving disk space I wanted to use the binary format
(kCFPropertyListBinaryFormat_v1_0) with  CFPropertyListWriteToStream. I have
all the code working, but what I'm finding is that due to the way CF
converts
my property list to its binary representation, as the size  of the property
list grows, the conversion to the binary representation  takes more and more
time, as if it's growing by O(n^2). It's bad enough that  it becomes
completely unusable for even moderately sized (by my definition ;-)  lists.

I'm writing out an array in which every element is another array. The
element
arrays are small, just a few numbers and a string. The main  array can get
large, as in hundreds of thousands of elements. With 200  items it's almost
instantaneous. 2000 take a few seconds. 20,000 items take,  well, I'm not
sure. I finally gave up after a few minutes. Has anyone else  experienced
this? How should I write up a bug about this, as it looks like the  problem
is that one or more parts of the code doing the conversion are not  written
very efficiently. Thoughts?

Larry
        

 _______________________________________________
Do not post admin requests to the list. They will be ignored.
Carbon-dev mailing list      (email@hidden)
Help/Unsubscribe/Update your Subscription:
http://lists.apple.com/mailman/options/carbon-dev/email@hidden

This email sent to email@hidden