• Open Menu Close Menu
  • Apple
  • Shopping Bag
  • Apple
  • Mac
  • iPad
  • iPhone
  • Watch
  • TV
  • Music
  • Support
  • Search apple.com
  • Shopping Bag

Lists

Open Menu Close Menu
  • Terms and Conditions
  • Lists hosted on this site
  • Email the Postmaster
  • Tips for posting to public mailing lists
Re: NSDocument packages and incremental writing using NSFileWrapper
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: NSDocument packages and incremental writing using NSFileWrapper


  • Subject: Re: NSDocument packages and incremental writing using NSFileWrapper
  • From: Thomas Zoechling <email@hidden>
  • Date: Tue, 05 Feb 2013 11:38:20 +0100

Thanks for your response,
> Given the right conditions, NSFileWrapper can make writing vastly more efficient by writing hard links for unchanged files, rather than recreating them afresh. Have you determined whether this is happening at all?
Currently I am trying to figure out what those conditions are :)

I ran the "Directory I/O" instrument in Instruments.app and it seems that NSFileWrapper links all large files within my document package:
   0 libsystem_kernel.dylib link
   1 Foundation -[NSFileWrapper writeToURL:options:originalContentsURL:error:]
   2 Foundation -[NSFileWrapper _writeContentsToURL:path:originalContentsURL:tryHardLinking:didHardLinking:error:]
   3 Foundation -[NSFileWrapper writeToURL:options:originalContentsURL:error:]
   4 AppKit -[NSDocument writeToURL:ofType:error:]
   5 AppKit -[NSDocument writeToURL:ofType:forSaveOperation:originalContentsURL:error:]
   6 AppKit -[NSDocument _writeSafelyToURL:ofType:forSaveOperation:forceTemporaryDirectory:error:]
   7 AppKit -[NSDocument _writeSafelyToURL:ofType:forSaveOperation:error:]
   8 AppKit -[NSDocument writeSafelyToURL:ofType:forSaveOperation:error:]
   9 AppKit __block_global_90

But nonetheless, NSDocument preserves versions for the whole package content at every save (call stack obtained with the "Reads/Writes" instrument):
Read:
   0 libsystem_kernel.dylib read
   1 libcopyfile.dylib copyfile
   2 libcopyfile.dylib copyfile
   3 GenerationalStorage GSAddPathAsGeneration
   4 GenerationalStorage _AddGenerationInternal
   5 GenerationalStorage GSLibraryAddGenerationWithOptionsByCopyingFile
   6 Foundation +[NSFileVersion(NSPrivate) _addVersionOfItemAtURL:withContentsOfURL:options:temporaryStorageIdentifier:error:]
   7 AppKit -[NSDocument _preserveContentsOfURL:forURL:reason:comment:options:error:]
   8 AppKit -[NSDocument _preserveContentsIfNecessaryAfterWriting:toURL:forSaveOperation:version:error:]
   9 AppKit __block_global_98

Write:
   0 libsystem_kernel.dylib write
   1 libcopyfile.dylib copyfile
   2 libcopyfile.dylib copyfile
   3 GenerationalStorage GSAddPathAsGeneration
   4 GenerationalStorage _AddGenerationInternal
   5 GenerationalStorage GSLibraryAddGenerationWithOptionsByCopyingFile
   6 Foundation +[NSFileVersion(NSPrivate) _addVersionOfItemAtURL:withContentsOfURL:options:temporaryStorageIdentifier:error:]
   7 AppKit -[NSDocument _preserveContentsOfURL:forURL:reason:comment:options:error:]
   8 AppKit -[NSDocument _preserveContentsIfNecessaryAfterWriting:toURL:forSaveOperation:version:error:]
   9 AppKit __block_global_98

> In my experience Versions tends to be pretty inefficient about its work.
I wonder what NSDocument's private _preserveContentsIfNecessaryAfterWriting (see above call stack) uses to determine whether a file has to be copied or not.

>  Are you seeing it block the main thread though? That can generally be avoided.
No. Saving and Version preservation are off the main thread.
That's why I almost didn't notice the vast amount of copying that's going on after each save.

My usage of NSDocument & file packages seems to be straightforward.
I didn't overwrite any of the write* methods, I keep a root file wrapper for the lifetime of each NSDocument instance, ...

What am I doing wrong? There must be a way to avoid this unnecessary copying...

with kind regards,
Thomas

On 04.02.2013, at 15:12, Mike Abdullah <email@hidden> wrote:

>
> On 30 Jan 2013, at 15:53, Thomas Zoechling <email@hidden> wrote:
>
>> Hello,
>>
>> My NSDocument based app uses packages with the following structure:
>>
>> - document.package
>> +- metadata.plist (small, mutable)
>> +- large0.file (large, immutable)
>> +- large1.file (large, immutable)
>> +- large2.file (large, immutable)
>>
>> While the metadata.plist file can change any time, all large files are created once and then stay as they are.
>> As my "large0-2" files can be several gigabytes, I try to carefully avoid unnecessary IO by using document packages with NSFileWrapper.
>> My package reading/writing methods resemble the ones in the "Document Package with iCloud" sample (But I don't use iCloud): http://developer.apple.com/library/mac/#samplecode/PackagedDocument/
>> I appended the relevant parts of my code at the end of this message.
>>
>> With this setup, I was hoping that the occasions where the document architecture has to copy the whole bundle are reduced to:
>> - File duplication
>> - Moves to another volume
>>
>> But after investigating file activity with Instruments.app, it turned out that my app is reading and writing chunks of my unmodified, immutable & large files during each save.
>> It seems that the copying is related to document revisions (a.k.a. Versions).
>> Instruments logs hundreds of IO operations in the form of:
>> #	Caller	Function	FD	Path	Bytes
>> ...
>> 70	copyfile	read	22	document.package/large0.file	1048576
>> 71	copyfile	write	23	/.vol/16777218/2/.DocumentRevisions-V100/staging/adding.Wohcjo4i/4772FAAA-78D3-44A9-9412-A2D651B6EB5A.package/large0.file	1048576
>> 70	copyfile	read	22	document.package/large0.file	1048576
>> 71	copyfile	write	23	/.vol/16777218/2/.DocumentRevisions-V100/staging/adding.Wohcjo4i/4772FAAA-78D3-44A9-9412-A2D651B6EB5A.package/large0.file	1048576
>> 70	copyfile	read	22	document.package/large0.file	1048576
>> 71	copyfile	write	23	/.vol/16777218/2/.DocumentRevisions-V100/staging/adding.Wohcjo4i/4772FAAA-78D3-44A9-9412-A2D651B6EB5A.package/large0.file	1048576
>> ...
>>
>> How can I tell Versions that the only file that constantly changes within my document.package is metadata.info? (Without turning off document revisions altogether)
>> I am targetting 10.8 (with sandboxing enabled) and also adopted async saving and autosavesInPlace.
>
> Given the right conditions, NSFileWrapper can make writing vastly more efficient by writing hard links for unchanged files, rather than recreating them afresh. Have you determined whether this is happening at all?
>
> In my experience Versions tends to be pretty inefficient about its work. Are you seeing it block the main thread though? That can generally be avoided.
>

_______________________________________________

Cocoa-dev mailing list (email@hidden)

Please do not post admin requests or moderator comments to the list.
Contact the moderators at cocoa-dev-admins(at)lists.apple.com

Help/Unsubscribe/Update your Subscription:

This email sent to email@hidden

  • Follow-Ups:
    • Re: NSDocument packages and incremental writing using NSFileWrapper
      • From: Mike Abdullah <email@hidden>
References: 
 >NSDocument packages and incremental writing using NSFileWrapper (From: Thomas Zoechling <email@hidden>)
 >Re: NSDocument packages and incremental writing using NSFileWrapper (From: Mike Abdullah <email@hidden>)

  • Prev by Date: Re: Interface Builder button appearance
  • Next by Date: Re: NSTask Explodes. Clueless.
  • Previous by thread: Re: NSDocument packages and incremental writing using NSFileWrapper
  • Next by thread: Re: NSDocument packages and incremental writing using NSFileWrapper
  • Index(es):
    • Date
    • Thread