Re: Large "Data" support
Re: Large "Data" support
- Subject: Re: Large "Data" support
- From: Alastair Houghton <email@hidden>
- Date: Tue, 06 Dec 2016 08:32:31 +0000
On 6 Dec 2016, at 01:15, Jens Alfke <email@hidden> wrote:
>
>> On Dec 5, 2016, at 4:12 PM, Daryle Walker <email@hidden> wrote:
>>
>> For the Swift 3 "Data" type, if I want to represent a multi-gigabyte file, it won't try to do it all in memory, would it?
>
> The Data type specifically represents in-memory data, so yeah, it would. (Where by “memory” I mean “address space”, which is not at all the same thing as physical RAM on macOS, though it is on iOS. You haven’t specified which platform you’re interested in.)
>
>> Or would I have to manage a memory-mapped NSData and somehow connect it to a Data object?
>
> On Mac this would be of limited use; all it changes is that when your data gets paged out it will be written to your custom file instead of the OS’s usual swap file. It probably won’t improve performance although it would help keep the swap file from growing as much.
Strictly speaking, it may well improve performance, for two reasons:
1. If you mmap() data, you don’t actually have to read all of it; it’s read from disk on demand.
2. If the system needs to page your data out and you haven’t dirtied it, if it’s mmapped() it can just dispose of the pages; no need to hit the disk.
Compare with malloc()ing a buffer and reading a file into it (which NSData will do if you just read a file into it):
1. You need to read all the data from disk into your buffer.
2. If it’s too large to fit in memory, or some of it needs to be paged out, it will need to be written to the swap file.
Note that these are mainly advantages for large data sets, particularly those that are sparsely accessed and/or written to. If you’re dealing with small files, mmap() may well be slower because of the overhead of page fault handling.
(Aside: a historical irony here is that on 32-bit platforms, software that dealt with exactly those kinds of data sets often couldn’t use mmap() because there wasn’t enough address space, but on 64-bit systems that’s unlikely to be true for most files most people care about.)
Kind regards,
Alastair
--
http://alastairs-place.net
_______________________________________________
Cocoa-dev mailing list (email@hidden)
Please do not post admin requests or moderator comments to the list.
Contact the moderators at cocoa-dev-admins(at)lists.apple.com
Help/Unsubscribe/Update your Subscription:
This email sent to email@hidden