dng image file corrupting
dng image file corrupting
- Subject: dng image file corrupting
- From: "Millers' Photography L.L.C." <email@hidden>
- Date: Sun, 28 Apr 2013 20:04:01 -0700
I have never understood the phrase 'checksums'. I know for you all, it's important.
From all your support, I have given up MIRROR RAID. And, of course I have to no way to use ECC in my two MBP Pro 8,3's.
Just one of these MBPro's could, maybe, have bought me a Mac Pro. Don't really know. Oh well. I like the idea of taking the imaging work with me. Yes….I could have could have and could have…
Now, I am manually using SuperDuper. I have dedicated two WD MyBook 1TB HD's for image file libraries. Another WD MyBook 1 TB HD is used with SuperDuper to clone the active internal HD. Time Machine is also active.
Don't want to make backup of backups. Have not yet figured out what chain of events I want to use to keep image files
secure, archived. Having one main HD with image files, and not rotating, but keep cloning that one HD. Does that read good?
I had been using the image files right from the MIRRORED RAID, manipulate and save in the same spot on the RAID. Sometimes I bring the image files onto the internal HD for manipulation. Once ready for client, or printing, I think I will send the file back to the RAID. Think I will do the same with the pair of cloned HD's. Of course, I am expecting suggestions from the group. I can use those suggestions.
Bringing me to another topic, FW400, or USB 2.0. No FW800 in this topic. Why? I only have one FW800 enclosure. All other external enclosures are USB 2 AND FW400. Not about to go out and purchase FW800 enclosures. The time cloning is not bothering me.
I have been looking at essays regarding which clones faster. USB 2 or FW400. I know I said clone, I really mean replicate image files that are plus or minus 200 mb's. SuperDuper tells me when using FW400 effective copy speed is about 5 mb's. With USB, effective copy speed ranges between 11 and 18 mb's.
I really have no idea what this all means. Clue me in, please.
Chris asks me how do I know the image files are corrupt. Photoshop CS5 gives me a message. Same with LR4. Once in a while I can view an image in preview. But not anywhere else.
David
On Apr 28, 2013, at 12:00 PM, email@hidden wrote:
> Send Colorsync-users mailing list submissions to
> email@hidden
>
> To subscribe or unsubscribe via the World Wide Web, visit
> https://lists.apple.com/mailman/listinfo/colorsync-users
> or, via email, send a message with subject or body 'help' to
> email@hidden
>
> You can reach the person managing the list at
> email@hidden
>
> When replying, please edit your Subject line so it is more specific
> than "Re: Contents of Colorsync-users digest..."
> Today's Topics:
>
> 1. Yes (STEVEN COTTON)
> 2. Re: dng image file corrupting (Chris Murphy)
>
>
>
> From: STEVEN COTTON <email@hidden>
> Subject: Yes
> Date: April 27, 2013 7:23:40 PM PDT
> To: colorsync users request <email@hidden>, colorsync users <email@hidden>, andrewn <email@hidden>, dreenecotton <email@hidden>, helen bird <email@hidden>, paul reeves <email@hidden>, kevinsolan <email@hidden>, john r cook <email@hidden>
> Reply-To: STEVEN COTTON <email@hidden>
>
>
>
> "I basically make $6,000-$8,000 a month online."
>
> http://www.miimage.net/hwork56.html
>
>
>
>
>
>
> 4/28/2013 3:23:39 AM
>
>
>
>
>
>
>
>
>
> From: Chris Murphy <email@hidden>
> Subject: Re: dng image file corrupting
> Date: April 28, 2013 8:39:16 AM PDT
> To: email@hidden, "Millers' Photography L.L.C." <email@hidden>
>
>
>
> On Apr 6, 2013, at 3:45 PM, Millers' Photography L.L.C. <email@hidden> wrote:
>
>> I am finding dng image files are corrupting.
>
> How do you know they're corrupt?
>
> DNG checksums are supported only since the implementation of the April 2008, DNG v1.2 spec. Without the checksum it's possible they're corrupted, yet open without error. And with the checksum, it's possible you're notified of corruption, but the corruption is minor (may affect a pixel in a non-visible manner, or may only affect metadata).
>
>> Canon's cry seem okay. crw image files seem to last.
>>
>> psd files seem to last. Some tiff files are corrupting.
>
> How do you know the TIFFs are corrupt?
>
> Some file formats are more sensitive to corruption than others. On the one hand a TIFF is more prone to corruption because the file sizes are larger, so there's a bigger footprint for corruption to occur. However, if random corruption occurs with uncompressed TIFF and a JPEG, the TIFF will be less affected than the JPEG because corruption of compressed data has a larger negative effect. So you may just be running into some bad luck with certain file formats, I don't off hand see a pattern.
>
>> These are files on a mirrored RAID with three 1 GB HD's. and also on an external HD. As well as some on the computer's internal HD.
>
> This is a three way mirror using Apple software RAID?
>
> In any case, the mirror instantly replicates corrupt data in memory to all disks in the mirror. The disk sees the corrupt data as valid as any other data. And it's replicated into your backups. And there's no work around for this except to do important work only on computers with ECC memory, which for some people is a show stopper (including myself). So you just have to accept that some corruption is a possibility.
>
> Further it's unclear how Apple software RAID deals with the very long error recovery times in most consumer hard drives, when it comes to repairing bad sectors (drive ECC detected but uncorrectable errors). I've asked a couple times on the filesystem forum but haven't received a response from anyone. So the RAID may be accumulating bad sectors on one or more disk, if they aren't being repaired by the RAID layer.
>
>> Both are MacBook Pro 8,3, Mt Lion, 17inch non-glare display. Both have external monitors.
>
> This model can't use ECC memory, so as Graeme mentioned, this can be a source of the initial corruption. Apple's MacPros are the only computers they sell that use ECC memory.
>
> But there are many other sources for such corruption between the data on the disk and the data in memory (AHCI, the disk controller, the cable, the drive itself which contains a number of components each one of which can induce corruption silently). Consumer drives experience uncorrectable errors statistically at one order magnitude higher than nearline drives, and two orders magnitude more than enterprise drives, and three to four orders magnitude more errors than tape. It's such a problem that RAID 5 with high density consumer drives is a really bad idea, there's a rather significant chance of losing all data on the array with merely a one disk failure. (Most manufacturers proscribe the use of consumer drives with RAID 5 BTW).
>
> And there are user behaviors that induces replication of corruption. e.g. rotation of all back up media, without an archive. And the even worse, but quite common, case of making backups from backups.
>
> The dirty secret of digital photography absolutely is the storage stack.
>
>
> Chris Murphy
>
>
>
>
>
>
>
> _______________________________________________
> Colorsync-users mailing list
> email@hidden
> https://lists.apple.com/mailman/listinfo/colorsync-users
Millers' Photography L.L.C
David B Miller, Pharm. D., member
3809 Alabama Street
Bellingham, Washington 98226
360 739 2826
_______________________________________________
Do not post admin requests to the list. They will be ignored.
Colorsync-users mailing list (email@hidden)
Help/Unsubscribe/Update your Subscription:
This email sent to email@hidden