April 2013 - colorsync-users

Re: Colorsync-users Digest, Vol 10, Issue 65
by Morgan Gordon 30 Apr '13

30 Apr '13

Graeme, I haven't heard any horror stories, but even if I had they'd merely be anecdotes. > And it's getting worse as the manufacturers try and cram more data in smaller and smaller cells. > Some of the earlier Flash devices took 10000 write cycles or so to wear out, but this is dropping - > there's talk of making SSD's out of flash that wears out in as little as a few hundred writes ! My understanding is that MLC NAND has a much shorter lifecycle than SLC NAND. At the beginning of SSD market penetration, all SSDs utilized SLC NAND cells which could write to one block at a time versus MLC NAND cells which required writing to five blocks at a time even if you only needed to record one blocks worth of data. I do agree that hard drives give some warning. Just the other day, one of my seagate backups was running at 85 degrees C... I returned it. Overprovisioning does do the trick and I don't think of it so much as a hack to make them stable as a necessary consequence of writing to a medium inherently less stable. The overprovisioning is monitored in SMART data and that, I think, is an adequate means of safetly establishing their remaining lifespan. On most MLC, cheaper SSDs, the overprovisioning is just a few percent of the total memory on the drive. With better companies such as Samsung, OCZ, and OWC, you can expect the drive to have between 5 and 7 percent over provisioning, potentially giving you an MTBF of over 3 million hours. And enterprise caliber SSDs that will really destroy your budget from companies like HP, Seagate, and OWC will use upwards of 12% of the total memory for over provisioning. The worn sectors are presented in the SMART data for the drive, so you can tell when it's reaching end of life. That said, each drive relies on a microprocessor to handle all writes and reads from the drive, abstracting block level access from the computer. If you were to have a failure in the processor or lose power before a write operation had finished, it could conceivably result in an immediate loss of the device. MTBFs for enterprise level SSDs exceed those of nearline SAS HDDs. Most of us use neither though, unless we're working with a large company and leveraging servers. I've had a lot of experience with HDDs and SSDs but am certainly not an expert on the topic. Of the six SSDs I've purchased in my life since they first became readily available in the 2007-2008 (?) range, I've never had a failure even when using them as personal servers that were constantly being written to. Whereas, the longest lived HDDs I currently own are the ones in my Promise Pegasus, I've had such bad luck with HDDs and especially the 2.5 inch laptop (not enterprise) variety that even if presented with a study that demonstrated with 6 sigma certainty that HDDs were longer lived and more reliable than SSDs, I would honestly probably rationalize the results away as errors in testing methodology. To this day, the price per megabyte ratio is still inferior for SSDs. But right after the floods in the far east where most HDD manufacturing occurs, SSDs became much more competitive. I don't just do still photography. I do a lot of video work. Purely as a function of time, I can significantly increase my productivity during video ingest achieving sustained transfer rates on my two year old laptop in excess of 250 MB / S. They're silent, typically (though not always) draw less power than comparable HDDs and arent subject to problems as a result of head to platter contact during movement which is a real concern with a laptop you're moving while using despite the laptops sudden motion sensor that parks the hard drives in anticipation of a shock that would damage the HDD. Finally, running a web browser, lightroom, photoshop, the OneOne suite, DxO, Bridge, and various plugins fragments my memory and for whatever reason results in progressively less stable systems despite the solid underpinnings of darwin. When I can restart my computer and achieve a relatively fresh state or, in a worst case scenario, start in single user mode, run fsck, repair permissions, and then restart again, all in less then 20 seconds, I feel much less stressed than I do with an HDD based system where the process of restarting and shutting down can sometimes takes five or more minutes. Things become more responsive and as such, I can rule out the causes of a slowdown or error much more quickly. This is a colorsync listserve though and I readily admit that my SSDs have no influence whatsoever on how well my spectro calibrates the limited gamut on my internal Mac Display. :P Morgan P.S. Graeme, until recently, SSDs did have a horrible reputation for not being a secure storage medium. The technology has advanced quickly both in garbage collection (the continuous process that manages the distribution of data on the NAND chips for longevity and to maintain speed) and the quality of the NAND memory integrated into most of these devices. For my mother's computer, I was able to outfit her with a SanDisk 480 GB SSD for $300 dollars. I put it in her 2007 mac mini and even though the computer is limited to 2 gigabytes of ram and she doesn't know how to close tabs in safari, the increased responsiveness of the system (I attribute it to the OS using the SSD for paging files) is so much greater than it was prior to the upgrade that she decided against buying a new iMac and instead spent the money on a really, really old and used RV... now that I think about it, maybe I did her a disservice :P > Message: 4 > Date: Tue, 30 Apr 2013 08:50:37 +1000 > From: Graeme Gill <graeme2(a)argyllcms.com> > To: ColorSync <colorsync-users(a)lists.apple.com> > Subject: Re: Colorsync-users Digest, Vol 10, Issue 64 > > Morgan Gordon wrote: > >> images I choose to retain that arent being worked on are removed from the SSD and spend their >> days on the Pegasus. Then I have a seagate backup plus thunderbolt adapter and three 4 TB drives >> that use with time machine to backup everything on the SSDs on the laptop and iMac and the >> photos I keep on the Pegasus. > > Hmm. SSD has a bit of a reputation for being extremely fast, and quitting without any warning. > And it's getting worse as the manufacturers try and cram more data in smaller and smaller cells. > Some of the earlier Flash devices took 10000 write cycles or so to wear out, but this is dropping - > there's talk of making SSD's out of flash that wears out in as little as a few hundred writes ! > > Yes, they're pulling redundancy tricks (over provisioning and wear levelling) to cover this up, > but I have a nasty suspicion that there will be an assumption that "you'll buy a new machine/SSD > every two years anyway", so that it "doesn't matter" if new SSD's fall over very quickly. > > In contrast, hard drives typically give some warning when they are on the way out. > > Graeme Gill. > >

1 0

Re: Colorsync-users Digest, Vol 10, Issue 64
by Morgan Gordon 30 Apr '13

30 Apr '13

> Message: 1 > Date: Sun, 28 Apr 2013 20:04:01 -0700 > From: "Millers' Photography L.L.C." <digitalimaging(a)dnmillerphoto.com> > > I have never understood the phrase 'checksums'. I know for you all, it's important. A checksum is essentially a hash. It's a mathematical algorithm that generates a unique result based on the contents of the file being checksummed. You can use that checksum as a tool to determine whether there has been any change to a file as even the smallest alteration in data of a checksummed file will generate a different unique result. So if the checksum originally calculated for a file is the same as one that later verified, you have a very high degree of certainty that the two files are identical. Of course, this requires that you checksum the original file in anticipation of verifying that the file is unchanged at a future time. As Chris indicated, checksums are now part of DNG. He was suggesting as one possibility that your data may still be intact or at least useful to you and a checksum mismatch could be interfering with you're ability to open your image. I personally do not know how to go about determining if that's what's happening. > From all your support, I have given up MIRROR RAID. And, of course I have to no way to use ECC in my two MBP Pro 8,3's. > > Just one of these MBPro's could, maybe, have bought me a Mac Pro. Don't really know. Oh well. I like the idea of taking the imaging work with me. Yes….I could have could have and could have… > > Now, I am manually using SuperDuper. I have dedicated two WD MyBook 1TB HD's for image file libraries. Another WD MyBook 1 TB HD is used with SuperDuper to clone the active internal HD. Time Machine is also active. > > Don't want to make backup of backups. Have not yet figured out what chain of events I want to use to keep image files > secure, archived. Having one main HD with image files, and not rotating, but keep cloning that one HD. Does that read good? There are two concerns: 1) your backups could become corrupted or damaged; 2) file corruption could occur on the drive you're creating a backup from and in the act of creating a new backup, you're just copying the corrupted file. The easiest way to prevent something like that from happening is to have iterations of backups -- essentially a bunch of backups from different points in time so that if you discover a corrupted file, you haven't already replaced the one good copy of the file you had in a backup when you go to backup things again. Time machine is an example of an iterative backup system. Time machine has it's own problems but conceptually is a good start. It even allows for automatically rotating drives so you automatically have a backup of a backup. It's not always the greatest backup method for photographers though as its methods for determining whether a file has changed can fail, resulting in unnecessary file duplication. Further, it's slow and can't be used as a live backup like a clone can. If you are on a timetable with a client, nothing quite brings on the stress like searching trying to restore things from a time machine backup. For protecting their backups, some people employ multiple hard drives, tape drives, etc. so they can keep one drive offsite to ensure the safety of your backup from fire or theft. This is not my area of expertise, so I will refrain from offering advice. This is what I personally do (I have a MacBook Pro 8,2 and an iMac ... the particular model number evades me): I've replaced all my internal drives with OWC SSDs (primarily for the speed benefit but also because, for the most part, they're reliable and failure can be anticipated). If I'm on the road, the SSD and memory cards are my backup. When I get home, I copy the files to a Promise Pegasus (it's a RAID-5 enclosure with a thunderbolt connection) but work from the SSD. All the images I choose to retain that arent being worked on are removed from the SSD and spend their days on the Pegasus. Then I have a seagate backup plus thunderbolt adapter and three 4 TB drives that use with time machine to backup everything on the SSDs on the laptop and iMac and the photos I keep on the Pegasus. One of the three time machine drives I keep at my parents' house in a fireproof safe and I typically rotate the drives every two weeks. I do not clone my drives although, it's more because I'm lazy than because it's not important. I've replaced the internal optical drive in the MBP with another SSD that has a fresh system on it in case the primary SSD has a failure and I need to work immediately. The iMac has another SSD in it as well (got to love OWC). In the past, I've experimented with software based RAIDs and always had issues with them, so now I try and stick with a hardware controller like the one in the pegasus. If those exceed your budget, possibly look into drobo. > > I had been using the image files right from the MIRRORED RAID, manipulate and save in the same spot on the RAID. Sometimes I bring the image files onto the internal HD for manipulation. Once ready for client, or printing, I think I will send the file back to the RAID. Think I will do the same with the pair of cloned HD's. Of course, I am expecting suggestions from the group. I can use those suggestions. Keep in mind, a mirrored raid is not intended to ensure data integrity. It's purpose is to have a live clone of your drive that you can seamlessly move to if you experience a hardware failure. I suppose you could periodically rotate the secondary drive in the RAID (I'm not sure apple's implementation supports this; soft raid, if they're still in business, does). > Bringing me to another topic, FW400, or USB 2.0. No FW800 in this topic. Why? I only have one FW800 enclosure. All other external enclosures are USB 2 AND FW400. Not about to go out and purchase FW800 enclosures. The time cloning is not bothering me. > I have been looking at essays regarding which clones faster. USB 2 or FW400. I know I said clone, I really mean replicate image files that are plus or minus 200 mb's. SuperDuper tells me when using FW400 effective copy speed is about 5 mb's. With USB, effective copy speed ranges between 11 and 18 mb's. That's unusual. I typically find fw400 to be faster than usb 2.0. If it says it's faster using USB 2, go with what's faster. > > I really have no idea what this all means. Clue me in, please. > > Chris asks me how do I know the image files are corrupt. Photoshop CS5 gives me a message. Same with LR4. Once in a while I can view an image in preview. But not anywhere else. Maybe Chris has a suggestion as to how to determine what the underlying process is. I could guess that you might be able to open console and see if Photoshop or Lightroom are giving you error messages. Incidentally, if you see I/O error anywhere, that's a sign that your drives are failing and that's why you can't open your images. Good luck! Morgan

4 3

help getting into the 10.8.3 without the user name password.
by Millers' Photography L.L.C. 29 Apr '13

29 Apr '13

How do I get into the 'back door' of 10.8.3? Somehow I must have changed the user password! I need help getting into the 10.8.3 without the user name password. This has nothing to do with firmware password. Thanks in advance. David Millers' Photography L.L.C David B Miller, Pharm. D., member 3809 Alabama Street Bellingham, Washington 98226 360 739 2826

6 7

Re: dng image file corrupting
by Morgan Gordon 29 Apr '13

29 Apr '13

I just joined this list so I've missed the OP. That said, in my personal experience, Canon raw files absolutely can corrupt (and have done so on a RAID-5 using enterprise drives, too.) Always have redundant backups that you rotate a varying intervals long enough to identify corrupted data before you inadvertently propagate the corrupted files throughout your backups. The prior poster accurately noted that file size is a variable in potential file corruption. He quoted the OP who specifically mentioned CRW as being stable. Since only the earliest Canon DSLRs used CRW, it's fair to assume that they were not large files. Losing your work is always hard. No system is truly foolproof. For those images truly important that you don't lose, there is going to be a great amount of effort and complexity required to ensure the permanence of your data. One other thought, I've had some corruption issues in the past with non Adobe software created linear DNG files. I've never had an issue with DNGs converted from Adobe's own software.

2 1

dng image file corrupting
by Millers' Photography L.L.C. 29 Apr '13

29 Apr '13

I have never understood the phrase 'checksums'. I know for you all, it's important. From all your support, I have given up MIRROR RAID. And, of course I have to no way to use ECC in my two MBP Pro 8,3's. Just one of these MBPro's could, maybe, have bought me a Mac Pro. Don't really know. Oh well. I like the idea of taking the imaging work with me. Yes….I could have could have and could have… Now, I am manually using SuperDuper. I have dedicated two WD MyBook 1TB HD's for image file libraries. Another WD MyBook 1 TB HD is used with SuperDuper to clone the active internal HD. Time Machine is also active. Don't want to make backup of backups. Have not yet figured out what chain of events I want to use to keep image files secure, archived. Having one main HD with image files, and not rotating, but keep cloning that one HD. Does that read good? I had been using the image files right from the MIRRORED RAID, manipulate and save in the same spot on the RAID. Sometimes I bring the image files onto the internal HD for manipulation. Once ready for client, or printing, I think I will send the file back to the RAID. Think I will do the same with the pair of cloned HD's. Of course, I am expecting suggestions from the group. I can use those suggestions. Bringing me to another topic, FW400, or USB 2.0. No FW800 in this topic. Why? I only have one FW800 enclosure. All other external enclosures are USB 2 AND FW400. Not about to go out and purchase FW800 enclosures. The time cloning is not bothering me. I have been looking at essays regarding which clones faster. USB 2 or FW400. I know I said clone, I really mean replicate image files that are plus or minus 200 mb's. SuperDuper tells me when using FW400 effective copy speed is about 5 mb's. With USB, effective copy speed ranges between 11 and 18 mb's. I really have no idea what this all means. Clue me in, please. Chris asks me how do I know the image files are corrupt. Photoshop CS5 gives me a message. Same with LR4. Once in a while I can view an image in preview. But not anywhere else. David On Apr 28, 2013, at 12:00 PM, colorsync-users-request(a)lists.apple.com wrote: > Send Colorsync-users mailing list submissions to > colorsync-users(a)lists.apple.com > > To subscribe or unsubscribe via the World Wide Web, visit > https://lists.apple.com/mailman/listinfo/colorsync-users > or, via email, send a message with subject or body 'help' to > colorsync-users-request(a)lists.apple.com > > You can reach the person managing the list at > colorsync-users-owner(a)lists.apple.com > > When replying, please edit your Subject line so it is more specific > than "Re: Contents of Colorsync-users digest..." > Today's Topics: > > 1. Yes (STEVEN COTTON) > 2. Re: dng image file corrupting (Chris Murphy) > > > > From: STEVEN COTTON <cotton334(a)btinternet.com> > Subject: Yes > Date: April 27, 2013 7:23:40 PM PDT > To: colorsync users request <colorsync-users-request(a)lists.apple.com>, colorsync users <colorsync-users(a)lists.apple.com>, andrewn <andrewn(a)easyodds.com>, dreenecotton <dreenecotton(a)yahoo.co.uk>, helen bird <helen.bird(a)avnet.com>, paul reeves <paul.reeves(a)eizo.co.uk>, kevinsolan <kevinsolan(a)hotmail.com>, john r cook <john.r.cook(a)gsk.com> > Reply-To: STEVEN COTTON <cotton334(a)btinternet.com> > > > > "I basically make $6,000-$8,000 a month online." > > http://www.miimage.net/hwork56.html > > > > > > > 4/28/2013 3:23:39 AM > > > > > > > > > > From: Chris Murphy <lists(a)colorremedies.com> > Subject: Re: dng image file corrupting > Date: April 28, 2013 8:39:16 AM PDT > To: colorsync-users(a)lists.apple.com, "Millers' Photography L.L.C." <digitalimaging(a)dnmillerphoto.com> > > > > On Apr 6, 2013, at 3:45 PM, Millers' Photography L.L.C. <digitalimaging(a)dnmillerphoto.com> wrote: > >> I am finding dng image files are corrupting. > > How do you know they're corrupt? > > DNG checksums are supported only since the implementation of the April 2008, DNG v1.2 spec. Without the checksum it's possible they're corrupted, yet open without error. And with the checksum, it's possible you're notified of corruption, but the corruption is minor (may affect a pixel in a non-visible manner, or may only affect metadata). > >> Canon's cry seem okay. crw image files seem to last. >> >> psd files seem to last. Some tiff files are corrupting. > > How do you know the TIFFs are corrupt? > > Some file formats are more sensitive to corruption than others. On the one hand a TIFF is more prone to corruption because the file sizes are larger, so there's a bigger footprint for corruption to occur. However, if random corruption occurs with uncompressed TIFF and a JPEG, the TIFF will be less affected than the JPEG because corruption of compressed data has a larger negative effect. So you may just be running into some bad luck with certain file formats, I don't off hand see a pattern. > >> These are files on a mirrored RAID with three 1 GB HD's. and also on an external HD. As well as some on the computer's internal HD. > > This is a three way mirror using Apple software RAID? > > In any case, the mirror instantly replicates corrupt data in memory to all disks in the mirror. The disk sees the corrupt data as valid as any other data. And it's replicated into your backups. And there's no work around for this except to do important work only on computers with ECC memory, which for some people is a show stopper (including myself). So you just have to accept that some corruption is a possibility. > > Further it's unclear how Apple software RAID deals with the very long error recovery times in most consumer hard drives, when it comes to repairing bad sectors (drive ECC detected but uncorrectable errors). I've asked a couple times on the filesystem forum but haven't received a response from anyone. So the RAID may be accumulating bad sectors on one or more disk, if they aren't being repaired by the RAID layer. > >> Both are MacBook Pro 8,3, Mt Lion, 17inch non-glare display. Both have external monitors. > > This model can't use ECC memory, so as Graeme mentioned, this can be a source of the initial corruption. Apple's MacPros are the only computers they sell that use ECC memory. > > But there are many other sources for such corruption between the data on the disk and the data in memory (AHCI, the disk controller, the cable, the drive itself which contains a number of components each one of which can induce corruption silently). Consumer drives experience uncorrectable errors statistically at one order magnitude higher than nearline drives, and two orders magnitude more than enterprise drives, and three to four orders magnitude more errors than tape. It's such a problem that RAID 5 with high density consumer drives is a really bad idea, there's a rather significant chance of losing all data on the array with merely a one disk failure. (Most manufacturers proscribe the use of consumer drives with RAID 5 BTW). > > And there are user behaviors that induces replication of corruption. e.g. rotation of all back up media, without an archive. And the even worse, but quite common, case of making backups from backups. > > The dirty secret of digital photography absolutely is the storage stack. > > > Chris Murphy > > > > > > > > _______________________________________________ > Colorsync-users mailing list > Colorsync-users(a)lists.apple.com > https://lists.apple.com/mailman/listinfo/colorsync-users Millers' Photography L.L.C David B Miller, Pharm. D., member 3809 Alabama Street Bellingham, Washington 98226 360 739 2826

1 0

dng image file corrupting
by Millers' Photography L.L.C. 28 Apr '13

28 Apr '13

I am finding dng image files are corrupting. Canon's cry seem okay. crw image files seem to last. psd files seem to last. Some tiff files are corrupting. Ideas and suggestions are welcome, These are files on a mirrored RAID with three 1 GB HD's. and also on an external HD. As well as some on the computer's internal HD. Actually both computers. Both are MacBook Pro 8,3, Mt Lion, 17inch non-glare display. Both have external monitors. Anyone else having this issue? Cheers David Millers' Photography L.L.C. digitalimaging(a)dnmillerphoto.com David B. Miller, Pharm. D. member 3809 Alabama Street Bellingham, WA 98226-4585 360 739 2826 Sent from my MacBook Pro 17 inch

6 8

Yes
by STEVEN COTTON 28 Apr '13

28 Apr '13

"I basically make $6,000-$8,000 a month online." http://www.miimage.net/hwork56.html 4/28/2013 3:23:39 AM

1 0

Re: iMac Monitor calibration
by Jim Lennon 27 Apr '13

27 Apr '13

> > On Thu, 25 Apr 2013 09:27:50 -0700 David Miller wrote: > > > Regarding the critical profiling of the new iMac display, it will never every match the NEC display. It is just not built the way the independent NEC is. Is it possible to use a current iMac with an NEC like the one being described (or a Lacie 526, like I have) where the iMac is simply a location for tools, etc. and only the other monitor is calibrated? Can an Eye One Display2 be used to calibrate the other monitor? Thanks, Jim

4 6

Re: iMac Monitor calibration
by Paul Schilliger 26 Apr '13

26 Apr '13

I had actually read so much praise stuff on the NEC that I was expecting it to be a jewel from outer space. Now that it is sitting on my desk, I realize how good my old iMac 24 non glare screen is! Its matte finish is unnoticeable, contrarily to the NEC Multisync PA sandy whites (I'm told though that the current SV Reference has a different panel). I have used that iMac to produce many printed materials with quite good results, as well as an Apple Cinema before, and so have many much more knowledgeable people than I. I have no means of comparing the NEC with the newer iMac — is it still the shiny mirror glass? — but put side by side with the old 2006 iMac and the new MBP Retina, they look pretty much in sync. Only when it comes to very saturated hues, the so called low end screens show some posterization, where the NEC still has gradient. In very dark hues as well, the NEC is capable of showing smoother transitions. But if you'd ask me, from a pragmatic point of view, a very small percentage of the images that are usually treated contain those highly saturated and/or very dark hues, and will actually benefit from being edited in the extended gamut, but take that as an uneducated point de vue. The quality of the calibration tools on the other hand is primordial and I don't regret the amount spent on upgrading to an i1DisplayPro. Paul Schilliger colorsync-users-request(a)lists.apple.com wrote: > Regarding the critical profiling of the new iMac display, it will never every match the NEC display. It is just not built the way the independent NEC is. > > Cheers, David > > > Millers' Photography L.L.C. > David Miller, member > Spinnaker Photo Imaging Center > spinnakerphotoimagingcenter(a)dnmillerphoto.com >> Message: 4 >> Date: Tue, 23 Apr 2013 16:19:56 -0400 >> From: David Ramsey<david(a)davidramseyphotography.com> >> To: Spinnaker Photo Imaging Center >> <spinnakerphotoimagingcenter(a)dnmillerphoto.com> >> Cc:colorsync-users@lists.apple.com >> Subject: Re: iMac Monitor calibration >> Message-ID: >> <F5A3F205-277C-4E3D-845C-4A72ACAD3C99(a)davidramseyphotography.com> >> Content-Type: text/plain; charset=utf-8 >> >> Apologies if I wasn't clear. I am wanting to calibrate best as possible my iMac screen. Is this realistic? I will still be using the NEC along side as the bellwether but having both in somewhat sync would be great. Since the iMac has it's brightness control as part of the keyboard, like notebooks, is there any way to override this so those of us who type badly do not constantly change the brightness not on purpose? >> >> David Ramsey

1 0

OS 10.6.8 again 1st post had a typo
by Dick Busher 26 Apr '13

26 Apr '13

Are there any color management issues with Mac OS 10.6.8? I'd like to upgrade from 10.4.11. Thanks, Dick Dick Busher, Publisher Cosgrove Editions 7042 20th Place NE Seattle, WA 98115 206-524-6726 888-507-7375 dick(a)cosgroveweb.com

2 2