For those of you following this thread, we sorted out the cert encryption problem this morning. The problem was complex to solve, due to the limited information on how certs work on OS X and the lack of good diagnostic messages for the end users. The root cause of the failure was the sender not having a non-expired "encryption" cert to use to send email coupled with the failure of OS X to select the correct cert when multiple certs are present. In this situation the receiver was a national lab employee (LANL) where they use ENTRUST. For some reason they issue to certs to each person, one for signing, one for encryption. So the receiver has to manually email their encryption cert for addition to the senders keychain to allow encrypted messages to be sent. Other national labs that we deal with seem to have ENTRUST certs that allow encryption of returned email and signatures for sending email. This situation may be unique to the way LANL has implemented ENTRUST. We do not know this for certain since we only deal with 5 of the national labs.
Here is the way we see OS X handling certs. This may not be exactly correct but it is close from our observations of how certs are handled by OS X:
OS X stores certs on the keychain at the OS level. Programs such as ADDRESS BOOK and MAIL make a call to the OS to match a cert for a user. The match is by email address and the match is case sensitive.
OS X does not thoroughly validate a cert for "cert usage". OS X tells ADDRESS BOOK that a cert is "valid" but if the cert is a "signing cert" and you don't have an "encryption" cert OS X leads you to believe you have a legitimate cert from the information displayed in Key Chain Access and Address Book. However, Apple Mail won't encrypt the message and you don't get any indication from OS X or Apple Mail why you cannot encrypt. In fact KEYCHAIN ACCESS and ADDRESS BOOK tell you the cert is "valid". This is extremely confusing to the end users. At a minimum, Key Chain Access and Address Book should show not only the validity of the cert, but also the usage of the cert (encryption, signing, etc.) End users should not be expected to expand the entire cert and read through the cert to see if it is a signing cert or an encryption cert or a cert that can do both.
When an application such as APPLE MAIL or Key CHAIN ACCESS ask OS X for a cert for an individual, only one cert is identified by the OS and passed back to the application. If multiple certs are stored in the OS, the OS should return a list of certs with information such as expired or valid, and valid cert use. If the application is going to interact with the cert for encryption, then the user should be able to pick a cert that is non-expired and suitable for encryption. If the user wishes to sign an email and has multiple signing certs, then the OS should return a list of valid "signing" certs and let the user choose which cert to use for signing.
The "rules" OS X is following to determine which cert to present when an application requests cert information are not clearly documented or understood. It appears that OS X is choosing the oldest non-expired cert. However, this is not always consistent. We have seen instances when OS X returns an expired cert that was added to the key chain after an unexpired cert. This is INCREDIBLY confusing to everyone. I filed a bug report on this over a year ago and have been complaining to Apple about this behavior, but have not received any indication that anything is being done to clean this area up. At the least, the OS should default to the newest unexpired cert. Optimally, the user should be given a choice of which cert to use if there are multiple unexpired certs. The OS should NEVER default to an expired cert if an unexpired cert is present. Users should never have to manipulate certs within the keychain to change the storage order so the OS presents the desired cert.
Hope this helps others that are wrestling with cert problems.
Paul
-- Paul Derby Chief Enterprise Architect supporting BioWatch Systems Program Office as IT Lead Department of Homeland Security 703-647-2745
-
Date: March 8, 2009 10:14:59 PM EDT
Subject: Re: [Fed-Talk] mail uses old x.509 certificate
Shawn,
I also found the certificate selection situation somewhat non-intuitive and confusing. I suspect that Paul's issue is that he didn't "Show Expired Certificates", and that's why he can't find them. I've run into the same thing a number of times before.
Your example of the "security" command is good, but I would recommend adding the "-a" option, to show ALL certificates, and not just the first match. Since the first match is often NOT the one that you want, it can be confusing, because it will often give you an expired cert even though there are valid certs in the keychain. It would be a lot less confusing if it returned "best match" instead of "first match", so a valid cert would have precedence over an expired or otherwise invalid cert.
The address book has the same problem. It is just as bad as the "security" command, in that it always seems to prefer invalid certs over ones that are valid. In order for Address Book to display the correct cert for a given email address, you have to rummage around all your keychains and delete all expired certs that match that email address. That really seems backwards. Why is this manual keychain maintenance required in order to get the right certs to display in Address Book?
Also, can you explain the relationship between what Apple Mail chooses for a certificate, vs. what Address Book displays? Is there any correlation? Paul's email implies that a user would intuitively assume that the cert used in signing an email would be the one displayed in the Address Book. Is there really any relationship, or do these separate applications have independent logic in how they choose which cert to use?
If Apple Mail is "never allowed (by the OS) to utilize any certificate / identity that does not successfully pass the various tests", why wouldn't you make Address Book operate the same way, so that a user trying to diagnose problems like this won't be confused by the different results of these apps? It would be nice if Address Book displayed the same cert that Apple Mail would choose for digital signature or encryption for a given email address.
--Ron
Date: March 8, 2009 10:14:59 PM EDT
Subject: Re: [Fed-Talk] mail uses old x.509 certificate
Shawn,
I also found the certificate selection situation somewhat non-intuitive and confusing. I suspect that Paul's issue is that he didn't "Show Expired Certificates", and that's why he can't find them. I've run into the same thing a number of times before.
Your example of the "security" command is good, but I would recommend adding the "-a" option, to show ALL certificates, and not just the first match. Since the first match is often NOT the one that you want, it can be confusing, because it will often give you an expired cert even though there are valid certs in the keychain. It would be a lot less confusing if it returned "best match" instead of "first match", so a valid cert would have precedence over an expired or otherwise invalid cert.
The address book has the same problem. It is just as bad as the "security" command, in that it always seems to prefer invalid certs over ones that are valid. In order for Address Book to display the correct cert for a given email address, you have to rummage around all your keychains and delete all expired certs that match that email address. That really seems backwards. Why is this manual keychain maintenance required in order to get the right certs to display in Address Book?
Also, can you explain the relationship between what Apple Mail chooses for a certificate, vs. what Address Book displays? Is there any correlation? Paul's email implies that a user would intuitively assume that the cert used in signing an email would be the one displayed in the Address Book. Is there really any relationship, or do these separate applications have independent logic in how they choose which cert to use?
If Apple Mail is "never allowed (by the OS) to utilize any certificate / identity that does not successfully pass the various tests", why wouldn't you make Address Book operate the same way, so that a user trying to diagnose problems like this won't be confused by the different results of these apps? It would be nice if Address Book displayed the same cert that Apple Mail would choose for digital signature or encryption for a given email address.
--Ron
|