From the imminent version 90, Chrome will show a certificate error when a user tries to access any website with a certificate signed by Camerfirma. Perhaps it is not the most popular CA, but it is very present in Spain in many public organisations, for example, in the Tax Agency. If this “banning” of Chrome is not solved for the income tax campaign, there may be problems accessing official websites.
Many other organisations in Spain (including the COVID vaccination campaign website, vacunacovid.gob.es) also depend on the certificate. But what happened, and why exactly did Chrome stop trusting this CA? Microsoft and Mozilla still trust it, but of course Chrome’s decision will create a chain effect that will most likely make it impossible to trust anything issued by this CA from the main operating systems or browsers.
In other news regarding this issue, there has been talk of Camerfirma’s failures and its inability to respond and solve them, but to be fair, we need to know a little about the world of certificates. The first thing is to understand that all CAs make mistakes: a lot of them, always. Just take a look at Bugzilla. The world of cryptography is extremely complex, and the world of certificates… as well.
Following the requirements is not always easy, and that is why the CA/Forum organisation and many researchers are responsible for ensuring that the certification authorities function perfectly and comply strictly and rigorously with these standards, so they are very used to these failures, mistakes and oversights and tolerate problems to a certain extent as long as they are reversed in time and corrected. It is a question of showing willingness and efficiency in management, rather than being perfect.
Incidents occur on a daily basis and are varied and complex, but CAs usually react by solving them and increasing vigilance, which improves the system on a daily basis. But sometimes, trust in a CA is lost because a certain limit is crossed in terms of its responses and reactions. In the case of Camerfirma, it seems that the key is that they have been making mistakes for years, some of them repeatedly, and that they have shown too many times that the remedies and resolution practices of this trusted authority cannot be trusted. Moreover, it seems that their excuses and explanations do not add up.
Chrome’s reaction thus demonstrates that cryptographic security must be taken seriously, and that it will not accept CAs that confess that they are understaffed, ignore specifications, etc. These moves are necessary. But with decisions like this, Chrome is on its way to becoming a de facto CA. We have already mentioned that traditional CAs are losing control of certificates. This could be one of the possible reasons why Chrome will have a new Root Store.
We will describe the reasons very briefly and in order of importance or relevance (subjective). The text in quotation marks is verbatim from what we have found in the Bugzilla tracker, which seems to gloat over the fragility of Camerfirma’s excuses. To be honest, they have to be read in their full, particular context in order to understand them. But even so, what emerges on the one hand is a certain inability on the part of Camerfirma to do the job entrusted of being a serious CA capable of responding in time and form… and on the other, a significant weariness on the part of those who ensure that this is the case.
- One: In 2017, the world stopped trusting WoSign/StartCom as a CA for different reasons. Camerfirma still had a relationship with StartCom as a way to validate certain certificates, and it did so under the criteria of “other methods”, which is the strangest (and last) way to achieve this and, therefore, raises suspicions. The CA/Forum did not want these “other methods” to be used (which came from an outdated specification) and did not want certificate validation to be delegated to StartCom. Camerfirma did not rectify the situation and continued the relationship with StartCom without making it clear how.
- Two: They did not respect the CAA standard. This DNS record should contain which CAs are the preferred CAs for a website. For example, I do not want CA X to issue a certificate for me ever… or I only want CA Y to issue certificates for my domain. Camerfirma thought that if certificate transparency existed, they could avoid respecting CAA standards, because “they were in a hurry and misunderstood the requirements”.
- Three: OCSP responses (to quickly revoke) did not comply with the standards.
- Four: It was discovered that the Subject Alternative Names fields of many of their certificates were wrong. When they reported this to Camerfirma, they got no response, because these reports “went to only one person” who did not respond. Camerfirma never “intentionally” fixed certificates of this type and even after revoking some of them, Camerfirma reissued them incorrectly.
- Five: Intesa Sanpaolo, one of Camerfirma’s sub-CAs, also made several mistakes when it came to timely revocation. It even issued a certificate for “com.com” by “human error”.
- Six: They made certain revocations by mistake, confusing serial numbers in valid and invalid certificates. Camerfirma decided to do a “de-revocation”, which is intolerable in the world of certificates, but they implemented it inconsistently. In the midst of all the trouble, they claimed that they would use EJBCA management software to mitigate this in the future, but then they didn’t… then they confirmed that they would develop their own software with similar features. As not much more was heard about this afterwards, they claimed that they were in “daily meetings to discuss these issues”.
- Seven: Camerfirma infringed a rule related to the inclusion of the issuer’s name and serial number in the key ID field (you must not). All Camerfirma certificates had been doing this wrong since 2003. They claimed they had got it wrong and fixed it at the end of 2019. But they did not revoke the previous certificates issued. In 2020 they reissued certificates that infringed this policy, which they did not revoke either.
- Eight, nine and ten: They are not supposed to issue certificates with underscores in their names. According to a “human error” in their issuing and detection, they were not able to detect them in time and some of them were missed. It also happened with a domain name with the character “:”. And with a domain that existed but they spelled it wrong in the certificate.
- Eleven: Camerfirma (and others) issued sub-CAs that could give OCSP responses for Camerfirma itself, because they had not included a suitable restriction in the certificate’s EKUs (EKUs are fields to limit the certificate’s power and use). They argued that they were not aware of this security flaw and did not revoke them in time. The reason for not revoking is that one of the sub-CAs was used in the healthcare smartcard sector and if they were to revoke them, these smartcards would have to be replaced. The problem was so important that they had to escalate the issue to higher bodies at a national level. On 2 October 2020, it appears that the keys on these cards were destroyed, but this destruction was neither supervised nor witnessed by a qualified auditor nor by Camerfirma itself.
- Twelve: They issued a subCA for the use in S/MIME to the government of Andorra, which they did not audit. When they did, it was found that they made quite a few mistakes. In the end they had to revoke it, and claimed that as they were TLS certificates, they thought they were outside the scope of the audits. Again, the problem seemed to be that they did not have sufficient and necessary staff.
- From thirteen to twenty-six: We have cheated here to put together all the other reasons that are very similar. For example, dozens of technical failures in other certificate fields that they were unable to revoke in time. And the excuses for this were varied. From the fact that local legislation obliged them to certain formulas that did not comply with the standards (things that they did not prove) … to the fact that their system had worked well for 17 years, but then, as it grew too much, some internal controls failed. Sometimes there were no excuses. They just did not respond to requests. In one incident, they were supposed to disclose the existence of a sub-CA within a week of its creation, but they did not do it. What was happening according to them was that “the person in charge was not available”. Neither was the person’s back-up. Camerfirma tried to solve this by saying that they would put “a backup for the backup person in charge of this communication”. To solve other problems, they claimed that their staff was completely ” overloaded”, or “on holiday”. Basically, of all the common errors in many certificates (insufficient entropy, incorrect extensions…), Camerfirma always failed to revoke certificates in a time and form.
It is not easy to be a CA. Camerfirma is not the first or the last to be revoked. Even Symantec suffered a setback in this respect. FMNT also had a hard time getting Firefox to include its certificate in its repository and it took several years. At some points in this incredible story with FMNT, there is also a sense of downtime, where one senses a lack of adequate staff to meet Mozilla’s demands.
The world of certification is a demanding one. But it must be. The internet that has been built literally depends on the good work of the CAs. Tolerating the operation of a CA that deviates one millimetre from continuous vigilance, control and demand, or fails to respond in a timely manner, is like allowing condescension towards a policeman or a judge who commits any hint of corruption. It should not be tolerated for our own sake and because of the significant consequences it would entail.