More recently, websites running Let’s Encrypt (225 million domains) risked running out of traffic from older Android devices (34% of all devices). Let’s Encrypt said they found a way out and IdenTrust issued a new three-year cross-mark for the ISRG X1 root from its DST X3 root CA for them. But can Let’s Encrypt website owners really relax?
The expiring Let’s Encrypt root certificate means that one in three Android devices (over 34% of devices use version 7.1.1 and below) will be blocked from millions of sites protected by Let’s Encrypt.
This issue is a result of Let’s Encrypt no longer cross-signing with a third-party root certificate. Due to this, website visitors on older versions of Android will be blocked from accessing sites protected with Let’s Encrypt.
This is definitely not what users want to see when they visit a website. Likewise, site owners are not eager for users to see any warnings that could raise doubts about the safety of their pages.
What exactly led to this turn of events? Who may be affected by the change? What can website owners do to minimize damage?
Background of the problem
The roots of the problem go back to 2015, when Let’s Encrypt was founded by the Internet Security Research Group (ISRG) and their partners. Since it can take years for all OSs and browsers to start trusting their own certificate, they cross-signed their certificates with the existing CA’s trusted root (IdenTrust and their DST Root X3 certificate), which is typical behavior for new CAs.
This allowed Let’s Encrypt to immediately begin issuing certificates that could be trusted on the Internet. But fast forward to our days: IdenTrust DST Root X3 is approaching an expiration date of September 30, 2021.
Let’s Encrypt tried to prepare for expiration by issuing its own ISRG Root X1 root certificate. Unfortunately, it does not yet have the overarching trust that their IdenTrust root had, so users on some older platforms will be blocked from accessing any website that uses a Let’s Encrypt SSL / TLS certificate.
Who exactly will suffer?
Users of older versions of Android
The problem only affects devices on older platforms, which would seem not so bad, but, unfortunately, they are still used by a large number of people.
It is worth clarifying that the main problem appeared not through the fault of Let’s Encrypt, but in the slow software update for many platforms.
Why are Android devices not updating?
Mobile phone manufacturers and operators usually customize the OS to their needs before installing it on devices and giving it to end users. When Google releases a new Android update, manufacturers and carriers need to first make changes to their own software versions before updating their customers. In addition, manufacturers are constantly releasing new devices that need to be sold, and therefore, OS updates often simply do not work on older models. This is why there are millions of Android devices with legacy operating systems nowadays.
Older Java versions are also prone to root directory changes. Any clients running Java versions prior to 1.8.0_141-b15 will receive warnings or errors when Let’s Encrypt certificates are found.
So far, these are the main platforms where compatibility issues are expected, but expiration of root privileges may show others.
Solution to the problem: two certificates instead of one
IdenTrust has agreed to issue a three-year cross-mark for ISRG X1. The new cross sign is a bit unusual as it is valid after the DST Root CA X3 expires. This solution works because Android does not intentionally set expiration dates for certificates used as trust anchors. Thus, Let’s Encrypt is going to provide its subscribers with a chain containing both ISRG Root X1 and DST Root CA X3, ensuring uninterrupted service to all users and avoiding potential disruptions.
The chain switchover previously scheduled for January 11 did not take place, and the transition to the new chain will take place in early February. However, the following guidelines can be used to avoid potential compatibility issues:
Prompt visitors to update their Android
Alternatively, you can warn visitors with older versions of Android to update them before using your site, but:
- The user is a lazy creature, and the request to update Android may lead the visitor to go to another site.
- Android usually isn’t updated, not because they don’t want to, but because they can’t. For example, a tablet or phone does not support new versions, and asking the user to buy a new device is clearly too much.
Suggest Firefox Mobile
Firefox Mobile relies on its own (regularly updated) list of root certificates, not a list of operating systems. But getting the user to switch to a different browser is perhaps more difficult than getting them to update.
Discontinue support for older versions
It is possible to end support for older versions, but this can lead to:
- Reducing traffic
- Increased support requests
Site owners on ACME
Site owners using ACME can change their client settings to continue using Let’s Encrypt cross-signed certificates. However, this will only work until September 2021. However, it will buy you time to come up with a long-term solution.
The most practical solution, without requiring any user action, is to switch to an ubiquitous CA that is trusted by all major platforms (including legacy systems). Certificates from trusted and authoritative authorities have been using their own trusted roots for years and cross-signed using their old roots to ensure full compatibility.