5 August 2022: Invalid command processing on HSM firmware
Summary
During a standard security assessment for the Vault, a security audit company commissioned by Ledger discovered a vulnerability in our product. We chose to deploy a hotfix as soon as the vulnerability was identified, as it broke our threat model.
The vulnerability has not been exploited. Client funds have never been at risk.
Description
A vulnerability has been discovered by a company commissioned by Ledger to perform a pentest on the Vault. During this pentest, they identified an important vulnerability on a critical component of our solution. This vulnerability would allow an attacker with high privileges to perform business operations on our hardware security modules (HSMs).
Our pentesting policy is to perform only white box assessments. We gave full access (full access to servers and product source code) to the team in charge of the audit, so that they can perform the most comprehensive security tests.
Audit is performed in a test environment, completely separate from our own environments (different networks and secrets). The company that discovered the vulnerability would not have been able to exploit it on our environments, as the vulnerable component cannot be reached without valid credentials and network access.
Exploiting this vulnerability requires first to take full control of our HSM host servers (our most critical servers). This requires SSH access to our servers. These servers are accessible only through a VPN, restricted to a small subset of our infrastructure team, with IP restriction and hardware authentication.
No malicious connection has been detected, and the mandated security company never got access to our production servers.
We preferred to immediately deploy a hotfix as the discovered vulnerability violates our threat model: even employees with the most privileged access to our servers should never be able to directly communicate with our HSM for business operations.
After the first update, a variant of this vulnerability has been identified. We decided to perform a deeper analysis of this component. No other variant has been found, and code has been hardened to prevent exploitation of that problem, as a defense-in-depth mechanism. A second update that fixes this variant and hardens code has been deployed.
We are totally confident in the fact that both vulnerabilities have not been exploited on our servers.
Timeline
| Date | Action |
|---|---|
| 28 July 2:25 PM | An external security company mandated by Ledger for a regular security assessment informs us about a vulnerability. |
| 28 July 4:35 PM | Internal security team confirms the vulnerability. |
| 28 July 5:28 PM | Fix is written. |
| 28 July 6:31 PM | Fix is validated by firmware team and internal security team. |
| 29 July morning | Fix is validated on our pre-production environment. |
| 29 July 2:04 PM | Vault servers start being upgraded. |
| 29 July 4:39 PM | All the Vault servers have been upgraded. |
| 1 August 1:30 PM | A variant of the vulnerability has been discovered. |
| 1 August 7:30 PM | Fix is written by firmware team and internal security team. Code hardening is done. |
| 2-3 August | Internal security team reviews fixes and looks for other variants. |
| 3 August | New version is validated in pre-production environment. |
| 4 August 7:00 AM | Vault servers start being upgraded. |
| 4 August 9:43 AM | All the Vault servers have been upgraded. |
All hours are given in CEST time.