On 27 August 2019, security vendor Imperva told customers that data from its Cloud WAF product, formerly called Incapsula, had been exposed. Imperva had learned of the incident from a third party on 20 August. The data covered Cloud WAF accounts that existed up to 15 September 2017 and included email addresses and hashed and salted passwords, and for a subset of customers, API keys and customer-provided SSL certificates. In an October 2019 update, Imperva explained the cause. During its 2017 move to Amazon Web Services (AWS), staff created a database snapshot for testing, and an internal compute instance holding an AWS API key was accidentally left reachable from the internet. In October 2018 an attacker compromised that instance, took the key and used it to access the snapshot. Imperva said it was not aware of malicious activity on customer accounts. Customers changed passwords, rotated certificates and regenerated API keys in large numbers.
Key takeaways
- An AWS API key on an internal compute instance that had been left exposed to the internet was stolen in October 2018 and used to access a Cloud WAF database snapshot, according to Imperva's October 2019 update as reported by Security Affairs, Dark Reading and TechTarget.
- The snapshot held email addresses and hashed and salted passwords for Cloud WAF accounts up to 15 September 2017, plus API keys and SSL certificates for a subset of customers, Imperva said.
- Imperva learned of the breach from a third party on 20 August 2019, about ten months after the key was used, and disclosed it on 27 August.
- According to chief technology officer Kunal Anand, customers changed more than 13,000 passwords, about 13,500 SSL certificates were rotated and over 1,400 API keys were regenerated. Imperva said it was not aware of malicious activity on accounts.
- The identity lesson: one forgotten cloud key on a test machine can expose the keys and certificates customers trusted a security vendor to hold.
At a glance
| Organisation | Imperva (Cloud WAF, formerly Incapsula) and its Cloud WAF customers |
|---|---|
| When | AWS API key stolen and snapshot accessed in October 2018; Imperva told by a third party on 20 August 2019; disclosed 27 August 2019; root cause published October 2019 |
| Attacker | Unknown; not publicly identified |
| Entry point | An internal compute instance in Imperva's AWS environment that had been accidentally exposed to the internet |
| Identities abused | An administrative AWS API key stored on the compute instance; exposed data included customers' Cloud WAF API keys and SSL certificates |
| Impact | Cloud WAF database snapshot accessed; customer emails, password hashes, API keys and SSL certificates exposed; mass rotation by customers; Imperva not aware of malicious account activity |
| Category | NHI. Incident class: confirmed NHI breach (a stolen AWS API key used to copy customer credentials) |
What happened
Imperva sells web application firewall (WAF) and other security services. Its Cloud WAF sits in front of customers' websites, filtering traffic, which means it holds customers' SSL certificates and gives them API keys to manage their settings. On 27 August 2019, Krebs on Security reported that Imperva was telling customers about a breach of Cloud WAF data. "We want to be very clear that this data exposure is limited to our Cloud WAF product," Imperva said. The company advised customers to change their passwords, enable multi-factor authentication, reset API keys and generate and upload new SSL certificates.
The cause was explained in an update in October 2019. According to Security Affairs, Imperva said that when it began moving infrastructure to AWS in 2017, developers created a database snapshot for testing. An internal compute instance that was accidentally exposed to the internet contained an AWS API key. Attackers compromised the instance, took the key and used it to reach the snapshot. Imperva described the event as "an unauthorized use of an administrative API key" in a production AWS account, in October 2018, and said the data was taken without exploiting a vulnerability in its products. Dark Reading reported that Imperva was decommissioning inactive compute instances, rotating credentials, strengthening credential management and placing internal compute instances behind a VPN by default.
The remediation numbers came from chief technology officer Kunal Anand. TechTarget reported his figures of "more than 13,000 passwords" changed, about 13,500 SSL certificates rotated and over 1,400 API keys regenerated. It was not stated how many customers were affected in total. Imperva's chief executive, Chris Hylen, left the company as of 21 October 2019 in what a statement called "a mutual decision" with the board, according to TechTarget, which noted it was unclear whether his departure was connected to the breach.
Analysts explained why the exposure mattered more than the passwords. Rich Mogull of DisruptOps told Krebs on Security that with a customer's API key and certificate, "Attackers could whitelist themselves and begin attacking the site without the WAF's protection". In the worst case, he said, an attacker could intercept or redirect a customer site's traffic.
Timeline
| Date | Event |
|---|---|
| 2017 | Imperva begins moving infrastructure to AWS; a database snapshot is created for testing. |
| 15 September 2017 | Cut-off date: Cloud WAF accounts that existed up to this date are in the exposed data. |
| October 2018 | An attacker takes an AWS API key from an exposed internal compute instance and uses it to access the snapshot. |
| 20 August 2019 | A third party tells Imperva about the data exposure. |
| 27 August 2019 | Imperva notifies customers and discloses the breach publicly. |
| October 2019 | Imperva publishes its root cause analysis and remediation figures. |
| 21 October 2019 | Chief executive Chris Hylen steps down. |
How it happened: the identity attack path
- Test data in production. During the 2017 AWS migration, a snapshot of the Cloud WAF customer database was created for testing.
- Exposed compute instance. An internal compute instance was accidentally left reachable from the internet and held an AWS API key with administrative reach.
- Key stolen. An attacker compromised the instance and took the key in October 2018.
- Snapshot accessed. The key gave access to the database snapshot, with customer emails, password hashes, API keys and SSL certificates.
- Late discovery. Imperva only learned of the exposure from a third party in August 2019, about ten months later, and then rotated its own credentials and pushed customers to rotate theirs.
Impact
- Confirmed: email addresses and hashed and salted passwords for Cloud WAF accounts up to 15 September 2017, and API keys and customer-provided SSL certificates for a subset of customers.
- Response at scale: more than 13,000 passwords changed, about 13,500 SSL certificates rotated and over 1,400 API keys regenerated, according to Imperva's CTO.
- Not reported: Imperva said it was not aware of malicious activity on customer accounts as a result of the incident.
- Potential: with a customer's API key and certificate, an attacker could weaken WAF settings, exempt its own traffic or intercept a site's traffic, according to Rich Mogull.
What this means for NHI governance
The Imperva breach links two layers of non-human identity. The first was Imperva's own: an administrative AWS API key left on a test instance that nobody was tracking, on a machine that should never have been reachable from the internet. The second was its customers': the API keys and certificates they had handed to a security provider. Losing the first exposed the second, and the theft went unnoticed for about ten months. It is a reminder that machine credentials created for migrations and tests often outlive the projects that needed them.
The lesson repeats across cloud breaches. A stolen cloud credential was also central to the Uber breach 2016 and, through a role rather than a static key, the Capital One Breach 2019 the same year Imperva disclosed. Keeping an inventory of every cloud key, removing those on idle or test infrastructure, and preferring short-lived role credentials are covered in our Cloud PAM and CIEM Guide and API Key Management Guide.
Recommendations
- Inventory and expire cloud keys created for migrations and tests. Every key needs an owner and an end date, and keys on idle instances should be removed. See our Cloud PAM and CIEM Guide.
- Use short-lived role credentials instead of static API keys on compute. Instances should get temporary credentials from the platform, scoped to their task. See the Cloud Workload Identity Guide.
- Never use production customer data for testing. Test with synthetic or masked data, and delete snapshots when the test ends.
- Keep internal compute off the internet. Place internal instances behind a VPN or private network by default, as Imperva later did, and scan for exposed services.
- Protect customer keys and certificates you hold. Encrypt them with keys managed separately from the database, and make customer rotation easy. See our API Key Management Guide.
- Detect key misuse quickly. Alert on snapshot access, copying and API calls from unusual locations, so a stolen key is found in hours rather than months. See the Leaked Credential Response Playbook.
Frequently asked questions
How was Imperva breached in 2019?
According to Imperva's October 2019 update, an internal compute instance created during its 2017 move to AWS was accidentally exposed to the internet and held an AWS API key. In October 2018 an attacker compromised the instance, took the key and used it to access a database snapshot of Cloud WAF customer data.
What Imperva customer data was exposed?
Email addresses and hashed and salted passwords for Cloud WAF (Incapsula) accounts that existed up to 15 September 2017, and API keys and customer-provided SSL certificates for a subset of those customers.
Why were the exposed API keys and SSL certificates dangerous?
Cloud WAF customers use API keys to manage their firewall settings, and the SSL certificates secure their sites' traffic. With both, an attacker could change a site's WAF protection or intercept its traffic, so customers were told to regenerate keys and replace certificates.
Related NHI Mgmt Group resources
Uber Breach 2016 · Capital One Breach 2019 · Tesla Kubernetes Cryptojacking 2018 · Cloud PAM and CIEM Guide · API Key Management Guide
How NHI Mgmt Group can help
Cloud migrations leave behind keys, snapshots and instances that nobody owns. We help organisations find and retire them, move workloads to short-lived credentials and protect the customer secrets they hold. See our NHI and AI agent security training.
References
- Krebs on Security: Cybersecurity Firm Imperva Discloses Breach (27 August 2019)
- Dark Reading: Imperva Details Response to Customer Database Exposure (11 October 2019)
- Security Affairs: Imperva explains how hackers stole AWS API Key and accessed to customer data (14 October 2019)
- TechTarget: Imperva CEO steps down following breach investigation (30 October 2019)