On 19 July 2020, someone modified the TaskRouter JavaScript SDK that Twilio hosts for its customers, adding code that sent visitors' browsers to a malicious advertising redirector. They did not steal a password or key. The Amazon S3 bucket path that served the library had an access policy allowing any AWS principal to read and write it, a permission Twilio says was added while troubleshooting a build system about five months after the path was created in 2015, and never removed. The altered file went up at 1:12 PM Pacific time and was replaced at about 10:30 PM, after Twilio received an alert. The Register reported the incident on 21 July, and Twilio published an incident report on 22 July. Twilio said only version 1.20 of the SDK was affected, that it had no evidence any customer data was accessed, and that no attacker reached its internal systems. It locked down the bucket, audited every S3 bucket it ran and told customers how to check their copies.
Key takeaways
- An S3 bucket policy granted read and write on Twilio's TaskRouter SDK path to any AWS principal, so anyone on the internet could replace the library, according to Twilio.
- The write permission dated from a change made while troubleshooting a build system about five months after the path was created in 2015, and was never reset.
- A modified TaskRouter JS SDK v1.20 was served for about nine hours on 19 July 2020, with cached copies possible for up to 24 hours afterwards; the injected code pointed to a malvertising redirector linked by RiskIQ to the Hookads campaign.
- Twilio called the attack opportunistic, said it had no evidence of customer data access, and found other buckets with improper write settings during its audit.
- The identity lesson: a resource policy is an identity decision, and a wildcard principal granted for a build job turns a software distribution channel into a public upload point.
At a glance
| Organisation | Twilio, and customers who loaded or downloaded TaskRouter JS SDK v1.20 |
|---|---|
| When | Modified file uploaded 19 July 2020 at 1:12 PM PDT and replaced at about 10:30 PM PDT; reported by The Register on 21 July 2020; Twilio incident report 22 July 2020 |
| Attacker | Unknown. Twilio believes the attack was opportunistic; RiskIQ linked the redirector to the Hookads malvertising campaign |
| Entry point | An S3 bucket policy on the media.twiliocdn.com/taskrouter/ path that allowed any principal to read and write objects |
| Identities abused | No credential was stolen. The bucket's resource policy granted write access to every AWS principal ("AWS": "*"), a permission added for a build system and never reset |
| Impact | Malicious code served to users of TaskRouter JS SDK v1.20 for about nine hours; no evidence of customer data access or of access to Twilio's internal systems, according to Twilio |
| Category | NHI. Incident class: confirmed NHI breach (cloud access policy granting write to any principal used to alter a customer SDK) |
What happened
Twilio served its TaskRouter JavaScript library, used by customers building contact centre and task routing applications, from an Amazon S3 bucket behind the domain twiliocdn.com. According to Twilio's incident report, the bucket policy for the TaskRouter path allowed any principal to perform both GetObject and PutObject. "That meant that anybody on the Internet could read and write to that specific path," Twilio wrote. The path had not been publicly writable when it was created in 2015. Twilio told Dark Reading: "We implemented a change 5 months later while troubleshooting a problem with one of our build systems," and the permissions were not reset afterwards.
At 1:12 PM Pacific time on Sunday 19 July 2020, a modified copy of taskrouter.min.js was uploaded to that path. The added code made the user's browser load an extra URL that Twilio said was associated with Magecart-style attacks. It set a cookie named jqueryapi1oad, fetched a URL that returned another URL, redirected users to other sites, tried to break the back button and used events aimed at mobile devices. Twilio believes it was designed to serve malicious advertising to mobile users. RiskIQ researchers, quoted by Dark Reading, linked the redirector to the Hookads malvertising campaign, and The Register reported that RiskIQ had seen the same URL in other S3 buckets targeted by attackers.
Twilio received an alert at about 9:20 PM, assembled its product and security teams within 15 minutes and replaced the file at about 10:30 PM, locking down the bucket within about an hour of the alert. It warned that the modified version may have stayed in its content delivery network or in browser caches for up to 24 hours. A Twilio spokesperson told The Register: "We have no evidence at this time that any customer data was accessed by a bad actor." Twilio added in a statement reported by Help Net Security: "We do not believe this was an attack targeted at Twilio or any of our customers."
Twilio then reviewed all of its S3 buckets and found others with improper write settings, including a backup holding a copy of the same policy. It said those buckets held no production or customer data, showed no sign of tampering, and that no other hosted SDK was affected. It committed to serving content only through known content delivery networks, monitoring bucket policy changes and offering integrity checks for its SDKs. Jordan Herman of RiskIQ told Dark Reading: "The Twilio compromise was another example of misconfigured Amazon S3 buckets used as an attack vector."
Timeline
| Date | Event |
|---|---|
| 2015 | Twilio creates the TaskRouter path in its S3 bucket without public write access; about five months later a build troubleshooting change adds it. |
| 19 July 2020 | A modified taskrouter.min.js (v1.20) is uploaded at 1:12 PM PDT; Twilio is alerted at about 9:20 PM and replaces the file at about 10:30 PM. |
| 20 July 2020 | End of the window (10:30 PM PDT) in which Twilio says cached copies may still have served the modified file. |
| 21 July 2020 | The Register reports the incident with a statement from Twilio. |
| 22 July 2020 | Twilio publishes its incident report, with integrity hashes and customer guidance. |
| 23 July 2020 | Help Net Security and Dark Reading report Twilio's findings and RiskIQ's link to the Hookads campaign. |
How it happened: the identity attack path
- Wildcard grant added. While troubleshooting a build system, Twilio changed the bucket policy so that any AWS principal could write to the TaskRouter path, and never reverted it.
- Bucket found. Attackers running opportunistic campaigns against writable S3 buckets found the path, according to Twilio and RiskIQ.
- Library replaced. With no credential needed, the attacker uploaded a modified
taskrouter.min.jscontaining a malvertising redirector. - Code served to customers. Browsers and applications that loaded v1.20 from Twilio's CDN ran the attacker's code until it was replaced, and cached copies could persist for up to 24 hours.
- Detected and contained. An alert led Twilio to replace the file, lock down the bucket and audit every other bucket.
Impact
- Confirmed: TaskRouter JS SDK v1.20 served from Twilio's CDN contained attacker code for about nine hours on 19 July 2020, according to Twilio.
- Not found: Twilio said it had no evidence of customer data access and that no attacker reached its internal systems, code or data. Twilio Flex was not affected.
- Uncertain: Twilio said it could not determine what individual users experienced, because the script did not run on all platforms and the linked URLs changed often.
- Wider: Twilio's audit found other buckets with improper write settings, which it said held no production or customer data and showed no tampering.
What this means for NHI governance
No key was stolen here, which is exactly why the case matters for machine identity. In a cloud account, a resource policy decides which identities can act on a resource. Twilio's policy named every identity at once. The grant was made for a machine purpose, helping a build system publish files, and it outlived that purpose by years. Over-broad permissions created for automation and never revisited are one of the most common non-human identity failures, and here the result was a software supply chain compromise without any credential theft at all.
The lesson is to govern the permissions on publishing paths as carefully as the credentials that use them. A build pipeline should write to a distribution bucket through its own scoped role, the bucket should deny every other writer, and any policy change that widens access should raise an alert. Integrity hashes let customers detect a swapped file, which Twilio added afterwards. Our Cloud PAM and CIEM Guide and CI/CD Pipeline Identity Security Guide cover these controls. The incident is unrelated to the 2022 Twilio breach, which began with SMS phishing of employees.
Recommendations
- Remove wildcard principals from write permissions. No storage policy should let
"*"write objects; review every bucket policy and access control list for public write. See our Cloud PAM and CIEM Guide. - Give build systems their own scoped publishing identity. Let only a dedicated pipeline role write to distribution paths, with short-lived credentials. See the CI/CD Pipeline Identity Security Guide.
- Expire troubleshooting changes. Temporary permission changes should carry an owner and an end date, and be reverted automatically. See the JIT Access Guide.
- Alert on policy changes that widen access. Monitor bucket policy changes continuously and flag any that grant public read or write. See the ISPM Guide.
- Publish integrity hashes and use subresource integrity. Customers loading third-party scripts should pin them with hashes so a swapped file fails to load.
- Serve code only through controlled delivery paths. Block direct access to origin buckets and deliver libraries through a CDN you control, as Twilio committed to do.
Frequently asked questions
What happened to the Twilio TaskRouter SDK in 2020?
On 19 July 2020, an attacker replaced Twilio's hosted TaskRouter JavaScript SDK v1.20 with a version containing a malicious advertising redirector. They could do so because the S3 bucket path serving the library allowed anyone to write to it. Twilio replaced the file about nine hours later.
Was customer data stolen in the Twilio S3 bucket incident?
Twilio said it had no evidence that any customer data was accessed and that no malicious party reached its internal systems, code or data. It could not determine what individual users who loaded the modified script experienced.
How was the Twilio S3 bucket misconfigured?
Its access policy allowed any AWS principal to read and write objects in the TaskRouter path. Twilio said the write permission was added about five months after the path was created in 2015, while troubleshooting a build system, and was never reset.
Related NHI Mgmt Group resources
Codecov Breach 2021 · Microsoft SAS Token Exposure 2023 · Twilio Breach 2022 · Cloud PAM and CIEM Guide · CI/CD Pipeline Identity Security Guide
How NHI Mgmt Group can help
Permissions granted for automation tend to outlive the job they were made for. We help teams map which machine identities can write to their software delivery paths, remove wildcard grants and put expiry on temporary changes. See our NHI and AI agent security training.
References
- The Register: Twilio: Someone waltzed into our unsecured AWS S3 silo, added dodgy code to our JavaScript SDK for customers (21 July 2020)
- Twilio: Incident Report: TaskRouter JS SDK Security Incident (July 19, 2020) (22 July 2020)
- Help Net Security: Attackers exploit Twilio's misconfigured cloud storage, inject malicious code into SDK (23 July 2020)
- Dark Reading: Twilio Security Incident Shows Danger of Misconfigured S3 Buckets (23 July 2020)