Once an AWS access key is known to be exposed, the question that decides the size of the incident is simple: what did that key actually do, and what did it create that will outlive it? AWS CloudTrail holds the answer, but only if you query it the right way: by key ID across every account and Region, then following each temporary credential the key minted, because those sessions carry a different key ID. This guide is the investigation half of the response: the fields to anchor on, the queries to run in Event history, Athena and CloudTrail Lake, and how to turn the results into a scoped list of affected resources and credentials. It is written for cloud security engineers and incident responders. Containment and rotation are covered in the Cloud Access Key Leak Playbook and How to rotate an AWS IAM access key without downtime; this page assumes the key is already deactivated or about to be.
This is an independent guide from NHI Mgmt Group, based on AWS's public CloudTrail and IAM documentation as of November 2026. It is not sponsored or reviewed by AWS. Product features change, so confirm settings against AWS's current documentation.
Key takeaways
- Anchor on the key ID, then follow its children. A long-term key starts with
AKIA; everyAssumeRoleorGetSessionTokencall it makes returns a newASIAkey inresponseElements, and later activity appears under that new ID, not the original. - Event history is a starting point, not the investigation. It covers 90 days of management events, one account and one Region, one filter at a time, at 2 requests per second. Use the organisation trail in S3 (Athena) or an existing CloudTrail Lake event data store for the real scope.
- Do not trust a blank
accessKeyId. AWS says the field "might not be present, or might be displayed as an empty string", so pivot on the principal ARN, source IP and user agent as well. - Failed calls are evidence.
AccessDeniederrors show what the attacker tried; key validation calls such asGetCallerIdentityfrom new IPs often mark the first use. - Scope means persistence plus data. The deliverable is two lists: every credential or identity the key created (to revoke) and every resource it read or changed (to assess), each with timestamps.
When you need this
Run this investigation whenever an AWS access key may have been used by someone other than its owner. Typical triggers:
- Exposure found: a key in a public repository, container image, mobile app, paste site or log file.
- Detection alert: an Amazon GuardDuty finding such as
UnauthorizedAccess:IAMUser/InstanceCredentialExfiltration.OutsideAWSor one of theIAMUser/AnomalousBehaviorfindings. See the Amazon GuardDuty Guide. - AWS notification: an AWS Support case or a quarantine policy attached to the user by AWS.
- Unexplained cost or resources: instances in Regions you do not use, model invocations, or new email sending identities.
A key seen in a public place with no evidence of use still needs this work: the absence of activity has to be shown from the logs, not assumed.
Before you start
- Record the key ID and exposure window. The window starts at the earliest possible exposure (commit time, image push, log write), not the time it was found. Every query below is bounded by it.
- Identify the owner.
aws sts get-access-key-inforeturns only the account that owns the key. Inside that account,aws iam get-access-key-last-usedreturns the owningUserNameplus the last used date, service and Region; the IAM credential report lists every user's keys. - Know where your logs are. Confirm the organisation trail bucket, which Regions it covers, whether data events are enabled for S3, Lambda, DynamoDB or Secrets Manager, and whether an Athena table or Lake event data store already exists. Logging scope is set out in the AWS CloudTrail Guide.
- Use a separate investigation identity. Query from a security or log archive account with read access to the trail, not from an account the key could reach.
- Snapshot the principal's permissions. Run
list-attached-user-policies,list-user-policiesandlist-groups-for-userbefore anyone changes them, so you can say what the key was able to do during the window.
| Query layer | Covers | Limits to know | Use it for |
|---|---|---|---|
Event history (console or lookup-events) | Management events, last 90 days, per account and Region | One lookup attribute per request; 50 results per page; 2 requests per second per account per Region; no data events | The first ten minutes: is the key in use right now, and from where |
| Athena over the S3 trail | Everything the trail delivered, all accounts and Regions in an organisation trail, including data events if enabled | Needs a table (the CloudTrail console can create one); cost depends on how much you scan, so filter on time and partitions | The full investigation for most organisations |
| CloudTrail Lake | Events in your event data stores, cross account and Region, SQL queries | AWS documents that Lake is closed to new customers from 31 May 2026; existing customers continue as normal | The full investigation where an event data store already exists |
Step-by-step investigation
The queries below use the column names of the Athena CloudTrail table that AWS documents. Replace the key ID, table or event data store ID and timestamps. That table declares eventtime as a string holding the ISO 8601 timestamp, and requestparameters and responseelements as JSON strings read with JSON functions, so string ranges in the form '2026-11-01T00:00:00Z' work for bounding queries.
Step 1: Confirm current use in Event history
Run a lookup on the key in each Region you use, including us-east-1, where calls to the STS global endpoint and IAM are recorded:
aws cloudtrail lookup-events \
--lookup-attributes AttributeKey=AccessKeyId,AttributeValue=AKIAEXAMPLEKEYID \
--start-time 2026-11-01T00:00:00Z --max-results 50 --region us-east-1
Any event after the time the key was deactivated means either propagation delay (allow a few minutes) or a second credential in play. Write down the source IPs and user agents you see; they seed every later query.
Step 2: Pull every call made with the key
SELECT eventtime, eventsource, eventname, awsregion, recipientaccountid,
sourceipaddress, useragent, errorcode, requestparameters
FROM cloudtrail_logs
WHERE useridentity.accesskeyid = 'AKIAEXAMPLEKEYID'
AND eventtime BETWEEN '2026-10-01T00:00:00Z' AND '2026-11-09T23:59:59Z'
ORDER BY eventtime;
In CloudTrail Lake the same query reads FROM <event-data-store-id> with userIdentity.accessKeyId; AWS recommends bounding Lake queries with start and end eventTime values to control cost.
Step 3: Separate the owner from the attacker
SELECT sourceipaddress, useragent, COUNT(*) AS calls,
MIN(eventtime) AS first_seen, MAX(eventtime) AS last_seen
FROM cloudtrail_logs
WHERE useridentity.accesskeyid = 'AKIAEXAMPLEKEYID'
AND eventtime BETWEEN '2026-10-01T00:00:00Z' AND '2026-11-09T23:59:59Z'
GROUP BY sourceipaddress, useragent
ORDER BY first_seen;
Mark each IP and user agent pair as known (corporate egress, CI runners, the application's own hosts, AWS service addresses) or unknown. The earliest first_seen for an unknown pair is your provisional time of first malicious use. Look closely at it: attackers often open with cheap validation calls. Calls such as GetCallerIdentity, used to test whether a key works, are a common first step.
Step 4: Follow the temporary credentials the key created
This is the step most investigations miss. For AssumeRole, AssumeRoleWithSAML, AssumeRoleWithWebIdentity, GetFederationToken and GetSessionToken, IAM's documentation states that CloudTrail includes the response elements (for the three AssumeRole calls, the full responseElements except secretAccessKey), so the new temporary key ID is in responseElements:
SELECT eventtime, eventname, recipientaccountid, sourceipaddress,
json_extract_scalar(responseelements, '$.credentials.accessKeyId') AS child_key,
json_extract_scalar(responseelements, '$.assumedRoleUser.arn') AS session_arn
FROM cloudtrail_logs
WHERE useridentity.accesskeyid = 'AKIAEXAMPLEKEYID'
AND eventsource = 'sts.amazonaws.com'
AND eventname IN ('AssumeRole','GetSessionToken','GetFederationToken')
AND eventtime BETWEEN '2026-10-01T00:00:00Z' AND '2026-11-09T23:59:59Z';
Then rerun Steps 2 and 3 with each child_key (they start with ASIA) and repeat for any further AssumeRole calls those sessions made. Role chaining across accounts shows up as a different recipientaccountid; include the target accounts' logs. Where accessKeyId is blank, query on the session ARN instead, using useridentity.arn or useridentity.sessioncontext.sessionissuer.arn.
Containment note: temporary credentials stay valid for their duration, which for STS can be up to 36 hours. AWS documents that a deny on the IAM user's unique ID (aws:userid, which starts AIDA) also denies sessions created with GetSessionToken. For assumed roles, use the role's Revoke active sessions action. See the AWS STS and AssumeRole Guide.
Step 5: Find persistence
Search the key and every child key for calls that create a new way in. Our recommended list, built from AWS's credential compromise playbook queries:
| What to look for | Event names | Why it matters | ATT&CK |
|---|---|---|---|
| New credentials | CreateAccessKey, CreateLoginProfile, UpdateLoginProfile | Survive deactivation of the original key | T1098.001 Additional Cloud Credentials |
| New identities and trust | CreateUser, CreateRole, UpdateAssumeRolePolicy, CreateOpenIDConnectProvider, CreateSAMLProvider | Give the attacker a principal you have not seen yet | T1078.004 Cloud Accounts |
| Privilege changes | AttachUserPolicy, AttachRolePolicy, PutUserPolicy, PutRolePolicy, CreatePolicyVersion | Widen what any of the above can do | T1098.001 |
| Code and triggers | CreateFunction, UpdateFunctionCode, AddPermission, CreateEventSourceMapping | Execution that runs without the attacker present | T1078.004 |
| Logging changes | StopLogging, DeleteTrail, UpdateTrail, PutEventSelectors | Gaps in the very evidence you are reading | T1685 Disable or Modify Tools |
| Resource policy changes | PutBucketPolicy, DeleteBucketPublicAccessBlock | Access that needs no credential at all | T1530 Data from Cloud Storage |
Read requestParameters and responseElements for each hit: a CreateAccessKey response holds the new key ID, which becomes another anchor for Steps 2 to 4.
Step 6: Establish data access and resource abuse
SELECT eventtime, eventsource, eventname, requestparameters, sourceipaddress
FROM cloudtrail_logs
WHERE useridentity.accesskeyid IN ('AKIAEXAMPLEKEYID','ASIACHILDKEYID1')
AND eventname IN ('GetObject','CopyObject','PutObject','GetSecretValue',
'GetParameter','GetParameters','GetItem','BatchGetItem',
'RunInstances','InvokeModel','CreateEmailIdentity')
AND eventtime BETWEEN '2026-10-01T00:00:00Z' AND '2026-11-09T23:59:59Z';
S3 object reads, DynamoDB item reads and similar calls are data events, which CloudTrail records only if the trail or event data store was configured to log them. If they were not, record that gap explicitly in the scope statement and use other sources (S3 server access logs if enabled, cost and usage data, the resource's own logs). For resource hijacking, also query RunInstances and model invocation calls across every Region, including ones you never use, and compare with billing. Where buckets were rewritten, look for bulk CopyObject with customer-provided encryption keys.
Step 7: Review what was denied
SELECT eventname, errorcode, COUNT(*) AS attempts, MIN(eventtime) AS first_try
FROM cloudtrail_logs
WHERE useridentity.accesskeyid = 'AKIAEXAMPLEKEYID'
AND errorcode IS NOT NULL AND errorcode <> ''
AND eventtime BETWEEN '2026-10-01T00:00:00Z' AND '2026-11-09T23:59:59Z'
GROUP BY eventname, errorcode ORDER BY attempts DESC;
A burst of denied List* and Describe* calls is discovery (T1580 Cloud Infrastructure Discovery, T1087.004 Cloud Account). Denied CreateUser or AttachUserPolicy calls show escalation attempts and tell you which guardrails worked.
Step 8: Write the scope statement
Close the investigation with a short record: first and last malicious use; source IPs and user agents; every key, session and identity in the chain; persistence found and removed; data and resources touched; log gaps (Regions without trail coverage, data events not logged, tampering events); and the confidence level for each conclusion.
Decision points
- Event history or Athena first: Event history if you need an answer in minutes and the key is younger than 90 days; Athena or Lake as soon as more than one account, Region or data events are involved, which is nearly always.
- Exposure window when the start is unknown: use the key's creation date from the credential report. Our recommendation: never shorten the window just to reduce query cost; filter by partition instead.
- Call AWS: for confirmed misuse of a privileged key, AWS's playbook suggests engaging AWS Support or the AWS Security Incident Response service; you do not need certainty first.
- Small versus large estates: a single account with Event history and one trail can be investigated by one engineer in hours. Organisations with many accounts need the organisation trail, a pre-built Athena table with partition projection and saved queries, prepared before an incident.
Verify it worked
- No further activity: rerun Step 1 and Step 2 for the key and every child key for at least 30 minutes after containment, as AWS's playbook suggests; any new event reopens the investigation.
- Every anchor is closed: each key ID, session ARN and identity on your list has a status (deactivated, deleted, denied or confirmed legitimate).
- Log integrity checked: run
aws cloudtrail validate-logsover the window before relying on S3 log files as evidence, and confirm noStopLoggingor selector changes left gaps. - Second reviewer: another analyst reruns the persistence query independently and gets the same list.
Common errors and how to avoid them
- Searching one Region: attackers launch resources where you are not looking, and IAM and global STS calls land in
us-east-1. Query all Regions. - Stopping at the AKIA key: activity from assumed roles is logged under
ASIAkeys and role session ARNs. Skipping Step 4 misses most of what happened in multi-account estates. - Deleting the key first: AWS's playbook prefers deactivation so the key ID stays available for queries and the change can be reversed.
- Assuming no data events means no data access: it means no evidence either way. Say so in the scope statement.
- Forgetting created keys: a key the attacker created with
CreateAccessKeyis a fresh anchor; trace it as you traced the original key.
Measuring readiness
- Time from key exposure alert to a complete Step 2 result (our target: under one hour).
- Percentage of accounts and Regions covered by the organisation trail (target: 100%).
- Percentage of sensitive buckets and secrets stores with data events logged.
- Number of saved, tested investigation queries in the security account.
- Percentage of incidents where every child credential was traced and closed.
Framework mapping
| Framework | Reference | How this guide applies |
|---|---|---|
| NIST CSF 2.0 | RS.AN Incident Analysis | Steps 2 to 8 determine what happened and the scope |
| NIST SP 800-53 Rev. 5 | AU-6 Audit Record Review, Analysis, and Reporting; IR-4 Incident Handling | Query-driven review of CloudTrail during handling |
| NIST SP 800-53 Rev. 5 | AU-9 Protection of Audit Information | Log integrity validation before relying on evidence |
| MITRE ATT&CK | T1078.004, T1098.001, T1580, T1530, T1496 Resource Hijacking | Behaviours the queries are built to find |
Practitioner checklist
- Record the key ID, owner and exposure window before changing anything.
- Snapshot the principal's policies and group memberships.
- Check current use in Event history in every active Region and
us-east-1. - Pull every call for the key from the organisation trail with Athena or Lake.
- Classify each source IP and user agent as known or unknown.
- Trace every
ASIAchild key and session ARN from STSresponseElements. - Run the persistence query on every key in the chain and treat new keys as new anchors.
- Establish data access, record any data event gaps, and check all Regions for resource abuse.
- Validate log integrity and write the scope statement with confidence levels.
Standards and references
- AWS: CloudTrail userIdentity element
- AWS: Logging IAM and AWS STS API calls with AWS CloudTrail
- AWS: CloudTrail LookupEvents API reference
- AWS: Create or edit a CloudTrail Lake query
- AWS: CloudTrail concepts (global service events in us-east-1)
- AWS: Create a table for CloudTrail logs in Athena
- AWS: CloudTrail Lake availability change
- AWS: IAM identifiers and unique ID prefixes
- AWS: Disabling permissions for temporary security credentials
- AWS Samples: IRP-CredCompromise IAM credential compromise playbook
- MITRE ATT&CK: T1098.001 Additional Cloud Credentials
Related NHI Mgmt Group resources: Cloud Credential Compromise Playbook · AWS Access Keys Security Guide · AWS CloudTrail Guide · Cloud Forensics Guide