The Kubernetes audit log is the only record of who asked the API server to do what, and most clusters either do not collect it or collect it with a policy that hides the events that matter. Every kubectl command, controller action and service account call passes through kube-apiserver, so a well tuned audit policy turns the API server into a sensor for privilege escalation, secret theft, exec into pods and persistence through RBAC or CronJobs. This guide is for detection engineers and platform teams who already run Kubernetes and need to know what the audit log contains, how to configure it on self-managed clusters and on EKS, GKE and AKS, which detections give the best return, and how to measure coverage. It goes deeper than the Kubernetes Security Guide and assumes you know the basics of RBAC and pods.
This is an independent guide from NHI Mgmt Group, based on the Kubernetes project's public documentation as of November 2026. It is not sponsored or reviewed by the Kubernetes project. Product features change, so confirm settings against the Kubernetes project's current documentation.
Key takeaways
- No policy, no log. On a self-managed cluster kube-apiserver records nothing unless
--audit-policy-fileis set, and on EKS all control plane log types are off by default. - The first matching rule wins. Audit policy rules are evaluated in order, so a broad
level: Nonerule placed early can silently drop the events you depend on. - Service accounts are the main actors. Most API calls come from workloads and controllers, so detections must baseline
system:serviceaccount:users rather than only human identities. - Five event families cover most attacks. RBAC changes, secret reads, pods/exec and attach, privileged or hostPath pod creation, and new CronJobs or DaemonSets map to the ATT&CK container techniques that matter most.
- Never log secret bodies. Log secrets, configmaps and token requests at
Metadatalevel only, or the audit log becomes a second copy of your credentials.
What the audit log records
Kubernetes documentation describes auditing as a chronological, security relevant set of records of actions by users, by applications that use the API and by the control plane itself. Each request generates events at defined stages, and each event is a JSON object in the audit.k8s.io/v1 Event schema.
| Field | What it tells you | Detection use |
|---|---|---|
user.username, user.groups | Authenticated identity, for example system:serviceaccount:ns:name or an OIDC user | Baseline per identity; alert on system:anonymous succeeding |
impersonatedUser | Identity assumed through impersonation headers | Impersonation of system:masters members or admins |
verb | get, list, watch, create, update, patch, delete | Separate reads (discovery, secret access) from writes |
objectRef | Resource, subresource, namespace, name, API group | Find pods/exec, secrets, clusterrolebindings |
sourceIPs, userAgent | Client address and tool string | A service account token used from an unexpected IP or with a kubectl user agent |
responseStatus.code | HTTP result | Bursts of 403 responses show permission probing |
requestObject, responseObject | Bodies, only at Request or RequestResponse level | Inspect pod specs for privileged or hostPath |
annotations | Including authorization.k8s.io/decision, authorization.k8s.io/reason and pod-security.kubernetes.io/audit-violations | Which RBAC binding allowed a call; Pod Security Admission violations |
stage, auditID, timestamps | Request lifecycle and correlation ID | De-duplicate and join stages of the same request |
The four stages are RequestReceived, ResponseStarted (only for long running requests such as watch), ResponseComplete and Panic. Most policies omit RequestReceived to halve volume, because ResponseComplete carries the outcome. The documentation also warns that auditing increases API server memory use, which is a reason to tune rather than to log everything at the highest level.
Writing an audit policy that supports detection
The audit policy is a Policy object in the audit.k8s.io/v1 group with an ordered list of rules. The first rule that matches an event sets its level: None, Metadata (user, timestamp, resource, verb, no bodies), Request (metadata and request body) or RequestResponse (metadata, request and response bodies). A policy with zero rules is invalid.
Recommended rule order
- Drop known noise first, narrowly: watch requests from
system:kube-proxyon endpoints and services, leader election configmaps or leases, and health URLs such as/healthz,/readyzand/livez. Scope everyNonerule by user and resource, never by verb alone. - Protect credentials:
secrets,configmapsandserviceaccounts/tokenatMetadataonly, so reads are visible but values are not stored. - Capture bodies where the content is the signal:
pods,deployments,daemonsets,cronjobs,roles,rolebindings,clusterroles,clusterrolebindings,mutatingwebhookconfigurationsandvalidatingwebhookconfigurationsatRequestorRequestResponse. - Keep exec and attach visible:
pods/exec,pods/attachandpods/portforwardatMetadataor higher. These are subresources, and a rule forpodsdoes not match them. - Catch-all last: a final
level: Metadatarule so nothing new falls through unlogged when CRDs or API groups are added.
The upstream example policy in the Kubernetes documentation follows this pattern and points to the policy generated for Google Container-Optimized OS as a starting point. Store the policy in version control and review changes to it like code, because editing it is a way to blind the SOC.
Backends on self-managed clusters
- Log backend:
--audit-log-pathwrites JSON lines to a file (-means standard out), rotated with--audit-log-maxage,--audit-log-maxbackupand--audit-log-maxsize. When kube-apiserver runs as a static pod, mount both the policy file and the log directory ashostPathvolumes or events are lost on restart. - Webhook backend:
--audit-webhook-config-filetakes a kubeconfig style file pointing at a collector. Batching and buffering flags control behaviour under load; test what happens when the collector is down. - Shipping: forward the file with a node agent to the SIEM and keep the local file as a short buffer only. See the Log Management and Retention Guide.
Managed Kubernetes: EKS, GKE and AKS
On managed services you do not write the API server flags. The provider runs its own policy and you choose whether to export the result.
| Service | How audit is enabled | Where it lands | Points to watch |
|---|---|---|---|
| Amazon EKS | Control plane logging types api, audit, authenticator, controllerManager, scheduler; all off by default, enabled per cluster | CloudWatch Logs group /aws/eks/<cluster>/cluster, audit streams kube-apiserver-audit-<id> | Enable authenticator too: it shows which IAM principal mapped to which Kubernetes user. Standard CloudWatch ingestion and storage charges apply; delivery is best effort |
| Google GKE | Admin Activity audit logs always on and cannot be disabled; Data Access logs off by default | Cloud Logging, cloudaudit.googleapis.com/activity and data_access, service k8s.io, resource k8s_cluster | Reads such as get and list on secrets are Data Access events, so secret read detection needs Data Access enabled. Reading Data Access logs in _Default needs Private Logs Viewer |
| Azure AKS | Diagnostic settings categories kube-audit and kube-audit-admin (plus guard for Entra ID authentication) | Log Analytics, Storage or Event Hubs | kube-audit-admin excludes get and list events, which lowers cost but removes secret read and discovery visibility |
GKE adds useful labels such as command.gke.io/command on exec requests. For identity mapping across cloud and cluster, see the Kubernetes Service Account Token Guide and the AKS Security Hardening Guide.
High-value detections mapped to ATT&CK
MITRE ATT&CK v20 lists kubernetes:audit and kubernetes:apiserver as log sources for data components such as Pod Creation, Container Creation, User Account Modification, Scheduled Job Creation, Process Creation and Pod Enumeration. The rules below are the ones most teams should build first.
| Detection | Audit log logic | ATT&CK |
|---|---|---|
| Exec or attach into a running pod | objectRef.subresource in exec, attach; alert on production namespaces, on service account users, or outside change windows | T1609 Container Administration Command |
| Cluster-admin or wildcard binding created | create or patch of clusterrolebindings or rolebindings where roleRef.name is cluster-admin or the role has * verbs or resources | T1098.006 Additional Container Cluster Roles |
| Secret enumeration | list on secrets cluster wide, or get on many secrets by one identity in a short window, especially from a workload service account | T1552.007 Container API |
| Privileged or host-mounting pod | create of pods or workload controllers where requestObject sets privileged: true, hostPID, hostNetwork or a hostPath volume; also Pod Security Admission audit violations | T1611 Escape to Host, T1610 Deploy Container |
| New CronJob or DaemonSet | create of cronjobs (group batch) or daemonsets by an identity that has not created them before | T1053.007 Container Orchestration Job |
| Discovery burst | many list calls across namespaces, nodes and RBAC objects, often with a mix of 403 responses | T1613 Container and Resource Discovery |
| Token minted or used off-cluster | create on serviceaccounts/token; a service account username seen from a sourceIPs value outside node and pod ranges | T1528 Steal Application Access Token, T1078.004 Cloud Accounts |
| Anonymous access succeeding | user.username = system:anonymous with a 2xx response on anything other than allowed health endpoints | T1613 Container and Resource Discovery |
| Audit or admission controls weakened | delete or patch of validating or mutating webhook configurations, policy engine resources or logging agents | T1685 Disable or Modify Tools |
Write the rules in a portable format where you can. See the Sigma Rules Guide and the Cloud Detection Engineering Guide. For what admission control can block before it ever needs to be detected, see the Kubernetes Admission Control Guide.
Where identity fits
The audit log is fundamentally an identity log. Every event is attributed to a user string, and in a typical cluster most of those strings belong to non-human identities: controllers, operators, CI/CD deployers and application service accounts. That has three practical consequences.
- Baseline each service account: a deployment controller that only ever patches deployments in one namespace and suddenly lists secrets cluster wide is a stronger signal than any human anomaly rule.
- Map external identities: on EKS the authenticator log, on AKS the guard log and on GKE the principal email show which cloud identity became which Kubernetes user. Without that join, an alert ends at a generic group name.
- Treat CI deployers as privileged: pipeline identities often hold broad create rights. Alert when their tokens appear from an address that is not the runner pool.
See the Kubernetes NHI Security Guide and the Kubernetes RBAC Guide.
Rolling it out
- Inventory clusters and current state: for each cluster record whether audit is on, the level, the backend, the destination and the retention. Managed clusters with export off are the common gap.
- Start with a Metadata catch-all: enable a minimal policy that logs everything at
Metadataon one non-production cluster, measure events per second and storage per day, then add the noise exclusions. - Add body capture for the high-value resources listed above, confirm the secret rule is still Metadata only, and check a sample of events to prove no secret values are stored.
- Pilot detections in alert-only mode for two to four weeks, tuning per namespace and per service account. Record every suppression with a reason.
- Roll out by cluster tier: production and clusters holding sensitive data first, then the rest. On managed services set the export through infrastructure as code so new clusters inherit it.
- Rollback: keep the previous policy file versioned. If the API server shows memory pressure or latency, revert to the last known policy rather than disabling audit, and lower body capture levels before removing rules.
- Test end to end: run scripted, authorised test actions (create a test ClusterRoleBinding, exec into a test pod, list secrets with a test service account) in a lab namespace each quarter and confirm each one raises the expected alert.
Detection and measurement
Monitor the pipeline as well as the cluster: an audit stream that silently stops is the most common way detections fail.
- Coverage: percentage of clusters exporting audit events to the SIEM, target 100 percent of production.
- Freshness: maximum lag between event time and SIEM ingestion per cluster, with an alert when any cluster sends nothing for 30 minutes.
- Detection coverage: number of the nine detections above deployed and passing quarterly tests.
- Signal quality: true positive rate per rule and alerts per cluster per week after tuning.
- Attribution rate: share of audit events where the actor can be resolved to an owner or workload, including external identity mapping.
- Volume and cost: events per day per cluster, so policy changes that double volume are visible.
Framework mapping
| Framework | Reference | How the audit log supports it |
|---|---|---|
| NIST SP 800-53 Rev. 5 | AU-2 Event Logging, AU-12 Audit Record Generation | Audit policy defines and generates the event set |
| NIST SP 800-53 Rev. 5 | AU-6 Audit Record Review, Analysis, and Reporting; SI-4 System Monitoring | Detection rules and triage |
| NIST SP 800-53 Rev. 5 | AU-11 Audit Record Retention | Retention of exported events |
| NIST CSF 2.0 | DE.CM-01, DE.AE-02 | Monitoring and analysis of adverse events |
| CIS Controls v8.1 | Control 8 Audit Log Management | Collection, centralisation, retention and review. See the CIS Control 8 Audit Log Management Guide |
| ISO/IEC 27001:2022 | Annex A 8.15 Logging, 8.16 Monitoring activities | Logging and monitoring of the cluster control plane |
| MITRE ATT&CK v20 | T1609, T1610, T1611, T1613, T1552.007, T1053.007, T1098.006 | Container techniques detectable from API server events |
Common failure patterns
- Export never enabled on managed clusters: teams assume the provider logs everything. On EKS nothing is exported until you turn it on, and on GKE secret reads need Data Access logs.
- Choosing the cheaper category without knowing what it drops: AKS
kube-audit-adminremoves get and list, so secret enumeration and discovery disappear. - Secret values in the SIEM: a policy that logs
secretsat Request or RequestResponse writes credentials into a system with far wider access than the cluster. Scope the secrets rule to Metadata and test it. - Broad None rules: excluding all get requests or all requests from
system:serviceaccountsto cut cost removes exactly the workload identities attackers use. The Kubeflow cryptomining attacks used a notebook's mounted service account to deploy miners, which is the pod creation pattern a per-identity baseline catches. - Exposed consoles and dashboards treated as out of scope: in the Tesla Kubernetes cryptojacking case an unprotected console led to cloud credentials in a pod. Anonymous and unexpected source IP detections, together with cloud credential monitoring, cover this path.
- No join to the cloud identity: alerts that stop at a group such as
system:mastersor an IAM role mapping cannot be acted on quickly. Collect the authenticator or guard logs alongside the audit log.
Practitioner checklist
- Confirm every production cluster exports API server audit events to the SIEM, including managed clusters.
- Keep the audit policy in version control with ordered, narrowly scoped None rules and a Metadata catch-all.
- Log secrets, configmaps and token requests at Metadata only and prove no values are stored.
- Capture request bodies for pods, workload controllers, RBAC objects and admission webhooks.
- Enable EKS authenticator, AKS guard or equivalent identity mapping logs.
- Deploy detections for exec, RBAC escalation, secret enumeration, privileged pods, new CronJobs and anonymous access.
- Baseline every service account and CI deployer identity by verb, resource and source IP.
- Alert when any cluster stops sending audit events.
- Test each detection quarterly with authorised actions in a lab namespace.
Standards and references
- Kubernetes: Auditing
- Kubernetes: kube-apiserver Audit Configuration (v1)
- Kubernetes: Audit Annotations
- AWS: Send control plane logs to CloudWatch Logs (Amazon EKS)
- Google Cloud: GKE audit logging
- Microsoft Learn: Monitoring data reference for Azure Kubernetes Service
- MITRE ATT&CK: Container Administration Command
- NIST: SP 800-53 Rev. 5
Related NHI Mgmt Group resources: Kubernetes Security Guide · Kubernetes Cluster Compromise Playbook · Log Source Coverage Matrix Template · Detection Engineering Guide