Kafka Credentials Security Guide: SASL, mTLS and ACLs for Producers and Consumers

Every producer, consumer, connector and stream processor that talks to Apache Kafka is a non-human identity, and a freshly installed cluster asks none of them to prove who they are. The Apache Kafka security model states that security is off by default: listeners accept unauthenticated PLAINTEXT connections and no authorisation is applied. This guide covers how Kafka authenticates clients (SASL mechanisms, mutual TLS and delegation tokens), how principals map to ACLs, how to store and rotate client credentials, and what to log and measure. It is written for platform engineers, streaming teams and the identity and security staff who govern the service accounts behind event pipelines, and reflects the KRaft-based Kafka documentation. For cluster-wide hardening beyond credentials, see the Kafka Security Guide.

Key takeaways

• Authentication without an authoriser is identity with no limits. Kafka's documentation is explicit that if no authoriser is configured, all authenticated principals have full access. Configure StandardAuthorizer alongside SASL or mTLS.
• Pick the mechanism by credential lifecycle, not convenience. SCRAM stores salted verifiers in cluster metadata, mTLS rides on your PKI, OAUTHBEARER hands credential issuance to an identity provider, and PLAIN is acceptable only inside TLS.
• The unsecured OAUTHBEARER default is for testing only. Out of the box it creates and accepts unsigned JWTs; production needs a token endpoint, JWKS validation and the org.apache.kafka.sasl.oauthbearer.allowed.urls allowlist.
• Super users and wildcard ACLs are the privileged accounts of Kafka. Principals in super.users bypass ACL checks entirely, and allow.everyone.if.no.acl.found=true opens every resource without ACLs to all principals.
• Kafka Connect concentrates secrets. A worker authenticates as one principal for all its connectors, and inline connector secrets are written to the config topic; use ConfigProvider references and lock down that topic.

How identity and access work in Kafka

Each broker listener is mapped to one of four security protocols through listener.security.protocol.map. Authentication produces a KafkaPrincipal (for example User:orders-producer), and the authoriser configured in authorizer.class.name decides what that principal may do.

Authentication mechanisms compared

Choosing a mechanism

• Choose mTLS when you already issue workload certificates automatically (a service mesh or internal CA with short-lived certificates) and want no shared secrets at all. Write ssl.principal.mapping.rules such as RULE:^CN=(.*?),OU=ServiceUsers.*$/$1/ so ACLs reference short, stable names rather than full DNs. See the mTLS Guide.
• Choose OAUTHBEARER when an enterprise identity provider already manages your workload identities and you want short-lived tokens and central revocation. Prefer client assertion with a locally held private key over a client secret: Kafka's documentation notes the private key never leaves the client and assertions are short-lived.
• Choose SCRAM for smaller estates or managed environments without an IdP integration. It is simple and keeps passwords off the wire, but it is a static shared secret per client that you must rotate yourself.
• Choose Kerberos only where a KDC is already part of the platform, typically Hadoop-adjacent estates. See the Kerberos Security Guide.
• Avoid PLAIN with the default JAAS file in production. If you must use it, run it only on SASL_SSL with a callback handler backed by an external password store.

Admin roles and privileged access

Kafka has no built-in role model; privilege comes from ACLs on the Cluster resource, the super.users list and access to the hosts themselves.

• super.users: a semicolon-separated list (for example super.users=User:Bob;User:Alice) of principals that bypass all ACL checks. The Kafka security model advises treating it like a root password. Keep it to the inter-broker and controller identities plus a break-glass administrator held in your PAM tool. See the Privileged Access Management Guide.
• Cluster-level operations: Alter, AlterConfigs and ClusterAction on the Cluster resource let a principal change ACLs and broker configuration. ACL management through kafka-acls.sh or AdminClient createAcls is itself gated by Cluster ACLs, so grant these to an automation identity, not to application principals.
• Delegation token powers: CreateTokens and DescribeTokens on a User resource allow one principal to create or describe tokens for another. Grant them only to the orchestration framework that needs them.
• Host and disk access: the security model states that anyone with shell access to a broker, controller or its disks can read every topic, forge any principal and rewrite ACLs. Broker SSH access is therefore equivalent to super-user rights and belongs under privileged access controls.
• Separate the planes: producer and consumer traffic, the Admin API and JMX each need their own authentication. Securing client listeners does not secure JMX, and the controller listener must never be reachable from client networks.

Integrations, API tokens and service accounts

Treat each application, connector and stream processor as its own principal. Shared principals make ACLs coarse, make rotation a coordinated outage and destroy attribution in audit logs.

Least-privilege ACLs for producers and consumers

The --producer shortcut includes Create on the topic; remove it where topics are provisioned by a pipeline, so applications cannot create arbitrary topics. Avoid --topic '*' grants for applications, and use --resource-pattern-type match when reviewing access because a plain --list on a topic does not show wildcard or prefixed ACLs that also apply.

Storing client credentials

• Keep secrets out of properties files: Kafka supports ConfigProvider indirection, with FileConfigProvider, DirectoryConfigProvider and EnvVarConfigProvider built in and custom providers for external stores. Set allowed.paths on the file-based providers; if unset, all paths are allowed. See the Secrets Management Guide.
• Connect config topic: a secret placed directly in a connector configuration is written to config.storage.topic as an ordinary Kafka record. A ConfigProvider reference keeps the value out of the topic, but providers are resolved through the shared Connect REST API, so that API needs authentication too.
• Connect identity: by default all connectors on a worker share the worker's principal and ACLs. Restrict connector.client.config.override.policy so connectors cannot substitute their own credentials, or run sensitive connectors on dedicated workers.
• Keystores: use distinct keystores for the inter-broker listener and client-facing listeners, so a leaked client-facing key cannot impersonate a broker, and keep keystore passwords in the same secret store as SASL credentials.
• Delegation tokens: pass them to workers through the job framework's secret mechanism, not command-line arguments. Tokens cannot be used to request further tokens, which limits chaining.

Rotating Kafka credentials

Automate this through the Secrets Rotation Runbook Template and track certificate lifetimes with the practices in the Machine Identity, PKI and Certificate Lifecycle Guide.

Logging and detection

• Authoriser log: Kafka emits authorisation decisions to the kafka.authorizer.logger log4j logger. INFO records every denied request and DEBUG also records allowed ones. Ship it off-broker to durable, append-only storage: Kafka has no built-in tamper-evident audit trail.
• Request log: kafka.request.logger gives per-call detail for forensics but is verbose and off by default; enable it temporarily during investigations.
• Authorisation error codes: clients receive TOPIC_AUTHORIZATION_FAILED (29), GROUP_AUTHORIZATION_FAILED (30), CLUSTER_AUTHORIZATION_FAILED (31) and TRANSACTIONAL_ID_AUTHORIZATION_FAILED (53). Spikes of these from one principal suggest a misconfigured client or a stolen credential being used to explore.
• What to alert on: denied requests from a principal against topics it has never touched, any use of a super-user principal outside change windows, new ACLs on * or broad prefixes, SCRAM credential changes, delegation token creation for other owners, and authentication failures from unexpected hosts.
• Credential leaks: scan repositories and container images for sasl.jaas.config lines with inline passwords. See the Git Secret Scanning Guide.

Metrics

• Listener coverage: percentage of listeners using SASL_SSL or SSL with client authentication (target 100% outside test).
• Principal uniqueness: ratio of client applications to distinct principals (target 1:1).
• Wildcard exposure: number of ACLs on * held by non-administrative principals (target zero).
• Credential age: days since rotation for SCRAM users and OAuth client secrets, and days to expiry for client certificates.
• Super-user count: principals in super.users, reviewed quarterly.

Hardening baseline

1. Inventory: list every client application, its principal, mechanism, topics and owner. Capture these in the Service Account Inventory Template.
2. Add an authenticated listener alongside the existing one: introduce SASL_SSL on a new port, keep inter-broker and controller traffic on a dedicated internal listener protected by mTLS, and move clients across one by one. Rollback is pointing a client back to the old listener.
3. Enable the authoriser in transition mode: configure StandardAuthorizer with allow.everyone.if.no.acl.found=true only while you write ACLs, using the authoriser log to find which principals need which resources. The security model calls this a transitional aid that should not remain set.
4. Write least-privilege ACLs per principal using the table above, then remove allow.everyone.if.no.acl.found so default-deny applies. Test in a pilot environment first and keep the flag change as the rollback lever.
5. Close plaintext: remove PLAINTEXT and SASL_PLAINTEXT listeners, disable TLS below 1.2, and keep ssl.endpoint.identification.algorithm=https on clients (the default since Kafka 2.0).
6. Externalise secrets with ConfigProvider and secure the Connect REST API and config topic.
7. Reduce super users and put the break-glass identity under PAM.
8. Turn on audit shipping for kafka.authorizer.logger and set up the alerts above.

Small vs large estates: a single cluster with a few applications can run SCRAM over SASL_SSL with credentials in a secrets manager and quarterly rotation. Large or multi-tenant estates benefit from OAUTHBEARER or mTLS with automated issuance, prefixed ACLs per team namespace, ACLs managed as code through a pipeline identity, and dedicated Connect workers per trust boundary.

Framework mapping

In ISO/IEC 27001:2022 Annex A the closest controls are 5.15 Access control, 5.17 Authentication information, 8.2 Privileged access rights and 8.5 Secure authentication.

Common misconfigurations

• SASL without an authoriser: teams enable authentication, see clients succeed and assume access is controlled, while every principal still has full access.
• allow.everyone.if.no.acl.found left on: any new topic without ACLs is open to every authenticated principal, which defeats the default-deny model as soon as someone creates a topic.
• Unsecured OAUTHBEARER in production: the default handler accepts unsigned JWTs, so any client can claim any sub. Kafka's documentation also warns that a mistyped JAAS option starting with unsecuredLogin makes the handler create an unsecured token.
• One shared principal for many applications: a single SCRAM user or certificate reused across services, so one leak exposes all their topics and rotation needs a coordinated outage.
• Secrets inline in connector configs and JAAS files: passwords in sasl.jaas.config committed to repositories or written into the Connect config topic. Credentials kept in repositories and documents are a known failure mode: in the Red Hat Consulting GitLab Breach 2025, the Belgian CCB reported that stolen engagement reports contained authentication tokens and keys for customer systems.
• Client-facing listener shares broker keys: one keystore used for clients and inter-broker traffic lets a leaked client-facing key impersonate a broker.

Practitioner checklist

• Remove PLAINTEXT and SASL_PLAINTEXT listeners from every environment reachable by untrusted networks.
• Configure StandardAuthorizer and confirm allow.everyone.if.no.acl.found is not set in steady state.
• Give every application, connector and stream processor its own principal and least-privilege ACLs.
• Use mTLS for inter-broker and controller traffic on a dedicated internal listener with its own keystore.
• Replace the unsecured OAUTHBEARER handler with IdP-issued tokens, JWKS validation and an allowed URL list.
• Move SASL passwords, keystore passwords and connector secrets behind ConfigProvider references with allowed.paths set.
• Keep super.users to broker identities and a PAM-managed break-glass account, reviewed quarterly.
• Rotate SCRAM credentials, OAuth secrets and certificates on a schedule with an overlap plan.
• Ship kafka.authorizer.logger off-broker and alert on denied-request spikes and ACL changes.

Standards and references

• Apache Kafka: Security documentation
• Apache Kafka: Authentication using SASL
• Apache Kafka: Authorization and ACLs
• Apache Kafka: Security Model
• IETF: RFC 5802 Salted Challenge Response Authentication Mechanism (SCRAM)
• IETF: RFC 7523 JWT Profile for OAuth 2.0 Client Authentication and Authorization Grants
• NIST: SP 800-53 Rev. 5
• NIST: Cybersecurity Framework 2.0

Related NHI Mgmt Group resources: Kafka Security Guide · Service Account Security Guide · Secrets Management Guide · mTLS Guide · Webhook Secrets Security Guide