Kafka event streams and API platforms: what IAM teams need to know

TL;DR: As organisations scale real-time data across teams, external consumers, and cloud environments, Kafka becomes harder to secure, share, and govern, according to Kong. The core issue is not Kafka throughput but control, visibility, and access discipline across event streams.

NHIMG editorial — based on content published by Kong: 7 Signs Your Kafka Environment Needs an API Platform

By the numbers:

• 80% of organisations report their AI agents have already performed actions beyond their intended scope.
• When AWS credentials are exposed publicly, attackers attempt access within an average of 17 minutes.

Questions worth separating out

Q: How should security teams govern Kafka access when multiple teams and partners share event streams?

A: Security teams should place a policy layer in front of Kafka, then govern access by consumer identity, topic scope, and approved use case.

Q: Why do Kafka ACLs become harder to manage as event-driven architectures expand?

A: Kafka ACLs become harder to manage because they were built for a smaller, more stable set of internal clients.

Q: What breaks when external consumers are given direct access to Kafka topics?

A: Direct access breaks the assumption that the broker is only serving trusted internal clients.

Practitioner guidance

• Map every Kafka consumer to an identity owner Inventory internal teams, external partners, and cloud workloads that consume event streams, then assign an accountable owner for each access path and topic set.
• Move authorization to the gateway layer Enforce access through a policy layer that can apply OAuth, JWT, API key, or SCRAM controls centrally, rather than relying only on broker ACLs.
• Separate public exposure from broker exposure Keep Kafka brokers private and expose only the gateway surface to external consumers.

What's in the full article

Kong's full blog post covers the operational detail this post intentionally leaves for the source:

• How Kong Event Gateway applies Virtual Clusters and Virtual Topics to specific Kafka sharing scenarios.
• How policy enforcement works for OAuth, JWT, API keys, and SCRAM authentication at the gateway layer.
• How observability, audit, and access visibility are handled when external consumers join the event stream.
• How to proxy Kafka clusters while keeping brokers private and limiting data exposure by consumer.

👉 Read Kong's analysis of seven signs your Kafka environment needs an API platform →

Kafka event streams and API platforms: what IAM teams need to know?

Explore further

View Full Forum →  |  NHI Foundation Course →