Subscribe to the Non-Human & AI Identity Journal

Notifications
Clear all

Kafka governance at scale: what Kong Event Gateway changes


(@nhi-mgmt-group)
Member Moderator
Joined: 1 year ago
Posts: 12212
Topic starter  

TL;DR: Teams can reduce Kafka clusters, topic sprawl, glue code, and ACL complexity by shifting isolation, filtering, protocol mediation, access control, and discovery into Kong Event Gateway, according to Kong. The underlying problem is not Kafka performance alone, but the governance overhead created when event access, identity, and distribution are managed inconsistently across teams.

NHIMG editorial — based on content published by Kong: 5 Steps to Immediately Reduce Kafka Cost and Complexity

Questions worth separating out

Q: How should teams govern Kafka access across humans and workloads?

A: Teams should treat Kafka as an identity-governed service, not a raw transport.

Q: When does Kafka topic duplication become a security problem?

A: Topic duplication becomes a security problem when it is the default way to express access boundaries.

Q: What do security teams get wrong about centralising Kafka controls?

A: The common mistake is assuming that one gateway automatically improves governance.

Practitioner guidance

  • Map Kafka consumers by identity type Separate human users, service accounts, applications, and external partners before applying gateway policy.
  • Replace duplicate topics with policy-defined access paths Review whether topic duplication exists only to satisfy consumer segmentation.
  • Audit gateway policy semantics and logs Confirm that the gateway records who requested access, what was exposed, and which policy decision was applied.

What's in the full article

Kong's full blog covers the operational detail this post intentionally leaves for the source:

  • Implementation guidance for Virtual Clusters and when logical isolation is preferable to spinning up new Kafka clusters
  • Specific examples of policy-based filtering for reducing topic sprawl across different consumer groups
  • Protocol mediation details for exposing Kafka streams as REST APIs or Server-Sent Events without custom glue code
  • How Kong positions AsyncAPI and the Developer Portal for discovery, reuse, and self-service access

👉 Read Kong's guide to reducing Kafka cost and complexity with Event Gateway →

Kafka governance at scale: what Kong Event Gateway changes?

Explore further

View Full Forum →  |  NHI Foundation Course →



   
Quote
(@mr-nhi)
Member Moderator
Joined: 2 months ago
Posts: 11787
 

Kafka governance is becoming an identity control problem, not just a streaming problem. Once teams rely on shared topics, gateway mediation, and layered access policy, the real question is who or what is authorised to consume which event data. That moves Kafka into the same governance conversation as APIs, service accounts, and internal platforms. Practitioners should stop treating event access as a separate technical silo.

A few things that frame the scale:

  • 92% agree governing AI agents is critical to enterprise security, yet only 44% have implemented any policies to do so, according to AI Agents: The New Attack Surface report.
  • Only 52% of companies can track and audit the data their AI agents access, leaving 48% with a complete blind spot for compliance and breach investigation.

A question worth separating out:

Q: What should IAM and platform teams review before exposing Kafka through a gateway?

A: They should review how identities are represented, how filtering rules are maintained, and whether access reviews include event-stream consumers. They should also verify that the gateway does not flatten workload and human access into the same entitlement model. If it does, least privilege becomes difficult to defend in practice.

👉 Read our full editorial: Kong Event Gateway for Kafka: cutting cost without losing control



   
ReplyQuote
Share: