AI fragmentation tax is turning speed into a margin problem

At a glance

What this is: This is Kong's analysis of how fragmented AI infrastructure creates hidden cost and governance overhead, with most companies already reporting gross margin erosion from AI costs.

Why it matters: For IAM, NHI, and platform teams, the lesson is that visibility and enforcement must span the full AI connectivity stack or cost controls will remain partial and easy to bypass.

By the numbers:

• 84% of companies report more than a 6% hit to gross margin from AI costs.
• 58% see a 6-15% reduction in gross margin from AI costs.
• 26% report erosion of 16% or more from AI costs.
• Only 15% of companies can forecast AI costs within ±10% accuracy.

Context

AI fragmentation tax is the hidden cost created when AI workloads, APIs, LLMs, event streams, and control points are spread across disconnected environments without shared visibility or enforcement. In practice, that means organisations can scale AI usage faster than they can measure, attribute, or govern its cost.

Kong's article argues that the governance problem is not just technical sprawl but financial control failure across the AI connectivity layer. The article ties that fragmentation to margin erosion, unpredictable forecasting, and limited ability to enforce consumption limits before costs hit COGS.

The identity security relevance is straightforward: when AI applications span multiple gateways, servers, and runtimes, access, usage, and entitlement boundaries become harder to observe and harder to govern consistently.

Key questions

Q: How should teams control AI costs when workloads span APIs, LLMs, and MCP servers?

A: They should treat AI connectivity as one governed control surface instead of a set of separate platforms. The practical goal is to connect metering, attribution, and enforcement so a request can be traced from the runtime that generated it to the team or product that owns it. Without that end-to-end view, cost controls remain partial and reactive.

Q: Why do fragmented AI deployments create margin risk?

A: Fragmented AI deployments create margin risk because each team can consume premium models, duplicate capabilities, and move data without shared visibility. The result is hidden spend that accumulates across many small decisions. Once the quarter closes, the organisation discovers that AI growth has outpaced its ability to explain or recover the cost.

Q: What are the signs that AI governance is failing in the enterprise?

A: Common warning signs include rapid growth in AI use without matching policy coverage, sensitive files being copied into personal accounts, and a large share of AI apps carrying high or critical risk. Another indicator is weak visibility into who is using which tools and what data they are sending. If teams cannot answer those questions, governance is not working as intended.

Q: Should organisations prioritise unified visibility or enforcement first for AI governance?

A: They need both, but unified visibility is the first prerequisite because enforcement without attribution tends to push costs around rather than control them. Once usage is measurable across the full AI stack, teams can apply thresholds, ownership, and policy consistently. The right sequence is visibility, then attribution, then enforcement.

Technical breakdown

Why fragmented AI connectivity breaks cost governance

AI connectivity is no longer a single application path. Modern deployments can include developers, agents, MCP servers, LLMs, APIs, event streams, gateways, and service meshes, often split across cloud, on-premises, and hybrid environments. That creates multiple cost and control boundaries, each with its own telemetry and policy model. When those layers are managed separately, organisations lose the ability to correlate usage with business outcome, which is why fragmented reporting and fragmented enforcement become the same problem from a governance perspective.

Practical implication: Practitioners need a single control view that spans AI connectivity and usage, not isolated dashboards for each platform.

Why attribution matters more than raw visibility

Visibility without attribution tells you that spend exists, but not which team, product, customer, or workflow created it. In AI environments, that matters because token usage, network egress, and upstream API calls may be generated indirectly through intermediate services such as MCP servers or gateways. Without attribution, cost governance remains aggregate-only, which is too coarse for chargeback, showback, or enforcement decisions. The article's core point is that cost control must follow the path of consumption, not just the location of the invoice.

Practical implication: Build metering that can attribute AI consumption to the service or business owner closest to the source of usage.

What unified enforcement changes in an AI stack

A unified control plane changes governance from after-the-fact review to real-time limit setting. Instead of waiting for finance to discover overruns, teams can apply usage thresholds to APIs, LLMs, MCP servers, and event-driven workflows as they operate. That matters because AI cost growth is often nonlinear: a small increase in agent activity can multiply downstream API calls and token consumption. The technical lesson is that AI governance must be embedded at the connectivity layer where consumption is actually triggered.

Practical implication: Enforce usage limits at the point where AI requests become billable consumption, not only in monthly reporting.

NHI Mgmt Group analysis

AI cost fragmentation is an identity governance problem when access and usage controls are split across runtime layers: the same organisational sprawl that fragments billing also fragments entitlement oversight. When agents, APIs, gateways, and data streams are governed in separate silos, no single team can explain who or what consumed which resource. The practitioner conclusion is that governance must track runtime consumption as a control surface, not just a finance metric.

Unified visibility is the minimum viable control, but it is not sufficient without enforcement: organisations can see partial AI spend and still fail to stop runaway consumption. That is the same failure pattern identity teams see when discovery exists without policy enforcement, because observation alone does not constrain access or usage. The practical takeaway is to design AI governance so metering and policy act together.

Fragmented AI infrastructure creates an identity blast radius across APIs, LLMs, and MCP servers: the more places a workload can call out to external services, the harder it becomes to define effective ownership and least-privilege boundaries. This is not just an operational inefficiency; it is a governance gap that increases control drift across the stack. Teams should treat AI connectivity sprawl as a boundary-management problem.

The margin problem exposes a new named concept: AI fragmentation tax: this is the cumulative cost premium created when disconnected visibility, attribution, and enforcement allow AI consumption to scale faster than governance. It is not caused by AI itself but by the lack of a coherent control plane across the AI estate. The implication is that programme design must shift from isolated controls to unified operating models.

AI programmes will increasingly be judged on cost governability, not just feature velocity: as AI becomes embedded in revenue-generating products, cost control becomes part of identity and platform governance rather than a finance afterthought. That means practitioners need cross-functional ownership for usage policy, telemetry, and limit enforcement. The conclusion is that sustainable AI delivery now depends on control-plane maturity.

From our research library:

• Organisations maintain an average of 6 distinct secrets manager instances, creating fragmentation that undermines centralised control, according to the State of Secrets in AppSec.
• Read next: AI Agent Identity Security Buyer's Guide

What this signals

AI fragmentation tax changes the governance problem from optimisation to control-plane design: when organisations run AI across hybrid environments, the question is no longer how fast they can deploy agents or models, but whether they can observe and govern what those systems consume. That makes metering, attribution, and enforcement part of the same operating model, not separate workstreams.

Fragmented visibility is the point where cost governance and identity governance intersect: once AI workloads call out to multiple APIs and LLMs, teams need a way to decide who owns the usage and which policy boundary applies. Organisations maintain an average of 6 distinct secrets manager instances, creating fragmentation that undermines centralised control, according to the State of Secrets in AppSec. The same pattern appears in AI connectivity when control points are spread across too many layers.

For practitioners

• Define a single AI cost governance boundary Map every place AI consumption can occur, including LLMs, APIs, MCP servers, gateways, event streams, and service meshes, then assign one governance owner for the full path.
• Meter consumption at the runtime edge Capture token usage, request counts, and downstream API calls where they are generated so that chargeback and enforcement use the same source of truth.
• Tie limits to business owners Attribute AI consumption to teams, products, or customers so that thresholds can be applied to the correct accountable party instead of to the whole platform.
• Unify reporting across hybrid environments Include on-premises components, public cloud services, and third-party LLMs in one reporting model so forecast variance and leakage are visible in the same view.
• Separate telemetry from enforcement Use the same metering data for both cost reporting and policy enforcement so overruns can be stopped before they appear in monthly financial review.

Key takeaways

• AI fragmentation can turn faster deployment into lower margin when cost visibility, attribution, and enforcement are split across the AI stack.
• The article's evidence shows that many organisations still lack reliable forecasting and unified reporting across hybrid AI environments.
• The operational response is to govern AI consumption through a single control plane that meters usage, assigns ownership, and enforces limits in real time.

Key terms

• AI Fragmentation Tax: AI fragmentation tax is the hidden cost created when model usage spreads across disconnected tools, providers, and environments without unified visibility. It is not a formal accounting line, but a useful term for the spend inefficiency and governance loss that accumulate when AI consumption cannot be managed as one system.
• Usage Attribution: Usage attribution is the process of linking AI consumption to a known owner such as a team, tenant, product, or application. In governance terms, it turns anonymous traffic into accountable activity and gives both finance and security a shared basis for action.
• Unified Control Plane: A unified control plane is an identity architecture where discovery, access governance, audit, and response operate across humans, machines, and AI agents together. It reduces blind spots caused by siloed tooling and gives security teams context for decisions about permissions, data, and containment.
• Request-cost Governance: The practice of setting policy around how expensive a request may be, based on endpoint semantics, parameter ranges, and expected business use. It shifts control design from raw traffic limits to a more precise model of safe application behaviour.

Deepen your knowledge

NHI governance, agentic AI identity, and machine identity lifecycle are core topics in our NHI Foundation Level course, the industry's only accredited NHI security programme. If you are building or maturing an IAM programme, it is worth exploring.