Topic starter
24/06/2026 8:47 pm
TL;DR: AI costs are eroding gross margin for 84% of companies, with 58% reporting a 6 to 15% hit and 26% seeing 16% or more, according to Mavvrik research cited by Kong. The real governance issue is not AI adoption itself but fragmented visibility, attribution, and enforcement across the AI stack.
Editorial analysis by NHI Mgmt Group, based on content published by Kong: “The Hidden AI Fragmentation Tax: AI Innovation Speed and Program Margins”.
By the numbers:
- 84% of companies report more than a 6% hit to gross margin from AI costs.
- 58% see a 6-15% reduction in gross margin from AI costs.
- 26% report erosion of 16% or more from AI costs.
Key questions
Q: How should teams control AI costs when workloads span APIs, LLMs, and MCP servers?
A: They should treat AI connectivity as one governed control surface instead of a set of separate platforms.
Q: Why do fragmented AI deployments create margin risk?
A: Fragmented AI deployments create margin risk because each team can consume premium models, duplicate capabilities, and move data without shared visibility.
Q: What are the signs that AI governance is failing in the enterprise?
A: Common warning signs include rapid growth in AI use without matching policy coverage, sensitive files being copied into personal accounts, and a large share of AI apps carrying high or critical risk.
Practitioner guidance
- Define a single AI cost governance boundary Map every place AI consumption can occur, including LLMs, APIs, MCP servers, gateways, event streams, and service meshes, then assign one governance owner for the full path.
- Meter consumption at the runtime edge Capture token usage, request counts, and downstream API calls where they are generated so that chargeback and enforcement use the same source of truth.
- Tie limits to business owners Attribute AI consumption to teams, products, or customers so that thresholds can be applied to the correct accountable party instead of to the whole platform.
Bottom line: AI fragmentation can turn faster deployment into lower margin when cost visibility, attribution, and enforcement are split across the AI stack.
Explore further
View Full Forum → | NHI Foundation Course → | Our Services → | Read the full analysis →
25/06/2026 5:36 am
AI cost fragmentation is an identity governance problem when access and usage controls are split across runtime layers: the same organisational sprawl that fragments billing also fragments entitlement oversight. When agents, APIs, gateways, and data streams are governed in separate silos, no single team can explain who or what consumed which resource. The practitioner conclusion is that governance must track runtime consumption as a control surface, not just a finance metric.
A few things that frame the scale:
- Organisations maintain an average of 6 distinct secrets manager instances, creating fragmentation that undermines centralised control, according to the State of Secrets in AppSec.
A question worth separating out:
Q: Should organisations prioritise unified visibility or enforcement first for AI governance?
A: They need both, but unified visibility is the first prerequisite because enforcement without attribution tends to push costs around rather than control them. Once usage is measurable across the full AI stack, teams can apply thresholds, ownership, and policy consistently. The right sequence is visibility, then attribution, then enforcement.
👉 Read our full editorial: AI fragmentation tax is turning speed into a margin problem