Grounded context cuts AI agent cost and improves answer consistency

At a glance

What this is: Apiiro argues that agentic systems should query a shared context layer instead of rereading source code for every question, and its proof-of-value showed sharply lower cost, token use, and latency.

Why it matters: For IAM, NHI, and broader security programmes, this matters because agents that depend on repeated discovery are expensive to scale and harder to govern than agents that retrieve from a controlled source of truth.

By the numbers:

• Apiiro says its proof of value cut total cost across 10 questions from $30.69 to $2.89, a reduction of about 91%.
• Apiiro reports that the same test reduced total tokens from 197K to 64K, which is roughly 68% fewer tokens.
• Apiiro says wall-clock time fell from 35.8 minutes to 16.1 minutes, or about 55% faster.

👉 Read Apiiro's analysis of grounded context for AI agents

Context

Agentic AI systems become expensive when they repeatedly rebuild the same understanding of a codebase, policy set, or application environment for each query. In security terms, the problem is not just compute cost. It is whether the system can ground decisions in a stable source of truth that is auditable, current, and reusable across workflows. That is especially relevant where AI touches IAM, secrets, runtime evidence, or compliance questions.

This article is about the economics and reliability of grounded context for AI agents, not about a single tool feature. Apiiro uses a codebase benchmark to show why repeated re-analysis creates unpredictable cost and output variance, then argues that a graph-based memory layer changes the operating model for security, compliance, and development teams. The starting position is typical for organisations moving from pilots to scaled agentic use cases.

Key questions

Q: How should security teams reduce the cost of AI agents that keep rereading the same systems?

A: Use a shared, governed context layer so agents retrieve application and risk knowledge once instead of reconstructing it on every invocation. The goal is not only lower spend. It is more stable answers, less model drift, and a clearer audit trail for security and compliance decisions.

Q: Why do AI agents become harder to govern as they scale across more repositories?

A: Because every additional repository multiplies the amount of context the agent must rediscover, which increases cost, latency, and output variance. At scale, the problem is not intelligence. It is whether the system can provide a consistent, current source of truth that all agents can reuse.

Q: What breaks when agents rebuild context from source code each time?

A: Answer stability breaks first, followed by budget predictability and auditability. If the model takes a different path through the code on each run, it may surface different conclusions for the same question, which makes automation difficult to trust for security or compliance workflows.

Q: How do organisations know whether their AI context layer is working?

A: They should test whether the system consistently retrieves current, governed and policy-approved sources rather than merely relevant ones. Strong signals include fewer stale answers, fewer policy exceptions, and higher agreement between business definitions and model outputs. If the model still guesses when context is missing, the layer is incomplete.

Technical breakdown

Why repeated codebase reconstruction drives agent cost up

When an AI agent answers from raw source, it must rediscover structure, dependencies, trust boundaries, and control gaps each time. That means repeated token consumption, variable reasoning paths, and inconsistent answers when prompts, model versions, or context windows change. In practical terms, the model is spending budget on discovery rather than judgment. For security use cases, that is a poor fit because the same question should resolve to the same grounded answer if the underlying system state has not changed.

Practical implication: reduce repeated discovery by giving agents a persistent context layer instead of letting each invocation rebuild its own view.

How a software graph and risk graph change the retrieval model

A software graph is a structured representation of application components, APIs, packages, data flows, and related assets. A risk graph overlays findings from scanners and control data onto that structure so the agent can retrieve context that is already correlated and deduplicated. The architectural shift is from free-form inference over source code to retrieval from an always-current model of the environment. That matters because the answer is then tied to lineage, not just to the model's current interpretation of a prompt.

Practical implication: use shared graphs or inventories as the system of record for agents that need repeatable, auditable security answers.

Where MCP fits into grounded agent workflows

MCP, or Model Context Protocol, gives agents a standard way to connect to tools and data sources. In this pattern, the protocol is not the control itself. It is the transport layer that lets a coding agent, CI workflow, or security harness ask the same grounded source for facts rather than re-inspecting the environment independently. That makes governance more realistic because teams can centralise what the agent can see, retrieve, and reuse, while keeping the context layer separate from the model.

Practical implication: treat MCP integrations as governed access paths into trusted context, not as a substitute for data quality or access control.

NHI Mgmt Group analysis

Grounded context is becoming a governance control, not just an efficiency tactic. The article's core finding is that agents become economically and operationally unstable when they reconstruct context on every query. That creates a governance problem because consistency, auditability, and cost predictability all degrade at the same time. For security programmes, the important shift is to treat shared context as infrastructure that enables control, not a convenience layer.

Agentic AI exposes a memory problem that traditional IAM does not solve. IAM can tell you who or what may access a source, but it does not by itself ensure the agent uses stable, lineage-aware context once access is granted. When the article describes a graph answer as more deterministic, it is really pointing to the need for governed context as part of the control plane around AI systems. Practitioner conclusion: agent access and agent knowledge both need governance.

Context reuse will become a named design pattern for scaled AI operations. A useful concept here is grounded context reuse: computing application understanding once and serving it repeatedly through a controlled retrieval layer. That pattern matters in security, compliance, and engineering because it lowers cost while also reducing answer drift. Practitioner conclusion: if the same question should produce the same answer, build the environment so the answer comes from a governed source, not from fresh improvisation.

The agentic AI stack is moving toward system memory layers that resemble identity and control registries. The more enterprises scale agents across repos, pipelines, and policy questions, the more they need a durable record of what the agent knows and how it got that knowledge. That does not make the graph an identity system, but it does make the graph a governance object with access, provenance, and change-management implications. Practitioner conclusion: align agent context design with the same discipline used for critical identity and access records.

Security teams should be cautious about equating better answers with better control. Faster and cheaper retrieval can improve adoption, but it also increases the blast radius if the underlying context layer is incomplete or stale. The real question is whether the data fabric reflects current software state, current risk findings, and current permissions. Practitioner conclusion: if the context layer is wrong, the agent will be confidently wrong at scale.

What this signals

AI programmes are moving from prototype prompts to governed operational dependencies, which means the next bottleneck is context quality rather than model capability. Teams that expose applications, policies, and risk evidence through governed retrieval will get more predictable automation than teams that rely on repeated inference from raw source.

Grounded context reuse: this is the design pattern that will separate scalable agent deployments from expensive demos. Where the context layer is current and auditable, agents can be measured and trusted; where it is stale, the organisation simply automates uncertainty.

The governance signal for security leaders is simple: if an agent can answer consistently, the environment that feeds it must be controlled with the same seriousness as any other production dependency. That aligns directly with NIST AI Risk Management Framework expectations around governance, measurement, and ongoing management.

For practitioners

• Inventory the questions agents answer repeatedly Map the security, architecture, and compliance questions that are asked across repos, pipelines, and review workflows. Prioritise the ones that consume the most tokens or produce inconsistent answers, then decide which should come from a governed context layer instead of repeated source-code inspection.
• Separate retrieval from reasoning Design agent workflows so the expensive part is once-only context creation, while the model focuses on interpretation and next-step decisions. This reduces duplication, makes outputs more stable, and lets teams measure whether the source of truth is actually current.
• Govern MCP access as a data path Treat Model Context Protocol integrations as privileged connections into authoritative systems, not as informal shortcuts. Apply access control, logging, and change management to the context source, especially where agents can query secrets, APIs, or compliance evidence.
• Measure variance, not only token cost Track whether the same query returns materially different answers across runs, models, or prompt order. Low unit cost is useful, but stable output and lineage are what let security, audit, and compliance teams trust the automation.

Key takeaways

• AI agents become expensive and unreliable when they rebuild the same context on every query.
• Graph-based retrieval improves consistency, auditability, and cost predictability at the same time.
• Security teams should govern the context layer as part of the control plane, not as a convenience feature.

Key terms

• Grounded Context: Grounded context is the trusted set of application, risk, or policy facts an AI system uses to answer questions. It is built from authoritative sources and kept current so outputs are repeatable, auditable, and less dependent on the model improvising from raw input.
• Software Graph: A software graph is a structured model of code, APIs, packages, data flows, and related components. It gives agents and analysts a reusable view of the application landscape so they do not have to reconstruct architecture from scratch on each query.
• Risk Graph: A risk graph overlays security findings, scanner output, and control data onto a software model. It helps teams connect vulnerabilities, sensitive data, and exposure patterns to the systems they affect, which makes retrieval and decision-making more consistent.
• Model Context Protocol: Model Context Protocol is an open protocol that lets AI agents connect to tools and data sources. It expands what an agent can reach, so governance has to cover not only the model and its prompts, but also every system that can receive or return agent-driven data.

What's in the full article

Apiiro's full article covers the operational detail this post intentionally leaves for the source:

• The full benchmark table showing cost, token, and runtime deltas across the 10-question proof of value.
• The description of how the Software Graph and Risk Graph are constructed and kept current across commits.
• The MCP-based retrieval pattern used to let agents query precomputed context instead of rereading source code.
• The example security and architecture questions that were used to test the system across repositories.

👉 Apiiro's full post covers the benchmark setup, graph architecture, and token economics in more detail.

Deepen your knowledge

The NHI Foundation Level course, the industry's only accredited NHI security programme, covers NHI governance, machine identity security, and secrets management. It helps practitioners connect identity control to the broader security and automation programmes they operate.