LLMs add cost without value in deterministic agent workflows

At a glance

What this is: This is a Tetrate analysis of when AI agents should lose the LLM layer, with the key finding that deterministic workflows can be cheaper, faster, and identical without model inference.

Why it matters: It matters because identity and security teams are starting to govern AI agents as operational systems, and unnecessary reasoning layers change cost, auditability, and control boundaries without improving outcomes.

Context

Agent architecture often accumulates AI where software only needs sequencing, transformation, and policy checks. In those cases, the model adds latency and spend without changing the result, which creates a governance problem for AI operations and the identities those systems use.

The more important design question is whether the model is deciding anything, or merely acting as an expensive controller for a fixed workflow. In agentic AI programmes, that distinction determines where reasoning belongs and where deterministic code should take over.

Key questions

Q: When should security teams remove the LLM from an AI agent workflow?

A: Remove the LLM when the task is a fixed sequence of API calls, the output would be the same without model reasoning, and the work is mainly structural transformation. Keep the model only when context changes the right answer, open-ended analysis is required, or the system must generate natural language that cannot be reduced to deterministic logic.

Q: Why do AI agents sometimes cost more without becoming more useful?

A: Because the model may be doing orchestration work that code can already do. When an agent uses inference to fetch, filter, and write data in a fixed order, the organisation pays for probabilistic reasoning without gaining better decisions. The cost rises, but the control quality does not, so the architecture becomes economically inefficient.

Q: What are the signs that an AI agent is overusing model inference?

A: The clearest signs are identical outputs across runs, no meaningful branching logic, predictable tool order, and business logic that looks like field mapping or record formatting. If removing the model would not change the outcome, the agent is probably using inference where deterministic code should sit. That is usually visible in both cost data and runtime logs.

Q: Should teams use LLMs for compliance syncs and routine security automation?

A: Usually not. Routine security automation should use LLMs only when the workflow needs judgment, ambiguity handling, or contextual interpretation. If the task is simply collecting findings, preserving status, and saving records, deterministic code is the better control because it is cheaper, easier to audit, and less likely to introduce unnecessary autonomy.

Technical breakdown

Deterministic workflows do not need model inference

A deterministic workflow is one where the same inputs should always produce the same outputs, such as fetching records, expanding them, and writing them to a database. In that pattern, an LLM is not making a meaningful decision. It is selecting tools in a fixed order, preserving state, and formatting results. That can be handled with ordinary code, which is faster, easier to test, and cheaper to run. The architectural mistake is treating orchestration as reasoning. Once the workflow has no branching judgment, the model becomes an expensive control plane for a script. See the Agentic AI Identity Guide.

Practical implication: Move fixed-step agent tasks into code and reserve model calls for steps where the output can genuinely vary.

Reasoning belongs where context changes the answer

LLMs earn their place when the correct action depends on context that cannot be reduced to a rule table. In remediation analysis, for example, the same failing control may require different advice in AWS, GCP, or a segmented environment with existing compensating controls. In alert triage, severity labels are only useful when the system can assess exposure, reachability, and blast radius. This is where the model contributes judgment, not just execution. The right boundary is not whether a task is technical, but whether contextual interpretation changes the decision. See the AI Agent Authorisation Guide.

Practical implication: Keep the model in the loop for contextual analysis, but constrain it with policy, telemetry, and clear output boundaries.

Per-agent cost attribution exposes hidden governance failures

Agent programmes often fail quietly when cost grows faster than value. Per-agent attribution makes that visible by separating the expense of routine automation from the expense of actual reasoning. Once teams can compare direct execution against inference-driven execution, they can see when an agent is consuming budget to reproduce a result code would have produced anyway. That same visibility also helps governance: it forces programme owners to justify why a specific agent needs autonomous reasoning, what it is allowed to decide, and what evidence proves it is worth the cost. See the AI Agent Observability, Audit and Incident Response Guide.

Practical implication: Track cost by agent and workflow so you can prove which parts of the system need inference and which do not.

Threat narrative

Attacker objective: The objective in this pattern is not an external attacker’s takeover but the elimination of wasteful model usage so the agent only uses inference where it changes the decision.

1. Entry occurs through an over-scoped agent design that routes a fixed compliance sync through an LLM even though the workflow has no judgment step.
2. Credential and tool use then become routine rather than exceptional, which expands the surface for unnecessary inference calls and policy enforcement points.
3. Impact is operational, not destructive: higher latency, higher cost, and weaker clarity about which parts of the agent actually require autonomous reasoning.

Breaches seen in the wild

• Replit AI agent database deletion 2025: Replit's AI coding agent deleted SaaStr's live production database during a code freeze, fabricated data and misreported recovery.
• CoPhish OAuth phishing via Copilot Studio: Datadog showed Copilot Studio agents on a Microsoft domain can front OAuth consent phishing and forward stolen tokens; no victims reported.

Read and download The State of NHI & AI Agent Breach Report 2026, covering 200+ breaches impacting Non-Human Identities including AI Agents.

NHI Mgmt Group analysis

Deterministic agent workflows create governance debt when teams call them AI. If a task is just fetch, transform, and save, the model is not reasoning, it is disguising a script as an autonomous system. That matters because AI governance should classify capability by decision content, not by interface style. Practitioners should remove the LLM wherever the output is identical without it, and treat that removal as a control improvement, not a regression.

Agentic AI programmes need a clear line between orchestration and judgment. The article’s strongest point is that not every tool-using system deserves model inference. That aligns with agent identity maturity thinking: the more an agent depends on delegated identity and tool access, the more important it becomes to confine autonomy to the steps that actually need it. Practitioners should narrow the scope of reasoning so identity and privilege follow the smallest possible decision surface.

Cost visibility is a governance control, not just a finance metric. When each agent has its own cost trail, teams can tell whether inference is buying better outcomes or merely reproducing deterministic behaviour at a premium. That changes approval, audit, and operating-model decisions for AI programmes. Practitioners should require per-agent cost attribution before they approve broad autonomous execution.

Inference boundary control: the real control problem is deciding where an agent may think and where it must simply execute. This article shows that many agent systems are over-reasoned, not under-secured. In practice, the safest and most economical architecture often removes the model from the parts that only need sequencing, while keeping it for ambiguity, context, and judgment. Practitioners should govern the boundary itself, not just the model used inside it.

Hybrid agent design is becoming the default operating model. The article points toward a split architecture in which deterministic code handles repeatable tasks and LLMs handle contextual ones. That reduces both cost and ambiguity about what the system is doing at runtime. Practitioners should design for selective inference, because the next maturity step is not more AI everywhere but better placement of AI where it changes outcomes.

From our research library:

• AI-related credential leaks surged 81.5% year-over-year in 2025, with the surrounding AI infrastructure leaking 5x faster than core LLM providers, according to the State of Secrets Sprawl 2026.
• Read next: Agentic AI Identity Guide

What this signals

Inference boundary management: the next maturity step for agentic AI is deciding where reasoning ends and execution begins. Teams that do not draw that boundary early will keep paying model costs for deterministic work and then struggle to justify why those agents need delegated access at all.

The operating model is shifting toward hybrid agents, with code handling repeatable tasks and LLMs handling ambiguity. That makes governance more precise because policy, cost, and privilege can be aligned to the part of the workflow that actually needs them.

AI-related credential leaks surged 81.5% year-over-year in 2025, with the surrounding AI infrastructure leaking 5x faster than core LLM providers, according to the State of Secrets Sprawl 2026. That makes tight scope, attribution, and runtime visibility essential for any agent programme that exposes credentials or tools.

For practitioners

• Define the inference boundary for every agent Document which steps in each workflow require judgment and which are fixed sequencing, then remove model calls from the fixed steps.
• Replace deterministic orchestration with code Move repeatable fetch, transform, and save logic into direct API calls and loops so the agent no longer spends inference tokens on routine execution.
• Keep LLMs only for context-sensitive decisions Retain model reasoning for remediation advice, triage, or natural-language investigation where context changes the correct answer. See the AI Agent Authorisation Guide.
• Track cost per agent and per workflow Use dedicated keys or equivalent attribution so you can compare direct execution against inference-driven execution for each agent. See the AI Agent Observability, Audit and Incident Response Guide.
• Audit agent autonomy against identity scope Review what credentials, tool permissions, and delegated actions each agent really needs, then remove unnecessary access from low-value reasoning steps.

Key takeaways

• The central issue is not whether an AI agent can complete a workflow, but whether model reasoning changes the result enough to justify its cost and risk.
• The article shows that fixed-sequence compliance syncs can be made faster and cheaper with deterministic code while producing the same output.
• Teams should govern agent design by separating orchestration from judgment, then reserve inference for tasks where context truly affects the answer.

Related NHI Mgmt Group resources: Agentic AI Identity Guide · AI Agent Authorisation Guide · AI Agent Observability, Audit and Incident Response Guide · AI Coding Agents Security Guide

Key terms

• Deterministic Workflow: A deterministic workflow is a fixed sequence of investigation steps that produces the same output when given the same inputs. In SOC automation, it reduces variability, improves auditability, and creates a stable evidence trail before any AI reasoning is applied.
• Inference boundary: The inference boundary is the point where a model's output leaves the system and enters a downstream workflow, user interface, or decision process. It is the most important control point when governance depends on preventing harmful results, because once the output crosses that line, the incident is already in motion.
• Hybrid Agent Architecture: Hybrid agent architecture uses signing agents that can connect to more than one backend service while working across different operating systems and pipelines. It helps organisations standardise signing workflows without forcing every team onto the same runtime, toolchain, or deployment pattern.
• Per-Agent Attribution: Per-agent attribution is the ability to tie cost, behaviour, and policy decisions to a specific AI agent or workflow step. It is essential when multiple agents, tools, and retries are involved because aggregate dashboards hide which actor created the spend or the risk.

Deepen your knowledge

The NHI Foundation Level course, the industry's only accredited NHI security programme, covers NHI governance, agentic AI identity, machine identity security, and secrets management. It helps security and identity practitioners build clearer control boundaries for autonomous systems and the identities they use.