LLM-orchestrated versus deterministic agent design at scale

At a glance

What this is: This is a comparison of two AI agent architectures for cloud cost analysis, showing that LLM-orchestrated workflows improve judgment while two-phase pipelines improve scale and consistency.

Why it matters: It matters because practitioners designing AI agents, workflows, and governance controls need to decide which parts of an agent should reason and which should remain deterministic, especially when operational scale, auditability, and repeatability matter.

Context

AI agents create a governance problem when the same system is expected to discover issues, reason about them, and produce decisions in one continuous loop. In cloud operations, that design can hide where the model is actually adding value and where a simpler control path would be more reliable.

The article contrasts emergent LLM control with deterministic discovery, which is a useful lens for agentic AI security because runtime governance depends on separating judgment from mechanical execution. That distinction is typical for teams moving from prototype agents to production systems.

Key questions

Q: How should teams split deterministic work from model judgment in AI agents?

A: Teams should push enumerable, repeatable checks into deterministic code and reserve the model for ambiguous cases where context changes the meaning of a result. That split reduces token cost, improves coverage, and makes the agent easier to test. If the same decision can be expressed as a rule, it usually should not depend on the model.

Q: When does an AI agent architecture become too dependent on the LLM?

A: It becomes too dependent when the model is responsible for discovery, prioritisation and explanation in the same loop, especially at scale. At that point, variability, cost growth and context bloat start to undermine repeatability. A healthy design keeps the model inside a narrow decision boundary and lets code handle the rest.

Q: What breaks when an AI agent tries to discover and assess everything at once?

A: Coverage becomes inconsistent, runtime costs increase non-linearly, and the system becomes harder to reproduce or audit. The model may skip paths it covered before or prioritise different items on each run, which weakens confidence in the output. The fix is architectural separation, not simply more prompting.

Q: What is the difference between LLM-orchestrated and two-phase agents?

A: LLM-orchestrated agents let the model choose actions as it goes, which is useful for open-ended reasoning but expensive and variable. Two-phase agents discover data first with deterministic logic, then use the model for assessment. The difference is where uncertainty lives: in the workflow itself or only in the final interpretation.

Technical breakdown

Why LLM-orchestrated agents become expensive and inconsistent

In an LLM-orchestrated agent, the model carries the running context, chooses the next action, and interprets intermediate results. That makes the architecture flexible, but the cost rises as the conversation grows because every new tool call reprocesses more history. The same design also introduces variability, since the model may prioritise different resources or skip different paths on each run. In cost analysis, that means judgment improves at the expense of repeatability, which is tolerable for exploratory work but weak for large, scheduled assessments. See the Agentic AI Security Guide.

Practical implication: Separate exploratory reasoning from scheduled assessment if you need stable coverage across repeated runs.

How two-phase discovery reduces token load and improves coverage

Two-phase architectures split the workflow into deterministic discovery and model-based assessment. The first phase enumerates resources with code, threads, or API calls and produces structured findings without natural-language reasoning. The second phase asks the model to rank, validate, and explain those findings in one pass. That reduces token use, makes the process easier to test, and limits failure modes because the scan still succeeds even if the model call fails. This pattern works best when the resource set and detection criteria are enumerable.

Practical implication: Move known checks into deterministic discovery when the environment is large and the rules are stable.

Where the model still matters in cloud optimisation

Deterministic checks can tell you that a resource is idle or overprovisioned, but they cannot always decide whether the resource is genuinely wasteful in context. That is where the model helps: it can interpret account purpose, traffic shape, and operational exceptions, then separate expected behaviour from a real finding. The point is not that the model is better at everything. The point is that judgment work and discovery work are different control problems, and the architecture should reflect that split.

Practical implication: Reserve model judgment for ambiguous cases where context changes the meaning of a finding.

Breaches seen in the wild

• AI LLM hijack breach: attackers used stolen AWS access keys to hijack Anthropic LLM models on Bedrock.
• Moltbook AI agent keys breach: Moltbook breach exposed 1.5M AI agent keys.

Read and download The State of NHI & AI Agent Breach Report 2026, covering 200+ breaches impacting Non-Human Identities including AI Agents.

NHI Mgmt Group analysis

LLM orchestration is a governance choice, not just an engineering style: when a model is allowed to drive discovery, prioritisation and interpretation in one loop, the programme inherits variability that is difficult to audit. That is acceptable when the goal is exploration, but it becomes a control weakness when teams need repeatable evidence for operations or assurance. The safer pattern is to make the decision boundary explicit and treat model judgment as a bounded control point, not the whole workflow.

Task enumeration is the real dividing line in agent design: the article is most useful where it shows that many agent tasks are not truly emergent. If a team can define what to check, what counts as idle, and what data sources matter, the agent should not rely on open-ended reasoning to discover those items every time. That reduces hidden dependency on the model and makes the operating model easier to defend.

Context-aware reasoning still has a place in cloud operations: cloud cost analysis is not a pure rules problem because account purpose, workload role and traffic pattern change the meaning of a signal. That is why deterministic discovery and LLM judgment should coexist rather than compete. For identity and access teams, the lesson is familiar: automate the enumerable, but keep human or model judgment where context genuinely changes outcome.

AI governance debt grows when teams cannot explain which part of an agent is deterministic: this article exposes a common failure mode in operational AI programmes, where model involvement is treated as a blanket capability rather than a scoped function. As the system scales, that ambiguity becomes harder to govern, harder to test and harder to attribute. Teams should treat architectural clarity as part of control design, not as an implementation detail.

Runtime supervision must match the agent architecture, not the marketing label: an agent that calls tools, reasons over cloud billing data and emits findings needs different controls from a chatbot with retrieval. Runtime policy, audit logging, cost attribution and fallback logic all matter because the model is participating in a workflow, not merely answering questions. The practical conclusion is that governance must be designed around the actual execution model, not the term 'agent'.

From our research library:

• AI-related credential leaks surged 81.5% year-over-year in 2025, with the surrounding AI infrastructure leaking 5x faster than core LLM providers, according to the State of Secrets Sprawl 2026.
• Read next: AI Agent Identity Security Buyer's Guide

What this signals

Agent architecture debt: the more work a model is asked to do inside one continuous loop, the harder it becomes to prove what was deterministic and what was inferred. That debt shows up first as cost, then as inconsistency, and finally as governance friction when teams need to explain decisions to operations or audit stakeholders.

For programmes building AI agents across cloud, data and identity workflows, the next control question is not whether to use a model, but where to remove it. That design choice will increasingly separate prototypes that merely function from production systems that can be governed, attributed and repeated.

For practitioners

• Map discovery work to deterministic checks List every resource type, signal and threshold that your agent is expected to inspect, then move those checks into code or API-driven discovery before invoking the model. See the AI Agent Identity Security Buyer's Guide.
• Limit model involvement to judgment calls Use the model only where context changes the outcome, such as distinguishing expected idle resources from genuine waste or ranking ambiguous findings.
• Separate scan outputs from narrative assessment Emit structured findings first, then ask the model to validate, prioritise and explain them in one bounded pass so that failures do not block discovery.
• Measure how often the model changes the decision Track which findings are confirmed, rejected or reclassified by the model so you can prove where LLM judgment adds value and where it does not.

Key takeaways

• The article shows that AI agents work best when model reasoning is confined to the part of the workflow that truly needs judgment.
• Deterministic discovery improves repeatability, lowers cost and makes large-scale scanning more predictable than a fully LLM-driven loop.
• The main operational lesson is architectural clarity: separate enumeration, assessment and explanation so the agent can be governed and measured.

Related NHI Mgmt Group resources: Agentic AI Security Guide · AI Agent Authorisation Guide · AI Agent Identity Security Buyer's Guide · AI Agent Observability, Audit and Incident Response Guide

Key terms

• LLM Agent: An LLM agent is a language model system that can plan, choose tools, remember context, and carry out tasks with limited human intervention. It goes beyond text generation by connecting reasoning to actions, which makes governance, access control, and monitoring much more important than in a basic chatbot.
• Two-Phase Agent Architecture: A design that separates deterministic discovery from model-based assessment. Code or APIs gather structured findings first, then the model interprets, ranks or explains them in a bounded second pass. This pattern reduces token use and improves auditability when the discovery work is enumerable.
• Deterministic discovery: Deterministic discovery is a method of identifying AI assets from explicit code and configuration signals such as imports, manifests, file paths, and settings. It is preferred for governance because it creates repeatable, auditable evidence rather than relying on probabilistic inference.
• Context window bloat: The accumulation of intermediate data, prompts, and outputs that an AI model must carry while completing a task. As bloat grows, reliability falls and the agent becomes harder to govern because more of the work happens in transient model memory rather than in the source system.

Deepen your knowledge

NHI Foundation Level course, the industry's only accredited NHI security programme, covers NHI governance, agentic AI identity and machine identity security. It helps practitioners connect identity controls to the operational realities of AI-enabled systems.