TL;DR: AI cost-optimization agents work best when model reasoning is confined to the parts of the workflow that need judgment, according to Tetrate. Cost analysis stays more measurable when discovery, assessment and explanation are separated.
Editorial analysis by NHI Mgmt Group, based on content published by Tetrate: “Two Ways to Build a Cost Agent (And Why We Use Both)”.
Key questions
Q: When does an AI agent architecture become too dependent on the LLM?
A: It becomes too dependent when the model is responsible for discovery, prioritisation and explanation in the same loop, especially at scale.
Q: What breaks when an AI agent tries to discover and assess everything at once?
A: Coverage becomes inconsistent, runtime costs increase non-linearly, and the system becomes harder to reproduce or audit.
Q: How should teams split deterministic work from model judgment in AI agents?
A: Teams should push enumerable, repeatable checks into deterministic code and reserve the model for ambiguous cases where context changes the meaning of a result.
Practitioner guidance
- Map discovery work to deterministic checks List every resource type, signal and threshold that your agent is expected to inspect, then move those checks into code or API-driven discovery before invoking the model.
- Limit model involvement to judgment calls Use the model only where context changes the outcome, such as distinguishing expected idle resources from genuine waste or ranking ambiguous findings.
- Separate scan outputs from narrative assessment Emit structured findings first, then ask the model to validate, prioritise and explain them in one bounded pass so that failures do not block discovery.
Bottom line: The article shows that AI agents work best when model reasoning is confined to the part of the workflow that truly needs judgment.
Glossary terms: LLM Agent, Two-Phase Agent Architecture, Deterministic discovery, Context window bloat
Explore further
View Full Forum → | NHI Foundation Course → | Our Services → | Read the full analysis →
LLM orchestration is a governance choice, not just an engineering style: when a model is allowed to drive discovery, prioritisation and interpretation in one loop, the programme inherits variability that is difficult to audit. That is acceptable when the goal is exploration, but it becomes a control weakness when teams need repeatable evidence for operations or assurance. The safer pattern is to make the decision boundary explicit and treat model judgment as a bounded control point, not the whole workflow.
A few things that frame the scale:
- AI-related credential leaks surged 81.5% year-over-year in 2025, with the surrounding AI infrastructure leaking 5x faster than core LLM providers, according to the State of Secrets Sprawl 2026.
A question worth separating out:
Q: What is the difference between LLM-orchestrated and two-phase agents?
A: LLM-orchestrated agents let the model choose actions as it goes, which is useful for open-ended reasoning but expensive and variable. Two-phase agents discover data first with deterministic logic, then use the model for assessment. The difference is where uncertainty lives: in the workflow itself or only in the final interpretation.
👉 Read our full editorial: LLM-orchestrated versus deterministic agent design at scale