LLM-driven or two-phase agents: what changes at scale?

TL;DR: AI cost-optimization agents can be built either as LLM-orchestrated systems or as two-phase pipelines, and the trade-off is between contextual judgment and scalable, reproducible discovery, according to Tetrate. The governance lesson is that teams should isolate where the model adds real decision value and remove it from work that can be enumerated deterministically.

Editorial analysis by NHI Mgmt Group, based on content published by Tetrate: “Two Ways to Build a Cost Agent (And Why We Use Both)”.

Key questions

Q: How should teams split deterministic work from model judgment in AI agents?

A: Teams should push enumerable, repeatable checks into deterministic code and reserve the model for ambiguous cases where context changes the meaning of a result.

Q: When does an AI agent architecture become too dependent on the LLM?

A: It becomes too dependent when the model is responsible for discovery, prioritisation and explanation in the same loop, especially at scale.

Q: What breaks when an AI agent tries to discover and assess everything at once?

A: Coverage becomes inconsistent, runtime costs increase non-linearly, and the system becomes harder to reproduce or audit.

Practitioner guidance

• Map discovery work to deterministic checks List every resource type, signal and threshold that your agent is expected to inspect, then move those checks into code or API-driven discovery before invoking the model.
• Limit model involvement to judgment calls Use the model only where context changes the outcome, such as distinguishing expected idle resources from genuine waste or ranking ambiguous findings.
• Separate scan outputs from narrative assessment Emit structured findings first, then ask the model to validate, prioritise and explain them in one bounded pass so that failures do not block discovery.

Bottom line: The article shows that AI agents work best when model reasoning is confined to the part of the workflow that truly needs judgment.

Explore further

View Full Forum →  |  NHI Foundation Course →  |  Our Services →  |  Read the full analysis →