Just-in-time discovery reduces context bloat and can lower token use, but it still creates risk if the search layer cannot consistently find the right tool. When nearly half the searches fail before selection or parameterization, the agent cannot be trusted to complete routine work. Reliability depends on both discovery quality and downstream action execution.
Why just-in-time discovery can still fail reliability
Just-in-time discovery trims preloaded context, but it also makes runtime search quality part of the control surface. If the discovery step misses the correct tool, returns an ambiguous match, or cannot consistently rank the right capability first, the agent may behave “efficiently” while still being operationally unreliable. That is a failure of dependable action selection, not just a prompt-size problem.
Discovery quality matters because the agent is only as good as the options it can find at the moment it needs them. A search layer that is brittle under naming variation, incomplete metadata, or tool sprawl creates avoidable misses even when the downstream tool itself is healthy. In practice, reliability depends on both finding the right tool and handing it off with the right parameters.
When discovery is used as a gate, the quality bar shifts from “can the agent theoretically act?” to “can it repeatedly locate the correct action path in real conditions?” That is why search relevance, tool catalog hygiene, and parameterization readiness are part of the system’s operational reliability, not optional implementation details. NHIMG’s Ultimate Guide to NHIs, lifecycle processes for managing NHIs is a useful reference point for the broader lifecycle discipline behind discoverability and control.
Where the failure usually appears in practice
Most teams notice the problem first as intermittent task failure: the agent succeeds on familiar requests, then stalls or chooses the wrong tool when the request wording changes, the schema is updated, or the tool catalogue grows. That is a classic reliability smell because the failure is non-deterministic. A system that only works when search happens to land on the right tool is not predictable enough for routine operations.
Discovery also creates a hidden coupling between tool naming and task completion. If the agent depends on search over descriptions, tags, or embeddings, then small documentation gaps can become execution failures. The result is not just slower behavior, but inconsistent behavior across similar requests, which is especially damaging when the agent is expected to perform repeatable operational work.
That is why tool governance matters as much as tool existence. Just-in-Time Access and Zero Standing Privilege Guide helps frame the broader control problem, while Privileged Access Management Guide covers the access and privilege side of making runtime actions both selectable and constrained.
What reliable agents need beyond discovery
Reliable action depends on two separate checks: first, the agent must find the right tool; second, it must invoke that tool with the right scope, credentials, and parameters. If either layer is weak, the overall workflow is weak. Discovery can reduce context bloat, but it cannot compensate for unclear tool contracts, poor parameter validation, or incomplete authorization design.
The practical goal is not to preload every tool into the model context, but to make discovery deterministic enough that the same intent maps to the same tool choice under normal operating conditions. That usually requires clean metadata, stable naming, explicit action boundaries, and a fallback path when discovery confidence is low. Without those, just-in-time discovery becomes a source of hidden variance rather than a control.
For that reason, the best operating model is a narrow, well-governed tool set with strong search semantics and explicit permissions. AI Agent Authorisation Guide is relevant where the missing piece is not discovery itself, but the rule that determines what the agent is allowed to do once it has found a tool. AI Agent Observability, Audit and Incident Response Guide is equally important when you need to prove whether failures came from discovery, selection, or execution.
Standards & Framework Alignment
This section maps relevant standards and security frameworks to the operational risks and controls described in this guidance.
OWASP Agentic AI Top 10 addresses the attack and risk surface, while NIST SP 800-53 Rev 5 sets the governance and control requirements practitioners need to meet.
| Framework | Control / Reference | Relevance |
|---|---|---|
| OWASP Agentic AI Top 10 | ASI02 — Tool Misuse | Discovery failures affect whether agents select the right tool. |
| ASI03 — Identity & Privilege Abuse | Reliable action depends on the authority applied after tool discovery. | |
| Recommendation — Constrain tool discovery and selection to reduce wrong-tool execution. Bind discovered tools to least-privilege, task-scoped permissions. | ||
| NIST SP 800-53 Rev 5 | AU-3 — Content of Audit Records | Telemetry is needed to separate discovery failure from execution failure. |
| AC-6 — Least Privilege | Discovery should not expose broader action rights than the task requires. | |
| CM-8 — System Component Inventory | A complete, current tool inventory underpins dependable discovery. | |
| Recommendation — Log tool search, selection, and execution events with enough detail to diagnose failures. Limit tool permissions to the minimum required for the task. Maintain an accurate inventory of tools, endpoints, and callable actions. | ||
Practitioner Guidance
What to verify: Measure discovery success separately from action success. If the agent can find a tool but still fails to parameterize or execute it correctly, you have an execution-design problem as well as a search problem.
What to measure: Track search recall, top-one tool match rate, and end-to-end task completion rate. The meaningful signal is not just how often a tool appears in results, but how often the correct tool is selected and used without human correction.
Common mistake: Treating just-in-time discovery as a context-saving optimization only. In production, it is also a reliability dependency, so catalogue quality, metadata discipline, and fallback behavior need the same attention as the model prompt.
Decision rule: If a routine task cannot be completed consistently from discovery through execution, tighten the tool catalogue before expanding autonomy. Narrower, better-governed tool access is usually safer than broader discovery with uncertain selection quality.
Practitioner takeaway: Just-in-time discovery is useful when it improves selection without weakening determinism, but it becomes a reliability liability the moment search uncertainty starts deciding whether the agent can act at all.
Related resources from NHI Mgmt Group
Deepen Your Knowledge
Reviewed and updated by the NHIMG editorial team on September 30, 2026.
NHI Mgmt Group — the #1 independent authority on Non-Human Identity, IAM, and Agentic AI security. nhimg.org
