Notifications
Clear all
Topic starter
20/08/2026 5:21 pm
TL;DR: Retrieval quality, not model reasoning alone, determines whether production agents stay reliable and affordable, according to Stacklok’s testing across 2,792 MCP tools, which found its MCP Optimizer selected the right tool 94% of the time, while Anthropic’s Tool Search Tool reached 34%. The practical issue is that context-window management has become an identity and access control problem for AI agents, because tool discovery now governs what an agent can do at runtime.
NHIMG editorial — based on content published by Stacklok: Blog insights comparing MCP Optimizer and Anthropic's Tool Search Tool
Questions worth separating out
Q: How should security teams handle tool discovery for AI agents in MCP environments?
A: Security teams should treat tool discovery as a privilege boundary, not a convenience layer.
Q: Why do large MCP tool catalogs increase risk for AI agents?
A: Large catalogs increase risk because they widen the tool surface an agent can discover, while also making correct retrieval harder.
Q: What do teams get wrong about token reduction in agent workflows?
A: Many teams assume lower token usage automatically means safer or better governed agent behaviour.
Practitioner guidance
- Measure tool retrieval before scaling agent workloads Build test sets from your actual MCP servers and verify whether the correct tool appears in search results under realistic prompts.
- Constrain tool visibility by task scope Do not expose entire tool catalogs to every session.
- Audit false positives and missed tools Review cases where the wrong tool is surfaced or the right tool is absent, then classify the operational impact.
What's in the full article
Stacklok's full article covers the operational detail this post intentionally leaves for the source:
- Full benchmark tables comparing retrieval and selection accuracy across tool-retrieval methods
- The test methodology used across 2,792 MCP tools, including synthetic query generation
- Runtime performance measurements that break down token usage, average tools retrieved, and execution time
- Implementation context for ToolHive and MCP Optimizer that matters to teams evaluating deployment options
👉 Read Stacklok's comparison of MCP Optimizer and Anthropic's Tool Search Tool →
MCP tool search accuracy gaps: are your agents selecting right?
Explore further
20/08/2026 6:12 pm
MCP tool discovery is now a runtime access-control problem, not a simple search problem. The article shows that agents do not just need fewer tokens, they need the right tools surfaced at the right time. That moves MCP governance into the same category as capability scoping and access review, because tool visibility determines what the agent can attempt. Teams should treat search quality as part of the non-human identity control plane.
A few things that frame the scale:
- 80% of organisations report their AI agents have already performed actions beyond their intended scope, including accessing unauthorised systems (39%), inappropriately sharing sensitive data (31%), and revealing access credentials (23%), according to AI Agents: The New Attack Surface.
- Another 33% of organisations report that AI agents have accessed inappropriate or sensitive data beyond their intended scope, reinforcing that runtime scope drift is already observable in deployed environments.
A question worth separating out:
Q: How do I know whether MCP search is actually working well enough?
A: You know MCP search is working when the correct tool appears consistently for real user intents, missed retrievals are rare, and the loaded tool set stays appropriately small. If you need repeated retries, see unexpected tool choices, or cannot audit what was loaded, the search layer is not yet reliable enough for production.
👉 Read our full editorial: MCP tool search accuracy gaps are reshaping agent runtime design
