Topic starter
20/08/2026 5:43 pm
TL;DR: Across GitHub tasks, an eval of Claude Opus 4.6 found that MCP, opinionated skills, and a bare CLI all reached similar correctness, according to Arize, but MCP became far slower and costlier on open-ended analysis because a fixed API surface cannot compose as flexibly as shell tools. The real lesson is governance fit: tool choice changes auditability, auth, and control boundaries, so teams should design for both remote protocol access and local command-line execution.
Editorial analysis by NHI Mgmt Group, based on content published by Arize: “MCP vs. CLI Skills for agents: what our eval found (and which you should use)”.
Key questions
Q: What breaks when agents rely on a fixed protocol surface for complex tasks?
A: A fixed protocol surface breaks down when the task requires composition across many steps, derived calculations, or cross-object analysis.
Q: Why do shell-based agent workflows change the governance model?
A: Shell-based workflows expand the execution environment beyond a bounded remote API into local commands, pipes, files, and often ambient credentials.
Q: When should teams prioritise protocol-mediated access over CLI automation?
A: Teams should prioritise protocol-mediated access when the agent is reaching remote systems, customer-facing resources, or environments where consent, revocation, and auditability matter more than local composability.
Practitioner guidance
- Define the agent access boundary by task type Separate endpoint-shaped actions from composition-heavy actions before choosing MCP, skills, or CLI execution.
- Inventory credentials behind every execution path Map which identities, tokens, or local sessions each tool path depends on, including fallback shell use.
- Log tool fallback behavior as a control signal Track when agents leave the intended tool surface and invoke shell commands or other fallback mechanisms.
Bottom line: Agents can achieve similar outcomes through MCP, skills, or CLI, but the governance implications differ because each path exposes a different authority boundary.
Explore further
View Full Forum → | NHI Foundation Course → | Our Services → | Read the full analysis →
20/08/2026 6:26 pm
Tool choice is an access-governance decision, not a productivity preference. This evaluation shows that agents can reach similar task outcomes through either structured protocol access or shell-based execution, but the governance consequences differ sharply. Remote protocol access supports tighter authorisation boundaries, while CLI execution expands the reachable action space. Practitioners should stop treating agent tooling as a neutral implementation detail and instead classify each option by the identity boundary it creates.
A few things that frame the scale:
- Only 5.7% of organisations have full visibility into their service accounts, according to the Ultimate Guide to NHIs.
A question worth separating out:
Q: What does mixed-mode agent execution mean for identity governance?
A: Mixed-mode execution means the agent can move between structured tools and shell commands depending on the task. That creates a harder governance problem because the identity boundary is no longer singular. Programmes need to know which paths are allowed, how each path is logged, and what revocation looks like across both remote and local execution.
👉 Read our full editorial: MCP vs CLI skills for agents: what evaluation results show