MCP vs CLI skills for agents: what evaluation results show

At a glance

What this is: Arize's eval showed that agents could reach similar correctness through MCP, skills, or a bare CLI, but the control surface changed latency, cost, and governance fit.

Why it matters: IAM and NHI teams need to treat tool connectivity as an access-design decision because the way an agent reaches systems affects auth, auditability, and operational control.

Context

MCP and CLI skills are different control surfaces for the same agentic task. One exposes tools through a protocol with structured discovery, while the other relies on shell commands and local automation. For identity teams, that difference matters because the access model changes even when the task outcome looks similar.

Arize's evaluation compared Claude Opus 4.6 across GitHub tasks using an official MCP server, two skills packages, and a bare CLI baseline. The article's central claim is not that one interface always wins, but that tool choice shapes cost, latency, and the governance boundary around what the agent can do.

The practical question for practitioners is whether their agent programme is optimised for composable remote tools, locally executed commands, or both. That is an identity and access design decision, not just an engineering preference.

Key questions

Q: What breaks when agents rely on a fixed protocol surface for complex tasks?

A: A fixed protocol surface breaks down when the task requires composition across many steps, derived calculations, or cross-object analysis. The agent compensates by making more calls, carrying more context, and generating larger payloads, which raises cost and latency. That is a governance signal, not just a performance issue, because it shows the control surface is too narrow for the work.

Q: Why do shell-based agent workflows change the governance model?

A: Shell-based workflows expand the execution environment beyond a bounded remote API into local commands, pipes, files, and often ambient credentials. That changes the risk profile because the agent can reach more than the intended tool boundary. For IAM and NHI teams, the key issue is not speed, but whether the access path remains clearly authorisable and revocable.

Q: When should teams prioritise protocol-mediated access over CLI automation?

A: Teams should prioritise protocol-mediated access when the agent is reaching remote systems, customer-facing resources, or environments where consent, revocation, and auditability matter more than local composability. CLI automation is useful for internal workflows, but it usually depends on broader ambient authority. The deciding factor is governability, not benchmark convenience.

Q: What does mixed-mode agent execution mean for identity governance?

A: Mixed-mode execution means the agent can move between structured tools and shell commands depending on the task. That creates a harder governance problem because the identity boundary is no longer singular. Programmes need to know which paths are allowed, how each path is logged, and what revocation looks like across both remote and local execution.

Technical breakdown

Why fixed MCP surfaces get expensive on open-ended tasks

MCP works well when a task maps cleanly to a tool endpoint, because the agent can call a defined method and receive structured output. The problem appears when the task requires composition across steps, such as grouping issues, comparing milestones, or calculating derived values. A fixed API surface cannot improvise new operations, so the agent fans out into more calls, larger payloads, and repeated context accumulation. That increases both latency and token cost. In other words, the protocol is efficient for bounded actions but less efficient for analytic work that requires cross-step reasoning over remote state.

Practical implication: classify which agent tasks are endpoint-shaped and which require composition before deciding whether MCP is the right control surface.

How CLI skills change the trust and audit model

A CLI gives an agent access to a broader execution environment than a narrow protocol does. That can improve composability because shell commands, pipes, and filters can combine into new workflows without waiting for a dedicated API endpoint. But the governance trade-off is that the agent may step outside the intended tool boundary, especially when it needs to parse files, call local commands, or use ambient credentials. For identity teams, that means the control boundary shifts from explicit tool authorisation to terminal-level execution governance. The question is not just what the agent can do, but what surrounding commands and credentials it can reach.

Practical implication: treat shell access as a separate privilege domain and monitor the commands, files, and credentials that sit behind it.

Why auth design matters more than benchmark speed

The article shows that both MCP and CLI can succeed, but the access path changes who can use the system and how access is granted. MCP's OAuth-oriented model is better suited to managed, consented access across remote resources, while CLI workflows often depend on locally stored tokens or manually configured credentials. That means the same agent capability can sit inside very different identity patterns. In NHI terms, the question is not whether the agent succeeded, but whether the access mechanism supports clear issuance, revocation, and scope control. Governance fails when tool performance is treated as the only selection criterion.

Practical implication: evaluate agent tooling with the same rigor you would apply to service-account authentication and token governance.

Breaches seen in the wild

• Gemini CLI prompt injection flaw 2025: Tracebit showed a poisoned README could make Gemini CLI run hidden commands and exfiltrate developer secrets; Google fixed it in 0.1.14.
• Nx s1ngularity attack 2025: Attackers stole Nx's npm token via a GitHub Actions flaw and shipped malware that stole 2,349 secrets and abused developers' AI CLIs.

Read and download The State of NHI & AI Agent Breach Report 2026, covering 200+ breaches impacting Non-Human Identities including AI Agents.

NHI Mgmt Group analysis

Tool choice is an access-governance decision, not a productivity preference. This evaluation shows that agents can reach similar task outcomes through either structured protocol access or shell-based execution, but the governance consequences differ sharply. Remote protocol access supports tighter authorisation boundaries, while CLI execution expands the reachable action space. Practitioners should stop treating agent tooling as a neutral implementation detail and instead classify each option by the identity boundary it creates.

MCP and CLI expose different control assumptions: MCP presumes bounded, pre-shaped operations, while CLI workflows presume composable local execution and broader ambient authority. That distinction matters because the access review, logging, and revocation model changes with the tool surface. The practical implication is that organisations must design governance around the execution path, not just the model behind it.

Agent tooling now sits at the intersection of NHI, IAM, and operational telemetry. The article's strongest lesson is that correctness alone does not tell you whether an agent is governable. A tool surface that is slightly slower but tightly bounded may be safer than one that is faster but harder to constrain, depending on the environment and data sensitivity. Teams should evaluate agent tools by control boundary, not by benchmark convenience.

The right question is which identity pattern the tool enables. CLI access may suit tightly managed internal automation, while MCP is better aligned to remote resource mediation and consent-driven access. The governance issue is not whether agents can use both, but whether the organisation can clearly inventory, authorise, and revoke the identities behind each path. Practitioners should align tool selection to the access model they can actually govern.

Composable execution creates measurable governance debt when the access path is not explicit. As soon as agents can fall back from a protocol into shell commands, the apparent simplicity of the benchmark disappears. That is where audit trails fragment, credentials sprawl, and policy expectations become less reliable. Identity teams should assume mixed-mode agent execution unless the environment forbids it by design.

From our research library:

• Only 5.7% of organisations have full visibility into their service accounts, according to the Ultimate Guide to NHIs.
• Read next: NHI Lifecycle Management Guide

What this signals

Composable execution is the real governance fault line: once an agent can fall back from a fixed protocol into shell commands, auditability and revocation become path-dependent rather than universal. Security teams should assume that access control now lives in the execution route as much as in the model or app layer.

Tool choice is also a trust-design choice. MCP-style access works best where consent, scope, and remote resource mediation need to be explicit, while CLI execution works best where local composability is the priority and the surrounding environment is already tightly governed.

For practitioners

• Define the agent access boundary by task type Separate endpoint-shaped actions from composition-heavy actions before choosing MCP, skills, or CLI execution. If the task needs only bounded retrieval or simple writes, keep the access surface narrow. If it requires multi-step analysis, decide explicitly whether shell execution is permitted or whether the workflow must stay inside a protocol boundary.
• Inventory credentials behind every execution path Map which identities, tokens, or local sessions each tool path depends on, including fallback shell use. Treat local command execution as a distinct privilege class because it often carries broader authority than protocol-mediated access.
• Log tool fallback behavior as a control signal Track when agents leave the intended tool surface and invoke shell commands or other fallback mechanisms. That behavior often indicates the primary surface is too rigid for the job, or that the agent is operating outside its designed control envelope.
• Compare access governance before comparing latency Use auth model, revocation ability, and audit clarity as the first decision criteria, then evaluate cost and speed. A faster tool is not easier to govern if it expands the execution boundary beyond what your programme can control.

Key takeaways

• Agents can achieve similar outcomes through MCP, skills, or CLI, but the governance implications differ because each path exposes a different authority boundary.
• When tasks require composition, a fixed protocol surface tends to increase calls, context, latency, and cost, which is a sign the tool model and the work pattern are misaligned.
• Identity teams should evaluate agent tooling by auth model, revocation path, and audit clarity before they optimise for speed or convenience.

Key terms

• MCP: Model Context Protocol, an open way for AI agents to connect to tools and data sources. It improves interoperability, but it also introduces a shared integration layer that must be governed carefully because the protocol can widen access across many systems at once.
• CLI skill: A CLI skill is a prompt or instruction set that teaches an agent how to operate command-line tools for a specific task or environment. It can improve composability and task performance, but it also broadens the execution surface because the agent may rely on shell commands, local files, and ambient credentials.
• Tool Fidelity: The degree to which an agent stays within the tools it was intended to use. High tool fidelity is useful, but only when the allowed tools themselves are properly scoped and the agent is not escaping into unmanaged execution paths.
• Execution boundary: The point at which an authorised task turns into a real system change, such as writing data, deleting records, spending money, or invoking a downstream tool. In AI governance, controlling the execution boundary matters more than simply approving access, because harm occurs when actions are allowed to complete unchecked.

Deepen your knowledge

NHI governance, agentic AI identity, and machine identity lifecycle are core topics in our NHI Foundation Level course, the industry's only accredited NHI security programme. If you are building or maturing an IAM programme, it is worth exploring.