Teams should test MCP-capable models against their own repositories, workflows, and tool chains before routing production traffic. Benchmark scores can indicate likely strengths, but they do not replace workload-specific validation. Focus on tool invocation accuracy, latency, cost per run, and how the model behaves under long, multi-step debugging or refactoring sessions.
Why This Matters for Security Teams
MCP-capable models can look impressive in generic coding benchmarks and still fail when they are connected to real repositories, ticketing systems, secrets stores, and deployment tooling. For agentic coding, the risk is not only bad code output. It is also incorrect tool selection, unsafe file access, unintended privilege use, and silent drift across multi-step tasks. Security teams should evaluate these systems as operational actors, not just as text generators, using guidance from the OWASP Agentic AI Top 10 and the NIST AI Risk Management Framework.
The practical question is whether the model can preserve task boundaries while operating in your environment. That means checking whether it requests only the tools it needs, whether it respects least privilege, whether it can recover from bad intermediate steps, and whether its outputs remain safe when prompts, code, and external context are mixed together. This is especially important where MCP servers expose repositories, CI/CD systems, cloud resources, or secrets through shared interfaces.
In practice, many security teams discover unsafe tool behavior only after an agent has already modified code, exposed data, or triggered an irreversible workflow, rather than through intentional pre-production testing.
How It Works in Practice
Evaluation should start with a representative workload set, not a single demo prompt. Build cases that reflect your real agentic coding flows: bug fixing, refactoring, dependency updates, test generation, issue triage, and incident-response support. Each case should be run against the actual MCP tools the model will use, with logging for tool calls, arguments, retries, refusals, and completion quality. The goal is to measure whether the model can complete work accurately without overreaching into unrelated systems.
A useful assessment usually combines functional, security, and operational checks:
- Tool invocation accuracy, including whether the model selects the correct MCP server and action.
- Permission discipline, including whether it avoids unnecessary access to repos, secrets, or production endpoints.
- Output integrity, including whether code changes compile, pass tests, and avoid unsafe patterns.
- Session resilience, including behavior across long debugging chains and context resets.
- Cost and latency, including how often the model loops, retries, or escalates tool use.
For threat modeling, map likely abuse paths to MITRE ATLAS adversarial AI threat matrix tactics and the relevant agentic controls in OWASP Top 10 for Agentic Applications 2026. If the model is allowed to authenticate to internal services, the workload identity should be explicit and auditable, which is where the SPIFFE workload identity specification becomes relevant for separating the agent’s identity from the human operator’s identity.
Teams should also test adversarial prompts that try to induce tool misuse, prompt injection via repository content, and hidden instructions in code comments or tickets. Where the model is used for code security triage, it should be assessed on false positives, false negatives, and whether it can justify recommendations with traceable evidence from the repo or build outputs. These controls tend to break down in highly dynamic monorepos with loosely governed MCP servers because tool boundaries, permissions, and context sources change faster than evaluation baselines can be updated.
Common Variations and Edge Cases
Tighter evaluation often increases testing overhead, requiring organisations to balance speed of adoption against confidence in the agent’s behaviour. That tradeoff is real for teams that want rapid developer productivity gains while still controlling code and infrastructure risk.
Best practice is evolving for MCP-capable agentic coding, and there is no universal standard for scoring one model against another yet. Some teams prioritise pass rate on coding tasks, while others weight security outcomes more heavily, such as safe refusal rates, tool-minimisation, or containment when prompts are hostile. A model that is excellent in isolated code completion may still be unsuitable for autonomous workflows if it cannot handle chained tool use safely.
Edge cases matter. For example, an internal model may perform well with read-only repository access but fail once given write permissions or access to CI/CD tokens. Similarly, a model may look compliant in a sandbox but behave differently when connected to long-lived context, production-like secrets, or developer convenience tooling. If the use case includes regulated software delivery or critical infrastructure, teams should align evaluation with governance expectations in the NIST AI Risk Management Framework and the CSA MAESTRO agentic AI threat modeling framework, especially where autonomous actions could affect deployment pipelines or security controls.
Where agentic coding is tied to real identities, access governance must be explicit. The model should not inherit broad human permissions by default, and security teams should decide whether separate credentials, scoped tokens, or workload identities are required for each MCP-connected system. That distinction becomes critical when debugging sessions span multiple tools, because a single overbroad credential can turn a coding assistant into a privileged operator.
Standards & Framework Alignment
This section maps relevant standards and security frameworks to the operational risks and controls described in this guidance.
OWASP Agentic AI Top 10, MITRE ATLAS and CSA MAESTRO address the attack and risk surface, while NIST AI RMF and NIST CSF 2.0 set the governance and control requirements practitioners need to meet.
| Framework | Control / Reference | Relevance |
|---|---|---|
| OWASP Agentic AI Top 10 | A2 | Agent tool misuse and unsafe autonomy are central risks in MCP coding workflows. |
| NIST AI RMF | GOVERN | Model evaluation needs documented accountability, risk decisions, and oversight. |
| MITRE ATLAS | AML.TA0002 | Prompt injection and adversarial manipulation map to known AI attack paths. |
| NIST CSF 2.0 | PR.AC-4 | MCP evaluations must verify least-privilege access to repositories and tools. |
| CSA MAESTRO | T2 | Agentic AI threat modeling supports workflow-specific testing of tool and identity risk. |
Test whether the model can use tools safely, refuse unsafe actions, and stay within intended autonomy.
Related resources from NHI Mgmt Group
- How should security teams evaluate agentic coding tools when model behaviour changes frequently?
- How should security teams govern MCP servers used by AI coding assistants?
- How should security teams govern MCP access in agentic workflows?
- How should security teams govern MCP workflows that mix models, servers, and users?
