Cyber-focused models accelerate AI testing, but not AI security

At a glance

What this is: Mindgard says cyber-focused models help with established security workflows, but they are not sufficient to assess the probabilistic and agentic risks in AI systems.

Why it matters: For IAM and security teams, the key issue is that AI behaviour, tool use and changing context create governance gaps that point-in-time testing will miss.

Context

Cyber-focused models can improve security analysis, but AI systems introduce a different problem space: behaviour changes with context, tools, memory and workflow state. In practice, that means a strong result on code review or vulnerability explanation does not translate into reliable coverage for AI-specific attack paths, especially where prompts, tool permissions or retrieval sources change over time.

The governance gap is not whether an AI model can help test a system. It is whether security teams can keep pace with a moving target that blends software logic, model behaviour and delegated actions into a single attack surface. That makes continuous validation more important than one-off assurance.

Key questions

Q: What breaks when AI security is treated only as model security?

A: Model-only security misses the part of the system that actually touches tools, data, and workflows in production. A secure model can still produce unsafe outcomes if the surrounding agent, connectors, or permissions are not governed. Practitioners need controls that follow the operational identity, not just the model artefact.

Q: Why do AI agents create governance risk even when they are meant to help testing?

A: AI agents create risk because they can follow unexpected paths, misread context, or act on malicious instructions without needing human intent. In testing workflows, that can mean contacting the wrong target, exposing data, or drifting into production-like systems. Governance has to assume behavioural drift and contain it technically before the agent acts outside scope.

Q: How should security teams test AI agents that can call tools and APIs?

A: They should test the agent's actual execution paths, not just its text responses. That means covering tool calls, protocol handling, approvals, and state transitions with adversarial scenarios that mirror real misuse. If the test cannot reach the access point where the agent acts, it does not meaningfully assess the security risk.

Q: Should organisations rely on cyber-focused models for AI red teaming?

A: They should use them as accelerators, not as a substitute for offensive expertise. Cyber-focused models are helpful for summarisation, code review and finding known issues, but AI security still needs proprietary attack intelligence, behavioural testing and continuous validation. The right model improves the work, but it does not replace the security programme.

Technical breakdown

Why probabilistic behaviour breaks traditional security testing

Traditional application testing assumes repeatable behaviour: the same input under the same conditions should produce the same result. AI systems are probabilistic, so context, prior interactions, prompt wording and model state can change the outcome. That means a single assessment can only describe one moment in time, not a stable security boundary. When the model, prompt, guardrails or retrieval sources shift, the system can expose new behaviours without any code change. Security testing must therefore treat the model and its operating context as part of the attack surface, not just the application wrapper.

Practical implication: Security teams should design assessments around change detection and revalidation, not static test results.

How agentic workflows expand the attack surface beyond model output

Once an AI system can use tools, memory and external data sources, the risk is no longer limited to text generation. The system can be manipulated into taking actions, delegating tasks or chaining tool use in unintended ways. That creates a behavioural attack surface where indirect prompt injection, context poisoning and tool manipulation matter as much as classic vulnerabilities. The security question becomes whether the system can be coerced into a harmful sequence of actions, not just whether it can answer a prompt incorrectly. This is why AI security has to consider identity, privilege and delegated authority inside the workflow. Read Red Teaming AI Agents for Identity Abuse.

Practical implication: Map every tool, memory store and delegated permission to the actions an AI system can trigger.

Why point-in-time AI assessments age out quickly

AI systems change too often for periodic testing to provide durable assurance. New prompts, new integrations, new retrieval sources and new permissions can all alter behaviour and create fresh attack paths. A one-time red team exercise may find real issues, but those findings can decay quickly if the model or workflow changes. The operational problem is not just discovery, but maintaining confidence as the system evolves. Continuous testing and monitoring are therefore part of the control plane, not an optional follow-up activity. In AI governance terms, coverage is temporary unless it is repeatedly refreshed. See the AI Security Maturity Model.

Practical implication: Re-test AI systems after model, prompt, tool or data-source changes, not only on a calendar cycle.

Threat narrative

Attacker objective: The attacker aims to turn a legitimate AI workflow into a controllable action path that exposes data, bypasses guardrails or triggers unauthorized operations.

1. Entry occurs when an attacker interacts with the AI system through prompts, context, retrieval content or connected tools.
2. Credential_harvested is not the only goal; the more realistic step is coercing the system into using its own delegated permissions or tool access in unintended ways.
3. Escalation happens when manipulated context, memory or tool chaining turns a benign AI workflow into an action path the defender did not intend.
4. Impact is achieved when the AI system leaks data, executes harmful actions or enables downstream compromise through its connected environment.

Breaches seen in the wild

• DeepSeek database exposure 2025: An unauthenticated DeepSeek ClickHouse database exposed over a million log lines with plaintext chat history and API keys in 2025.
• Microsoft SAS token exposure 2023: An over-permissive Azure SAS token in a Microsoft AI GitHub repo exposed 38TB, including workstation backups and Teams messages, for 3 years.

Read and download The State of NHI & AI Agent Breach Report 2026, covering 200+ breaches impacting Non-Human Identities including AI Agents.

NHI Mgmt Group analysis

Cyber-focused models are accelerators, not control systems. They can improve code review, vulnerability explanation and other established workflows, but those gains do not equal AI security assurance. The security problem changes once the target becomes a probabilistic system with tools, memory and context dependence. Practitioners should treat model-assisted analysis as one input to governance, not the governance model itself.

AI security creates a psycho-technical attack surface that classic application testing misses. The article usefully frames the problem as more than code defects, because prompt manipulation, indirect prompt injection and tool coercion all change the threat model. That means security teams need to assess how behaviour can be steered, not just whether a vulnerability exists. For identity teams, delegated authority inside AI workflows becomes a governance question, not a feature detail.

Continuous validation is the right operating assumption for AI systems. If prompts, model state, retrieval sources and permissions can change behaviour, then no assessment can claim full coverage. That is a governance debt problem as much as a testing problem, and it aligns closely with AI red teaming as a repeatable practice rather than a one-off event. Practitioners should plan for revalidation whenever the system changes.

Named concept: ephemeral AI attack paths describe the way AI risk changes when the attack surface shifts with context, tools and workflow state. Once that is accepted, assurance moves from periodic evaluation to continuous control verification. Teams that still frame AI security as a static test cycle will miss the conditions under which the system actually fails. The practical conclusion is to govern change, not just output.

From our research library:

• Only 5.7% of organisations have full visibility into their service accounts, according to the Ultimate Guide to NHIs.
• Only 23% of IT leaders were very confident in their organisation's ability to manage security and governance for GenAI deployments, according to a 2025 Gartner survey of 360 IT leaders.
• Read next: AI Red Teaming Guide

What this signals

Ephemeral AI attack paths: this is the governance gap that emerges when AI behaviour changes with prompts, context, tools and workflow state. Security teams should stop treating AI assurance as a one-time checkpoint and instead build revalidation into every meaningful system change.

The same governance model that protects static applications will not hold for agentic workflows. Once delegated actions, external data and memory enter the loop, continuous red teaming and change-triggered reassessment become the only defensible way to keep pace.

The most useful external reference point here is the {{L2|AI red teaming guide}}, because the practical challenge is not just finding flaws but proving that they still matter after the next system change.

For practitioners

• Build continuous AI revalidation into change management Re-test models, prompts, guardrails, retrieval sources and tool permissions whenever the system changes, because each change can create new attack paths.
• Inventory every AI tool and delegated permission Document which tools, memory stores and external data sources each system can reach, then map those actions to specific business and security risks.
• Separate model assistance from security decision authority Use cyber-focused models to accelerate analysis, but keep final judgments, escalation decisions and governance ownership with human reviewers.
• Test for behavioural manipulation as well as vulnerabilities Include indirect prompt injection, context poisoning, sycophancy and tool misuse in red team scenarios so the assessment covers how the system can be steered.

Key takeaways

• AI security is not solved by better model output alone because the system itself changes under different prompts, tools and context.
• The article’s core warning is that periodic testing creates false confidence when behaviour can shift without obvious code changes.
• Continuous validation is the control that matters most, because governance has to follow the evolving system rather than the latest assessment snapshot.

Related NHI Mgmt Group resources: Red Teaming AI Agents for Identity Abuse · AI Red Teaming Guide · AI Security Maturity Model · AI Security Platform Buyer's Guide

Key terms

• Psycho-technical attack surface: The combined technical and behavioural exposure created when an AI system can be influenced through prompts, tools, memory and context. It captures risks that do not exist in classic software testing because the system can be socially manipulated as well as technically exploited.
• Indirect Prompt Injection: Indirect prompt injection is an attack where malicious instructions are hidden inside content that an AI system reads later. The model may treat that content as context rather than as hostile input, which can influence tool use, data access, or workflow actions if controls are weak.
• Context Poisoning: Context poisoning is the manipulation of information that an AI agent reads before acting. The malicious content does not need to be code. If it changes the agent’s instructions, tool choices, or assumptions, it can alter behaviour and expand the impact of a compromised delivery path.
• Continuous validation: Continuous validation is the practice of re-checking user, device, or session risk after login instead of trusting access indefinitely. It recognizes that identity assurance can drift during a session, especially when endpoint state or user context changes after authentication.

Deepen your knowledge

NHI Mgmt Group covers identity security, NHI governance, and agentic AI through independent research, practitioner guides, and the NHI Foundation Level course. It is designed for teams that need a practical baseline for governing identities, secrets, and delegated access in modern security programmes.