AI systems create risk because they introduce new attack surfaces, including prompt abuse, model misuse, and vulnerabilities unique to large language model applications. Mature application security still matters, but it is not enough on its own. Organisations need continuous testing and specialised review because AI can change behavior, expose unexpected outputs, and interact with other systems in ways traditional controls do not fully cover.
Why mature appsec does not remove AI-specific exposure
Mature application security still helps, but AI systems change the threat model in ways that traditional appsec does not fully cover. The new risk comes from how the model can be steered, how outputs can be manipulated, and how the application may act on model-generated content or tool requests that were never part of the original web or API security baseline.
That means the security problem is not just “is the code secure?” but “can the system be induced to behave unsafely through the model interface, the prompts, the retrieval layer, or the connected tools?” In agentic designs, this can also extend into action execution, where a harmful instruction becomes a harmful system operation.
Organisations that already know how to secure code paths, sessions, and APIs often still need separate review for model behavior, prompt handling, data boundaries, tool permissions, and abuse-resistant fail-safes. Mature appsec is a foundation, not a complete control set for AI.
Where the new attack surface actually appears
AI systems introduce several places where the control objective changes. Prompt injection and prompt abuse target the instruction layer rather than the code layer. Model misuse can produce unsafe content, flawed decisions, or unauthorized actions. RAG pipelines, plugins, connectors, and tool calls add trust relationships that may not exist in conventional applications. If the model can reach other systems, the blast radius can expand beyond the AI component itself.
This is why a secure SDLC can still miss the issue: the risk is often in the system composition. A well-tested API may still be exposed if the model can be tricked into requesting data it should not see, or into using a tool it should not control. For a practitioner, the key question is whether the AI component can influence confidentiality, integrity, or availability outside the bounds assumed by ordinary appsec testing. The OWASP ASVS baseline still matters, but AI-specific abuse paths need added review. For agentic systems, the failure modes called out in OWASP Agentic Applications Top 10 and the OWASP Top 10 for Agentic Applications 2026 are especially relevant because they map directly to prompt abuse, tool misuse, and privilege-related errors.
If you need a concrete example of how AI-linked access can fail outside standard appsec expectations, the DeepSeek breach shows how exposed secrets and logs can become an AI-era security problem, not just an application hygiene issue.
What practitioners should do differently
The practical shift is to test the system as a whole, not just the codebase. That means assessing prompt injection resistance, model output handling, connector permissions, retrieval controls, secret exposure, and the safety of any downstream action the system can take. Security teams should define which model outputs are advisory only and which can trigger automation, then require extra controls before any output can cross that boundary.
What to verify: Confirm that AI inputs are isolated from system instructions, that retrieved content cannot silently override policy, and that tool or API access is scoped to the minimum required action set. If a model can write, delete, send, approve, or retrieve, then the surrounding controls must be designed for those consequences, not for a normal read-only chat workflow.
What to measure: Track how often malicious or malformed prompts change behaviour, how often tool calls exceed expected intent, and how quickly risky model changes or connector changes are retested. The most useful signal is not model accuracy alone, but whether the system preserves its security boundaries when prompts, data, or integrations are adversarially shaped.
Practitioner takeaway: Mature appsec reduces ordinary software risk, but AI requires testing for instruction abuse, unsafe tool use, and emergent system behavior, because those failure modes can bypass controls that were never designed for a model-mediated workflow.
Standards & Framework Alignment
This section maps relevant standards and security frameworks to the operational risks and controls described in this guidance.
OWASP Agentic AI Top 10 address the attack and risk surface, while NIST AI RMF set the governance and control requirements practitioners need to meet.
| Framework | Control / Reference | Relevance |
|---|---|---|
| OWASP Agentic AI Top 10 | A1 — Prompt Injection | Prompt abuse is a core AI-specific attack path in agentic systems. |
| A3 — Tool Abuse and Unauthorized Actions | AI risk grows when model outputs can trigger connected tools or actions. | |
| A5 — Sensitive Information Disclosure | AI systems can expose secrets through prompts, logs, retrieval, or outputs. | |
| Recommendation — Test prompts and retrieved content for instruction hijacking and isolate system instructions. Restrict tool permissions and require explicit authorization before model-driven actions execute. Block secret exposure in prompts, logs, retrieval, and generated responses. | ||
| NIST AI RMF | GOVERN — Govern | AI systems need governance over intended use, oversight, and risk ownership. |
| MAP — Map | Risk assessment must identify AI-specific attack surfaces and integrations. | |
| MANAGE — Manage | AI controls must be continuously managed as models and integrations change. | |
| Recommendation — Establish governance for acceptable AI use, escalation, and accountability. Map model inputs, outputs, dependencies, and downstream impacts before deployment. Continuously monitor AI behavior and adjust controls when risk conditions change. | ||
Related resources from NHI Mgmt Group
- Why do MCP-based controls create new security risks in multi-agent AI systems?
- Why do generative AI systems create new incident response risks for enterprise security teams?
- Why do conversational AI systems create new identity and access risks?
- Why do chat-based AI systems create new identity risk for organisations?
