Why do AI systems create both safety and security risk?

Why This Matters for Security Teams

AI systems introduce two distinct failure paths. Safety failures emerge when the model behaves as intended but still produces harmful, biased, unsafe, or misleading outcomes. Security failures emerge when an attacker manipulates the system through prompt injection, data poisoning, compromised credentials, or tool abuse. That distinction matters because governance, testing, and incident response differ. NIST Cybersecurity Framework 2.0 helps anchor the security side, while AI risk management needs its own controls for model behaviour and output quality.

This is not just theoretical. NHIMG research in the Ultimate Guide to NHIs — Why NHI Security Matters Now shows how quickly exposed credentials can be abused in real environments, which becomes especially relevant when AI systems call external tools or APIs. In practice, many security teams encounter AI risk only after a model output, workflow, or credential has already caused damage, rather than through intentional governance design.

How It Works in Practice

In operational terms, safety and security risk overlap but do not map to the same control set. Safety focuses on whether the system’s outputs remain acceptable under expected use. Security focuses on whether an adversary can change inputs, context, training data, retrieval sources, or tool execution to force harmful behaviour. A model can be “safe” in testing and still be insecure if an attacker can inject instructions into a retrieved document or hijack an agent’s credentials.

For practitioners, the cleanest way to separate the two is to ask three questions:

• Can the system fail on its own, without an attacker?
• Can an attacker influence the model, its prompts, its context, or its tools?
• Can the system’s action path be observed, explained, and reversed if needed?

That is why AI governance often needs both AI-specific review and security engineering. Frameworks such as NIST Cybersecurity Framework 2.0 cover protection, detection, and response, while AI-focused guidance examines output reliability, model provenance, and misuse resistance. NHIMG’s OWASP NHI Top 10 is also relevant where AI agents use non-human identities, because compromised tokens or over-privileged service accounts can turn a model mistake into a breach. The practical control set usually includes prompt filtering, retrieval allow-listing, strong secrets handling, tool-scoped permissions, logging, and human approval for sensitive actions.

The biggest implementation gap is usually ownership: model safety may sit with AI teams, while authentication, secrets, and monitoring sit with security teams. These controls tend to break down when AI agents are allowed to execute high-impact workflows with shared credentials and weak audit trails because no single owner can see the full request-to-action chain.

Common Variations and Edge Cases

Tighter controls often increase latency, cost, and workflow friction, requiring organisations to balance model utility against operational containment. There is no universal standard for this yet, especially for agentic AI, so current guidance suggests using risk-tiered controls rather than treating every use case the same.

Some systems are mainly safety-sensitive, such as customer support summarisation or medical decision support, where the core concern is harmful output quality. Others are mainly security-sensitive, such as autonomous agents that can send emails, change tickets, or access cloud resources, where credential governance and tool isolation matter more. The hardest cases combine both: retrieval-augmented generation over sensitive data, AI copilots with write access, or workflows that touch customer identity, payments, or privileged infrastructure.

NHIMG’s The State of Non-Human Identity Security highlights that 45% of organisations cite lack of credential rotation as a top cause of NHI-related attacks, which is a useful reminder that AI risk often becomes security risk through weak machine identity hygiene. The same principle applies when AI tools use APIs, service accounts, or OAuth grants. Best practice is evolving, but the conservative approach is to treat model behaviour, access control, and transaction approval as separate control layers. That separation becomes essential in regulated environments, high-autonomy agents, and any deployment where the model can take actions outside the immediate visibility of a human operator.

Related resources from NHI Mgmt Group

• Why do agentic AI systems create more security risk than standard chatbots?
• Why do training data changes create security risk in AI systems?
• What is the core decision loop Agentic AI follows and why does it create security risk?
• How should security teams limit the risk from AI agents that have access to production systems?