Insecure AI models can turn ordinary prompts into a control failure that leaks data, generates unsafe code, or follows hostile instructions. When the model sits near sensitive systems, weak guardrails and poor access boundaries can expose credentials, business logic, or source code. The risk rises further if users can deploy unsanctioned models without governance.
Why This Matters for Security Teams
Insecure AI models become enterprise risk multipliers when they are wired into data stores, ticketing systems, code repositories, or workflow automation. The model is no longer just producing text; it is influencing decisions, retrieving sensitive context, and sometimes triggering actions. That makes prompt injection, data leakage, and unsafe code generation operational issues, not just model-quality concerns. Current guidance from NIST Cybersecurity Framework 2.0 and NHIMG research such as Ultimate Guide to NHIs — Key Challenges and Risks both point to the same operational reality: once an AI system can reach business data, its failure modes inherit the sensitivity of that environment.
This is especially dangerous because AI systems do not need to be “fully compromised” to cause damage. A model that sees secrets in context may repeat them, a model with tool access may call the wrong endpoint, and a model with weak approval gates may help an attacker pivot across workflows. NHIMG research in The State of Secrets in AppSec found that 43% of security professionals are concerned about AI systems learning and reproducing sensitive information patterns from codebases, which reflects a broader concern about data adjacency and training spillover. In practice, many security teams encounter the risk only after the model has already touched production data or automation paths, rather than through intentional testing.
How It Works in Practice
The risk increases when AI is connected to business workflows because the model’s inputs, outputs, and side effects all become part of the attack surface. A user prompt can carry hostile instructions, but the more serious issue is what the model is allowed to do after interpreting that prompt. If it can retrieve customer records, generate code, open tickets, or query internal systems, then a single bad interaction may expose data or drive an unsafe action downstream.
Security teams should think in terms of control boundaries, not just model accuracy. At minimum, separate the model from sensitive systems by default, then grant narrow, auditable access only when a workflow genuinely needs it. That usually means:
- Limiting what data the model can see, especially secrets, credentials, and regulated records.
- Using explicit authorization checks before tool calls or database access.
- Logging prompts, retrieved context, and actions taken so investigations can reconstruct impact.
- Applying policy at request time rather than assuming a safe model will always behave safely.
For implementation detail, NIST control guidance in NIST SP 800-53 Rev 5 Security and Privacy Controls remains useful for access restriction, auditability, and system integrity, while NHIMG’s Top 10 NHI Issues highlights how machine identities, tokens, and automation privileges often become the real escalation path. These controls tend to break down when AI agents are embedded into sprawling SaaS workflows because permissions, context retrieval, and tool execution are spread across too many systems for one team to govern cleanly.
Common Variations and Edge Cases
Tighter AI controls often increase latency and workflow friction, so organisations have to balance speed against containment. That tradeoff becomes visible in environments where business users want broad model access but data owners require strict separation.
One common edge case is shadow AI: employees route sensitive work through unsanctioned models because approved tools feel too restrictive. Another is retrieval-augmented generation, where the model itself is not trained on sensitive data but still surfaces it through connected search or document access. Best practice is evolving here, and there is no universal standard for how much contextual data a model should see by default.
The most reliable approach is to classify AI use by data sensitivity and action scope. A summarization model with no tool access is a very different risk from an agent that can modify records or deploy code. Organisations should also treat secrets as high-priority exposure points, because once credentials enter prompts, logs, or retrieved context, downstream abuse can happen quickly. For broader governance context, the Ultimate Guide to NHIs — Why NHI Security Matters Now explains why machine-to-machine access demands tighter control than traditional user workflows.
Standards & Framework Alignment
This section maps relevant standards and security frameworks to the operational risks and controls described in this guidance.
OWASP Non-Human Identity Top 10, OWASP Agentic AI Top 10 and CSA MAESTRO address the attack and risk surface, while NIST AI RMF and NIST CSF 2.0 set the governance and control requirements practitioners need to meet.
| Framework | Control / Reference | Relevance |
|---|---|---|
| OWASP Non-Human Identity Top 10 | NHI-01 | AI systems connected to data workflows often expose weak identity boundaries and secret handling. |
| OWASP Agentic AI Top 10 | A2 | Prompt injection and tool abuse are core risks when models act on business data. |
| CSA MAESTRO | MAE-2 | Maps to governance of agent and model exposure across enterprise workflows. |
| NIST AI RMF | AI RMF applies to managing model misuse, data leakage, and unsafe outcomes. | |
| NIST CSF 2.0 | PR.AC-4 | Enterprise AI access should be limited to authorized systems and workflows. |
Inventory every AI-connected NHI, then bind each to least-privilege access and short-lived secrets.
Related resources from NHI Mgmt Group
- Why do MCP connectors increase the risk of data exposure in enterprise AI workflows?
- Why do AI agents increase data exposure risk when they are connected to content repositories like Box?
- Why do LLM-based workflows increase privacy risk when they process raw business data and attachments?
- Why do AI agents and workflow automations increase operational risk when they interact with business data and third-party tools?
