AI application security is becoming an NHI governance problem

At a glance

What this is: This is an analysis of why AI application security requires different controls than traditional software, with the key finding that AI development and runtime expand the attack surface across data, tools, and autonomous actions.

Why it matters: For IAM and NHI practitioners, it shows why AI systems should be governed as identities with access and execution authority, not just as applications.

👉 Read Noma Security's analysis of AI application security and NHI risk

Context

AI application security has become an identity and access problem as much as a code-security problem. When model development depends on production data, shared notebooks, MLOps platforms, and tool-connected agents, the old assumption that development environments are low-risk no longer holds. That matters for NHI governance because AI workflows now create privileged, non-human access paths that need inventory, policy, and review.

The article argues that traditional security tooling misses the specific failure modes of AI systems, including prompt injection, poisoned retrieval data, model misuse, and agentic execution. That is a credible framing for practitioners: once AI systems can retrieve data and trigger workflows, they operate like NHIs with dynamic privileges rather than static software components.

Key questions

Q: How should security teams govern AI systems that can call tools and access data?

A: Treat them as non-human identities with execution authority. Give each agent the smallest possible tool set, separate read and write actions, and log every high-risk invocation. Governance should focus on approved workflows, scoped permissions, and continuous review rather than assuming the model itself is the control boundary.

Q: Why do AI development environments create more security risk than traditional dev environments?

A: Because they often contain production datasets, secrets, and shared experimentation assets instead of isolated code only. That collapses the normal dev-to-prod trust model and increases the chance that sensitive data, credentials, or model artefacts are exposed before release. Security teams should treat these environments as sensitive by default.

Q: What is the difference between securing AI content and securing AI execution?

A: Content security tries to block unsafe text or outputs. Execution security controls what the model or agent can actually do, including which tools it can call, which data it can retrieve, and which workflows it can trigger. For NHI governance, execution controls matter more because that is where real privilege lives.

Q: How can organisations reduce risk from prompt injection and tool misuse?

A: Use policy enforcement at the agent layer, not just filters at the prompt layer. Limit tool permissions, validate retrieved content, separate trusted and untrusted context, and require human approval for sensitive actions. That combination reduces the chance that manipulated language becomes unauthorised execution.

Technical breakdown

Why AI application development changes the attack surface

AI application development is not just software development with a new model plugged in. It brings data into the development environment, often through notebooks, data warehouses, and shared experimentation platforms. That breaks a core AppSec assumption: that dev environments contain little sensitive data and are easy to isolate. In AI pipelines, credentials, datasets, and model artifacts move across systems that were not originally designed for controlled software delivery. The result is a broader trust boundary and a larger set of assets that can be abused, leaked, or modified before anything reaches production.

Practical implication: Inventory AI development assets as privileged systems and apply access review, data classification, and secret handling controls before experimentation starts.

How autonomous agents create NHI-style execution risk

Autonomous AI agents extend the problem beyond inference. Once an LLM can call tools, retrieve context, and launch workflows, the system is no longer only producing outputs. It is exercising execution authority, which is the defining feature of a non-human identity in operational terms. Free-text prompts can alter behavior at runtime, and that means authorization is no longer tied only to the user session or the application role. The real security question becomes which tools the agent can reach, which data it can access, and how much damage it can do if its context is manipulated.

Practical implication: Treat agent permissions as scoped machine access and constrain tool use with least privilege, task boundaries, and explicit approval points.

Why standard AppSec and CloudSec controls miss AI-specific failures

Traditional scanners and runtime protections were built for deterministic software patterns. They do not inspect binary model files for poisoned training data, they do not understand prompt injection at the language layer, and they do not reliably see into specialized AI platforms such as notebooks, MLOps stores, or RAG pipelines. That gap matters because AI supply chain risk now spans models, datasets, prompts, and connectors. Security teams need controls that operate at the data, model, and agent layers rather than assuming the existing toolchain will catch AI-native abuse.

Practical implication: Add AI-specific validation, runtime protection, and red-teaming into the build and release path instead of relying on legacy AppSec gates.

Threat narrative

Attacker objective: The attacker wants to turn AI systems into trusted execution channels that expose data, move laterally through tools, and complete actions without raising immediate suspicion.

1. Entry occurs through exposed notebooks, shared data platforms, or compromised model supply chain artifacts that were never intended to hold sensitive credentials.
2. Escalation happens when an attacker abuses prompt injection, poisoned retrieval content, or over-privileged tool access to steer agent behaviour toward unintended actions.
3. Impact follows when the agent retrieves restricted data, triggers workflows, or leaks sensitive information at machine speed under legitimate-looking execution paths.

Breaches seen in the wild

• Moltbook AI agent keys breach — Moltbook breach exposed 1.5M AI agent keys.
• Cisco DevHub NHI breach — IntelBroker exploited exposed Cisco credentials, API tokens and keys in DevHub.

Read our 52 NHI Breaches Analysis report for a comprehensive view of breaches impacting Non-Human Identities including AI Agents.

NHI Mgmt Group analysis

AI application security is now an NHI governance issue, not just an AppSec issue. The article describes systems that can retrieve data, call tools, and execute workflows without direct human instruction. That is operationally closer to machine identity than to static application logic. Security teams should stop treating AI as a special-case workload and govern it through identity, privilege, and lifecycle controls.

Ephemeral AI context creates a new form of trust debt. The prompt, the model, the connector, and the retrieved data can all change runtime behaviour in ways traditional perimeter tools cannot see. That makes trust conditional and short-lived, which is exactly the environment where standing privilege becomes dangerous. Practitioners need to design for narrow, auditable access rather than broad ambient trust.

Data exposure in AI development environments is the real control failure. The article correctly notes that notebooks, training jobs, and AI platforms often hold secrets and production data in places developers consider temporary. That pattern is especially risky because it collapses development and production trust boundaries. The practical conclusion is to treat AI build systems as sensitive production-adjacent environments from the start.

Prompt injection and tool abuse should be managed as execution risks, not content risks. A model that is manipulated into calling a tool or revealing context has crossed from language safety into access control failure. That is where conventional content filters stop helping. Organisations need policy enforcement that governs what the agent can do, not just what it can say.

AI security programmes will converge on lifecycle governance. Discovery, data-flow mapping, secret handling, runtime guardrails, and pre-production red-teaming are not separate tasks anymore. They are the minimum components of an AI governance model that can support NHI-grade oversight. Teams that fail to connect those controls will accumulate hidden access paths faster than they can review them.

From our research:

• Only 1.5 out of 10 organisations are highly confident in their ability to secure NHIs, compared to nearly 1 in 4 for securing human identities, according to The State of Non-Human Identity Security.
• 85% of organisations lack full visibility into third-party vendors connected via OAuth apps, with 38% reporting no or low visibility and 47% reporting only partial visibility.
• That visibility gap is why practitioners should also review Top 10 NHI Issues as they map AI-connected access paths and third-party integrations.

What this signals

Ephemeral credential trust debt: AI systems accumulate short-lived but highly privileged access across notebooks, connectors, and agents, and that debt becomes operationally expensive fast. The right response is to reduce standing access, shorten approval paths, and bind every AI workflow to a named owner and a clear purpose.

With 85% of organisations lacking full visibility into third-party vendors connected via OAuth apps, the broader lesson is that AI governance will fail if teams cannot see delegated access clearly. That is why NHI programmes should align AI inventory work with the NIST Cybersecurity Framework 2.0 and external guidance such as MITRE ATLAS adversarial AI threat matrix.

Practical AI security now depends on connecting identity controls to runtime behaviour. Organisations that map data flow, privilege, and tool use together will be able to spot abuse earlier than teams that treat AI as another application tier.

For practitioners

• Inventory AI assets as privileged systems Map notebooks, training jobs, datasets, MLOps platforms, model registries, and agent tool connections as part of the privileged estate. Include owners, access paths, and data sensitivity so review cycles cover the full AI workflow.
• Restrict tool access for autonomous agents Apply task-scoped permissions, explicit approval points, and least-privilege rules to every tool an AI agent can invoke. Separate read-only retrieval from write or action permissions wherever possible.
• Embed AI-specific validation in delivery pipelines Add checks for prompt injection exposure, unsafe data retrieval, vulnerable packages, and untrusted model artifacts before release. Make the review cover both model inputs and tool-connected outputs.
• Treat development notebooks as sensitive workspaces Block plaintext secrets, limit open-access storage, and require stronger controls around experimentation environments that touch production data. Use access reviews on the same cadence as other high-value systems.
• Run continuous red-teaming against AI workflows Test model drift, adversarial prompts, retrieval poisoning, and agent misuse before production and after major workflow changes. Use the results to update guardrails, not just to produce a report.

Key takeaways

• AI application security now expands the NHI problem into data platforms, notebooks, and autonomous agents.
• Traditional AppSec and CloudSec controls miss the key failure modes because AI introduces probabilistic behaviour and tool-using execution.
• The practical answer is lifecycle governance: inventory, least privilege, runtime guardrails, and continuous red-teaming.

Key terms

• Non-Human Identity: A non-human identity is any digital identity used by software, services, workloads, or AI agents to authenticate and access systems. In practice, it includes API keys, tokens, certificates, service accounts, and agent credentials that can carry real privilege and require lifecycle governance.
• Agentic AI: Agentic AI refers to systems that can plan actions, call tools, retrieve data, and continue execution with limited human input. These systems behave like autonomous software entities, which means their access rights, audit trails, and approval boundaries must be managed like privileged identities.
• Prompt Injection: Prompt injection is an attack technique that manipulates an AI system through crafted input so it follows attacker intent instead of intended policy. The risk is not just incorrect output, but unauthorised tool use, data exposure, or workflow execution through the agent’s own privileges.
• RAG Poisoning: RAG poisoning is the corruption of retrieval-augmented generation inputs so an AI system pulls misleading or malicious context into its response path. Because the model trusts retrieved data as part of the working context, poisoned sources can change behaviour without directly compromising the model itself.

Deepen your knowledge

AI application security and NHI governance are core topics in our NHI Foundation Level course, the industry's only accredited NHI security programme. If your team is trying to bring AI into production without losing access control, this is a relevant starting point.

This post draws on content published by Noma Security: AI application security and the limits of traditional cybersecurity tools. Read the original.