Agentic AI changes security from static scans to live governance

At a glance

What this is: This opinion piece argues that generative AI turns security into a runtime governance problem, not a static pre-release check.

Why it matters: It matters because IAM, NHI, and AI governance teams now have to monitor live behaviour, access paths, and prompt-driven change as part of one operating model.

👉 Read Pillar Security's analysis of why static scanning fails for LLM-native applications

Context

Generative AI changes the security problem because the application itself now adapts through live prompts, live data, and rapid iteration. That shifts the control question from whether a system passed pre-release testing to whether it can be governed safely while it is already in use, especially where identity, data access, and runtime decision paths intersect.

For IAM and NHI teams, this is not just a model-quality issue. When prompts become the effective source code and outputs can change in minutes, the programme has to account for continuous validation, entitlement review, and monitoring of who or what can influence the runtime behaviour of the system.

Key questions

Q: How should security teams govern AI systems that change through live prompts?

A: Security teams should treat prompts as governed change objects, not informal text. That means version control, approval workflows, rollback capability, and audit trails for prompt edits that can affect output, data disclosure, or tool use. Without that discipline, the organisation cannot explain why the system behaved a certain way or who authorised the change.

Q: Why do static tests miss the real risks in generative AI applications?

A: Static tests miss the core risks because many failures only appear when the system is running with live users, live data, and adversarial inputs. Prompt injection, jailbreaks, and data leakage are emergent behaviours, so they need runtime monitoring and not just pre-release validation.

Q: What do security teams get wrong about deploying AI safely?

A: They often assume deployment marks the end of assurance, when it actually marks the beginning of continuous governance. AI systems need ongoing validation of outputs, access paths, and data exposure because behaviour can change with each interaction. Without that, the control model is already stale when the first live session starts.

Q: How can organisations reduce prompt-injection risk in production AI systems?

A: Organisations reduce prompt-injection risk by limiting what the model can see, validating untrusted input, and monitoring output for policy violations. They also need strict control over any tools or data sources the model can reach, because the attack becomes more dangerous when the model can act on manipulated context.

Technical breakdown

Why static scanning fails for LLM-native systems

Static scanning was built for software that behaves consistently after release. LLM-native systems behave differently because the same prompt, context, and retrieved data can produce different outputs depending on live user input and model state. That means traditional QA cannot reliably expose prompt injection, jailbreak attempts, or hidden data exposure paths. The real problem is not just code defect detection, but the way runtime context becomes part of the attack surface. In identity terms, the system is making decisions with inputs that are both dynamic and adversarial, which makes pre-deployment assurance incomplete.

Practical implication: treat static analysis as a baseline only and add runtime validation for prompt and output behaviour.

Prompt iteration has become a governance control surface

The article’s key point is that prompts now function like source code because small text changes can materially alter system behaviour. That creates a new governance surface for versioning, approval, and change control, especially when business users and product teams can modify prompts quickly. The security issue is not merely speed, but the collapse of the old assumption that meaningful behaviour changes require a formal release cycle. In practice, any system that can be reshaped in minutes needs controls that track prompt lineage, access, and rollback capability as tightly as application code.

Practical implication: apply change management, ownership, and rollback controls to prompts as if they were production code.

Continuous monitoring is now part of AI security architecture

The article reframes deployment as the start of security work, not the end. For AI systems, monitoring has to cover runtime outputs, data exposure, and model behaviour in real time because failures are often emergent rather than deterministic. That makes observability a security control, not just an operations function. For identity practitioners, the important connection is that access to models, tools, and live data must be monitored as part of one control plane. If the system can see sensitive data or act on behalf of users, governance has to extend into runtime execution and not stop at authentication.

Practical implication: build monitoring that links identity, data access, and output validation into a single runtime control process.

Threat narrative

Attacker objective: The objective is to redirect or weaken the model so it exposes data, bypasses guardrails, or produces harmful outputs inside a live application.

1. Entry occurs when a malicious or careless input is accepted into the live AI workflow and the system processes it as normal context.
2. Escalation follows when prompt injection or related manipulation changes the model’s behaviour and expands what data it can reveal or influence.
3. Impact appears when the system leaks sensitive information, produces unsafe content, or behaves outside the intended security boundary.

Breaches seen in the wild

• Moltbook AI agent keys breach — Moltbook breach exposed 1.5M AI agent keys.
• AI LLM hijack breach — attackers used stolen AWS access keys to hijack Anthropic LLM models on Bedrock.

Read our 52 NHI Breaches Analysis report for a comprehensive view of breaches impacting Non-Human Identities including AI Agents.

NHI Mgmt Group analysis

Static security controls were built for software that stays still, but LLM-native systems keep changing at runtime. This article shows why pre-release testing is no longer the centre of gravity when prompts, retrieval, and user input can reshape behaviour after deployment. The implication is that AI governance now has to assume live variability as the normal condition, not the exception.

Prompt versioning is now an identity and governance problem, not just an engineering convenience. When a few sentences can materially alter how a system behaves, the organisation needs a control model for authorship, approval, rollback, and accountability. That makes prompt lineage part of the security record, because the runtime policy surface has become as mutable as the application itself.

Runtime observability is the missing control when AI systems can ingest data and act on it in the same session. The article correctly points to continuous monitoring as the new security discipline, because the failure may only exist while the system is live. Practitioners should read that as a signal that monitoring, data access, and identity assurance must converge into one operating model.

Continuous AI security closes the gap between who can influence the system and what the system is allowed to reveal. That bridge matters across NHI, human, and AI governance because the same live workflow can involve users, service accounts, and agentic components. The practitioner conclusion is straightforward: if runtime influence is not governed, static assurance will keep missing the real control boundary.

Model behaviour is the asset, and the attack surface, in equal measure. The article’s strongest insight is that quality, safety, and security now share the same runtime substrate. That means identity teams can no longer separate access control from behavioural assurance, because the same entitlement can become either a productivity mechanism or a leakage path depending on live context.

From our research:

• 98% of companies plan to deploy even more AI agents within the next 12 months, despite documented rogue behaviour in 80% of current deployments, according to AI Agents: The New Attack Surface report.
• Only 52% of companies can track and audit the data their AI agents access, leaving 48% with a complete blind spot for compliance and breach investigation.
• That governance gap makes runtime monitoring a necessary control path, as shown in OWASP Agentic AI Top 10.

What this signals

Prompt governance is becoming a core control plane issue, not a content-management detail. As organisations move from experimentation to production, they will need clear ownership for who can edit prompts, approve changes, and roll back risky behaviour. The practical shift is toward governed runtime change, where identity, access, and model behaviour are reviewed together rather than separately.

AI security programmes should expect a convergence of application security, IAM, and observability. When a live model can ingest sensitive data and change behaviour in the same session, the old separation between security testing and operational monitoring stops holding. Practitioners should prepare for cross-functional control ownership, with security teams able to trace influence from user input to model action.

Our research shows 33% of organisations report AI agents accessing inappropriate or sensitive data beyond intended scope. That figure is a warning that behaviour drift is already a live governance issue, not a future concern. Teams that are still relying on static approval gates will need to move toward runtime policy enforcement and more explicit control over which inputs can shape output.

For practitioners

• Move security validation into runtime Add continuous checks for prompt injection, jailbreak attempts, and sensitive-data exposure after deployment, not just before release. Use production telemetry to spot behaviour drift that pre-release QA will miss.
• Version and approve prompts like production code Track prompt authorship, changes, and rollback paths in the same governance process you use for application releases. Require approval for prompt updates that can change access, disclosure, or tool-use behaviour.
• Bind identity controls to model and data access Review which service accounts, users, and applications can supply context, retrieve data, or trigger model actions. Limit the data the system can see and the actions it can take in the same control review.
• Create an incident path for emergent model behaviour Define who investigates when a live system starts leaking data, bypassing guardrails, or producing unsafe outputs. The response runbook should cover containment, rollback, and access revocation before the session continues.

Key takeaways

• Generative AI turns security into a live governance problem because the system’s behaviour can change after deployment.
• Prompt injection, jailbreaks, and data leakage are runtime failures that static QA will keep missing.
• Teams need versioned prompt control, identity-aware access limits, and continuous monitoring to govern AI safely in production.

Key terms

• Prompt Governance: Prompt governance is the set of controls used to manage who can create, edit, approve, and roll back prompts in a live AI system. It treats prompts as change-controlled artefacts because small text changes can materially alter model behaviour, data exposure, and tool use.
• Runtime Monitoring: Runtime monitoring is continuous observation of a system while it is serving real users and data. In AI environments it goes beyond uptime checks and looks for output drift, policy violations, prompt injection, jailbreak attempts, and unexpected disclosure of sensitive information.
• Model Behaviour Drift: Model behaviour drift is the change in an AI system’s outputs or actions over time as prompts, context, data, or model versions change. It matters because the same application can behave safely in testing and unsafely in production, especially when live data shapes the response.
• Behavioural Assurance: Behavioural assurance is the practice of proving that an AI system acts within acceptable boundaries under real operating conditions. It combines testing, monitoring, and governance so that quality and safety are assessed as live properties rather than as one-time release outcomes.

Deepen your knowledge

NHI governance, agentic AI identity, and machine identity security are core topics in our NHI Foundation Level course, the industry's only accredited NHI security programme. If you are responsible for identity security strategy or programme maturity, it is worth exploring.

This post draws on content published by Pillar Security: From Static Scanning to Recursive Loops: Lessons from a Decade in Data Science and AI. Read the original.