Text guardrails fail when models can hide intent in actions

At a glance

What this is: NHI Mgmt Group's analysis, drawing on Starseer's blog post: Text-based AI guardrails fail because they cannot reliably detect prompt injection, jailbreaks, or malicious model behaviour hidden behind normal-looking outputs.

Why it matters: For IAM and AI security teams, the implication is that governance has to extend beyond prompts to model provenance, tool execution, and the identity and authority of agent actions.

Context

AI guardrails are often evaluated as if the model were only a text interface, but that framing misses the real security boundary. Once an agent can browse, call tools, and change state, the control problem shifts from screening words to governing action. That makes model security part of the same dependency and trust problem seen in software supply chains, not a standalone chatbot issue.

The article also sits squarely in the intersection of AI governance and AI supply chain security because the risk is not only what the model says, but where it came from, how it was tuned, and what it is allowed to do at runtime. That is a more realistic way to think about security for agentic systems than filtering text at the perimeter.

Key questions

Q: What fails when AI agent guardrails exist only in prompts and logs?

A: The control fails because prompts and logs are not enforcement points. An agent can still attempt or complete risky actions if it is allowed to read files, call tools, or execute commands before any denial occurs. Effective governance has to block the action at runtime, not just document the policy after the fact.

Q: Why do agentic AI systems create more security risk than standard chatbots?

A: Agentic systems can turn model output into action, which means a bad instruction can affect code flow, tool use, and downstream state. Standard chatbots usually stop at text generation. Agentic AI adds execution authority, so compromise can lead to disclosure, unauthorized changes, or misuse of credentials.

Q: How should organisations govern poisoned or modified models?

A: Treat model provenance as a production control. Know where the model came from, what training or fine-tuning changed its behaviour, and whether any post-release modification could have introduced unsafe actions or refusals. If the lineage is unclear, the model should not be trusted for sensitive use until it is revalidated.

Q: How should security teams use an LLM judge to evaluate AI output without creating a brittle or gamable process?

A: Use an LLM judge as a scoring and screening layer, not as an unquestioned authority. Define a strict rubric, test it against sample outputs, and monitor for bias, hallucinated reasoning, and over-alignment with safe but unhelpful answers. The strongest implementations keep human review for edge cases and continuously validate that the judge still reflects real evaluation goals.

Technical breakdown

Why text filters fail against jailbreaks and prompt injection

Regex filters, classifiers, and LLM-as-a-judge systems all inspect the literal prompt or the generated output. That is useful for obvious abuse, but it breaks down when an attacker rewrites the same intent with synonyms, encoding, hidden text, or roleplay. Prompt injection is especially hard because malicious instructions can arrive through retrieved documents or webpages, so the application believes it is processing benign external content. The core problem is that these controls are text classifiers, not intent or behaviour controls. They can reduce noise, but they do not reliably capture what the model is being asked to do.

Practical implication: Treat text filtering as a narrow signal, not a boundary, and move high-risk decisions to controls that inspect runtime behaviour.

Why agent tool calls create a larger security surface than chat outputs

Traditional guardrails assume the primary risk is unsafe language. Agentic systems widen the attack surface because the model can trigger retrieval, browse websites, call APIs, and modify files. That means the dangerous act may be the tool invocation, not the sentence the model emits. A model can produce a normal answer while issuing a hidden fetch, exfiltration attempt, or other side effect in the background. Security therefore has to track the chain from intent to action, not just the visible dialogue. This is where agent identity, tool permissions, and action telemetry become security controls rather than implementation details. See the Agentic AI Security Policy Template.

Practical implication: Classify tool access by risk and inspect every agent action before execution, especially where data exfiltration or privilege escalation is possible.

Why model poisoning changes the trust model for AI supply chains

Internal poisoning means the model itself is compromised, either during training or after release through fine-tuning or tampering. In that case, a harmless prompt can still produce malicious behaviour because the unsafe logic is embedded in the weights. This defeats prompt-only controls because the attack is no longer in the input stream. The governance issue is provenance: if you cannot trust the model source, training lineage, or post-training modifications, you cannot assume benign behaviour from its output. That makes model acquisition and validation a supply-chain problem as much as an AI safety problem. See the AI Supply Chain Security and AI-BOM Guide.

Practical implication: Require provenance checks and release controls for models before deployment, not just prompt-layer screening after integration.

Threat narrative

Attacker objective: The attacker wants the model to bypass its intended safeguards and carry out hidden actions that expose data or manipulate downstream systems.

1. Entry occurs through either a crafted prompt or contaminated external content that reaches the model’s context at runtime.
2. Escalation happens when the model accepts hidden instructions, roleplay framing, or compromised weights that override the intended behaviour.
3. Impact occurs when the agent performs unauthorised tool calls, leaks data, or executes actions the organisation never approved.

Breaches seen in the wild

• LiteLLM PyPI package breach: LiteLLM PyPI supply chain attack, credentials stolen from users.

Read and download The State of NHI & AI Agent Breach Report 2026, covering 200+ breaches impacting Non-Human Identities including AI Agents.

NHI Mgmt Group analysis

Text-only guardrails are a false boundary for agentic systems: once a model can browse, call tools, and alter files, the meaningful security question is no longer only what it said. The control problem shifts to what it was trying to do and which side effects it triggered. That means prompt filters may still have a role, but they cannot be the primary governance layer for AI systems that act on their own inputs.

Model provenance is now part of access governance: a poisoned or altered model is functionally a compromised identity in the AI stack. If the model’s lineage, fine-tuning history, and release path are not controlled, the organisation is trusting an opaque actor to make decisions and take actions. That is why AI supply chain review belongs alongside identity and privilege governance, not as a separate niche concern.

Starseer’s core challenge to the market is conceptual, not product-specific: defenders need to stop treating AI risk as a content moderation problem and start treating it as an execution-control problem. The important distinction is between seeing words and governing effects. Practitioners should expect the strongest AI security programmes to converge with agent identity policy and runtime control rather than rely on text classification alone.

Malicious model behaviour is a governance failure before it is a detection problem: when refusal removal or backdoors are possible, the organisation has already lost part of the trust chain. The right response is to verify what a model is allowed to do before production use and to continuously validate that behaviour stays within bounds. In practice, that makes runtime inspection and release governance inseparable.

Read next: AI Supply Chain Security and AI-BOM Guide

What this signals

Action-based AI governance is becoming the real control plane: organisations will increasingly need to monitor what an agent is authorised to do, not just what it is allowed to say. That means the operational boundary moves from prompt filtering to tool permissions, provenance checks, and runtime enforcement.

AI security programmes will converge with identity governance: agent identity, delegated authority, and model provenance are becoming inseparable from access decisions. The practical consequence is that security teams will need policies for who or what can act, under which conditions, and with what evidence trail.

Behavioural inspection will outlast text moderation as the primary design pattern: the more autonomous the system becomes, the less useful static word filtering becomes. Future controls will focus on execution traces, constrained tool use, and whether the system’s actions match approved intent.

For practitioners

• Define AI model trust tiers Classify models by source, modification history, and allowed use case before they are connected to production workflows.
• Inspect agent tool calls before execution Log and evaluate every outbound action, especially web fetches, API calls, and file operations that can move data or change state.
• Separate prompt filtering from action control Use text filters only as one signal, and add runtime checks that can stop unsafe tool use even when the prompt appears benign.
• Validate model provenance after fine-tuning Require approval for new weights, post-training modifications, and third-party models before they are promoted into sensitive environments.

Key takeaways

• Text-only guardrails are too narrow for agentic AI because they miss hidden intent, tool use, and poisoned model behaviour.
• The article’s strongest evidence is that jailbreaks, prompt injection, and modified models can all bypass controls built around text inspection.
• The practical response is to govern model provenance, inspect actions before they execute, and treat agent authority as an access-control problem.

Related NHI Mgmt Group resources: AI Supply Chain Security and AI-BOM Guide · Agentic AI Security Policy Template · AI Security Maturity Model

Key terms

• Prompt Injection (Agentic): An attack where malicious instructions are embedded in content that an AI agent reads, causing the agent to execute unintended actions using its own legitimate credentials. A primary vector for agent goal hijacking and identity abuse.
• Jailbreak: A prompt or input transformation that bypasses a language model’s safety restrictions and causes it to produce output it would normally refuse. In operational settings, a jailbreak matters because the model may be embedded in a workflow, making the bypass a pathway to broader misuse, not just a bad response.
• Tool Poisoning: Tool poisoning is an attack in which malicious instructions are hidden inside tool descriptions, examples, or schemas that an AI agent reads when deciding what to do. The danger is not only in the tool's code, but in the metadata that shapes the agent's behaviour and trust decisions.
• Mechanistic interpretability: Mechanistic interpretability is the effort to explain how a model produces its outputs by understanding internal structure and behaviour. It is still a research area, not a dependable operational control. For security teams, its limitation is important because inspectable weights do not automatically reveal hidden triggers or backdoors.

Deepen your knowledge

NHI Foundation Level course, the industry's only accredited NHI security programme, covers NHI governance, machine identity security, and secrets management. It helps practitioners connect identity controls to the broader security architecture their programmes depend on.