Join our Newsletter — 33% off our NHI Course

Could prompt injection and jailbreaks still slip past guardrails?

 

(@nhi-mgmt-group)
Member Moderator
Joined: 2 years ago
Posts: 20225
Topic starter  

TL;DR: Prompt injection, jailbreaks and poisoned models can all bypass text-only guardrails, Starseer argues in its blog post. Agent authority needs runtime checks on tools, provenance and model behaviour before actions execute.

Editorial analysis by NHI Mgmt Group, based on content published by Starseer: “Malicious Prompts and Models: Why Text Guardrails Fail”.

Key questions

Q: Why do agentic AI systems create more security risk than standard chatbots?

A: Agentic systems can turn model output into action, which means a bad instruction can affect code flow, tool use, and downstream state.

Q: How should organisations govern poisoned or modified models?

A: Treat model provenance as a production control.

Q: What fails when AI agent guardrails exist only in prompts and logs?

A: The control fails because prompts and logs are not enforcement points.

Practitioner guidance

  • Define AI model trust tiers Classify models by source, modification history, and allowed use case before they are connected to production workflows.
  • Inspect agent tool calls before execution Log and evaluate every outbound action, especially web fetches, API calls, and file operations that can move data or change state.
  • Separate prompt filtering from action control Use text filters only as one signal, and add runtime checks that can stop unsafe tool use even when the prompt appears benign.

Bottom line: Text-only guardrails are too narrow for agentic AI because they miss hidden intent, tool use, and poisoned model behaviour.

Glossary terms: Prompt Injection (Agentic), Jailbreak, Tool Poisoning, Mechanistic interpretability

Explore further

View Full Forum →  |  NHI Foundation Course →  |  Our Services →  |  Read the full analysis →


This topic was modified 1 week ago by NHI Mgmt Group

   
Quote
(@mr-nhi)
Member Moderator
Joined: 6 months ago
Posts: 20715
 

Text-only guardrails are a false boundary for agentic systems: once a model can browse, call tools, and alter files, the meaningful security question is no longer only what it said. The control problem shifts to what it was trying to do and which side effects it triggered. That means prompt filters may still have a role, but they cannot be the primary governance layer for AI systems that act on their own inputs.

A question worth separating out:

Q: How should security teams use an LLM judge to evaluate AI output without creating a brittle or gamable process?

A: Use an LLM judge as a scoring and screening layer, not as an unquestioned authority. Define a strict rubric, test it against sample outputs, and monitor for bias, hallucinated reasoning, and over-alignment with safe but unhelpful answers. The strongest implementations keep human review for edge cases and continuously validate that the judge still reflects real evaluation goals.

👉 Read our full editorial: Text guardrails fail when models can hide intent in actions


This post was modified 1 week ago by NHI Mgmt Group

   
ReplyQuote
Share:

Free weekly newsletter

Subscribe to the NHI & AI Identity Journal

The latest on NHI and Agentic AI security – articles, research, breaches, news and events every week.

Bonus 33% off our NHI Course when you subscribe.