Notifications
Clear all
Topic starter
02/08/2026 12:15 pm
TL;DR: More than 80 new LLM attack scenarios, including prompt injection and jailbreak testing across AWS Bedrock and Azure OpenAI, are described in research on Exposure Validation to measure how models behave under adversarial pressure and where filtering, prompt isolation, and execution controls still fail, according to Cymulate. The analysis shows that LLM security cannot rely on content filters alone, especially as copilots, agents, and autonomous tools expand the attack surface.
NHIMG editorial — based on content published by Cymulate: Testing LLM Resilience, how Cymulate Exposure Validation simulates prompt injection and jailbreaks
By the numbers:
- Cymulate says its Exposure Validation now includes more than 80 new attack scenarios in private preview for existing customers.
Questions worth separating out
Q: How should security teams test enterprise LLMs for prompt injection risk?
A: Test the model inside the real application path, not in isolation.
Q: Why do prompt filters fail to stop many jailbreak attacks?
A: Prompt filters often fail because they look for known unsafe patterns, while attackers hide intent inside benign-looking wrappers such as code comments, markdown, roleplay, or layered instructions.
Q: What breaks when an LLM can override system instructions?
A: When an LLM can override system instructions, the application loses reliable policy enforcement.
Practitioner guidance
- Isolate trusted instructions from user content Store system policy outside the prompt stream where possible, and validate any model-facing instruction layer before runtime.
- Test for semantic jailbreaks in pre-production Run red-team scenarios that include obfuscation, markdown encapsulation, narrative camouflage, and multi-step override chains against every LLM release.
- Constrain tool and action permissions by default Treat every LLM-to-tool integration as a privileged path, and require explicit allowlists, scoped sessions, and output validation before any external action.
What's in the full article
Cymulate's full blog covers the operational detail this post intentionally leaves for the source:
- Modular YAML-driven attack scenario structure for prompt injection and jailbreak testing
- Execution flow examples across AWS Bedrock and Azure OpenAI test targets
- Verdict parsing and retry logic used to score malicious success consistently
- Examples of payload shaping strategies such as instruction commentary and narrative obfuscation
👉 Read Cymulate's analysis of LLM resilience testing and jailbreak validation →
Prompt injection and jailbreak testing: are your LLM controls keeping up?
Explore further
02/08/2026 1:17 pm
Instruction hierarchy is becoming a governance boundary, not just a prompt-engineering issue. When an LLM can be persuaded to ignore trusted instructions, the failure is structural: the application has not created a hard separation between policy and user content. That means security teams must treat prompt precedence as part of the control plane, especially where the model can initiate actions or expose data. The practitioner conclusion is simple: if instruction order can be negotiated, it is not yet enforced.
A question worth separating out:
Q: Who should be accountable when an LLM triggers an unauthorized action?
A: Accountability should rest with the team that granted the LLM its reach, defined its permissions, and allowed the action to proceed without adequate gating. The right governance model treats the model as a system component with delegated authority, not as a black box that sits outside ownership.
👉 Read our full editorial: LLM resilience testing shows why prompt injection still matters
