Security teams should turn legal prohibitions into concrete test policies and attack cases, then red-team the model for outputs that would violate them. The practical goal is to detect harmful generations, prompt manipulation, and guardrail gaps before release. Testing should cover both direct requests and bypass attempts, including social scoring, emotion inference, profiling, and illegal instruction generation.
Why pre-deployment testing matters for prohibited AI behaviors
Testing for eu ai act prohibited behaviors is not ordinary quality assurance. The issue is whether an LLM can be induced, directly or indirectly, to generate outputs or support behaviours that cross a legal line, such as manipulative profiling, emotion inference in restricted contexts, or illegal instruction generation. The EU AI Act creates a governance boundary that teams must translate into measurable test cases before deployment, rather than hoping a generic safety filter will catch everything. See the EU AI Act regulatory framework for the policy context that defines those boundaries.
Security teams often treat model testing as a content moderation exercise, but prohibited-behaviour testing is broader because it has to examine prompt sensitivity, instruction hierarchy, and jailbreak resistance. It also has to distinguish between intended capability and unacceptable use. That means the team needs a test catalogue that maps legal prohibitions into observable failure conditions, then validates whether the model, surrounding application controls, and post-processing layers actually prevent the disallowed outcome.
In practice, many teams only discover these gaps when users find a bypass path that the original test plan never covered.
How to turn legal prohibitions into red-team test cases
The practical method is to convert each prohibited behaviour into a test objective, then express that objective through prompts, adversarial variants, and expected failure signals. For example, if a prohibition concerns social scoring, the test should check whether the model helps create ranking logic, classification rules, or decision support that would enable that function. If the concern is illegal instruction generation, the test should include straightforward requests, euphemisms, role-play prompts, and layered bypass attempts.
This is where model evaluation and application testing must be joined. A model may refuse one prompt while still leaking dangerous content through system prompt confusion, multi-turn manipulation, tool-mediated workflows, or incomplete guardrails. Teams should therefore test the full chain, not just the base model. The NIST AI 600-1 Generative AI Profile is useful here because it frames generative AI risks in operational terms that can be turned into evaluation criteria.
- Start with a prohibition inventory that names the exact behaviour to block.
- Write direct prompts, disguised prompts, and multi-turn evasion prompts for each item.
- Test refusal quality, not just refusal presence, because weak refusals can still leak useful guidance.
- Include application-layer paths such as retrieval, tools, and orchestration logic if the model can reach them.
- Record the smallest prompt that reproduces the failure so the control can be retested later.
The guidance breaks down when teams evaluate only the standalone model and ignore the deployed workflow that actually exposes the user to risk.
Where the hard cases and edge conditions usually appear
Tighter prohibited-behaviour testing often increases evaluation effort, requiring organisations to balance coverage against the cost of maintaining realistic adversarial cases. The most difficult cases are usually not the obvious unsafe prompts, but borderline requests where the model is coaxed into supporting a prohibited outcome indirectly. That includes role-play, translation, summarisation of harmful material, or policy-adjacent outputs that become disallowed once combined with downstream human or automated decision-making.
There is also a genuine consensus gap in how far teams should test inferred harm versus explicit harm. Some organisations focus narrowly on direct generation of prohibited content, while others also test whether the model enables the workflow that would make the prohibited behaviour operational. For deployment decisions, that difference matters because a model that never states a banned conclusion may still materially assist an unlawful use case. The relevant question is whether the model creates a dependable path to the prohibited outcome, not whether it uses the forbidden words.
Another edge case is the difference between model capability and policy enforcement. A model might be technically able to infer sentiment, emotion, or sensitive attributes, but the deployed system may prevent that capability from being used in a prohibited context. Security teams should test both layers separately so they can tell whether the risk sits in the model, the wrapper, or the governance process.
For teams building agentic or tool-using systems, the OWASP Top 10 for Agentic Applications 2026 is a strong companion reference because prohibited behaviour can emerge through orchestration, tool use, and delegated action rather than through plain text generation alone.
Standards & Framework Alignment
This section maps relevant standards and security frameworks to the operational risks and controls described in this guidance.
OWASP Agentic AI Top 10 address the attack surface, NIST AI 600-1 and CIS Controls v8 set the technical controls, and EU AI Act and ISO/IEC 42001:2023 define the regulatory obligations.
| Framework | Control / Reference | Relevance |
|---|---|---|
| EU AI Act | Article 5 — Prohibited AI Practices | The question is about testing for legally prohibited AI behaviors before deployment. |
| Recommendation — Translate each prohibited practice into a concrete red-team test case before release. | ||
| NIST AI 600-1 | Map, Measure, Manage — Generative AI Risk Functions | Pre-deployment probing and refusal validation fit generative AI risk evaluation. |
| Recommendation — Use risk evaluations to measure harmful outputs, bypasses, and guardrail failures before deployment. | ||
| ISO/IEC 42001:2023 | 6.1 — Actions to Address Risks and Opportunities | The task requires AI governance controls that turn legal limits into operating requirements. |
| Recommendation — Document prohibited behaviors as governance requirements and verify they are enforced in testing. | ||
| CIS Controls v8 | 18 — Penetration Testing | Adversarial testing of jailbreaks and bypasses is a pen-test style validation activity. |
| Recommendation — Red-team the model and surrounding workflow with realistic bypass prompts before release. | ||
| OWASP Agentic AI Top 10 | A1 — Agentic Access Control | Agentic or tool-using LLM deployments can surface prohibited behavior through delegated actions. |
| Recommendation — Test tool-enabled paths for unsafe action chains, not just plain-text refusals. | ||
Practitioner Guidance
What to prioritise: Test the exact prohibited behaviour first, then the bypasses that make it operational. A team gets the most value by prioritising the behaviours that are both legally sensitive and easiest to expose through ordinary user prompts.
What to verify: Verify that refusal is durable across paraphrase, role-play, multi-turn context, and prompt injection. Also verify that surrounding controls do not re-enable the same outcome through retrieval, tools, or post-processing.
Decision rule: If a model can be induced to support a prohibited outcome in any realistic deployment path, treat that as a release blocker until the path is removed or the risk is explicitly accepted at the appropriate governance level.
What good looks like: Good testing produces a traceable mapping from each legal prohibition to concrete prompts, expected refusals, observed failures, and a remediation owner. That evidence should be repeatable, not anecdotal.
Practitioner takeaway: The right question is not whether the model can usually refuse unsafe requests, but whether the full deployed system can be made to fail in a way that would matter under the EU AI Act.
Related resources from NHI Mgmt Group
Deepen Your Knowledge
Reviewed and updated by the NHIMG editorial team on September 9, 2026.
NHI Mgmt Group — the #1 independent authority on Non-Human Identity, IAM, and Agentic AI security. nhimg.org
