Notifications
Clear all
Topic starter
24/07/2026 4:10 pm
TL;DR: Prompt extraction attacks succeeded 70% more often on NVIDIA Nemotron-3-Nano-30B than on Claude Opus 4.5 at roughly one-tenth the cost, according to Noma Security, while running fully self-hosted through NIM. The finding matters because red-team quality is now constrained by throughput, cost, and data handling, not just model cleverness.
NHIMG editorial — based on content published by Noma Security: an evaluation of NVIDIA Nemotron-3-Nano-30B versus Claude Opus 4.5 for AI red teaming
By the numbers:
- Noma Security reports that NVIDIA Nemotron-3-Nano-30B achieved a 14.17% attack success rate on prompt extraction, compared with 8.33% for Claude Opus 4.5.
- Noma Security says Nemotron ran at roughly 10× lower cost while staying fully self-hosted via NIM.
Questions worth separating out
Q: How should security teams use AI red teaming results in production governance?
A: Security teams should treat red team results as control evidence, not as a finished deliverable.
Q: Why do self-hosted AI testing environments matter for governance?
A: Self-hosted environments keep prompts, target responses, and policy logic inside the organisation, which reduces exposure to external APIs and supports tighter access control.
Q: What do teams get wrong about prompt extraction tests?
A: Teams often treat prompt extraction as a narrow benchmark rather than a recurring control.
Practitioner guidance
- Benchmark attacker models on coverage, not just cost Compare prompt generators on attack diversity, leak rate, and repeatability across multiple test classes, not only on one prompt-extraction benchmark.
- Classify red-team infrastructure as sensitive AI tooling Apply access controls, audit logging, and environment segregation to the red-teaming stack because it can expose target prompts, outputs, and policy logic during testing.
- Keep prompt-extraction tests in release gates Make prompt leakage one of the default checks for AI systems that embed hidden policies, tool instructions, or workflow rules.
What's in the full article
Noma Security's full analysis covers the operational detail this post intentionally leaves for the source:
- Benchmark methodology across additional attack plugins beyond prompt extraction, including the model selection criteria used for each test.
- The production integration path through Amazon Bedrock and the later self-hosted NIM plan, including deployment trade-offs.
- How Noma measures attack effectiveness across hundreds of adversarial prompts per deployment in its CI/CD workflow.
- The broader attack taxonomy that the red-teaming engine supports, including prompt injection, goal hijacking, and PII extraction.
👉 Read Noma Security's analysis of Nemotron versus Claude for AI red teaming →
Prompt extraction red teaming: are your AI controls keeping up?
Explore further
24/07/2026 4:13 pm
AI red teaming is becoming a throughput problem, not just a model-safety problem. When attack generation has to run inside CI/CD, the deciding factor is whether the test harness can keep pace with release velocity and still produce useful adversarial variation. That shifts assurance from periodic review to continuous adversarial simulation. Practitioners should treat red-team capacity as part of operational security, not a specialist side activity.
A question worth separating out:
Q: How do you decide between external APIs and self-hosted AI testing?
A: Choose external APIs when speed to start matters most and data sensitivity is low. Choose self-hosted testing when prompt content, outputs, or model interactions contain sensitive business logic, because privacy, latency control, and auditability become more important than convenience.
👉 Read our full editorial: AI red teaming needs cheaper attack generation, not cheaper assurance
