Notifications
Clear all
Topic starter
21/08/2026 4:54 pm
TL;DR: Microsoft Foundry AI red teaming, tracing, online evaluations, and prompt optimisation can be chained into a closed-loop workflow that turns failed jailbreak and safety probes into better system prompts, according to Arize. The article is less about a tool and more about operationalising continuous AI security testing with measurable regression handling and human review.
NHIMG editorial — based on content published by Arize: How To Improve AI Agent Security with Microsoft’s AI Red Teaming Agent in Microsoft Foundry
By the numbers:
- 80% of organisations report their AI agents have already performed actions beyond their intended scope, including accessing unauthorised systems, sharing sensitive data, and revealing credentials.
Questions worth separating out
Q: How should organisations red team AI models before production?
A: Use both automated and manual testing so you cover scale and adversarial nuance.
Q: Why do AI red team failures need to be tracked as governance evidence?
A: Because a failed prompt often reveals a repeatable weakness rather than an isolated error.
Q: What do organisations get wrong when they automate prompt optimisation?
A: They often confuse suggestion generation with safe deployment.
Practitioner guidance
- Instrument red-team runs with end-to-end tracing Capture the original prompt, model response, safety label, and remediation path so each failure can be analysed later as evidence rather than anecdote.
- Build a curated regression dataset from failed prompts Separate obvious refusals from subtle failures, then curate a golden dataset that reflects the specific patterns your red-team probes exposed.
- Require human approval before prompt changes reach production Use prompt optimisation to propose candidate updates, but keep human review in the loop for safety, policy, and rollback decisions before deployment.
What's in the full article
Arize's full article covers the operational detail this post intentionally leaves for the source:
- Step-by-step setup for the Microsoft Foundry red teaming agent and Arize tracing instrumentation.
- Example notebook flow for converting failed red-team runs into a curated regression dataset.
- Prompt Hub optimisation workflow showing how successive prompt versions are generated and compared.
- Before-and-after scoring results that quantify the effect of the optimisation loop.
👉 Read Arize's walkthrough of AI red teaming and prompt optimisation in Microsoft Foundry →
AI red teaming and prompt optimisation: are your controls keeping up?
Explore further
21/08/2026 5:12 pm
AI red teaming is becoming a governance control, not just a testing exercise. The article shows that the value is not in generating adversarial prompts alone, but in converting failures into auditable improvements. That is an AI security and model governance pattern, but it also intersects with identity because the behaviour of an AI system acting on behalf of users can create access, data, and privilege risk. Teams should treat red teaming output as control evidence, not as a one-off safety report.
A question worth separating out:
Q: How do you know whether AI red teaming is actually improving governance?
A: Look for repeatable reductions in high-severity findings, clearer ownership of model and tool permissions, and evidence that tests are blocking risky releases. If findings are interesting but do not change access scope, secrets handling, or deployment decisions, the programme is producing noise rather than control.
👉 Read our full editorial: AI red teaming exposes prompt failure modes in agent security
