Notifications
Clear all
Topic starter
05/07/2026 6:52 pm
TL;DR: TrojAI’s interview traces how adversarial ML moved from a niche model-testing problem to a runtime security issue as organisations push AI into production, including agentic AI systems, according to TrojAI. The practical shift is that security teams now need to govern build-time and runtime exposure together, not as separate concerns.
NHIMG editorial — based on content published by TROJ.AI: AI Security When AI Became the Target, an interview with James Stewart
Questions worth separating out
Q: How should security teams govern AI systems that behave unpredictably in production?
A: Security teams should govern production AI as a live control problem, not a one-time validation problem.
Q: Why do build-time AI tests fail to fully reduce production risk?
A: Build-time tests only prove how a model behaved in a controlled environment.
Q: When should organisations treat AI security as part of identity governance?
A: Organisations should treat AI security as identity governance whenever an AI system can access tools, call APIs, or trigger downstream actions.
Practitioner guidance
- Separate pre-deployment testing from runtime control Use red teaming to find weaknesses before launch, but keep runtime monitoring in place for live input abuse, misuse, and unexpected model behavior.
- Map tool reach for any AI system that can act Document every downstream system, API, or workflow an AI system can influence, then restrict that reach to the minimum necessary for the task.
- Treat agentic AI as an access governance issue Apply approval, logging, and scope boundaries to AI systems that can chain actions so execution authority is explicit and reviewable.
What's in the full article
TROJ.AI's full article covers the interview detail this post intentionally leaves for the source:
- James Stewart’s origin story from computer vision to AI security, including the specific turning points that changed his view of adversarial ML.
- The company’s view of how NIST, OWASP, MITRE ATLAS, and CSA shaped the AI security landscape.
- The split between TrojAI Detect and TrojAI Defend, including how the vendor positions build-time red teaming versus runtime defense.
- Examples of enterprise deployment scale, including AI environments with broad application coverage and audit scrutiny.
👉 Read TROJ.AI’s interview on AI security, adversarial ML, and runtime defense →
AI security at runtime: are your controls keeping up?
Explore further
05/07/2026 7:12 pm
AI security is no longer a model-quality problem alone. The article’s core lesson is that adversarial pressure moves the control point from training-time confidence to runtime governance. That matters because enterprises do not deploy AI in static conditions, and the threat surface grows once models are embedded in live workflows. Practitioners should treat AI security as an operating condition, not a pre-launch test result.
A few things that frame the scale:
- 85% of organisations lack full visibility into third-party vendors connected via OAuth apps, according to The State of Non-Human Identity Security.
- In the same study, only 1.5 out of 10 organisations are highly confident in their ability to secure NHIs, which shows how weak the governance baseline remains.
A question worth separating out:
Q: What should teams do if an AI system can chain actions across workflows?
A: Teams should restrict the system’s execution scope, require explicit approval for high-impact actions, and monitor for chained behavior that turns one model decision into multiple downstream events. The goal is to prevent a single bad interaction from compounding.
👉 Read our full editorial: AI security is moving from model testing to runtime defense
