Notifications
Clear all
Topic starter
11/08/2026 3:17 pm
TL;DR: Generative AI behaves probabilistically rather than deterministically, so the same prompt can produce different outputs as context, retrieval, or configuration changes, according to Noma Security. That means traditional software security assumptions do not hold, and AI-aware threat modeling and runtime controls are now a governance requirement.
NHIMG editorial — based on content published by Noma Security: why AI systems are not traditional software
Questions worth separating out
Q: What breaks when security teams treat AI like traditional software?
A: The main failure is assuming the system will behave the same way every time.
Q: When should organisations prioritise AI-specific controls over generic appsec checks?
A: They should prioritise AI-specific controls as soon as a model influences decisions, customer interactions, or connected workflows.
Q: How can organisations tell whether an AI agent is operating outside its intended boundary?
A: Look for inconsistent classifications, premature tool calls, fabricated inputs, and responses that ignore structured guardrails.
Practitioner guidance
- Map AI control dependencies before deployment Identify every model, retrieval source, tool connection, and permission that can influence or extend the AI system.
- Add behaviour-based testing to release gates Test prompts, contexts, and retrieval combinations that can drive unsafe or misleading output.
- Constrain delegated tool access tightly Limit what an AI system can read, write, or trigger through connected services.
What's in the full article
Noma Security's full article covers the operational detail this post intentionally leaves for the source:
- The article's leader-friendly explanation of why probabilistic model behaviour changes the security model.
- The article's examples of how prompt phrasing, retrieval, and fine-tuning can alter output and risk.
- The article's board-level framing for explaining AI governance to non-technical stakeholders.
- The article's suggested way to map AI controls to the broader lifecycle of design, build, and runtime.
👉 Read Noma Security's analysis of why AI systems are not traditional software →
AI systems are not traditional software - are your controls keeping up?
Explore further
11/08/2026 3:29 pm
Deterministic security assumptions are the first control failure in AI programmes. The article is right to frame generative AI as a different system category, because fixed-rule security models do not map cleanly to probabilistic output. That difference matters for governance, because risk review processes that assume stable behaviour will understate the probability of unsafe responses. For AI programmes, the practitioner conclusion is simple: security evidence must cover behaviour variance, not just code integrity.
A question worth separating out:
Q: Who is accountable when an AI system makes a harmful decision?
A: Accountability should follow the identity chain that authorized, configured, or triggered the action, including the human owner, the platform team, and any delegated agent or tool account. If the organisation cannot name that chain, the governance model is too weak for regulated AI use.
👉 Read our full editorial: AI systems are not traditional software: why security models break
