What breaks when model scanners rely on blocklists instead of structural analysis?

Why This Matters for Security Teams

Blocklist-based scanners create a false sense of safety because they only recognize known-bad names, not dangerous behaviour. That works until a model route, wrapper, or opcode is renamed, nested, or composed in a way the scanner has never seen. Security teams then approve code that looks inert on paper but becomes executable at runtime.

This is especially risky in NHI-heavy environments where model outputs, automation pipelines, and secrets handling converge. NHI Mgmt Group notes that 97% of NHIs carry excessive privileges, which means a missed bypass can quickly become a high-impact path to execution or data access in systems that already have too much authority. The broader lesson matches Ultimate Guide to NHIs: visibility and governance matter, but they have to be paired with analysis that understands structure and intent. Static allow and deny lists are brittle when the thing being inspected can generate new execution paths on the fly. In practice, many security teams encounter the bypass only after a benign-looking package has already been promoted into a trusted workflow.

How It Works in Practice

Structural analysis evaluates the shape of the model, the relationships between functions, and the runtime effects of wrappers, callbacks, and indirection. Instead of asking whether a known bad token appears, it asks whether the code can resolve into unsafe behaviour. That is a much better fit for scanning models, agent toolchains, and embedded orchestration logic where execution is often assembled dynamically.

Practitioners should treat this as a detection problem, not a naming problem. A stronger workflow usually combines several checks:

• Parse the model or artifact into its underlying structure, then trace call graphs and data flow.
• Resolve aliases, imports, decorators, and generated code before scoring risk.
• Flag behaviour such as shell execution, file system access, network calls, secret retrieval, and tool invocation even when wrapped in harmless names.
• Use policy rules from NIST SP 800-53 Rev. 5 Security and Privacy Controls to require review of code paths that can reach privileged actions.

For NHI and agentic workflows, the same logic applies to service identities and automation accounts: the scanner should assess what the workload can do, not whether it contains a banned string. The Ultimate Guide to NHIs is useful here because it frames identity risk around lifecycle, privilege, and visibility rather than single indicators. This guidance tends to break down in highly dynamic environments where code is generated at runtime and the scanner cannot reliably resolve the final execution path.

Common Variations and Edge Cases

Tighter blocklists often reduce obvious abuse patterns, but they also increase maintenance overhead and false negatives, requiring organisations to balance simplicity against coverage. That tradeoff becomes sharper when models are updated frequently, tool names change, or adversaries intentionally repackage dangerous behaviour into safe-looking abstractions.

There is no universal standard for structural model scanning yet, so current guidance suggests pairing behavioural analysis with staged approvals and human review for high-risk paths. In practice, a scanner should be conservative when it sees dynamic dispatch, reflection, code generation, or chained tool execution, because those patterns are exactly where a blocklist loses fidelity. The challenge is similar to the NHI problem described in Ultimate Guide to NHIs: the control that looks adequate in a static review can fail once privilege is exercised in context. For baseline detection expectations, NIST SP 800-53 Rev. 5 Security and Privacy Controls supports layered monitoring, but it does not replace structural inspection of the artifact itself.

Blocklists are still useful as a fast first pass, but they should be treated as one signal among several, not as the decision engine for whether a model or workload is safe.

Related resources from NHI Mgmt Group

• What breaks when security teams rely on model output instead of verifying the authorization event?
• What breaks when MCP servers rely on custom middleware instead of a standard interceptor model?
• What breaks when teams rely on symptom-level bug reports instead of root-cause analysis for AI workflows?
• What breaks when organisations rely on direct model access instead of a gateway?