Llama 4 red-teaming exposes model security and compliance gaps

At a glance

What this is: VirtueAI’s red-teaming analysis finds Llama 4 Scout and Maverick still show meaningful weaknesses in compliance, privacy, code generation, and multimodal security.

Why it matters: For AI governance, IAM, and security teams, the issue is not model adoption itself but whether controls, guardrails, and review processes can contain unsafe outputs and delegated misuse.

By the numbers:

• VirtueAI says its red-teaming platform uses over 100 specialized algorithms to test AI models across multiple safety and security domains.

👉 Read VirtueAI's red-teaming analysis of Llama 4 Scout and Maverick

Context

AI model red-teaming is the discipline of trying to break a model before users, attackers, or downstream workflows do. In this case, the primary governance gap is not only model quality, but whether an organisation can prove that safety, privacy, and abuse-resistant controls are strong enough for real deployment. That matters to AI governance, IAM, and NHI teams because model access increasingly connects to tools, data, and delegated actions.

VirtueAI’s analysis of Llama 4 Scout and Maverick is best read as a warning about operational readiness, not as a simple model ranking. Where a model can leak sensitive data, generate exploitable code, or respond poorly to jailbreaks, the surrounding control plane becomes as important as the model itself. That is typical of modern AI deployment risk, especially when models are embedded into business workflows and agentic systems.

Key questions

Q: How should security teams govern AI models that can call tools and access data?

A: Security teams should govern AI models as non-human identities with named owners, limited scope, short-lived credentials, and continuous authorization. The critical shift is to treat every tool call, data read, and update path as a privileged action that can be logged, revalidated, and revoked. Without that discipline, model risk becomes identity risk.

Q: Why do multimodal AI systems create a different governance problem from text-only models?

A: Multimodal systems create a different governance problem because the visual channel can alter internal activations before the final response is generated. That means the trust boundary is not only the prompt box. It also includes image uploads, scanners, and any upstream system that passes visual inputs into inference.

Q: What do enterprises get wrong about AI red teaming maturity?

A: Many teams stop at attack simulation and assume the test itself is the control. In practice, maturity depends on whether findings feed monitoring, policy enforcement, and audit-ready reporting in production. A strong programme reduces risk after the test, not just during the assessment window.

Q: How do organisations decide whether a model is safe enough to deploy?

A: They should tie deployment to explicit test evidence, not to model enthusiasm or a favourable benchmark alone. Safe enough means the model has passed cohort thresholds, invariance checks, and adversarial review for the decisions it will influence. If any of those fail, the model should stay out of production until the gap is remediated.

Technical breakdown

How red-teaming exposes privacy and compliance failure modes

Red-teaming is structured adversarial testing that probes whether a model can be induced to violate policy, reveal protected data, or behave in ways that create regulatory exposure. In practice, it combines prompt injection, extraction attempts, jailbreaks, and scenario testing across content, privacy, and safety domains. For governance teams, the key signal is not whether a model can answer well in normal use, but whether it resists adversarial pressure when the model is placed into real workflows, shared interfaces, or retrieval-augmented systems.

Practical implication: Use red-team findings as deployment gates, not post-launch commentary.

Why code generation risk is a security problem, not just a model-quality issue

Code generation risk matters because an AI system that emits exploitable code can accelerate insecure development, embed malicious logic, or reduce the time required to weaponise a weakness. That risk sits at the intersection of application security, AI governance, and software supply chain control. If a model cannot reliably reject unsafe code patterns or adversarial instructions, then downstream engineering teams inherit a new source of insecure-by-default output that must be inspected like any other untrusted artifact.

Practical implication: Treat AI-generated code as untrusted input and scan it before merge or execution.

Multi-modal jailbreaks and the expansion of the attack surface

Multi-modal jailbreaks exploit the fact that models processing images and text can be manipulated through visual prompts, embedded text, or cross-modal instruction conflicts. This expands the attack surface beyond the chat interface and into files, screenshots, documents, and image-based workflows. Security teams should view multimodal capability as a governance decision, because every new input mode increases the number of places where policy bypass, hidden instructions, or data extraction can occur.

Practical implication: Constrain multimodal permissions to use cases that can tolerate higher prompt-injection risk.

Threat narrative

Attacker objective: The attacker aims to turn the model into a reliable source of unsafe output, sensitive disclosure, or operational misuse at scale.

1. Entry begins when an attacker submits adversarial prompts, jailbreak content, or multimodal inputs designed to bypass model safeguards and policy filters.
2. Escalation occurs when the model is induced to reveal sensitive data, produce unsafe code, or follow instructions that override intended safety boundaries.
3. Impact follows when those outputs are used to leak information, accelerate abuse, or corrupt downstream application and governance decisions.

NHI Mgmt Group analysis

Model safety is now an access-control problem as much as a content-safety problem. When AI systems can retrieve data, call tools, or influence workflows, unsafe outputs become a governance issue for IAM, PAM, and policy enforcement. Red-teaming therefore needs to test not only what the model says, but what it can cause the surrounding system to do. Practitioners should treat model behaviour as part of the access perimeter.

Code generation weakness creates a new insecure software supply chain inside the development workflow. A model that emits exploitable code shifts risk from the prompt layer into the engineering pipeline, where it can be copied into applications, infrastructure scripts, or automation. That makes secure code review, policy checks, and approval gates essential. The practical conclusion is that AI-generated code must be governed like third-party code.

Multimodal capability expands policy bypass opportunities faster than most governance programmes can adapt. Image inputs, embedded text, and cross-modal instructions create more ways to hide malicious intent than text-only systems do. This is not just a model-safety issue, because the control gap often sits in the surrounding workflow and data handling. Teams should re-evaluate whether multimodal use is justified for each business process.

Named concept: safety-to-governance drift. This is the gap that appears when a model looks acceptable in benchmark settings but proves harder to control once connected to real tools, data, and users. It is a governance failure because the risk moves from evaluation into runtime operation. Practitioners should demand runtime evidence, not only red-team summaries.

For identity and AI governance programmes, the real question is who can delegate to the model and what the model can delegate onward. That includes secrets exposure, tool use, retrieval permissions, and downstream action boundaries. Where those controls are unclear, the model becomes an identity-adjacent risk surface that traditional app reviews will miss. Teams should map model privileges before they expand deployment.

What this signals

AI governance programmes will increasingly need to prove not just model performance, but model containment. As models gain tool access and wider input modes, the operational boundary matters more than the benchmark score, especially where retrieval or action can reach sensitive systems.

Safety-to-governance drift: once a model moves from test prompts into real workflows, the control problem changes. That means release management, logging, and access scoping need to sit alongside the AI review board, not after it.

For teams managing identity-linked AI workflows, this is a cue to align model permissions with least privilege and to use the NIST SP 800-63 Digital Identity Guidelines where human authentication or delegated access is part of the flow.

For practitioners

• Set red-team release gates for model deployment Block production use until the model has passed adversarial testing for privacy leakage, unsafe code generation, and multimodal jailbreak resistance under your own workflows.
• Restrict model tool and data access by use case Limit retrieval, file access, and external action permissions to the smallest set needed for the workflow, especially where the model can influence sensitive systems.
• Review AI-generated code as untrusted input Route generated code through the same security checks used for third-party code, including static analysis, secrets scanning, and human approval before deployment.
• Separate multimodal features from high-risk workflows Disable image or document inputs where hidden instructions or extraction attempts would materially increase business or regulatory exposure.

Key takeaways

• Llama 4 Scout and Maverick still show control gaps in privacy, code generation, and multimodal safety that matter for production governance.
• The strongest finding is not model ranking alone, but the need to test how a model behaves once it is connected to tools, data, and workflows.
• Practitioners should gate deployment on adversarial testing, least-privilege access, and continuous review of AI-generated output.

Key terms

• Red Teaming: Red teaming is structured adversarial testing used to find how an AI system fails under realistic misuse or attack conditions. In AI security, it is a discovery method, not a proof of safety, because probabilistic behaviour and changing models prevent any lasting guarantee.
• Multimodal Jailbreak: A bypass technique that uses images, documents, embedded text, or mixed-input prompts to evade a model’s safety rules. It matters because controls built for text-only systems often do not hold when the model can process multiple input types at once.
• Safety-to-Governance Drift: The gap that appears when a model seems acceptable in a test environment but becomes materially harder to govern once connected to tools, data, and users. The risk is operational rather than theoretical, because runtime context changes what the model can expose or influence.
• AI-Generated Code Risk: AI-generated code risk is the possibility that model-produced output introduces insecure credential handling, hidden secrets, or repeated patterns that weaken control design. The risk is operational, because the output can be merged quickly and at scale before humans catch the problem.

What's in the full report

VirtueAI's full analysis covers the model-by-model red-teaming detail this post intentionally leaves at summary level:

• Per-domain scoring for compliance, privacy, bias, hallucination, and multimodal resistance across Llama 4 Scout and Maverick.
• Comparative red-team observations against GPT-4.5 that show where the risk boundaries differ in practice.
• Examples of the prompt patterns and test categories used in VirtueRed assessments.
• Operational guidance for teams evaluating whether a model is ready for broader deployment.

👉 The full VirtueAI post includes the domain-by-domain findings and model comparison details.

Deepen your knowledge

The NHI Foundation Level course, the industry's only accredited NHI security programme, covers NHI governance, secrets management, and machine identity security. It helps practitioners connect identity controls to the broader security decisions that AI systems now depend on.