Lies-in-the-Loop attack

A Lies-in-the-Loop attack is a deception pattern that uses misleading code, dependencies, or feedback signals to persuade an AI assistant and human reviewer that unsafe behaviour is acceptable. The goal is to corrupt the decision process, not just hide malicious logic in source code.

Expanded Definition

Lies-in-the-Loop attack is an adversarial pattern in which false or misleading artefacts are inserted into the software development or AI review path so that an assistant, reviewer, or automated guardrail concludes unsafe behaviour is acceptable. The deception can appear in code comments, dependency metadata, prompt context, test fixtures, policy text, changelogs, or other feedback signals that shape judgement. In practice, the target is not only the system output but the reasoning process that authorises it.

This matters because AI-assisted development and agentic workflows increasingly rely on contextual evidence to approve code, execute tools, or recommend actions. That makes the attack broader than classic code obfuscation. It overlaps with social engineering, supply chain tampering, and prompt injection, but the defining feature is the use of lies inside the loop that determines trust. For threat modelling, the most useful adjacent reference is the adversarial AI framing used by MITRE ATLAS adversarial AI threat matrix, although industry usage is still evolving and no single standard governs this term yet.

The most common misapplication is treating Lies-in-the-Loop as ordinary malicious code hiding in a repository, which occurs when teams miss the misleading evidence that persuaded the reviewer or AI assistant to approve it.

Examples and Use Cases

Implementing review automation rigorously often introduces a trust tradeoff, requiring organisations to weigh faster AI-assisted decisions against the risk of deceptive context that nudges those decisions in the wrong direction.

• A pull request includes benign-looking tests and comments that claim a dangerous function is only used in internal diagnostics, causing an AI code reviewer to downgrade the risk.
• A dependency manifest or package description is altered to present a backdoored library as a routine utility, shifting human or machine review away from deeper inspection.
• An agent receives prompt context that quotes fabricated policy language, persuading it that a risky tool action is already authorised.
• A build pipeline feeds synthetic telemetry or misleading changelog notes into an approval workflow, so the final reviewer assumes the change has been validated elsewhere.
• An attacker embeds deceptive explanations in code adjacent to suspicious logic, making the unsafe behaviour appear justified during a fast triage cycle informed by MITRE ATT&CK Enterprise Matrix style analysis.

Defenders increasingly compare these scenarios with broader AI abuse patterns described in the Anthropic — first AI-orchestrated cyber espionage campaign report, especially where tool use and narrative manipulation combine.

Why It Matters for Security Teams

Lies-in-the-Loop attack is important because it undermines the assurance layer that modern software and AI operations depend on. If a review process can be steered by false context, then code signing, policy checks, secure SDLC gates, and AI agent approvals can all be bypassed without an obvious technical exploit. This is especially relevant where AI assistants are given authority to summarise risk, approve routine changes, or recommend access and deployment decisions.

Security teams should treat this as a governance problem as much as a detection problem. Controls from NIST SP 800-53 Rev 5 Security and Privacy Controls help by separating reviewer independence, integrity checks, and change approval duties, while advisory material from CISA cyber threat advisories is useful for operational awareness of active deception tactics. The key identity-adjacent issue is that NHI and agentic AI systems often act with delegated authority, so misleading context can create unauthorised action without credential theft.

Organisations typically encounter the business impact only after a compromised review result is shipped, at which point Lies-in-the-Loop attack becomes operationally unavoidable to investigate and contain.

Related resources from NHI Mgmt Group

• What is the core decision loop Agentic AI follows and why does it create security risk?
• Why does Agentic AI make NHI attack surface expand so significantly?
• What is the difference between attack surface management and NHI governance?
• Why do NHIs create a larger attack surface than human users?