TL;DR: Recent AI security incidents in April 2026, including an Anthropic leak and a Mercor supply chain attack, show how human error, insecure integrations, and compromised dependencies can expose source code, customer data, and API keys across AI environments, according to Proofpoint. The lesson is that AI security failures are now operational governance failures, not just model issues.
At a glance
What this is: NHI Mgmt Group's analysis, drawing on Proofpoint's blog post: Recent AI security incidents reveal a governance gap across human error, integrations, and software supply chains, not just model security.
Why it matters: For IAM, PAM, NHI, and AI governance teams, the lesson is that AI exposure often starts in access paths, release processes, and trusted dependencies that conventional controls do not fully see.
By the numbers:
- 82% of breaches involve human factors such as error, misdelivery, or misuse.
Context
AI security incidents often arise from the way people, integrations, and dependencies are allowed to move data, not from the model alone. When release packaging, third-party libraries, and collaboration channels are loosely governed, exposure can become permanent because code, tokens, and data cannot be recalled once they have escaped.
This article sits at the intersection of AI governance and identity security. It shows how authorised users, shared libraries, and connected services can become the real attack surface, especially where AI systems rely on application credentials, API access, and delegated integrations across the enterprise.
Key questions
Q: What fails when AI code or internal files are exposed before release?
A: The failure is durability. Once source code or internal files are public, the attacker can study system logic, data handling, and workflow design even if the original issue is later fixed. That is why release governance, packaging checks, and secret scanning must happen before publication, not after detection.
Q: Why does AI make software supply chain risk harder to control?
A: AI increases the amount of code produced, which reduces the time available for review and makes malicious or unauthorized changes harder to spot. It also introduces non-human actors into the development flow, so traditional assumptions about developer identity no longer hold. That combination expands the attack surface at the commit stage.
Q: What are the signs that AI governance is failing in the enterprise?
A: Common warning signs include rapid growth in AI use without matching policy coverage, sensitive files being copied into personal accounts, and a large share of AI apps carrying high or critical risk. Another indicator is weak visibility into who is using which tools and what data they are sending. If teams cannot answer those questions, governance is not working as intended.
Q: Should organisations separate AI agent monitoring from identity governance?
A: Yes. Organisations should separate AI agent monitoring from identity governance because they solve different problems. Monitoring answers what happened, while identity governance answers whether the action should have been possible. Keeping those functions distinct prevents teams from mistaking visibility for control and helps reduce over-permissioning.
Technical breakdown
Why release packaging errors create durable exposure
A release packaging error is not just a deployment bug. In AI environments, it can expose source code, internal files, prompts, or workflow details that reveal how a system processes data and enforces access. Once code or files are public, the attacker does not need continued access to keep the benefit. They can study logic, map dependencies, and look for downstream credential or data handling weaknesses. This is why exposure control has to happen before release, not after detection.
Practical implication: tighten pre-release review and artifact handling so exposed code and internal files never leave controlled build and release paths.
How supply chain compromise propagates through AI integrations
AI systems rarely operate alone. They depend on libraries, SDKs, connectors, and agent-facing services that move data between applications and model endpoints. If a widely used dependency is compromised, the malicious code inherits trust from the surrounding ecosystem and can harvest API keys, intercept data, or alter flows before defenders notice. The security problem is therefore not just code provenance, but dependency trust at runtime and in build pipelines. That makes inventory, provenance, and runtime visibility essential.
Practical implication: treat AI-connected dependencies as governed trust relationships and verify provenance for code that can touch credentials or model traffic.
Why human-driven exposure is amplified in AI environments
Human error remains central because AI workflows speed up the movement of sensitive information across chat, email, collaboration tools, and APIs. That reduces the margin for mistakes and expands the impact when a user shares the wrong file, uses the wrong integration, or approves an unsafe release step. In identity terms, the issue is not only who has access, but how that access is exercised across an AI-enabled workflow. The governance gap is the lack of visibility into those interactions.
Practical implication: monitor user and service interactions with AI systems as a single risk surface, not as separate data, application, and identity problems.
Threat narrative
Attacker objective: The attacker seeks durable access to sensitive code, credentials, and customer data moving through AI-enabled environments.
- Entry occurs through either public release exposure or a compromised open source dependency embedded in an AI-facing workflow.
- Credential or data access follows when the exposed code, internal files, or malicious library logic reveals API keys, customer data, or system internals.
- Impact is achieved when those secrets or data flows are harvested at scale, creating lasting exposure that cannot be undone by simple revocation.
Breaches seen in the wild
- Ultralytics PyPI compromise 2024: A GitHub Actions injection and a leftover PyPI token let attackers publish four Ultralytics releases that installed a cryptominer.
- Hugging Face API tokens exposed 2023: Lasso Security found 1,681 live Hugging Face tokens in public code, 655 with write access, reaching 723 organisations including Meta; all were revoked.
Read and download The State of NHI & AI Agent Breach Report 2026, covering 200+ breaches impacting Non-Human Identities including AI Agents.
NHI Mgmt Group analysis
AI security governance is now an identity problem as much as a model problem. The article shows that exposed code, shared integrations, and user-driven mistakes are where AI risk becomes real. That means the control plane is not just the model runtime, but also the people, credentials, and dependencies that move information around it. Practitioners should treat AI governance as a trust-boundary problem, not a model-only discipline.
Release packaging risk is the durable failure mode this article exposes. Once source code or internal files become public, the exposure cannot be cleanly revoked, which makes pre-release controls more important than post-release response. The practical lesson is that release governance for AI systems must be designed for irreversibility. Teams should assume that one packaging mistake can create long-lived intelligence for an attacker.
Supply chain trust is now part of AI security posture. When AI systems rely on open source libraries and shared connectors, a compromised dependency can inherit legitimate access to data and API traffic. This broadens the scope of governance from code review to provenance, inventory, and runtime control. Practitioners should evaluate AI supply chains with the same seriousness they apply to privileged access.
Human error amplification: AI systems do not remove the human attack surface, they magnify it by increasing the speed and reach of every mistake. That is the clearest pattern in the article and it aligns with broader breach behaviour across the sector. Security programmes should therefore shift from awareness-only thinking to measurable control over user actions, release paths, and data movement.
From our research library:
- Organisations that rely heavily on static credentials reported a 20-percentage-point increase in security incidents compared with those with low reliance, according to the 2026 Infrastructure Identity Survey.
- Organisations that describe themselves as confident in their AI deployment actually experience a 72% security incident rate, compared to 33% for those who remain cautious, according to the 2026 Infrastructure Identity Survey.
- Read next: AI Supply Chain Security and AI-BOM Guide
What this signals
AI governance debt: organisations accumulate risk when AI systems are allowed to expand through release shortcuts, untracked dependencies, and weak user controls. The result is not only model exposure but a broader inability to explain who can move sensitive data, where credentials are used, and which integrations are trusted.
The operating model needs to move from reactive review to preventative trust management across AI workflows. Where code, data, and API access cross the same environment, identity governance and data security must be treated as a single control problem, not separate after-the-fact reviews.
For practitioners
- Enforce pre-release exposure checks Block releases until source code, internal files, and embedded secrets are scanned and approved in controlled pipelines.
- Inventory AI dependencies and connectors Maintain a live register of libraries, SDKs, and services that can reach AI data flows or credentials, including open source components.
- Restrict API key reachability Scope API keys to the minimum required services and separate keys used for model access from keys used for business systems.
- Monitor user-to-AI data movement Track who shares data into AI tools, which channels they use, and which files or prompts contain sensitive material.
- Treat exposed code as permanent risk Assume leaked source code or internal files cannot be contained after publication, and prioritise prevention over post-event containment.
Key takeaways
- AI security incidents now show that governance failures in release processes, integrations, and dependencies can expose code, data, and credentials at enterprise scale.
- Human factors still dominate breach paths, with 82% of breaches involving error, misdelivery, or misuse, which means AI adoption amplifies familiar weaknesses rather than replacing them.
- The practical response is to govern AI-connected release pipelines, dependencies, and data movement together so exposed code or stolen keys do not become permanent risk.
Standards & Framework Alignment
This section maps relevant standards and security frameworks to the operational risks and controls described in this guidance.
OWASP Agentic AI Top 10, OWASP Non-Human Identity Top 10 and MITRE ATT&CK address the attack and risk surface, while NIST AI RMF and NIST CSF 2.0 set the governance and control requirements practitioners need to meet.
| Framework | Control / Reference | Relevance |
|---|---|---|
| OWASP Agentic AI Top 10 | ASI04 — Agentic Supply Chain Vulnerabilities | AI-connected dependencies and connectors are the core exposure path in this article. |
| ASI09 — Human-Agent Trust Exploitation | Human error and unsafe trust in AI workflows drive the incidents described here. | |
| Recommendation — Track AI dependencies and connectors as supply chain risk and verify provenance before they can handle sensitive data. Constrain human-driven AI actions that can expose data or approve unsafe release steps. | ||
| OWASP Non-Human Identity Top 10 | NHI-02 — Secret Leakage | API keys and source code exposure are central to the risk pattern discussed in the article. |
| NHI-03 — Vulnerable Third-Party NHI | Compromised libraries and shared integrations create third-party trust exposure around AI systems. | |
| Recommendation — Scan AI release paths for leaked secrets and revoke exposed credentials immediately. Review third-party components that can reach AI data flows and remove ungoverned trust. | ||
| NIST AI RMF | GOVERN — AI Governance and Accountability | The article is fundamentally about governance across AI systems, dependencies, and human behaviour. |
| Recommendation — Assign governance ownership for AI release, dependency, and data-control decisions. | ||
| NIST CSF 2.0 | PR.DS-01 — Data-at-rest is protected | Sensitive data exposure through AI workflows makes data protection a primary control concern. |
| Recommendation — Apply data protection controls to sensitive material moving through AI-connected systems. | ||
| MITRE ATT&CK | TA0006;TA0010 — Credential Access; Exfiltration | The compromise pattern centres on harvesting keys and exposing data through trusted workflows. |
| Recommendation — Map AI exposure paths to credential access and exfiltration techniques in your detection programme. | ||
Key terms
- AI Supply Chain Security: AI supply chain security is the practice of protecting the people, data, models, code, tools, and services used to build and run AI systems. It covers risks from training data, model artifacts, dependencies, prompts, connectors, and deployment pipelines, with controls for integrity, provenance, access, and change monitoring across the full lifecycle.
- Release Packaging Error: A release packaging error is a deployment failure that exposes code, files, or configuration outside the intended control boundary. In AI systems, this can reveal internal logic, data handling paths, or embedded secrets, creating an exposure that is often permanent once published.
- Human-Driven Exposure: Human-driven exposure occurs when authorised users accidentally disclose sensitive data, credentials, or system details through normal work processes. In AI environments, the speed and reach of collaboration tools and integrations make these mistakes more damaging than in traditional systems.
- Trusted Dependency: A trusted dependency is an external library, package, or service that is allowed to operate inside an application or workflow because it is presumed safe. When that trust is misplaced, the dependency can become a delivery path for credential theft, data interception, or code manipulation.
Deepen your knowledge
NHI Foundation Level course, the industry's only accredited NHI security programme, covers NHI governance, machine identity security, and secrets management. It helps practitioners connect identity controls to the wider security and AI adoption decisions their programmes now face.
Published by the NHIMG editorial team on July 14, 2026.
Updated on October 13, 2026.
NHI Mgmt Group, the independent authority on Non-Human Identity, IAM, and Agentic AI security. nhimg.org