TL;DR: Black Hat USA 2025 highlighted how AI, software supply chain risk, and cloud security are converging into a tougher governance problem, while Veracode’s GenAI Code Security Report found nearly half of AI-generated code samples failed security tests and 83% of organisations are using AI for software development. The practical lesson is that AI adoption is outpacing security assurance, so controls must shift from review at the end to governance across the pipeline.
At a glance
What this is: Black Hat 2025 surfaced a three-way governance problem across AI, software supply chain and cloud risk, with Veracode highlighting that AI-generated code remains insecure at a high rate.
Why it matters: IAM, NHI and security teams need to treat AI-assisted development, third-party code and cloud integrations as one control plane, because risk now enters through the pipeline as often as through the perimeter.
Context
Black Hat 2025 is being used here as a signal of where security governance is straining, not as the subject itself. The central issue is that AI-assisted development, third-party software dependencies and cloud-connected delivery are now creating overlapping risk paths that traditional end-stage code review and siloed cloud controls do not cover well.
For identity and access teams, the important intersection is not only code quality but the trust chain behind software creation and deployment. AI-generated output, open-source components and multi-cloud integrations all depend on credentials, service accounts and tool access that must be governed continuously rather than assumed safe at the point of use.
Key questions
Q: How should teams prevent AI code scanners from becoming the primary security control?
A: Treat AI scanners as advisory, not authoritative. Use deterministic SAST, SCA, and secret detection to establish the security baseline, then let AI help with explanation and remediation. The control decision is simple: if a finding cannot be reproduced on the same input, it should not decide whether code ships.
Q: Why do software supply chain attacks create such broad impact?
A: They exploit trust relationships that are already built into delivery pipelines. One malicious or compromised component can reach many applications at once, especially when release identities have permission to build, sign, and deploy across environments. That scale is why supply chain governance must focus on identities and provenance, not just vulnerability counts.
Q: Where does cloud security fail when identities and integrations are over-broad?
A: Cloud security fails when service accounts, delegated integrations and automation credentials have more access than the task requires and no one is watching their use continuously. In that state, attackers do not need to break the cloud first, they only need to abuse the access already embedded in the delivery chain.
A: Modern products depend on third-party libraries, APIs, CI/CD systems, and cloud infrastructure, so the attack surface extends far beyond source code. That creates exposure through compromised dependencies, insecure integrations, weak IAM policies, and misconfigured environments. Product security has to address these layers together, because attackers usually exploit the easiest trust boundary, not only the code itself.
Technical breakdown
AI-generated code and security assurance gaps
Large language models can generate syntactically valid code that still fails secure coding expectations, because they optimise for plausible output rather than policy-conformant implementation. In practice, that means common weaknesses can be reintroduced at scale unless code is checked against security requirements before merge and before release. The issue is not that AI always writes bad code, but that its output has no intrinsic assurance boundary. That makes developer workflow controls, automated scanning and policy enforcement part of the same control set, rather than separate stages.
Practical implication: Shift security checks left into the AI-assisted coding workflow and treat model output as untrusted until verified.
Software supply chain risk in third-party code
Modern software inherits risk from packages, build dependencies and other external components that are often trusted by default once they enter the pipeline. Supply chain attacks exploit that trust by targeting upstream code, malicious packages, compromised maintainers or build artifacts, then propagating downstream through automated delivery paths. The governance problem is compounded by the volume of dependencies and the speed of release cycles, which make manual review unrealistic. Integrity therefore depends on provenance, inventory and continuous monitoring, not just occasional vetting.
Practical implication: Establish dependency inventory, provenance checks and automated detection for malicious or compromised packages across the build pipeline.
Cloud security becomes an access-governance problem
Cloud risk is not only about misconfigured infrastructure. It also arises when LLMs, third-party tools and contributors gain broad connectivity into multi-cloud environments without tight identity scoping and continuous monitoring. In such environments, service accounts, integrations and delegated access paths can become the quiet route into production systems, especially when ownership is fragmented across teams. That is why cloud security, IAM and NHI governance overlap here: the technical failure is often excessive or poorly monitored access, not just a control-plane setting.
Practical implication: Map all cloud-facing identities and integrations to explicit ownership, least privilege and continuous review.
Threat narrative
Attacker objective: The attacker aims to turn trusted development and cloud delivery paths into scalable compromise channels that affect code integrity, environment access and downstream execution.
- Entry occurs through AI-assisted development pipelines, third-party packages or cloud-connected integrations that are trusted too early in the delivery chain.
- Credential and access abuse follow when service accounts, developer tooling or delegated integrations have more reach than their task requires.
- Escalation and spread occur as compromised code or packages move downstream into build systems, repositories and multi-cloud environments.
- Impact lands in insecure applications, exposed cloud resources and broader trust degradation across the software delivery lifecycle.
Breaches seen in the wild
- CISA Private-CISA GitHub leak 2026: A CISA contractor's public GitHub repo exposed AWS GovCloud admin keys, Artifactory credentials and plaintext passwords for six months.
- Sisense breach 2024: A credential in Sisense's GitLab reportedly opened S3 buckets of customer tokens, passwords and certificates; CISA urged a full reset.
Read and download The State of NHI & AI Agent Breach Report 2026, covering 200+ breaches impacting Non-Human Identities including AI Agents.
NHI Mgmt Group analysis
AI-assisted development has created a secure-by-review illusion. Review after code generation assumes the unsafe output can be reliably caught at the end of the pipeline, but Black Hat themes show that volume and speed are breaking that assumption. When nearly half of AI-generated code samples fail security tests, the governance question becomes how to prevent insecure patterns from entering the delivery chain in the first place. Practitioners should treat model output as an input requiring policy enforcement, not as draft code that later becomes safe by inspection.
Software supply chain security is now an identity and provenance problem as much as a code problem. Third-party packages, build artifacts and contributor paths are all trust relationships, and trust relationships need ownership, inventory and revocation logic. That is why supply chain governance increasingly overlaps with NHI discipline, because tokens, signing identities and automation credentials are what move malicious artefacts through the pipeline. Practitioners should align supply chain controls with identity lifecycle controls rather than handling them as separate programmes.
Cloud security exposure increasingly begins with over-broad delegated access. Multi-cloud complexity turns service accounts, integrations and automation into hidden entry points when access is not continuously scoped and monitored. The field should stop treating cloud compromise as only a configuration problem and start treating it as a governed access problem with identity at the centre. Practitioners should make cloud-facing identities auditable assets, not incidental plumbing.
Security posture management is becoming a coordination layer, not a point control. The article’s ASPM framing reflects a wider market shift toward joining code, cloud and runtime signals into one decision surface. That is consistent with where enterprise governance is heading: control fragmentation no longer matches the attack surface. Practitioners should evaluate whether their current tooling can correlate code provenance, application risk and cloud exposure well enough to support action.
What this signals
AI code governance is moving from quality assurance to trust assurance. The practical question is no longer whether a model can generate code, but whether the pipeline can prove that generated code has been screened, bounded and approved before it touches production. For practitioners, that means build-stage policy enforcement matters more than after-the-fact remediation.
Cloud and supply chain controls are converging on the same governance problem. If dependency trust, service-account scope and cross-platform access are not owned centrally, the organisation will keep discovering security issues one release too late. The programme response is to make identity, provenance and posture part of the same control conversation.
For practitioners
- Implement security gates for AI-generated code Require automated scanning, secure coding policy checks and human review before AI-generated code reaches merge or release.
- Inventory and classify third-party dependencies Track package provenance, maintainers and update paths so risky components can be identified and removed quickly.
- Tighten ownership of cloud-facing identities Assign explicit owners to service accounts, integrations and delegated access paths across every cloud environment.
- Correlate code-to-cloud risk signals Use posture management to connect application findings with cloud exposure so remediation prioritisation reflects real blast radius.
Key takeaways
- AI-generated code can move insecure patterns into delivery pipelines faster than manual review can catch them.
- Software supply chain risk now depends on dependency provenance, trust relationships and downstream propagation.
- Cloud security is increasingly an access-governance problem, which makes identity ownership and continuous monitoring essential.
Standards & Framework Alignment
This section maps relevant standards and security frameworks to the operational risks and controls described in this guidance.
MITRE ATT&CK and OWASP Agentic AI Top 10 address the attack and risk surface, while NIST CSF 2.0 and NIST AI RMF set the governance and control requirements practitioners need to meet.
| Framework | Control / Reference | Relevance |
|---|---|---|
| MITRE ATT&CK | TA0006;TA0008;TA0010 — Credential Access; Lateral Movement; Exfiltration | The article describes malicious packages and trusted pipelines as attack paths for compromise and theft. |
| Recommendation — Map supply chain abuse to credential access, lateral movement and exfiltration tactics in your detections. | ||
| NIST CSF 2.0 | PR.DS-10 — Confidentiality, Integrity, and Availability | The article centres on integrity and confidentiality loss across code, dependencies and cloud delivery. |
| PR.AA-05 — Access Permissions, Entitlements and Authorizations | Cloud and pipeline risk in the article depends on over-broad service and integration access. | |
| Recommendation — Use PR.DS-10 to align code integrity, dependency trust and cloud exposure controls in one governance view. Apply PR.AA-05 to scope service accounts and delegated access to the minimum needed for delivery. | ||
| NIST AI RMF | GOVERN — AI Governance and Accountability | The article frames AI code generation as a governance issue, not just a tooling issue. |
| Recommendation — Establish AI governance roles that define when generated code can enter production workflows. | ||
| OWASP Agentic AI Top 10 | ASI03 — Identity & Privilege Abuse | AI tooling and automated workflows can abuse identity and privilege boundaries inside delivery pipelines. |
| Recommendation — Model AI-assisted delivery paths for identity and privilege abuse before they touch production. | ||
Key terms
- AI-Generated Code Risk: AI-generated code risk is the possibility that model-produced output introduces insecure credential handling, hidden secrets, or repeated patterns that weaken control design. The risk is operational, because the output can be merged quickly and at scale before humans catch the problem.
- Software Supply Chain Risk: Software supply chain risk is the chance that code, build, deployment, or dependency paths are altered in ways that affect downstream systems. For AI code assistants, this risk rises when the tool can change source, trigger pipelines, or influence infrastructure definitions.
- Identity Security Posture Management: Identity security posture management is the continuous assessment of identity configuration, privilege, and exposure across an environment. It focuses on drift, overprivilege, and control gaps so teams can see where IAM, PAM, and NHI governance are failing before those gaps become incidents.
- Delegated access path: A delegated access path is the chain of identities, tokens, connectors, and approvals that lets one system act through another. It becomes a governance concern when the path outlives the original approval or can be reused for actions beyond the intended business purpose.
Deepen your knowledge
NHI Mgmt Group covers identity security, NHI governance, and agentic AI through independent research, practitioner guides, and the NHI Foundation Level course. It is designed for practitioners who need to govern machine identities, secrets and access paths as part of a broader security programme.
Published by the NHIMG editorial team on August 2, 2026.
Updated on October 12, 2026.
NHI Mgmt Group, the independent authority on Non-Human Identity, IAM, and Agentic AI security. nhimg.org