Why do AI and open source programmes increase identity risk in practice?

Why This Matters for Security Teams

AI and open source programmes expand the number of identities that can act on behalf of the business, often faster than teams can document them. That matters because the risk is not limited to a few extra logins. It includes service accounts, API tokens, automation keys, pipeline roles, package-signing credentials, and delegated permissions across platforms. Each one can be over-privileged, forgotten, or reused in ways that are hard to detect.

Security teams often treat these identities as implementation details, but attackers do not. If a token is exposed in a repository, or an automation account is trusted across multiple environments, the blast radius can exceed that of a single human account compromise. The control problem is therefore not just authentication. It is lifecycle governance, privilege scope, and evidence that access is continuously justified. That aligns closely with the NIST Cybersecurity Framework 2.0, especially around asset management, access control, and continuous risk management.

In practice, many security teams encounter identity sprawl only after a build pipeline, AI workflow, or third-party integration has already been abused, rather than through intentional inventory and review.

How It Works in Practice

The identity risk rises because AI and open source ecosystems are built on machine-to-machine trust. A single workflow may involve code repositories, model registries, package managers, CI/CD runners, cloud roles, secret stores, and external APIs. Each layer needs a credential or delegated permission, and each permission tends to outlive the original task unless there is explicit revocation discipline. The result is a dense web of non-human identities, many of which are invisible to traditional joiner-mover-leaver processes.

Practitioners should look at this as a control design problem, not a tooling problem. The objective is to know what each identity can reach, why it exists, who owns it, how it is rotated, and what event proves it is safe to keep. The relevant control family in NIST SP 800-53 Rev 5 Security and Privacy Controls is not one single safeguard but a combination of access enforcement, account management, audit logging, and configuration control.

• Inventory every service account, token, key, and workflow credential used by AI and open source tooling.
• Bind each identity to a named owner, purpose, and expiry or review date.
• Scope permissions to the narrowest repository, environment, dataset, or API that is genuinely required.
• Rotate secrets on a schedule and after every suspected exposure, then verify dependent systems still function.
• Monitor for anomalous use, especially around build systems, model deployment jobs, and package publication.

For AI programmes specifically, the risk also includes agentic workflows that can call tools or trigger actions autonomously. That creates an identity and authority question: what can the system do without human approval, and what proof exists that the action was allowed. Best practice is evolving here, but current guidance suggests treating AI execution rights as privileged access, not ordinary application access. These controls tend to break down when legacy CI/CD platforms, shared developer credentials, and unmanaged third-party integrations are all allowed to coexist because ownership and revocation responsibilities become ambiguous.

Common Variations and Edge Cases

Tighter identity control often increases operational overhead, requiring organisations to balance developer velocity against auditability and revocation certainty. That tradeoff becomes sharper in open source heavy environments, where contributors, automation bots, and package publishing systems may be distributed across teams and sometimes across legal entities. There is no universal standard for this yet, so teams should document local policy for who can approve machine access, how exceptions are granted, and how quickly unused credentials must be removed.

The hardest cases are usually not the obvious production services. They are ephemeral runners, fork-based pull request workflows, AI evaluation sandboxes, and vendor-managed integrations that inherit broad access for convenience. These environments can look low-risk because they are temporary, but they often hold the credentials that can reach source code, models, or release pipelines. In that sense, identity risk is amplified by speed and reuse, not just by scale.

Where open source programmes depend on external maintainers or automation, the trust boundary extends beyond the organisation. That means trust decisions should include code provenance, signing, dependency integrity, and how secrets are injected into build systems. If the question is how to reduce risk quickly, start with the identities that can publish code, deploy models, or modify secrets. Those are the ones that most often turn convenience into systemic exposure.

Related resources from NHI Mgmt Group

• Why do AI agents increase non-human identity risk in existing IAM programmes?
• Why do AI-driven attacks increase risk for identity and access management programmes?
• Why do AI agents increase non-human identity risk?
• Why do AI programmes create new identity risk for CISOs?