Shadow AI in development pipelines is breaking governance models

At a glance

What this is: This is an analysis of how unsanctioned AI use inside development teams creates hidden security, compliance, and accountability risks across the software lifecycle.

Why it matters: It matters because IAM, PAM, and governance teams now have to control not only human access to code and data, but also the tool paths and delegation chains that shadow AI introduces.

By the numbers:

• A 2024 Stack Overflow Developer Survey shows that 75% of developers now use AI assistants regularly.
• Independent research from Asisonline shows that 43% of workers plug sensitive work information into AI tools.

👉 Read Knostic's analysis of shadow AI risks in development pipelines

Context

Shadow AI is the use of unsanctioned AI tools by developers and engineers outside approved governance, monitoring, and logging. The primary issue is not experimentation itself, but the fact that sensitive code, credentials, and customer data can leave the controlled environment without any identity, data, or workflow oversight.

This is an identity governance problem because the actor is often a developer using a tool that sits outside approved access controls, policy enforcement, and provenance tracking. In DevSecOps environments, that creates a parallel decision layer for code, data, and dependencies that the security programme does not reliably see.

Key questions

Q: How should security teams control shadow AI use in development environments?

A: Security teams should control shadow AI by treating it as an unsanctioned data and identity path, not just an app choice. That means approved tooling lists, monitored gateways, prompt filtering, provenance tagging, and repo controls for AI-assisted code. The goal is to stop sensitive information and unreviewed changes before they cross the governed boundary.

Q: Why does shadow AI create compliance risk even when developers are trying to be productive?

A: Shadow AI creates compliance risk because prompts can contain regulated data, source code, or architecture details that leave the organisation’s approved processing boundary. Once external systems receive that content, the organisation may lose control over retention, residency, and reuse. Productivity does not reduce liability if the data path is ungoverned.

Q: What breaks when AI-assisted code is merged without provenance controls?

A: When AI-assisted code is merged without provenance controls, teams lose attribution, review evidence, and incident reconstruction capability. Investigators cannot tell whether a bug, policy breach, or vulnerability came from a developer, a plugin, or an external model. That makes audits incomplete and remediation slower.

Q: Who is accountable when shadow AI causes a data leak or insecure deployment?

A: Accountability sits with the organisation that allowed the unapproved tool path, not with the model itself. Security, engineering, and governance teams all need clear ownership for approved tools, sensitive data rules, and code provenance. Without named ownership, shadow AI becomes an unmanaged exception rather than a controllable risk.

Technical breakdown

Unapproved AI use creates shadow data egress

When developers paste code, credentials, diagrams, or customer records into external AI services, the data may be stored, reused, or incorporated into model training. That turns a local productivity action into an external disclosure event. The key technical problem is boundary loss: once the prompt leaves the trusted environment, traditional logging, DLP, and code review no longer govern what happens next. If the tool is not sanctioned, the organisation often has no audit trail for the content, the recipient, or the retention terms.

Practical implication: treat AI prompt paths as data egress channels and control them with policy, filtering, and monitored gateways.

AI-generated code introduces supply-chain and dependency risk

AI assistants can produce code that looks correct while introducing insecure logic, unapproved libraries, or outdated packages. In DevSecOps pipelines, that matters because the output can flow straight into source control and build systems unless validation gates stop it. This is not only a code-quality issue. It is a supply-chain problem because generated code can silently widen the attack surface, pull in risky dependencies, or create compliance exposure through licensing and provenance failures.

Practical implication: require dependency scanning, SBoM generation, and review gates for any AI-assisted code before merge.

Shadow AI obscures authorship and accountability

Code written or rewritten by unsanctioned AI tools often becomes indistinguishable from human-authored code once it enters the repository. Without metadata, commit tagging, or signed provenance, investigators cannot determine who introduced a defect, a vulnerability, or a policy violation. That makes audits incomplete and incident response slower. In practice, the security problem is not just that changes happened. It is that the organisation cannot prove what changed, who approved it, or whether the change was produced through governed tooling.

Practical implication: enforce provenance tagging, signed commits, and AI usage metadata in the development workflow.

Threat narrative

Attacker objective: The objective is to move sensitive code or credentials out of governed boundaries and into an unmonitored external system while also introducing insecure changes back into the software lifecycle.

1. Entry occurs when a developer uses an unsanctioned AI tool, plugin, or agent inside the development workflow and shares sensitive context with it.
2. Escalation occurs when that tool generates code, suggestions, or dependencies that are merged into repositories without validation, creating hidden supply-chain exposure and traceability loss.
3. Impact occurs when leaked data, insecure code, or unreviewed dependencies reach production, resulting in compliance violations, investigation delays, or downstream compromise.

Breaches seen in the wild

• Reviewdog GitHub Action supply chain attack — reviewdog/action-setup GitHub Action supply chain attack exposed secrets.
• CI/CD pipeline exploitation case study — full server takeover via exposed .git directory and mismanaged CI/CD pipeline secrets.

Read our 52 NHI Breaches Analysis report for a comprehensive view of breaches impacting Non-Human Identities including AI Agents.

NHI Mgmt Group analysis

Shadow AI is a governance failure before it is a tooling problem. The central issue is that developers can create a parallel approval path outside sanctioned identity, code, and data controls. Traditional IAM sees the developer account, but not the unsanctioned AI system receiving the sensitive context. Practitioners should treat shadow AI as an identity and workflow governance gap, not an isolated productivity choice.

Data egress through prompts is the new hidden boundary in DevSecOps. External AI systems can retain, reuse, or train on the information they receive, which turns a quick refactor or debugging step into a long-lived disclosure event. This is where the boundary between code assistance and data governance collapses. The implication is that teams must re-evaluate where controlled processing actually ends.

Traceability is now a security control, not an audit luxury. If AI-assisted code enters repositories without provenance metadata, the organisation loses the ability to attribute change, prove review, or reconstruct decision chains after an incident. That weakens both technical response and compliance assurance. The practical conclusion is that provenance must be treated as part of the control plane for software delivery.

Unapproved AI tooling expands the attack surface in the same way unmanaged software once did, but with faster propagation. Plugins, browser extensions, IDE assistants, and autonomous agents can all request broad permissions and move data or code across boundaries in ways developers do not fully observe. This is why the risk is systemic: every additional tool adds a new governance exception, and every exception weakens the identity model behind DevSecOps.

Shadow AI also exposes the limits of policy-only governance. Policies can define approved behaviour, but they cannot stop developers from using external tools unless technical enforcement exists at the point of action. That makes runtime monitoring, sanctioned gateways, and provenance enforcement necessary complements to policy. Practitioners should expect shadow AI to remain undercounted until controls move closer to the workflow.

From our research:

• 72% of organisations have experienced or suspect they have experienced a breach of non-human identities, according to The 2024 ESG Report: Managing Non-Human Identities.
• Two-thirds of enterprises have endured a successful cyberattack resulting from compromised non-human identities, with a quarter encountering multiple attacks, according to Oasis Security & ESG.
• If you are building governance for developer-held AI tools, the broader NHI lesson is that unmanaged identity paths become repeatable exposure, as explored in the 52 NHI Breaches Analysis.

What this signals

Shadow AI will increasingly be managed as an identity governance issue, not merely a software usage issue. Once developers can move prompts, code, and configuration through unapproved AI services, the real control question becomes who authorised the path, what data crossed it, and whether the event was visible at all. The control model needs to shift from tool approval to workflow accountability.

With 72% of organisations already reporting or suspecting NHI breaches, according to The 2024 ESG Report: Managing Non-Human Identities, shadow AI adds another unmanaged identity path to an already crowded governance landscape. The practical implication is that engineering security, IAM, and application governance must converge on the same policy boundary before AI usage becomes embedded in normal delivery practice.

Code provenance will become a baseline control for AI-assisted delivery. Teams that cannot identify whether a change was human-authored or AI-assisted will struggle to satisfy audit, compliance, and incident response requirements. That pressure will push provenance metadata, signed commits, and workflow telemetry into the core of modern software governance.

For practitioners

• Map unsanctioned AI entry points Inventory where developers are already using external AI tools, browser extensions, IDE plugins, and agents, then classify each route by the data types it can see and move. Start with DevSecOps and product engineering teams because they tend to create the largest hidden exposure surface.
• Block sensitive prompt content at the boundary Apply prompt filtering, DLP rules, and monitored gateways so code, credentials, personal data, and architecture details do not leave approved environments unchecked. The goal is to stop data egress before it becomes an external retention problem.
• Require provenance for AI-assisted code Tag AI-generated or AI-rewritten changes in commit metadata, enforce signed commits, and make provenance a merge requirement for high-risk repositories. This gives auditors and responders a way to separate human-authored work from tool-assisted changes.
• Gate plugins, extensions, and agents by permission scope Use default-deny approval for new developer tools, restrict filesystem and network permissions, and isolate agents with short-lived tokens and namespace limits. That reduces the chance that an approved convenience tool becomes an unmonitored path to code or secrets.

Key takeaways

• Shadow AI is dangerous because it creates unmonitored identity and data paths inside engineering workflows.
• The evidence in the article points to broad developer adoption, sensitive data exposure, insecure code, and lost traceability as the main failure modes.
• Governance must move from policy statements to runtime controls, provenance enforcement, and approval boundaries at the point of action.

Key terms

• Shadow AI: Shadow AI is the use of AI tools, plugins, or agents outside approved governance and monitoring. In practice, it creates unreviewed paths for code, data, and credentials to move beyond the organisation’s controlled environment, making both security and compliance blind spots more likely.
• Code Provenance: Code provenance is the record of where code came from, who changed it, and which tool or workflow produced it. For AI-assisted delivery, provenance matters because teams need to distinguish human-authored changes from generated or rewritten content during review, audit, and incident response.
• Shadow Data Egress: Shadow data egress is the unapproved transfer of sensitive information out of the organisation through prompts, extensions, or AI integrations. It matters because the data may be retained, reused, or exposed outside the enterprise even when the original user intended only a temporary workflow shortcut.
• Provenance Metadata: Provenance metadata is the machine-readable context attached to a change, such as origin, author, workflow, or AI-assistance flags. In AI-assisted development, it gives governance teams enough detail to enforce policy, support audits, and reconstruct the chain of custody for code changes.

What's in the full article

Knostic's full article covers the operational detail this post intentionally leaves for the source:

• Step-by-step controls for detecting shadow AI use in IDEs and CI/CD workflows.
• The article's full risk matrix for mapping governance, technical, and tooling controls to each shadow AI failure mode.
• Specific examples of runtime enforcement for plugins, extensions, and agent permissions.
• The vendor's detail on how Kirin validates MCP servers and inspects developer telemetry for unsafe behaviour.

👉 The full Knostic article covers the control matrix, runtime enforcement examples, and developer workflow details.

Deepen your knowledge

NHI governance, agentic AI identity, and machine identity security are core topics in our NHI Foundation Level course, the industry's only accredited NHI security programme. If you are responsible for identity security strategy or programme maturity, it is worth exploring.