AI-built applications outpace AppSec controls and expand exposure

At a glance

What this is: Pluto Security argues that AI-accelerated software development is creating applications that security teams cannot reliably review, inventory or govern with traditional AppSec methods.

Why it matters: This matters because AI-built code can introduce hidden secrets, unsafe dependencies and shadow tooling faster than security controls, which affects both application risk and the identity controls around sensitive data access.

By the numbers:

• Pluto Security is officially out of stealth with $42 million in total funding.

Context

AI application security is the problem of governing code, workflows and data paths that are partially or fully produced by AI systems. The security gap appears when development speed, hidden dependencies and unmanaged tooling outpace the organisation’s ability to review what was built and what it can reach.

In this article, the issue is not AI as a feature but the operational mismatch between faster code production and slower assurance. That mismatch becomes more serious when AI tools touch source code, secrets and sensitive data outside normal governance, which makes inventory and accountability harder to sustain.

Key questions

Q: How should security teams govern AI-generated code in production environments?

A: Security teams should treat AI-generated code as normal production code with extra provenance risk. Require architectural review, test coverage, static analysis, and approval before merge. Then bind the agent and the build pipeline to least privilege, short-lived credentials, and complete audit logging so implementation speed does not outrun control.

Q: Why do AI-generated code changes increase application security risk?

A: AI-generated code can increase risk because it accelerates output faster than review, testing, and secret hygiene can keep up. The issue is not only flawed logic. It is also the possibility that tokens, credentials, unsafe dependencies, or insecure defaults are reproduced at scale across the delivery pipeline.

Q: What are the first controls to add when developers start using AI coding tools?

A: Start with inventory and review. Identify which tools are approved, what data may be shared with them, and which security checks must run on generated code before merge or deployment. Then add secret scanning, dependency review and authorisation testing so the development pipeline can catch the most common AI-specific failure modes early.

Q: How do organisations stop shadow AI from creating access and data exposure risk?

A: Organisations need explicit usage policy, service discovery, and data access boundaries so employees know what is approved and what is not. Then they must monitor for unauthorised tools, unusual data flows, and unowned identities. Without that combination, shadow AI becomes invisible NHI sprawl with unclear accountability.

Technical breakdown

Why AI-generated code defeats normal application review

AI-generated code changes the review problem because the developer may not fully understand every library, logic branch or security assumption in the output. That creates a validation gap: static analysis can catch some syntactic issues, but it cannot reliably infer whether the generated logic is safe in context. When code is produced faster than human review can keep pace, security teams lose the chance to find issues before they reach production. This is not just a tooling problem. It is an assurance problem caused by the speed and opacity of machine-assisted development. See the Application Security Guide.

Practical implication: Treat AI-generated output as untrusted until it passes the same validation gates as human-written code.

Shadow AI and the loss of inventory control

Shadow AI is the use of unsanctioned AI tools or embedded AI features outside formal governance. In application development, that matters because code, prompts, data and outputs can move through third-party services without appearing in normal inventories or approval workflows. Once that happens, security teams lose line of sight over where sensitive inputs went, which tools processed them, and whether those tools retained or exposed the information. The governance failure is not only technical. It is also an ownership problem: if no one knows the tool is in use, no one can define its acceptable use, access scope or data handling rules.

Practical implication: Build a complete inventory of sanctioned and unsanctioned AI tooling before you can govern application risk.

Why AI-built applications often fail at secrets and access control

AI-generated applications can embed hardcoded credentials, weak authorisation logic or unsafe API handling because the model optimises for functional output rather than secure design. That means secrets may end up in code, repositories or build artefacts, while logic flaws can create privilege escalation or IDOR conditions. In identity terms, the risk is that machine credentials and access decisions get woven into software without lifecycle controls. The result is an application that looks complete but has insecure authentication assumptions built in from the start. Security teams need to assume these failures are possible whenever AI writes application logic that touches identity or sensitive data. See the Secure Coding Guide.

Practical implication: Scan generated code for embedded secrets, weak authorisation and unsafe token handling before merge or deployment.

Threat narrative

Attacker objective: The attacker’s objective is to exploit the speed and opacity of AI-built software to reach sensitive data, privileged functionality or production systems before security can inspect them.

1. Entry occurs when developers adopt unsanctioned AI tools or accept generated code without formal security review.
2. Credential access follows when AI-generated output embeds static secrets, tokens or sensitive data handling into code and workflows.
3. Escalation happens when weak logic, hidden dependencies or unsafe API chains expand the application’s attack surface.
4. Impact is realised when insecure code reaches production and exposes data, authorisation boundaries or third-party services.

NHI Mgmt Group analysis

AI application security is becoming a governance problem, not just a code quality problem. The article makes clear that the failure mode is organisational: teams are shipping software faster than they can inventory, assess and approve what AI has produced. That changes the control objective from after-the-fact review to governable generation, with AppSec, IAM and data handling all converging at the point of code creation. Practitioners should treat AI-assisted development as a managed risk domain, not a productivity side channel.

Shadow AI creates an identity and trust boundary problem inside development workflows. When unsanctioned tools process source code or sensitive data, the organisation no longer knows which systems had access, for how long, or under what contractual and technical conditions. That is especially important for IAM and NHI governance because tool access, API tokens and service integrations can proliferate outside normal lifecycle controls. third-party AI risk becomes the right lens when vendor tools are handling code, prompts or data outside approved workflows. Practitioners should extend governance to tool usage, not just to code repositories.

Hardcoded secrets in generated code are a machine-speed version of an old control failure. The new element is not the weakness itself but the rate at which it can be replicated across branches, services and environments. That makes secret leakage a production design issue rather than a post-deployment incident. When AI writes code that touches authentication or API calls, credential hygiene has to move upstream into generation, review and release gates. Practitioners should assume secrets will be introduced unless controls are explicitly tuned to detect them.

AI-built application sprawl is the right shorthand for the risk described here. The article shows how speed, opaque dependencies and unsanctioned tooling can create a software estate that expands faster than traditional assurance can map it. That makes governance, not just testing, the decisive control plane. Organisations that cannot account for where AI is used will struggle to control what those systems can change, call or expose. Practitioners should build inventory, policy and review around AI-assisted delivery before scale turns visibility into guesswork.

What this signals

AI-built application sprawl: the governance challenge is no longer whether teams can generate software faster, but whether they can still prove what was generated, what it touches and who approved it. Security programmes that stop at code scanning will miss the organisational controls needed to manage AI-assisted delivery end to end.

The operational implication is that AppSec, IAM and data governance now need a shared control plane for AI-assisted development. Security teams should expect more pressure to prove inventory, data-use limits and release gating as the use of AI coding tools becomes normal rather than exceptional.

For practitioners

• Build an approved AI tool inventory Track every sanctioned and unsanctioned AI system used in development, including embedded AI features in SaaS products, so security can define data handling, retention and access boundaries.
• Add security gates to AI-generated code Require pre-merge checks for secrets, weak authorisation, unsafe API calls and dependency risk before generated code can move into mainline builds or production release paths.
• Shift AppSec review left into CI/CD Embed automated testing and runtime checks into build pipelines so the same workflow that accelerates delivery also catches insecure patterns before deployment.
• Define data-use rules for AI inputs Publish rules for what developers may paste into models, which data classes are prohibited, and how sensitive code or business data must be handled when AI is involved.
• Map AI delivery to governance controls Align AI-assisted development with existing security and compliance controls so ownership, approval and evidence exist before the software reaches production.

Key takeaways

• AI-assisted development creates a control gap when software is produced faster than security can review, inventory and approve it.
• The article points to multiple exposure paths, including Shadow AI, opaque code generation, embedded secrets and unsafe dependencies.
• The practical answer is to move governance upstream with approved tools, pipeline checks, data-use rules and stronger ownership of AI-assisted workflows.

Related NHI Mgmt Group resources: Application Security Guide · AI Risk Assessment Guide · Third-Party AI Risk Guide · OWASP Agentic Applications Top 10

Key terms

• AI Application Security: AI application security is the discipline of protecting software that uses AI to make decisions, generate code, or interact with data and services. It goes beyond model safety to cover prompts, APIs, generated logic, runtime behaviour, and the permissions those components use in production.
• Shadow AI: AI agents, copilots, or connected tools operating without full visibility or governance from security teams. Shadow AI becomes an identity problem when those systems authenticate with unmanaged tokens, service accounts, or OAuth apps that can reach production resources.
• Generated Code Risk: Generated code risk is the possibility that machine-produced software contains insecure logic, hidden dependencies, weak authorisation or embedded secrets. It matters because the code may look functional while still carrying security defects that are difficult to detect quickly in normal development workflows.
• DevSecOps Pipeline Protection: DevSecOps pipeline protection is the practice of embedding security checks into the software delivery process before code reaches production. It covers repositories, builds, container images, and infrastructure templates. The aim is to catch vulnerabilities, secrets exposure, and misconfiguration early enough that teams can fix issues at the source.

Deepen your knowledge

The NHI Foundation Level course, the industry's only accredited NHI security programme, covers NHI governance, workload identity and secrets management in the context of modern development and automation. It helps security practitioners connect identity controls to the broader risks created by AI-assisted workflows.