Why do AI agents and citizen developers increase application security risk for engineering teams?

Why This Matters for Security Teams

AI agents and citizen developers change application risk because software creation is no longer limited to experienced engineering teams with established secure coding habits. Business users can assemble workflows, write prompts, connect APIs, and publish automations quickly, while autonomous agents can generate or modify code with limited human context. That speed increases attack surface, especially when review, testing, and ownership do not scale at the same pace.

The security issue is not simply that more code exists. It is that more code is produced under uneven governance, with unclear provenance, inconsistent input validation, and weak understanding of abuse paths such as prompt injection, secret leakage, and unsafe tool use. Current guidance from the NIST AI Risk Management Framework and the OWASP Agentic AI Top 10 makes clear that governance, traceability, and human oversight are core controls, not optional extras.

Security teams also inherit operational noise: more exceptions, more low-quality findings, and more untracked changes to critical business logic. In practice, many security teams encounter the breach path only after an agentic workflow has exposed a secret or a citizen-built application has been connected to production data, rather than through intentional design review.

How It Works in Practice

In practical terms, the risk grows at three points: code generation, integration, and release. AI agents can create functions, tests, infrastructure snippets, and API calls that appear valid but miss secure defaults. Citizen developers may then assemble those outputs into applications using low-code platforms or automation tools, often with broad permissions and limited segregation of duties. The result is not always broken software; often it is software that works while quietly failing security expectations.

Engineering teams need to treat these systems as production-grade software sources, not experimental shortcuts. That means applying policy, review, and telemetry to the full lifecycle, including prompts, model outputs, connectors, secrets handling, and post-deployment monitoring. The security objective is to make unsafe actions difficult to execute, easy to detect, and attributable to a specific owner or workflow.

• Define which use cases are allowed for AI-generated code and low-code automation.
• Require review for authentication, authorisation, data access, and secret handling logic.
• Scan generated code and workflow definitions before merge or publication.
• Log prompts, tool calls, and connector changes for later investigation.
• Restrict production access so agents and citizen developers cannot self-approve risky changes.

The NIST Cybersecurity Framework 2.0 is useful here because it maps governance, protection, detection, and response across both traditional applications and AI-assisted delivery. For the AI-specific threat model, the MITRE ATLAS adversarial AI threat matrix helps teams anticipate manipulation of models, prompts, and surrounding tooling.

These controls tend to break down in fast-moving SaaS environments where business users can connect third-party apps directly to production data without central identity governance or application security review.

Common Variations and Edge Cases

Tighter approval gates often slow delivery, requiring organisations to balance developer autonomy against the cost of remediation and incident response. That tradeoff is real, but best practice is evolving toward risk-based controls rather than blanket approval bans, especially where AI agents are used for repetitive tasks or where citizen developers operate inside controlled platforms.

Not every AI-assisted workflow needs the same level of scrutiny. A prototype that works on synthetic data is different from an agent that can call payroll, customer, or code deployment APIs. Similarly, a citizen-built dashboard with read-only access carries less risk than a workflow that can approve transactions or modify identity records. The question is not whether AI or low-code should exist, but which actions they are trusted to perform.

There is no universal standard for this yet, so security teams should define thresholds based on data sensitivity, privilege level, external connectivity, and blast radius. The NIST AI Risk Management Framework and the Anthropic first AI-orchestrated cyber espionage campaign report both reinforce the need to understand agent behaviour in context, not just code quality in isolation. Where applications process regulated or customer-sensitive data, the CSA MAESTRO agentic AI threat modeling framework is especially useful for identifying control gaps around autonomy, delegation, and oversight.

For engineering teams, the practical answer is to separate experimentation from production, limit privileges by default, and require explicit ownership for every agent or citizen-built workflow that can affect real systems.

Related resources from NHI Mgmt Group

• Why do AI agents increase browser security risk for IAM teams?
• How should security teams limit the risk from AI agents that have access to production systems?
• How should security teams reduce risk from AI agents and developer tools that use secrets locally?
• How should security teams reduce prompt injection risk in AI agents?