What do security and engineering teams get wrong about AI-assisted development?

Why This Matters for Security Teams

AI-assisted development changes the shape of software delivery because code can be produced faster than the organisation can review, test, and govern it. The core mistake is assuming that speed is a proxy for quality. It is not. A generated snippet may compile cleanly while still introducing insecure defaults, inconsistent error handling, weak dependency choices, or business logic that bypasses existing controls.

This matters because engineering teams usually absorb the change first, while security teams see the downstream effects later in code review, vulnerability findings, incident response, or audit exceptions. Guidance from NIST SP 800-53 Rev 5 Security and Privacy Controls is still relevant here because the control objective is not just to produce software, but to maintain secure configuration, change control, and accountability across the lifecycle. AI assistance can help teams move faster, but it also weakens assumptions about authorship, intent, and review rigor if no one explicitly reasserts them.

In practice, many security teams encounter the risk only after generated code has already been merged into production paths, rather than through intentional engineering governance.

How It Works in Practice

AI-assisted development becomes risky when teams treat the model as a trusted implementation partner instead of a suggestion engine. The output may reflect common internet patterns, not internal architecture, approved libraries, secure-by-design requirements, or regulatory constraints. That means the team must validate more than syntax. It must validate provenance, dependency selection, secret handling, data flow, and whether the generated logic fits the system’s threat model.

Current guidance suggests this is best handled with layered controls: secure prompts, code review, automated testing, policy checks, and traceability of where the code came from and who approved it. The OWASP Top 10 for Large Language Model Applications is useful because it highlights prompt injection, insecure output handling, and supply chain risks that matter when AI tools are part of the delivery path. Security teams should also align with NIST AI Risk Management Framework principles for governance, measurement, and monitoring, even when the “AI system” is a development assistant rather than a customer-facing product.

• Define which tasks AI tools may help with, and which remain human-only.
• Require review for generated code that touches authentication, authorisation, cryptography, or data handling.
• Scan outputs for secrets, unsafe dependencies, and policy violations before merge.
• Preserve traceability so reviewers can distinguish original work from model-generated suggestions.
• Test the final integrated change, not just the generated fragment.

This guidance tends to break down in fast-moving teams with weak branch protection and inconsistent review discipline because the model’s output is merged as if it were already vetted.

Common Variations and Edge Cases

Tighter governance often increases delivery overhead, requiring organisations to balance developer productivity against control assurance. That tradeoff becomes sharper when teams use AI assistants for infrastructure-as-code, security-sensitive refactoring, or rapid prototype-to-production paths. In those environments, a permissive workflow can look efficient until it creates cleanup work, hidden technical debt, or unclear ownership of defects.

There is no universal standard for this yet, but best practice is evolving toward risk-based segmentation. Low-risk tasks such as boilerplate generation may justify lighter review, while changes to identity, session management, payment flows, or data access logic deserve the same scrutiny as manually written code. Where AI-generated content is trained or tuned on proprietary code, supply chain integrity becomes a separate concern: teams need to know whether the tool is exposing internal patterns or reintroducing deprecated practices. That is where alignment with OWASP Application Security Verification Standard can help translate policy into testable checks, even if the tool itself is not formally certified.

The edge case that repeatedly trips teams up is when AI is used to accelerate remediation. If the fix is generated quickly but not validated against the original defect cause, the organisation may ship a new issue while believing the backlog is shrinking.

Related resources from NHI Mgmt Group

• What do security teams get wrong about review loops in AI-assisted development?
• What do security teams get wrong about prompt engineering for AI agents?
• What do security teams get wrong about AI-assisted webpage safety checks?
• What do security teams get wrong about shift-left and AI-assisted review?