AI coding tools can still produce insecure output because prompt discipline does not guarantee secure implementation. The model may generate vulnerable patterns, omit controls, or reproduce unsafe defaults. Organisations should treat generated code as untrusted until it has passed policy checks, vulnerability verification, and functional revalidation inside the workflow.
Why This Matters for Security Teams
Security-aware prompts reduce careless output, but they do not change the model’s underlying tendency to optimise for plausibility rather than assurance. AI coding tools can still emit unsafe defaults, weak secret handling, insecure deserialization, or missing access checks even when the developer explicitly asks for secure code. NHI Management Group has documented how secret exposure and AI-assisted code generation can amplify one another in The State of Secrets in AppSec, where 43% of security professionals said they are concerned about AI systems learning and reproducing sensitive information patterns from codebases.
The operational issue is not prompt quality alone. It is that generated code enters a real delivery pipeline with real permissions, real dependencies, and real data paths. A prompt can request parameterisation, but it cannot guarantee the model will preserve it across a larger refactor, a framework change, or a follow-up edit. That is why current guidance from the NIST Cybersecurity Framework 2.0 still maps well here: software must be governed, validated, and monitored as it moves from creation into production use. In practice, many security teams discover prompt-safe code is still unsafe only after it has already been committed, deployed, or copied into a high-trust service.
How It Works in Practice
Prompt-aware development should be treated as a first pass, not a control. The better pattern is to insert checks that evaluate the output itself, then verify whether the implementation matches the security intent. That means combining secure prompting with policy-as-code, dependency scanning, secret detection, and functional tests that exercise the risky path rather than trusting the model’s explanation of what it produced.
For teams building AI-assisted delivery pipelines, the most reliable controls are layered:
- Use prompts to express intent, such as required auth checks, input validation, and secret-free examples.
- Run generated code through static analysis and secret scanning before merge.
- Revalidate security controls after model-driven edits, because a later suggestion may remove a safeguard.
- Require human review for privilege boundaries, cryptographic code, and data handling logic.
- Track the provenance of generated code so reviewers know which sections were AI-assisted.
This aligns with the broader governance direction in OWASP NHI Top 10, which treats generated or autonomous behaviour as something that must be bounded by policy, not merely guided by prompts. It also fits the security lifecycle emphasis in NIST guidance, where controls must be verifiable at the point of use, not just declared in advance. These controls tend to break down when teams let AI tools write directly into production repositories because review pressure drops and unsafe patterns are copied faster than humans can inspect them.
Common Variations and Edge Cases
Tighter prompt controls often increase developer overhead, requiring organisations to balance speed against assurance. That tradeoff is real, especially in small teams that use AI coding assistants for routine scaffolding and assume security prompts are enough.
Best practice is evolving, but current guidance suggests several edge cases need extra scrutiny. Code generated for authentication, payment flows, or secrets management should be treated as high-risk regardless of prompt quality. The same applies when the tool is asked to “just fix” a failing test, because models may preserve test success while quietly weakening security logic. Another common exception is boilerplate generation from a vulnerable codebase: if the training context or local repository already contains unsafe patterns, the tool may reproduce them even when asked not to.
Security teams should also distinguish between prompts that describe desired outcomes and controls that can prove those outcomes. A prompt can request “secure by default,” but only enforcement can verify whether the output actually avoids hardcoded secrets, injection paths, or missing authorization checks. NHIMG’s reporting on secret exposure and AI-assisted risk in LLMjacking and The State of Secrets in AppSec shows why this matters: attackers and models both benefit from weak secret hygiene, so prompt discipline alone is not a control boundary.
Standards & Framework Alignment
This section maps relevant standards and security frameworks to the operational risks and controls described in this guidance.
OWASP Agentic AI Top 10, OWASP Non-Human Identity Top 10 and CSA MAESTRO address the attack and risk surface, while NIST AI RMF and NIST CSF 2.0 set the governance and control requirements practitioners need to meet.
| Framework | Control / Reference | Relevance |
|---|---|---|
| OWASP Agentic AI Top 10 | LLM-05 | Prompting does not prevent insecure generated code or unsafe model outputs. |
| OWASP Non-Human Identity Top 10 | NHI-03 | AI tools can reproduce secrets and unsafe patterns from codebases. |
| CSA MAESTRO | A2 | Autonomous generation needs runtime governance, not prompt-only guidance. |
| NIST AI RMF | MAP | Prompt-safe output still needs mapped risk assessment and validation. |
| NIST CSF 2.0 | PR.DS-1 | Generated code can mishandle sensitive data and secrets. |
Use data protection controls and scanning to stop AI output from exposing sensitive material.
Related resources from NHI Mgmt Group
- Why do AI coding tools create a security risk even when code looks correct?
- Why do AI coding agents create security risk even when they use the same model?
- Why does AI-assisted development increase security risk even when developers use familiar controls?
- Why do AI security tools create governance risk even when they only generate findings?
