Organizations should make security part of the development loop, not a separate phase. That means setting explicit security requirements in prompts, scanning running applications during development, and verifying that remediations actually hold. The goal is to preserve AI-driven velocity while keeping approval, testing, and accountability aligned with production risk.
Why This Matters for Security Teams
AI delivery pressure often pushes teams to treat security as a gate at the end of the pipeline, but that approach creates rework, hidden risk, and slow exceptions. The better model is governance that is proportionate to the application’s risk and embedded from design through release. That includes policy for model use, controls for code and prompt changes, and review paths that can keep pace with iterative builds. The NIST Cybersecurity Framework 2.0 is useful here because it frames security as an ongoing function, not a one-time approval.
Security teams also need to distinguish between the AI model, the application wrapper, and the data and secrets the application can reach. A fast-moving pilot may look safe in a demo while still exposing tokens, unsafe connectors, or weak output handling in production. Governance should therefore focus on where the application can make changes, what data it can retrieve, and what human review is required before those changes are trusted. In practice, many security teams encounter AI governance only after a rushed pilot has already exposed production data or created an unreviewed privilege path, rather than through intentional design.
How It Works in Practice
Balancing speed and security usually means shifting from heavyweight approvals to risk-based guardrails. For AI application development, the governance model should define what is allowed by default, what needs security sign-off, and what must be blocked entirely. That can include approved model sources, restrictions on training data, prompt and output logging, secrets handling, and checks for tool use when an application can call APIs or execute actions. For AI-specific threat thinking, MITRE ATLAS helps teams map likely attack paths such as prompt injection, data poisoning, and inference-time manipulation.
- Classify the AI application by data sensitivity, external exposure, and action authority.
- Set baseline controls for prompts, model endpoints, plugins, connectors, and retrieval sources.
- Automate checks in CI/CD for secrets, dependency risk, and unsafe configuration.
- Require runtime validation for high-impact outputs before an application can act on them.
- Track ownership so a model, prompt, or tool change has a clear approver and rollback path.
Operationally, the best results come from making security visible inside the same workflow developers already use. That means code review for application logic, policy-as-code for deployment rules, and ongoing validation after release so a fix does not disappear in the next sprint. Where agentic systems are involved, the identity of the agent matters too: tool permissions, service credentials, and delegation scope should be controlled like privileged access. Current guidance suggests this is where application security and NHI governance converge most strongly, especially when agents can reach production systems. These controls tend to break down when teams ship experimental agent workflows directly into broad network and data access because the approval boundary no longer matches the system’s real authority.
Common Variations and Edge Cases
Tighter governance often increases cycle time, so organisations have to balance developer autonomy against the cost of rework and the risk of unsafe release. That tradeoff becomes more visible in teams that ship frequent model updates, run multiple vendors, or allow business users to assemble AI features with low-code tools. Best practice is evolving here, and there is no universal standard for how much approval an AI change should require.
One common edge case is internal-only AI tools that later become customer-facing. A control set that felt adequate for a sandbox may be weak once regulated data, customer actions, or payment flows are introduced. Another is RAG-based applications, where model behaviour is often less risky than the retrieval layer and the documents it can surface. In those cases, governance should focus on source approval, document provenance, and output validation, not just the model itself. For identity-heavy environments, NHI controls become relevant when AI components use service accounts, tokens, or API keys to fetch data or trigger workflows.
Security leaders should also plan for exceptions. If a team needs to move quickly, the exception path should still require time-bound approval, scoped access, and measurable compensating controls. The aim is not to slow AI delivery for its own sake, but to keep the blast radius of a mistake small enough that speed remains sustainable.
Standards & Framework Alignment
This section maps relevant standards and security frameworks to the operational risks and controls described in this guidance.
MITRE ATLAS and OWASP Agentic AI Top 10 address the attack and risk surface, while NIST CSF 2.0, NIST AI RMF and NIST AI 600-1 set the governance and control requirements practitioners need to meet.
| Framework | Control / Reference | Relevance |
|---|---|---|
| NIST CSF 2.0 | GV.OV-01 | Governance and oversight are central to balancing AI speed with security. |
| NIST AI RMF | GOVERN | AI governance frames accountability, policy, and monitoring for AI delivery. |
| MITRE ATLAS | T0012 | Prompt injection and related adversarial AI attacks affect app security decisions. |
| OWASP Agentic AI Top 10 | LLM07 | Agentic tool misuse is a key risk when AI systems can act beyond chat. |
| NIST AI 600-1 | GenAI profile guidance fits secure development and output validation. |
Restrict tool permissions and validate actions for any AI system with execution authority.
