Because the organisation ends up automating an undefined workflow. That usually produces vague success criteria, unclear ownership, and weak data controls. The result is a system that looks innovative but does not reliably improve the underlying process or protect the information flowing through it.
Why This Matters for Security Teams
AI programmes fail fastest when the team starts with a model, chatbot, or agent and only later asks what business problem it is meant to solve. That reversal produces automation without process clarity, which means success criteria, data boundaries, and owner accountability stay vague. Security then inherits an unstable design surface: undefined inputs, overbroad access, and a growing set of secrets and integrations that nobody can justify cleanly.
This is especially visible when teams treat AI as a feature rather than an operating change. The State of Secrets in AppSec research shows how quickly weak operational discipline becomes security debt, and the same pattern appears in AI rollouts when credentials, prompts, and source data are assembled before the workflow is understood. Current guidance in ISO/IEC 42001:2023 AI Management System Standard supports governance-first thinking, but many programmes still invert that sequence.
In practice, many security teams encounter AI risk only after an undefined pilot has already connected to production data, rather than through intentional process design.
How It Works in Practice
A problem-first AI programme starts by defining the workflow, the decision it should support, and the boundaries around data use. Only then should the team decide whether the right answer is an LLM, an agent, a rules engine, or no AI at all. That order matters because an AI system can only be governed effectively when its purpose, inputs, and acceptable outputs are explicit.
For security teams, the practical test is simple: can the process be described without naming a vendor tool? If not, the programme is probably still at the solution-seeking stage. The strongest implementations define ownership, classify the data involved, and set review points for human approval before any model gets access to live records, code, or secrets. The DeepSeek breach is a reminder that AI systems become materially risky when data handling is unclear and controls are bolted on too late.
In practice, mature teams usually follow this sequence:
- Describe the business outcome in plain language.
- Map the workflow and identify where AI adds value.
- Classify the data and secrets the workflow would touch.
- Set success metrics, guardrails, and approval ownership before deployment.
- Reassess whether automation is still justified after the first pilot.
When teams do this well, AI becomes one control option among several, not the organising principle for the entire process. This guidance breaks down when the organisation insists on retrofitting AI into a heavily regulated workflow that already lacks process ownership, because the control baseline is too weak to support safe automation.
Common Variations and Edge Cases
Tighter AI governance often increases delivery time, requiring organisations to balance speed against control. That tradeoff is real, especially when leadership wants a visible pilot before the underlying process has been stabilised. Current guidance suggests that a narrow, well-scoped use case is better than a broad platform launch, but there is no universal standard for how much process maturity is enough.
One common edge case is the “AI assistant for everything” programme, where teams buy a general tool and then search for use cases. That approach often creates fragmented ownership, duplicated integrations, and unclear data retention. Another is automation of a broken workflow: if the manual process is already inconsistent, AI simply scales the inconsistency. Security leaders should also be cautious with pilots that touch secrets, customer data, or internal code, because the access model may be too broad for the actual task.
Best practice is evolving, but the safe default is to force a problem statement first, then choose the smallest possible technical solution. If the use case cannot be defended without the tool, it is probably not ready for automation.
Standards & Framework Alignment
This section maps relevant standards and security frameworks to the operational risks and controls described in this guidance.
OWASP Agentic AI Top 10 and CSA MAESTRO address the attack and risk surface, while NIST AI RMF and NIST CSF 2.0 set the governance and control requirements practitioners need to meet.
| Framework | Control / Reference | Relevance |
|---|---|---|
| NIST AI RMF | GOVERN 1 | Calls for clear AI accountability and purpose before deployment. |
| OWASP Agentic AI Top 10 | AA-01 | Prevents tool-first agent design that expands attack surface and uncertainty. |
| CSA MAESTRO | GOV-02 | Requires governance of agent purpose, scope, and oversight from the outset. |
| NIST CSF 2.0 | GV.OC-01 | Supports business-context definition so controls map to real objectives. |
Document intended outcomes, data boundaries, and human oversight before enabling automation.
Related resources from NHI Mgmt Group
- How should security teams handle risks from AI browser extensions?
- How should security teams govern API keys used for generative AI access?
- When do AI agents become an NHI governance problem instead of an automation tool?
- How should teams govern AI consumption when spend is spread across multiple tools?
Deepen Your Knowledge
Reviewed and updated by the NHIMG editorial team on July 9, 2026.
NHI Mgmt Group — the #1 independent authority on Non-Human Identity, IAM, and Agentic AI security. nhimg.org
