Traditional controls were built to watch endpoints, networks, and static data flows, not conversational decision-making. They can show that an action happened, but not why an AI produced it, what prompt influenced it, or whether a user pushed the system outside policy. That blind spot leaves prompt injection, unsafe data sharing, and agent overstepping under-detected.
Why This Matters for Security Teams
Traditional security controls were designed for bounded systems: endpoints, networks, applications, and well-defined human workflows. Human and AI collaboration breaks that model because the risky action is often produced through conversation, context reuse, and tool invocation rather than a single executable event. A team can log access, but still miss the prompt that induced the model, the retrieved context that altered output, or the user action that turned a suggestion into an automated decision.
This matters because the failure mode is not just data leakage. It also includes policy bypass, over-permissioned agent action, and unauthorised reliance on AI output in operational decisions. The right question is no longer only whether a control blocked a connection or copied a file, but whether the system preserved intent, constrained authority, and made decisions explainable enough for review. The NIST Cybersecurity Framework 2.0 is useful here because it pushes teams to connect governance, protection, detection, response, and recovery, not just perimeter monitoring.
In practice, many security teams encounter this only after a harmless-looking assistant interaction has already triggered unsafe sharing, a bad approval, or an agent action that should never have been possible.
How It Works in Practice
Security teams need to instrument the collaboration layer, not just the infrastructure underneath it. That means capturing prompts, retrieved context, model outputs, tool calls, and user confirmations as part of the audit trail, then correlating them with identity, role, and policy state. Without that chain of evidence, it is difficult to distinguish a legitimate decision from a manipulated one.
Operationally, the strongest approach combines prevention, monitoring, and post-action review. Current guidance suggests treating AI interactions like high-value decision points: validate inputs, limit what the model can see, constrain what an agent can do, and require human approval for high-impact actions. The OWASP Top 10 for Large Language Model Applications is a practical reference for risks such as prompt injection, insecure output handling, and excessive agency. For broader AI governance, the NIST AI Risk Management Framework helps teams connect those technical controls to accountability, transparency, and lifecycle oversight.
- Log prompts, retrieved documents, tool actions, and final outputs together.
- Apply least privilege to both users and AI agents, including scoped tool access.
- Separate advisory output from executable actions so recommendations do not become silent automation.
- Require policy checks before the model can access sensitive data or external systems.
- Review incidents for both malicious input and unsafe operator behaviour.
Where agentic systems are involved, identity becomes part of the control plane: the organisation must know which human approved an action, which agent executed it, and which privileges were active at the time. These controls tend to break down when organisations connect general-purpose models to live SaaS, ticketing, or code deployment systems without action scoping, because the model inherits real authority faster than governance can adapt.
Common Variations and Edge Cases
Tighter control often increases friction and slows workflows, requiring organisations to balance speed against assurance. That tradeoff is real, especially in teams that use AI for drafting, triage, or operational support. Best practice is evolving, and there is no universal standard yet for how much prompt logging, content inspection, or human review is enough in every environment.
Some environments need stronger constraints than others. In regulated sectors, AI-generated recommendations may need stronger traceability than ordinary productivity use. In software delivery, the main risk may be agents creating or changing code with insufficient review. In customer-facing use, the issue may be unsafe disclosure or hallucinated advice. In all of these cases, traditional controls still matter, but they need to be extended into model governance, tool governance, and human decision governance.
The practical test is simple: if an organisation cannot answer who influenced the AI, what data it used, what action it was allowed to take, and who approved the result, then the control set is not yet fit for human and AI collaboration. That gap is especially visible in environments that rely on fast-moving integrations, delegated credentials, or shadow AI use across business units.
Standards & Framework Alignment
This section maps relevant standards and security frameworks to the operational risks and controls described in this guidance.
OWASP Agentic AI Top 10 and MITRE ATLAS address the attack and risk surface, while NIST CSF 2.0, NIST AI RMF and NIST AI 600-1 set the governance and control requirements practitioners need to meet.
| Framework | Control / Reference | Relevance |
|---|---|---|
| NIST CSF 2.0 | GV.OV-01 | Oversight is needed when AI decisions and human approvals blend together. |
| NIST AI RMF | AI risk management addresses model behaviour, context use, and accountability. | |
| OWASP Agentic AI Top 10 | Agentic systems create risks from overreach, tool misuse, and prompt manipulation. | |
| MITRE ATLAS | ATLAS covers adversarial tactics such as prompt injection and model manipulation. | |
| NIST AI 600-1 | The GenAI profile focuses on safeguards for generative AI deployments. |
Define governance for AI use cases and review whether controls actually reduce decision risk.
Related resources from NHI Mgmt Group
Deepen Your Knowledge
Reviewed and updated by the NHIMG editorial team on August 26, 2026.
NHI Mgmt Group — the #1 independent authority on Non-Human Identity, IAM, and Agentic AI security. nhimg.org
