What should organisations do first when moving from chatbots to agents?

Start by defining the runtime boundaries before scaling the use case. Decide which tools are allowed, who approves execution, how context is managed, and what traces are retained. That sequencing matters because the main risk is not the model itself, but the permissions and automation around it.

Why This Matters for Security Teams

Moving from chatbots to agents changes the risk model from output quality to execution authority. A chatbot can produce a risky answer; an agent can take an action, call a tool, retrieve data, or chain steps across systems. That means the first design question is not whether the model is “good enough”, but what it is allowed to do and under what approval path. Current guidance from the NIST AI Risk Management Framework and the OWASP Agentic AI Top 10 both point to governance, observability, and access control as core controls, not optional add-ons.

Security teams often miss the transition point where a conversational interface becomes an operational actor. At that moment, prompts start influencing privileged workflows, context can leak across tasks, and logs become evidence rather than mere telemetry. The practical risk is broader than prompt injection: it includes tool misuse, over-scoped connectors, unreviewed autonomous retries, and weak human approval boundaries. In practice, many security teams encounter agent risk only after a production workflow has already executed an unintended action, rather than through intentional pre-deployment review.

How It Works in Practice

The first step is to define the agent’s runtime boundary in writing and then enforce it technically. That boundary should specify the allowed tools, the data sources the agent may read, the actions it may request, the approvals required for sensitive steps, and the exact traces that must be retained for review. This is where NHI-style discipline becomes relevant: any API key, token, service account, or delegated identity used by the agent should be treated as a non-human identity with scoped permissions, lifecycle control, and revocation paths.

A practical implementation usually starts with a narrow pilot use case. Teams should map each planned action to an explicit trust decision, then separate read, recommend, and execute permissions. The agent should not inherit broad user entitlements by default. Instead, use least privilege, step-up approval for high-impact actions, and deterministic logging for every tool call, retrieved document, and policy exception. The MITRE ATLAS adversarial AI threat matrix is useful here because it helps teams think through prompt injection, data poisoning, and downstream abuse paths that appear after the model has been connected to tools.

• Classify the use case as advisory, semi-automated, or autonomous before integration begins.
• Assign a named system owner for the agent, its tools, and its credentials.
• Separate retrieval, reasoning, and execution privileges where possible.
• Require approvals for actions that change records, move money, expose secrets, or alter access.
• Retain prompts, tool calls, outputs, and policy decisions for investigation and audit.

Where this breaks down is in legacy environments with brittle integrations, shared service accounts, or flat permission models, because the agent cannot be constrained cleanly without first refactoring the surrounding access architecture.

Common Variations and Edge Cases

Tighter runtime control often increases delivery overhead, requiring organisations to balance speed of automation against the cost of governance and engineering effort. That tradeoff is real, especially when business teams want immediate productivity gains. Best practice is evolving, but there is no universal standard for how much autonomy a first-generation agent should receive. For some workflows, a human-in-the-loop checkpoint is enough; for others, especially those touching customer data or privileged systems, the safer pattern is human-approval-by-default with narrow exceptions.

Edge cases often appear when the agent spans multiple systems or vendors. If context is passed between retrieval, planning, and execution layers, organisations need clear provenance controls so they can tell which source influenced which action. The CSA MAESTRO agentic AI threat modeling framework is helpful for identifying where control boundaries should sit, while the Anthropic report on an AI-orchestrated cyber espionage campaign illustrates how autonomy and tool access can be abused when safeguards are weak. For regulated environments, the governance record should also support internal assurance and policy review, not just technical debugging.

In short, the first move is to constrain the agent before expanding what it can touch. That applies whether the concern is data leakage, workflow abuse, or delegated identity misuse, and it is especially important when the agent is allowed to act on behalf of a person or service account.

Related resources from NHI Mgmt Group

• Should organisations prioritize visibility or least privilege first for AI agents?
• Should organisations prioritise tool scoping or skill governance first for AI agents?
• Should organisations prioritise secrets rotation or policy controls first for agents?
• Should organisations prioritise least privilege or lifecycle governance first for AI agents?