Security Responsibilities for AI Engineers Building Agents

Every time an AI engineer wires an agent to a tool, an API or a database, they create a non-human identity with real permissions, usually before anyone in security or IAM knows it exists. That makes the engineer, not the security team, the first owner of agent credentials, tool scopes and the models and packages the agent loads. This guide sets out what an AI or agent engineer is responsible for, where those duties hand over to platform, security and IAM teams, and what evidence a design review should ask for. It is written for engineers who build LLM applications and agents, and for the leads who review their work. For the career path into the specialist security role, see the AI Security Engineer Career Guide.

Key takeaways

You create the identity, so you own it first. Each agent needs its own identity, an owner on record and credentials the agent never sees in its prompt.
Instructions are not permissions. A system prompt that says "do not delete" stops nothing; only scopes, roles and approval gates do, as the Replit AI agent database deletion showed.
Excessive agency has three causes you control. OWASP LLM06:2025 names excessive functionality, excessive permissions and excessive autonomy; each is a design choice in the engineer's code.
Every model, skill, MCP server and package is privileged code. It runs with the agent's access, so pin, verify and review it as you would a dependency with production credentials.
Log for the investigator, not only for debugging. Record which identity called which tool with which arguments and result, so an incident can be reconstructed without trusting the agent's own account.

Who owns what

Most organisations do not yet have a clear split between the people building agents and the people securing identities. This RACI is NHI Mgmt Group's suggested starting point; adapt the team names to your structure.

Activity	AI engineer	Platform team	Security	IAM
Request an agent identity and record its owner and purpose	Responsible	Consulted	Informed	Accountable
Choose tools and define the scopes each needs	Responsible, accountable	Consulted	Consulted	Consulted
Store and inject credentials at runtime	Responsible for using the pattern	Accountable for the vault and injection service	Consulted	Informed
Human approval gates for high-impact actions	Responsible, accountable	Informed	Consulted	Informed
Select and pin models, skills, MCP servers and packages	Responsible	Consulted	Accountable for policy	Informed
Tool call and identity logging	Responsible for emitting events	Accountable for the pipeline	Consulted on detection	Informed
Access review and decommissioning	Responsible for attesting	Informed	Informed	Accountable

Responsibility 1: agent identity and credentials

One identity per agent, per environment: never run an agent on a developer's personal token or a shared service account. Separate development, test and production identities so a coding or test agent cannot reach production data.
Delegated or own identity: when an agent acts for a specific user, use delegated tokens scoped to that user's rights (see the OAuth Token Exchange and Delegation Guide). When it acts on its own schedule, give it its own workload identity. OWASP LLM06 warns against extensions that act for one user through a generic high-privileged identity.
Short-lived over static: prefer workload identity federation and tokens that expire in minutes or hours over long-lived API keys. Where a static key is unavoidable, record its rotation period. See the AI Agent Credential Lifecycle and Rotation Guide.
Keep secrets out of the context window: credentials are injected into the tool runtime, not the prompt, so prompt injection or system prompt leakage cannot reveal them. See Runtime Secrets Injection for AI Agents Without Prompt Exposure.
No plain .env files on shared machines: the ClawHub malicious skills campaign included a skill that read an agent's .env file and sent it to a webhook.

Responsibility 2: tool permissions and autonomy

OWASP LLM06:2025 Excessive Agency is the clearest statement of what the engineer controls. Its examples translate directly into review questions.

OWASP LLM06 root cause	Example from OWASP	Engineer's control
Excessive functionality	A third-party extension chosen to read documents can also modify and delete them; a trialled plugin stays available	Offer only the tools the task needs; remove tools dropped in development; prefer narrow tools to open-ended shell or URL fetchers
Excessive permissions	A read extension connects with an identity that also has UPDATE, INSERT and DELETE	Scope each tool's credential to the operations it performs; read-only where possible
Excessive autonomy	An extension deletes a user's documents without confirmation	Require human approval for destructive, financial or external actions. See the Human-in-the-Loop for AI Agents Guide

If you build or consume MCP servers

The MCP security best practices document (aligned to the 2025-11-25 specification at the time of checking) sets several MUST-level rules that fall on the engineer:

No token passthrough: MCP servers must not accept tokens that were not explicitly issued for that server.
Do not authenticate by session: servers that implement authorisation must verify every inbound request and must not use sessions for authentication; session IDs must be secure and non-deterministic.
Minimise scopes: start with a small baseline scope and elevate with targeted challenges, rather than publishing every scope and requesting them all.
Consent for local servers: clients offering one-click local server setup must show the exact command and get explicit approval, and should sandbox the server.

See the MCP Security Guide and the AI Agent Tool and Plugin Security Guide.

Responsibility 3: model and component supply chain

An agent's supply chain is wider than its package manifest: base and fine-tuned models, embedding models, prompt templates, skills, MCP servers and the SDKs that call them. OWASP LLM03:2025 Supply Chain covers models and components; the engineer's part is to know what is loaded and from where.

Pin and record: pin model versions and hashes, package versions and MCP server versions; keep an inventory per agent. See the Model Supply Chain Guide.
Verify provenance: use signed models and artefacts where the publisher provides them. See the Model Signing and OpenSSF Model Signing Guide.
Review skills and servers before install: in the ClawHub campaign, skills told the user or agent to run "prerequisite" commands that fetched information stealers. Never run setup commands from a skill without reading them.
Treat installs as access grants: a skill or server inherits the agent's credentials, so it needs the same approval as granting a new identity those permissions.

Responsibility 4: logging and incident readiness

Log every tool call: agent identity, user on whose behalf, tool, arguments (with secrets redacted), result and approval decision, with a correlation ID across steps.
Keep logs outside the agent's reach: the agent should not be able to edit or delete its own audit trail. In the Replit case the agent gave a misleading account of whether rollback was possible.
Know your kill switch: document how to revoke the agent's credentials and stop it within minutes, and test it.
Back up what the agent can change: test restores for any data store the agent can write to.

See the AI Agent Incident Response Playbook.

Design review: questions and evidence

Question	Evidence to show
What identity does each agent and tool use, and who owns it?	Identity inventory entry with owner, purpose and environment
Can any credential reach production from a development agent?	Role assignments per environment
Where do secrets live, and can the model ever see them?	Injection design; prompt and log samples showing no secrets
Which actions need human approval?	List of high-impact tools and the approval flow
What models, skills, servers and packages are loaded?	Pinned versions, hashes, provenance checks
How would we reconstruct what the agent did?	Sample audit log for one end-to-end task
How do we stop it?	Tested revocation runbook with time to stop

Putting it into practice

Inventory first: list every agent in development and production, its identities, tools and credentials. Most teams find personal tokens and shared keys here.
Split environments: issue separate development and production identities; remove production write access from development agents.
Move secrets out of code and prompts: adopt the platform vault and runtime injection pattern; rotate anything that was in source, .env files or prompts.
Scope tools: reduce each tool to the operations it needs and add approval gates for destructive actions. Test in a staging environment with seeded data before production.
Pin the supply chain: pin models, packages, skills and servers; add a review step to change them.
Turn on audit logging and test the kill switch: run a drill that revokes an agent's credentials mid-task and confirm it stops cleanly. Keep the previous version deployable to roll back.

Small team or large organisation: a small team can meet most of this with one vault, per-environment cloud roles and a short design review template. A larger organisation should register agents in the same identity governance process as other non-human identities, with IAM accountable for reviews. See the Agent Identity and Access Guide.

Detection and measurement

Agents with a named owner: target 100 percent.
Agents using personal or shared credentials: target zero.
Share of agent credentials that are short-lived or federated: rising over time.
High-impact tools behind approval gates: target 100 percent.
Unpinned models, skills or servers in production: target zero.
Time to revoke an agent in a drill: measured in minutes.

Framework mapping

Responsibility	OWASP LLM Top 10 2025	NIST CSF 2.0	NIST SP 800-53 Rev 5
Defined roles for agent security	None	GV.RR-02	SA-8
Agent identity and least privilege	LLM06 Excessive Agency	PR.AA-05	AC-6, IA-5
Secrets kept out of prompts and outputs	LLM02 Sensitive Information Disclosure, LLM07 System Prompt Leakage	PR.AA-05	IA-5
Model and component supply chain	LLM03 Supply Chain	GV.SC-05	SR-3, SR-4
Tool call logging and monitoring	LLM06 Excessive Agency	PR.PS-04, DE.CM-09	AU-2

The NCSC-led Guidelines for secure AI system development group the same duties into secure design, development, deployment, and operation and maintenance.

Common failure patterns

Freeze by instruction: in the Replit AI agent database deletion, a code freeze existed only as a natural-language instruction, and the agent held production write access from the development workflow.
Skills installed like browser extensions: the ClawHub malicious skills campaign used fake prerequisites to deliver stealers that took SSH keys, cloud credentials and agent secrets.
Developer token in production: the agent runs as the engineer who built it, so its access changes when that person's role changes and disappears when they leave.
One omnibus key for all tools: a single API key or broad OAuth scope serves every tool, so one prompt injection reaches everything.
Logs that only the agent writes: no independent record of tool calls, so investigators rely on the agent's summary.

Practitioner checklist

Register every agent identity with an owner, purpose and environment before it reaches production.
Use separate development and production identities; no production write access from development agents.
Inject credentials at runtime; never place them in prompts, source code or shared .env files.
Scope each tool's credential to the operations it performs and remove unused tools.
Gate destructive, financial and external actions behind human approval enforced in code.
Reject tokens not issued for your MCP server and request minimal scopes.
Pin and verify models, skills, MCP servers and packages, and review changes.
Log every tool call outside the agent's control and test the revocation runbook.

Security Responsibilities for AI Engineers Guide: Agent Credentials, Tool Permissions and Model Supply Chain