Why is inventorying AI agents harder than counting model deployments?

Counting AI agents means counting behavior, not only deployments

An AI agent inventory is harder because the unit of control is not the model endpoint alone. A meaningful count has to include registered agent definitions, execution paths, tool permissions, delegated sub-agents and the operational context that turns a shared model into a different system.

The same model can sit behind multiple agents with different prompts, policies, tools and human approval gates, so two deployments may look identical at the infrastructure layer while behaving very differently in production. That is why inventorying agents is closer to inventorying agent identity, registration and delegation than counting model instances.

In practice, the inventory has to answer what exists, who or what can invoke it, what it can do, and how far its delegated authority reaches. Without those fields, the inventory records a model presence but misses the actual control surface that matters for governance and security.

Why the operational layer creates inventory ambiguity

Agents are dynamic systems. They may spawn loops, call tools, hand work to sub-agents, or change behavior based on retrieved context and policy, which means the apparent deployment footprint can be much smaller than the effective runtime footprint. NHIMG’s AI Agents vs Agentic AI explains why autonomy level changes the object being governed, and that matters directly for inventory design.

Model catalogs usually stop at endpoint, version and environment. Agent inventories have to capture the orchestration layer as well, including harness logic, connectors, callbacks, memory, and any decision points where the agent can branch, retry, escalate or delegate. A single visible agent may therefore represent several hidden operational states.

That is also why inventory should be tied to observable action, not only code registration. If an agent can act through tools, shared credentials, or delegated access, the security team needs a record of those authorities even when the underlying model binary never changes.

What a useful AI agent inventory should contain

A practical inventory should identify each agent, its owner, environment, purpose, and control boundaries, then link that entry to the model it uses and the tools or systems it can reach. It should also show whether the agent acts directly, acts on behalf of a user, or passes work to another agent.

• Registered agent name or identifier, plus business owner
• Model or models used, including version or provider where relevant
• Connected tools, APIs, data sources and runtime permissions
• Delegation relationships, including sub-agents and handoffs
• Approval requirements, policy checks and exception paths
• Lifecycle state, such as active, paused, retired or shadow

NHIMG’s Shadow AI and AI Agent Discovery Guide is useful here because discovery and inventory are the same problem once agents start appearing through OAuth grants, API keys, SaaS integrations and other indirect signals. If you cannot find the runtime authorizations, you do not really have an inventory yet.

For teams building the inventory from scratch, the most important distinction is between a model deployment record and an agent record. The former answers where the model is hosted; the latter answers what the autonomous system can actually do.

Risk and Threat Considerations

Inventory gaps create blind spots in access governance, and blind spots are exactly what attackers and misuse paths exploit. An organisation may believe it has a small number of deployed agents while actually operating many runtime instances with different permissions, tool paths and delegation chains.

Failure mechanism: Shared models, dynamic tool use, and hidden sub-agent behavior cause the control plane to undercount active agentic systems, leaving unreviewed access paths, stale permissions and unknown business impact.

Impact: Missed inventory entries can translate into excessive privilege, undetected shadow agents, incomplete offboarding, and weaker incident response when an agent behaves badly or is compromised.

Practitioner Guidance

What to verify: Verify that every agent record maps to a real runtime principal, not just a code repository or model endpoint. If the system can invoke tools, reach data, or delegate work, the inventory should expose those permissions explicitly.

What good looks like: A good inventory lets you answer, for each agent, who owns it, what it can access, what it can delegate to, and how to retire it cleanly. If those answers require manual spelunking across logs and configuration files, the inventory is too shallow to trust.

Common mistake: Treating all uses of one model as one agent is the fastest way to undercount risk. The useful unit is the autonomous operating context, not the shared model asset.

Practitioner takeaway: Inventory AI agents by authority and runtime behavior first, because the security-relevant object is the executable system that can act, delegate and access tools, not the model endpoint that happens to power it.

Related resources from NHI Mgmt Group

• How should organizations approach the governance of AI agents?
• Why does identity security become harder when workloads and AI agents are part of the access model?
• Why do AI agents complicate trust and governance more than traditional model deployments?
• Why do AI agents become harder to secure when the model runtime is reachable from shared infrastructure or local networks?