Google ADK observability exposes the control gap in multi-agent systems

At a glance

What this is: This is a how-to article on tracing and evaluating Google ADK multi-agent systems, with the key finding that production observability is required to understand routing, tool use, and handoffs.

Why it matters: It matters because agentic workflows introduce governance and assurance gaps that are invisible in demos, and IAM and AI security teams need trace-level evidence before they can trust agent decisions in production.

Context

Multi-agent systems are defined by delegation, tool use, and stateful handoffs between specialised agents. That architecture can work in a demo while still hiding decision errors, routing mistakes, and brittle dependencies that only appear under production load.

The governance problem is that teams cannot certify or improve what they cannot observe. In agentic systems, the control point shifts from a single model response to the chain of prompts, tool calls, and inter-agent transfers, which makes traceability and evaluation part of the security baseline.

For identity and access teams, this creates a genuine intersection with agentic AI governance: each agent behaves like a non-human actor whose scope, instruction path, and tool permissions need to be measurable, not assumed.

Key questions

Q: How should teams govern AI agent handoffs across workflows?

A: They should define each handoff as a governed trust boundary, not a casual message exchange. That means assigning ownership, limiting inherited permissions, and validating the source of context before one agent can influence another. Without that discipline, a single compromised agent can contaminate the whole chain.

Q: Why do multi-agent systems need specialised evaluation beyond standard LLM testing?

A: Because the main failure modes are often procedural rather than linguistic. A system can generate a good answer while choosing the wrong agent, calling the wrong tool, or taking an inefficient trajectory. Specialised evaluation surfaces those issues by measuring handoff quality, tool selection, and path correctness instead of only response quality.

Q: What breaks when agent observability is missing in production?

A: Teams lose the ability to explain routing decisions, isolate bad tool calls, and identify where coordination failed. That makes debugging slower, cost control weaker, and governance evidence thin. In practice, the system may still work in visible cases while drifting into unreliable behaviour under real user traffic.

Q: What is the difference between tracing and evaluation in agent governance?

A: Tracing captures what happened during execution, while evaluation judges whether that behaviour met the standard you set. Good governance needs both. Traces provide the evidence, and evaluation turns that evidence into repeatable decisions about release readiness, drift, and control effectiveness.

Technical breakdown

How Google ADK structures multi-agent orchestration

Google ADK uses a code-first, modular pattern where a root orchestrator delegates work to specialist agents. That makes it easier to build deterministic flows for some steps and adaptive LLM-driven routing for others, but it also increases the number of decision points that can go wrong. In practice, the architecture is less about a single model answer and more about coordination, state management, callbacks, and routing logic across several task-specific agents. The security-relevant issue is that each handoff creates an execution boundary that must be governed, traced, and validated if the system is expected to behave predictably in production.

Practical implication: treat routing rules and sub-agent permissions as governed control points, not just application logic.

Why observability must capture every decision, tool call, and handoff

Observability for agent systems needs to record the full execution path, not just final outputs. OpenTelemetry-style tracing makes it possible to see why a planner agent was selected, which tool was invoked, how long it took, and where one agent passed context to another. That is essential because non-deterministic behaviour is not visible through static testing or simple response sampling. In an agentic system, the real failure may be a bad handoff, a redundant tool call, or a routing choice that only appears under specific user inputs. Without structured traces, those problems remain operational guesswork.

Practical implication: instrument agent traces early so investigation, tuning, and governance can use evidence rather than anecdote.

Why evaluation for agentic systems must go beyond standard LLM metrics

Standard LLM evaluation is not enough when a system is made of multiple cooperating agents. Multi-agent evaluation has to measure handoff quality, tool-selection accuracy, and trajectory correctness, because a system can produce a plausible final answer while still taking the wrong path to get there. That matters for reliability, cost, and security, since poor trajectories often correlate with unnecessary tool exposure or brittle behaviour under edge cases. For operational governance, evaluation should therefore assess the chain of action, not just the language quality of the answer.

Practical implication: define evaluation criteria around routing, tool use, and trajectory quality rather than only output correctness.

NHI Mgmt Group analysis

Observability is now a governance control for agentic systems, not just an engineering convenience. When a multi-agent workflow delegates across specialised sub-agents, the organisation is no longer governing one model response. It is governing a chain of decisions, tool invocations, and context transfers that can fail in ways a demo never reveals. The practitioner takeaway is that traceability becomes part of the control surface.

Agent handoffs create the kind of blind spot that traditional application testing does not cover. A workflow can appear functional while still routing tasks to the wrong agent, invoking the wrong tool, or accumulating hidden cost and latency. That makes evaluation data a security and reliability artefact, not a mere product metric. The practitioner conclusion is to treat trajectory inspection as a production requirement.

Agentic AI introduces a non-human identity problem even when the article is framed as observability. Each specialised agent behaves like a bounded software actor with its own permissions, instructions, and downstream effects. That means identity security thinking must extend from authentication into authorisation, delegation, and oversight of tool-enabled behaviour. The practitioner conclusion is that agent identity and observability have to be designed together.

Control gaps in multi-agent systems are usually coordination gaps first and model-quality gaps second. The article’s core lesson is that orchestration quality, not only model capability, determines whether the system is governable at scale. If evaluation only measures the final answer, teams miss the operational drift happening in the middle of the trace. The practitioner conclusion is to measure the process, not just the result.

What this signals

Agentic observability will become a baseline requirement for organisations that want to operationalise multi-agent systems. The practical shift is from inspecting outputs to governing execution paths, which means tracing, evaluation, and regression analysis sit closer to production control than traditional prompt testing. Teams that delay this work will accumulate invisible complexity in agent handoffs and tool selection.

Agent trajectory drift is a better warning signal than isolated output errors. In multi-agent environments, the same user request can trigger different paths depending on context, state, and routing logic, so the useful question is whether the system still follows a defensible sequence under load. That makes trace review and curated failure datasets central to continuous improvement.

Agent identity and observability have to be designed together. Once specialised agents are allowed to call tools, hand off work, and retain state, the organisation is managing non-human actors with distinct authority boundaries. The control question becomes whether each agent’s scope is observable, reviewable, and constrained before it acts.

For practitioners

• Instrument end-to-end agent traces Capture every routing decision, tool call, and agent handoff in production so you can investigate the full path a request took.
• Define agent-specific evaluation criteria Measure handoff quality, tool-selection accuracy, and trajectory correctness for each production workflow instead of relying on generic LLM output scoring.
• Review tool permissions by agent role Map each specialist agent to the minimum tools it needs and validate that routing logic cannot expand access beyond intended task scope.
• Establish regression datasets from failed traces Curate traces that show routing errors, impossible requests, or repeated tool failures so prompt and policy changes can be tested before release.

Key takeaways

• Multi-agent systems can appear reliable in demos while still hiding coordination failures, routing mistakes, and tool-use errors that only show up under production conditions.
• Trace-level observability and evaluation are the controls that turn agent behaviour from opaque execution into evidence that can be reviewed and improved.
• For agentic AI programmes, the main governance challenge is not whether agents can respond, but whether their decision paths, permissions, and handoffs are measurable and bounded.

Key terms

• Agent Orchestration: Agent orchestration is the coordination of multiple AI agents or workflows to complete a task set with limited human intervention. In identity terms, it creates delegated execution paths that need ownership, scope limits, and auditability because work is no longer performed only by a person in one session.
• Agent Handoff: Agent handoff is the point at which a system passes an unresolved case from structured automation to an AI agent. The handoff should occur only after the workflow has collected context, because the agent is then extending evidence rather than inventing a case from scratch.
• Agent Trajectory: The sequence of intermediate steps an AI agent takes while trying to complete a task, including reasoning, tool use, and decision timing. Evaluating trajectory matters because a system can appear correct at the end while taking unsafe, inefficient, or unauthorized actions along the way.
• Observability: Observability is the ability to understand the internal state of a system from the data it produces. In security and operations, that means combining logs, metrics, and traces so teams can explain why something happened, not just confirm that something changed.

Deepen your knowledge

NHI Mgmt Group covers identity security, NHI governance, and agentic AI through independent research, practitioner guides, and the NHI Foundation Level course. Explore the course to strengthen how your team governs machine identities, delegation, and access scope in modern AI systems.