GitHub Copilot observability exposes the governance gap in agentic AI

At a glance

What this is: Fiddler argues that GitHub Copilot telemetry can expose how the agent is actually used, including prompts, tool calls, and sensitive-data events, which reveals a governance gap in AI operations.

Why it matters: This matters because AI governance teams need evidence of agent behaviour, not just seat counts, to control risk, cost, and compliance across coding workflows.

Context

GitHub Copilot introduces a familiar governance problem in a new form: access does not equal observability. In AI-assisted coding, the question is no longer simply who has a license, but what the agent does after a prompt is issued, what data it touches, and whether sensitive context is being exposed or retained in ways teams cannot see.

That matters for AI governance, security, and engineering leaders because agentic workflows compress many actions into a single session. When a coding agent can call tools, read and write files, and execute commands, the control problem shifts from deployment to oversight. For teams running Copilot alongside other agentic tools, telemetry becomes part of the governance boundary, not just an operational nice-to-have.

Key questions

Q: What breaks when coding agents are deployed without telemetry?

A: Teams lose the ability to show what the agent did, what it touched, and whether it exposed sensitive data. That creates a governance blind spot where security, compliance, and engineering cannot reconstruct risky behaviour from seat counts or invoices alone. In practice, policy enforcement shifts from evidence-based oversight to assumption.

Q: When should organisations prioritise observability over broader agent autonomy?

A: They should prioritise observability first when agents can call tools, write files, or execute commands on shared codebases. Without runtime visibility, autonomy expands faster than governance can verify safe behaviour. The practical sequence is to measure, classify, and alert before granting wider operational reach.

Q: What are the signs that Copilot governance controls are failing?

A: Common warning signs include sensitive files appearing in Copilot responses, inconsistent label coverage across Teams or SharePoint content, and unlabeled non-Office files such as images or PDFs. A weak audit trail, excessive prompt activity, or repeated policy violations in compliance reviews also suggest control gaps. If users can easily retrieve restricted material through AI, governance is not working as intended.

Q: How should teams govern auth code written by coding agents?

A: Use a fixed development path with known skills, local validation, and traceable tool use, then require explicit checks for first-party origin, route matching, cookie forwarding, and runtime configuration. That gives reviewers evidence that the generated code followed the intended auth pattern rather than a plausible approximation.

Technical breakdown

Why Copilot telemetry changes the control point

OpenTelemetry gives teams a structured way to capture what a coding agent is doing at runtime. In this case, the telemetry includes prompts, responses, tool calls, token usage, models invoked, session duration, and repository or branch context. That matters because an agentic system can complete multiple actions inside one developer session, so post-hoc reviews often miss the sequence that created the risk. Observability makes the session auditable, but it does not itself enforce policy. The control point moves closer to execution time, where behaviour can be measured before it becomes a security or compliance issue.

Practical implication: treat agent telemetry as a governance input and require runtime logging before you can trust agent activity at scale.

Why environment variables are enough to create a governance path

The setup described here uses three environment variables rather than a custom SDK or collector, which lowers the barrier to capturing telemetry across developer seats. That simplicity is operationally useful, but it also means governance depends on how consistently the organisation controls those variables, endpoints, and capture settings. If telemetry can be enabled or redirected through local configuration, then the real control is policy around approved endpoints, environment provisioning, and workstation hygiene. For AI governance teams, this is the same lesson seen in other identity-adjacent controls: convenience without standardisation creates blind spots.

Practical implication: inventory and standardise agent telemetry configuration so approved observability cannot drift by workstation or team.

Semantic mappings matter when agent ecosystems do not agree

The article points to a common AI operations problem: different tools emit different attribute names, even when they speak the same underlying telemetry language. Semantic mappings are therefore a normalisation layer, translating varied OpenTelemetry fields into a common control view. Without that layer, teams end up with fragmented dashboards, incomplete policy signals, and inconsistent reporting across tools such as Copilot, coding agents, and AI gateways. This is where AI governance starts to resemble data governance: the organisation needs a shared schema before it can compare behaviour, enforce standards, or explain risk to leadership.

Practical implication: define a canonical telemetry schema before you expand observability across multiple agentic tools.

NHI Mgmt Group analysis

Observability has become the first governance control for agentic coding tools. Once a coding agent can plan, call tools, and write outputs, seat management no longer tells you what risk you have. The governance question becomes whether you can evidence prompts, actions, and sensitive-data events well enough to constrain behaviour before it affects code or data.

Telemetry without policy still leaves a control gap. Seeing prompts and tool calls is useful, but visibility alone does not prevent unsafe actions, credential exposure, or accidental data movement. The meaningful shift is that teams can now connect Copilot activity to monitoring, alerting, and policy enforcement, which is the minimum viable structure for agent governance.

Agent governance is converging on a shared control plane model. The article reflects a broader market direction in which coding agents, gateways, guardrails, and executive dashboards sit in one operational view. That is a strong signal that AI governance is moving from point tools to integrated oversight, and practitioners should plan for shared telemetry rather than isolated controls.

For identity and access teams, Copilot telemetry is a machine identity issue in disguise. The agent is not a user in the human sense, but it acts with delegated access, tool permissions, and repository reach that must be bounded and reviewed. That makes agent behaviour part of the broader identity governance problem, especially where prompts can surface secrets or internal data.

Governance debt accumulates quickly when agent adoption outpaces measurement. The article’s core warning is not that Copilot is present, but that enterprises have already deployed it widely without equivalent visibility. The practitioner lesson is to treat observability, classification, and policy enforcement as prerequisites for scaling agentic development, not as later-stage enhancements.

From our research library:

• Developers using GenAI tools like GitHub Copilot are reporting 35% productivity gains, according to IDC’s 2024 Generative AI Study.

What this signals

Observability is becoming the minimum governance layer for agentic development. Once coding agents can act across repositories and tools, the programme question changes from adoption to evidence. Security and AI governance teams should assume that usage data, prompts, and tool activity will increasingly be required for board reporting, incident reconstruction, and control assurance.

Agent telemetry should be treated as a control boundary, not a logging extra. If the organisation cannot see prompts, tool calls, and sensitive-data events in one place, it cannot reliably distinguish safe automation from unmanaged behaviour. The next step for practitioners is to standardise what gets captured before agent deployment expands further.

For practitioners

• Implement runtime telemetry for coding agents Capture prompts, tool calls, model usage, token consumption, and repository context before expanding agent access beyond pilot groups.
• Standardise agent telemetry schema and endpoints Define approved OTel attributes, field mappings, and ingestion endpoints so every team reports comparable session data.
• Apply policy checks to sensitive session events Alert on prompts or responses that contain secrets, credentials, or regulated data, and tie those alerts to response workflows.
• Measure AI productivity against verified usage Compare session volume, token spend, and developer output so leadership can distinguish adoption from actual value creation.

Key takeaways

• GitHub Copilot telemetry highlights a broader AI governance gap because agentic systems can act too quickly for seat-based oversight to provide meaningful control.
• The article shows that prompts, tool calls, token usage, and repository context are now observable, which changes how teams can measure risk and value.
• Practitioners should treat runtime observability, schema standardisation, and policy enforcement as the baseline for scaling coding agents safely.

Key terms

• Agent telemetry: Operational data describing what an AI agent did, which tools it called, how much it consumed, and what resources it touched. Telemetry is essential for audit, incident response, and behavioural governance because agent decisions are otherwise difficult to reconstruct.
• Control Plane: The control plane is the set of actions that create, configure, or manage a service. For AI workloads, it covers deployment and administration of the model platform, while data-plane permissions govern what the service and its identities can read or process.
• Semantic Mapping: Semantic mapping connects technical metadata to business meaning so users and systems interpret a data asset consistently. It reduces ambiguity across teams by aligning terms, definitions and relationships, which is essential when the same dataset supports reporting, analytics and AI use cases.
• Delegated Access: Delegated access is permission granted to one identity to act on behalf of another user, service, or system. In NHI environments, this usually appears in OAuth-connected apps and automation tooling. It is powerful, but it must be tightly scoped and reviewed because it can persist long after the original business need ends.

Deepen your knowledge

The NHI Foundation Level course, the industry's only accredited NHI security programme, helps practitioners connect identity governance, machine identity security, and agentic AI oversight. It is designed for teams that need a common control language across human and non-human access decisions.