Join our Newsletter — 33% off our NHI Course

Notifications
Clear all

GitHub Copilot telemetry: what it means for AI governance teams


(@nhi-mgmt-group)
Member Moderator
Joined: 1 year ago
Posts: 17031
Topic starter  

TL;DR: GitHub Copilot now exports OpenTelemetry traces that can surface prompts, tool calls, token usage, repository context, and sensitive-data events across developer sessions, according to Fiddler. The governance problem is no longer whether teams deployed Copilot, but whether they can actually see, measure, and constrain what the agent is doing.

NHIMG editorial — based on content published by Fiddler: See What GitHub Copilot is Actually Doing with Fiddler

By the numbers:

Questions worth separating out

Q: How should security teams govern AI coding assistants that can execute commands?

A: Treat them as delegated non-human identities with bounded execution authority.

Q: Why do AI assistants create a new trust problem for identity governance?

A: AI assistants create a new trust problem because they can read data, choose tools, and act on external text in ways traditional review processes do not expect.

Q: What breaks when teams rely on seat counts instead of agent telemetry?

A: Seat counts show adoption, not behaviour.

Practitioner guidance

  • Define Copilot as a governed runtime surface Classify coding-agent sessions as security-relevant activity, then decide what must be logged, retained, reviewed, and escalated before rollout expands across more developer seats.
  • Normalize telemetry before adding policy rules Map prompts, tool calls, repository context, and model identifiers into a consistent schema so policy engines and analysts can compare behaviour across assistants and gateways.
  • Add secrets and PII detection to session monitoring Scan prompts, responses, and tool outputs for credentials, customer data, and other sensitive content so leakage is caught while it is still inside the agent workflow.

What's in the full article

Fiddler's full blog covers the operational detail this post intentionally leaves for the source:

  • Exact environment-variable setup for streaming Copilot OpenTelemetry into Fiddler
  • Attribute-level filtering examples for token usage, model names, and repository context
  • Semantic mapping guidance for normalizing non-standard OpenTelemetry fields across agent tools
  • Dashboard views for response quality, frustration signals, and git activity volume

👉 Read Fiddler's analysis of GitHub Copilot observability and AI governance →

GitHub Copilot telemetry: what it means for AI governance teams?

Explore further

View Full Forum →  |  NHI Foundation Course →



   
Quote
Share: