How should security teams find AI workloads that were never tagged in Kubernetes?

Why This Matters for Security Teams

Untracked AI workloads in Kubernetes create a discovery problem before they become a governance problem. If a cluster contains model-serving pods, retrieval components, or agent services that were never tagged, inventory tools will miss them and policy enforcement will be partial. Current guidance suggests discovery should focus on observable behaviour, not naming conventions, because AI workloads often blend into ordinary application traffic until they begin calling model APIs, vector stores, or internal tool endpoints. That matters for exposure review, egress control, incident response, and cost allocation.

This is also an identity issue. AI workloads frequently need credentials, service account bindings, and secrets to reach models, data, and orchestration tools. When those dependencies are invisible, teams cannot tell which workload is authorised, which one is simply functional, and which one is shadow AI. The SPIFFE workload identity specification is relevant here because it shows how runtime identity can be attached to workloads even when metadata is incomplete.

In practice, many security teams discover AI workloads only after unusual egress, unexpected GPU usage, or a data handling incident has already revealed them.

How It Works in Practice

The most reliable approach is to combine cluster telemetry, process intelligence, and network inspection so the platform can infer AI activity from behaviour. A pod that loads transformer libraries, starts Python or inference runtimes, and opens outbound connections to LLM endpoints is far more likely to be an AI workload than a generic microservice. That is especially true when those signals appear together with access to embeddings services, prompt templates, or retrieval databases. The objective is not perfect classification on day one, but a defensible discovery pipeline that steadily reduces blind spots.

Security teams usually get better results when they correlate several signals:

• Container and process events showing AI frameworks, serving binaries, or notebook-style runtimes.
• Network telemetry showing calls to model APIs, internal inference gateways, or vector databases.
• Secret access and service account usage tied to model credentials or tool connectors.
• Admission, audit, and runtime policy data that show where workloads were deployed and what they reached.

This is where Kubernetes-native controls and workload identity help. If tags are missing, runtime identity can still provide a stable way to tie observed activity to a workload, especially when paired with Kubernetes service accounts and network policy logs. NIST guidance on cloud and identity control mapping also supports this style of evidence-based inventory, rather than assuming that declared metadata is complete.

Teams should also look for contextual markers: access to prompt stores, embeddings pipelines, fine-tuning jobs, feature stores, or tool execution services. Those dependencies often reveal that the workload is doing more than ordinary API processing. Once identified, the workload should be placed into a policy bucket for model risk review, secrets review, and egress restrictions. These controls tend to break down when clusters allow highly dynamic workloads with shared namespaces and weak observability because the same telemetry needed for discovery is missing or too noisy to trust.

Common Variations and Edge Cases

Tighter discovery often increases operational overhead, requiring organisations to balance deeper telemetry against storage cost, alert noise, and privacy constraints. That tradeoff becomes sharper in multitenant clusters, ephemeral job runners, and data science environments where AI and non-AI workloads share the same images, base runtimes, and namespaces.

There is no universal standard for this yet. Best practice is evolving toward behaviour-based discovery with policy enrichment, but teams should expect false positives when general-purpose Python services import AI libraries without actually serving models. The reverse also happens: some AI workloads use compiled runtimes or remote model endpoints and leave little local evidence beyond network patterns.

Two edge cases deserve attention. First, batch or event-driven AI jobs may appear only briefly, so scheduled snapshots miss them unless the platform watches continuously. Second, some workloads will be hidden behind internal APIs or service meshes, making outbound traffic less obvious and pushing greater emphasis onto process, admission, and identity logs. In these cases, the strongest signal is usually convergence across sources, not any single indicator. Where regulated data is involved, discovery should feed into NIST AI risk management and cloud security controls, rather than remaining a pure inventory exercise.

For practitioners, the real test is whether the organisation can answer three questions quickly: what is running, what identity does it use, and what external model or internal tool can it reach. If one of those answers is missing, the workload is not fully governed.

Related resources from NHI Mgmt Group

• How should security teams unify IAM for humans, workloads, and AI agents?
• How should security teams govern access to Azure AI workloads?
• How should security teams govern data access for AI workloads?
• How should security teams implement zero trust for workloads and AI agents?