Activation monitoring: what it means for AI security teams

TL;DR: Activations provide a more reliable view of what an AI model is actually doing than Chain of Thought alone, according to Starseer, because they capture internal routing rather than the story a model tells about its own reasoning. That shift matters because output-only monitoring can miss intent, manipulation, and misuse in enterprise AI deployments.

Editorial analysis by NHI Mgmt Group, based on content published by Starseer: “What Are AI Activations and What Do They Tell Us?”.

Key questions

Q: How should security teams govern AI models when output text is not trustworthy evidence?

A: They should treat output text as one signal, not the control boundary.

Q: When is activation monitoring more useful than Chain of Thought review?

A: It becomes more useful whenever the security question is not just what the model said, but what path it took to say it.

Q: How can organisations tell whether an AI system is behaving safely at inference time?

A: They need observable signals from the inference process, not just end-state output checks.

Practitioner guidance

• Map where inference runs Document whether each AI workload runs through a provider API, on owned infrastructure, or on endpoints under your control, because that determines whether activations are observable.
• Separate explanation from evidence Treat Chain of Thought and other model explanations as interpretive artefacts, not as the primary proof of why the model produced a result.
• Test internal behaviour with contrastive prompts Use paired prompts that share vocabulary but differ in meaning so you can see which layers or activation patterns move toward the risky concept.

Bottom line: Activations give security teams a deeper view of model behaviour than output text or model-generated explanations can provide.

Explore further

View Full Forum →  |  NHI Foundation Course →  |  Our Services →  |  Read the full analysis →