Security AI context management: are your agents pruning the right data?

TL;DR: Security investigation agents break traditional context management because logs are verbose, anomaly-driven, and dependent on surrounding evidence, according to Cotool’s analysis of token estimation and pruning trade-offs. The practical lesson is that security AI needs fast, provider-aware measurement and conservative pruning, not generic RAG or sliding-window assumptions.

NHIMG editorial — based on content published by Cotool: Security AI context management and token pruning for agentic investigations

By the numbers:

• Using OpenAI's Tiktoken, encoding a 673K-token conversation takes about 368ms on an M4 MacBook Pro.
• On production GCP Cloud Run instances, encoding 678K tokens took 3.3 seconds.

Questions worth separating out

Q: How should security teams use AI in SIEM without losing identity context?

A: Security teams should use AI to accelerate correlation, summarisation, and triage, but only after identity telemetry is fully part of the detection pipeline.

Q: Why do AI agents struggle with security investigations more than document Q&A?

A: Security investigations depend on surrounding events, timestamps, and repeated patterns, so the meaningful signal is often distributed across a long transcript.

Q: What breaks when sliding-window context management is used for agentic security workflows?

A: Sliding windows can remove messages that still anchor reasoning, tool-call sequencing, or provider-specific message requirements.

Practitioner guidance

• Measure token usage per provider and message class Track token counts separately for user prompts, assistant text, reasoning blocks, and tool results so estimator drift is visible before investigations fail.
• Preserve investigation structure before pruning content Keep message ordering, tool-call pairing, and the latest assistant turn intact, then prune oldest and largest tool outputs first.
• Test context controls against long, noisy SIEM transcripts Use replayable investigations with verbose logs, repeated tool calls, and mixed provider responses to confirm the agent still returns correct results under pressure.

What's in the full article

Cotool's full post covers the operational detail this post intentionally leaves for the source:

• Benchmark results comparing token estimation methods across OpenAI, Anthropic, and Google providers.
• The exact weighted-estimation approach used to keep accuracy within the 110% overestimation threshold.
• Phase-by-phase pruning rules for tool results, assistant turns, and user messages in long investigations.
• Implementation lessons from extended-thinking requirements and AI SDK tool-call pairing constraints.

👉 Read Cotool's analysis of context management for security AI agents →

Security AI context management: are your agents pruning the right data?

Explore further

View Full Forum →  |  NHI Foundation Course →