Threat detection for RAG pipelines is really a posture problem

At a glance

What this is: ARMO argues that threat detection for RAG pipelines is really a posture problem because the attack surface spans index-time, query-time, and context-assembly-time windows that most tools do not fully observe.

Why it matters: IAM and security teams responsible for AI systems need baseline-driven visibility across RAG ingestion, retrieval, and assembly stages, or they will miss the control gap that turns ordinary retrieval into a breach path.

Context

RAG pipelines create a governance gap because the security event often occurs before, during, or after retrieval, not just at the retrieval API itself. If a tool only watches the network destination for query-time activity, it misses poisoned content entering the index and the assembly logic that turns retrieved chunks into model context. That makes the real control problem behavioural posture across the whole pipeline, not alerting at a single chokepoint.

This matters to identity and access governance because retrieval, indexing, and context assembly all depend on distinct principals, permissions, and trust boundaries. Where those boundaries collapse into one generic data-access event, teams lose the ability to distinguish legitimate corpus maintenance from abusive RAG behaviour. The article's core point is that the baseline must reflect how the pipeline actually behaves, not how the control plane describes it.

Key questions

Q: What breaks when RAG security only monitors retrieval requests?

A: Teams lose visibility into the earlier and later stages where the real compromise often happens. Poisoning can occur at index time, and malicious instructions can become effective during context assembly even if the retrieval request itself looks ordinary. Retrieval-only monitoring creates false confidence because it captures the symptom, not the control failure.

Q: Why do RAG pipelines need behavioural baselines instead of generic AI alerts?

A: Generic baselines collapse retrieval into ordinary data access and hide the differences between writing to the index, querying it, and assembling context. RAG security only becomes actionable when each stage has its own expected pattern. Otherwise, alerting either becomes noisy or misses the deviation that actually matters.

Q: What are the signs that context assembly is becoming the weak link?

A: Look for a mismatch between clean retrieval logs and unexpected model behaviour, especially when reranking or middleware changes which chunks reach the prompt. If the retriever looks normal but the final context shifts, the failure is likely in assembly rather than retrieval. That is the point where most monitoring stacks go blind.

Q: How should security teams govern cross-tenant RAG boundaries?

A: Treat tenant separation as an end-to-end authorisation problem, not only a retrieval filter. If the filter is correct but the reranker or prompt assembler can still surface foreign content, the boundary is not actually enforced. Teams need controls that propagate tenant context through every stage that can influence the final prompt.

Technical breakdown

Index-time poisoning: why stored content is the first attack window

Index-time risk begins when content is written into the retrieval corpus before any user query exists. At that stage, poisoned text can sit dormant until a later retrieval surfaces it, which means the harmful payload is a stored-state problem rather than a live request problem. Security tooling that only inspects query traffic will miss source compromise, malicious document ingestion, and connector abuse. The operational challenge is to treat vector database writes, embedding jobs, and source provenance as security-relevant runtime events, because those are the mechanisms that decide what the model can later retrieve.

Practical implication: Instrument index writes and source provenance, not just retrieval traffic, so poisoned corpus entries can be detected before they are queried.

Query-time retrieval abuse and the limits of network-only detection

Query-time threats emerge when an agent asks the retriever for content and the returned set is shaped by prompt injection, retrieval flooding, or semantic manipulation. The difficulty is that every retrieval can look similar at the network layer, especially in service-to-service traffic where one call resembles the next. A useful baseline has to capture per-agent query frequency, top-k selection patterns, similarity score shifts, and correlations with recent index writes. Without that behavioural context, alerts either fire on normal traffic or miss the retrieval pattern that shows deliberate abuse.

Practical implication: Baseline query shapes per agent and correlate them with recent index changes so abnormal retrieval patterns become visible.

Context-assembly-time: the blind spot between retrieval and inference

Context assembly is the stage where retrieved chunks are reordered, filtered, or merged before reaching the model. That stage often happens inside framework middleware or in-process code, which is why standard infrastructure monitors miss it entirely. The security issue is not only what was retrieved, but what actually entered the final prompt and in what order. If the assembly layer can be altered, a clean retrieval can still produce a compromised context. This is why application-layer tracing and framework callbacks matter: they expose the last security decision before the model acts.

Practical implication: Trace the assembled prompt, not just the retrieval response, so manipulation inside middleware or rerankers cannot hide behind clean network logs.

Threat narrative

Attacker objective: The attacker wants poisoned content to survive ingestion, surface during retrieval, and influence the model's output or tool actions.

1. Entry occurs when a malicious or compromised source is indexed into the RAG corpus, or when a connector writes poisoned material into the vector store.
2. Credential or access abuse enables the attacker to place content that appears legitimate at ingestion time, so the malicious instruction survives until a later semantic match.
3. Impact occurs when a user query retrieves the poisoned chunk, the assembly layer includes it in model context, and the downstream agent performs the instructed action or leaks data.

Breaches seen in the wild

• reviewdog Action compromise 2025: A stolen maintainer token poisoned reviewdog/action-setup, leaking CI secrets including the tj-actions bot token used in the next attack.
• Shai Hulud npm malware campaign: Shai Hulud campaign: npm malware exposed secrets on GitHub.

Read and download The State of NHI & AI Agent Breach Report 2026, covering 200+ breaches impacting Non-Human Identities including AI Agents.

NHI Mgmt Group analysis

Behavioral posture is the real control plane for RAG security. Retrieval alerts without a baseline do not separate malicious activity from routine corpus access. The article shows that the breach-relevant behaviour is distributed across indexing, retrieval, and assembly, so a single alert layer cannot explain the event. Practitioners should treat posture as the source of truth and detection as the output of that posture, not the other way around.

Context-assembly-time is the named governance gap most teams still under-instrument. The security boundary is not complete when retrieval returns safe-looking chunks, because reranking and prompt assembly decide what the model actually sees. This creates a distinct trust boundary for AI systems that resembles NHI governance in one key respect: the identity and authority of the component making the final decision matters as much as the data itself. Practitioners should expose and monitor the assembly layer as a first-class control surface.

RAG security needs a five-signal model, not a collapsed data-access model. Index population, source provenance, retrieval query pattern, retrieval result pattern, and context-assembly pattern each produce different deviations. When they are merged into one generic access category, teams lose the ability to tell corpus maintenance from abuse. Practitioners should separate these signals so each stage can be baselined and investigated independently.

Multi-tenant RAG turns provenance and boundary propagation into governance issues, not just engineering issues. When retrieval is scoped correctly but context assembly still leaks cross-tenant material, the control failure sits in the propagation path between layers. That is the kind of failure identity and access programmes need to recognise as an authorisation problem, not a model quality problem. Practitioners should govern the whole retrieval-to-context chain as one access decision.

What this signals

RAG posture has to be measured where behaviour changes, not where traffic happens. The practical lesson for security teams is that index population and context assembly deserve the same operational scrutiny as retrieval itself. If those stages are invisible, the programme is not detecting threats so much as reacting to symptoms.

Pipeline visibility should be treated as an authorisation control, not just a logging exercise. Once assembled context is allowed to cross a trust boundary, the model can act on content no user intended to authorise. That makes retrieval-to-prompt propagation a governance issue for identity, access, and AI control owners alike.

For practitioners

• Build stage-specific baselines for the full RAG pipeline Separate index population, source provenance, retrieval query patterns, retrieval result patterns, and context-assembly patterns so each stage can be monitored on its own terms.
• Instrument vector database writes and connector principals Log which principal wrote which source into the index, when it happened, and whether the timing and metadata match the declared corpus refresh pattern.
• Trace the assembled prompt before inference Capture which retrieved chunks reached the model, in what order, and through which reranking or middleware path they passed.
• Correlate retrieval anomalies with recent index changes Flag sudden top-k expansion, similarity floor collapse, or first-time retrieval of very recent content when those events line up with new or unusual index writes.

Key takeaways

• RAG threats are split across ingestion, retrieval, and assembly, so a single alert layer cannot provide reliable detection.
• The main failure is behavioural blindness, not missing telemetry volume, because many normal-looking retrievals mask a poisoned corpus or altered prompt path.
• Teams that baseline each stage separately can distinguish ordinary corpus maintenance from the attack patterns that turn retrieval into impact.

Key terms

• Behavioural posture: Behavioural posture is the real operating state of a security control as shown by live activity, not its declared configuration. It measures whether identities, access paths, and enforcement points are being used as intended in production, which is often different from what policy or compliance tools report.
• Context assembly: The act of combining live data from multiple systems into a single working session or tool context. This matters because it expands the effective trust boundary at runtime, making the session itself the unit of governance rather than any one connector or endpoint.
• Index-Time Poisoning: The insertion of malicious or misleading content into a retrieval index before a query ever occurs. In RAG systems, this can happen through compromised sources, connector abuse, or manipulated embeddings, creating a dormant risk that only appears later at runtime.
• Access Provenance: Access provenance is the record of how an identity was created, approved, used, and withdrawn. In NHI governance, it is the evidence trail that lets teams prove an account is legitimate, explainable, and still within its intended access boundary.

Deepen your knowledge

NHI Mgmt Group covers identity security, NHI governance, and agentic AI through independent research, practitioner guides, and the NHI Foundation Level course. It is suitable for practitioners who need a governance lens on identity, access, and machine-driven systems across the wider security programme.