Threat detection for RAG pipelines is really a posture problem

At a glance

What this is: This is an analysis of why threat detection for RAG pipelines fails when teams only watch retrieval events, and how the three attack windows require behavioral posture across indexing, query, and context assembly.

Why it matters: It matters to IAM practitioners because RAG pipelines rely on source provenance, connector identity, and runtime access patterns, which means NHI, workload identity, and access controls all influence whether poisoned content can be retrieved and acted on.

By the numbers:

• 79% of organisations have experienced secrets leaks, with 77% of these incidents resulting in tangible damage.
• Only 5.7% of organisations have full visibility into their service accounts.
• 96% of organisations store secrets outside of secrets managers in vulnerable locations including code, config files, and CI/CD tools.
• 71% of NHIs are not rotated within recommended time frames, increasing the risk of compromise over time.

👉 Read ARMO's analysis of threat detection for RAG pipelines and the three blind spots

Context

Threat detection for RAG pipelines fails when teams treat retrieval as the only meaningful security event. RAG systems create a wider governance problem because the risk can begin at index time, emerge at query time, or surface later during context assembly, and each stage depends on different identity and control assumptions.

In practice, the question is not whether a retrieval happened but whether the pipeline had trustworthy source provenance, connector identity, and runtime oversight before the model saw the assembled context. That makes this a posture issue as much as a detection issue, especially where NHI governance, workload identity, and secrets control determine who can write into the knowledge layer.

Key questions

Q: What breaks when RAG security only monitors retrieval requests?

A: Teams lose visibility into the earlier and later stages where the real compromise often happens. Poisoning can occur at index time, and malicious instructions can become effective during context assembly even if the retrieval request itself looks ordinary. Retrieval-only monitoring creates false confidence because it captures the symptom, not the control failure.

Q: Why do RAG pipelines need identity and provenance controls as well as detection?

A: Because the system’s trust boundary includes the principals that populate sources, not just the user who asks a question. If a connector, service account, or third-party integration can write poisoned content into the index, detection at query time arrives too late. Identity and provenance controls reduce the attack surface before runtime behaviour is affected.

Q: How do security teams know if a RAG baseline is actually useful?

A: A useful baseline separates index population, source provenance, retrieval shape, result pattern, and context assembly instead of collapsing them into one generic data-access event. If every retrieval looks similar in the dashboard, the baseline is too coarse to explain whether the issue is poisoning, abuse, or assembly-stage manipulation.

Q: How should teams respond when poisoned content reaches the model context?

A: They should contain the ingestion path, identify the connector or source that introduced the content, and instrument the assembly layer before treating the model output as the main problem. The correct response is to stop further propagation, not just filter the visible response after the fact.

Technical breakdown

Index-time poisoning and vector database writes

Index time is the first blind spot because the harmful payload can be stored long before any user query triggers it. In a RAG pipeline, that means the attack lands through source ingestion, connector writes, embedding jobs, or vector database updates that look normal at the network layer. The core problem is that many security tools watch reads and retrieval calls, but not the write path that determines what future retrievals can surface. If the runtime-derived index is poisoned, the later model response is only the symptom. Provenance, connector identity, and write cadence are the security signals that matter here.

Practical implication: instrument vector database writes and source provenance before any enforcement decision on retrieval content.

Query-time retrieval abuse and agent access patterns

Query time is where most tools stop, but even here they often only see a generic service-to-service request. That misses the shape of the retrieval pattern, which is what distinguishes routine use from abuse. A benign query may pull a predictable top-k set from stable sources, while an attacker may drive retrieval flooding, target a specific semantic neighborhood, or steer the agent toward content it would not normally access. The useful signal is not simply that retrieval occurred, but how the query pattern, similarity distribution, and returned documents differ from the agent’s established baseline.

Practical implication: baseline retrieval shape per agent and alert on top-k expansion, similarity collapse, and unusual source selection.

Context assembly is where the final control gap appears

Context assembly is the least visible window because it happens after retrieval and before inference, often inside framework middleware or application code. This is where retrieved chunks are reranked, merged, reordered, or filtered, and where a poisoned instruction can become operational even if the original retrieval looked legitimate. Network tools see separate calls, not the in-process logic that decides which chunks enter the model context. That makes assembly a distinct security boundary, not a continuation of retrieval. If teams do not instrument the pre-inference path, they are blind to the step that turns a suspicious chunk into an actionable prompt.

Practical implication: add framework-level tracing or in-process inspection around reranking and prompt assembly, not just the vector database.

Threat narrative

Attacker objective: The attacker wants the RAG pipeline to treat poisoned content as trusted context so the model will leak data or execute an unwanted downstream action.

1. Entry occurs when a malicious or compromised source is indexed into the RAG corpus, or when a connector write introduces poisoned content into the vector database.
2. Escalation happens when a normal retrieval query surfaces the poisoned chunk and the assembly layer promotes it into the model’s active context.
3. Impact follows when the model acts on the hidden instruction and discloses data, triggers downstream tools, or sends information to an attacker-controlled destination.

NHI Mgmt Group analysis

RAG security is a posture problem before it is a detection problem. Retrieval alerts are too late when the real compromise occurred during indexing or context assembly. Behavioural posture across source provenance, connector identity, and runtime access patterns is what makes retrieval signals meaningful. Practitioners should treat alerting as the last mile of control, not the control itself.

Index-time poisoning creates a governance gap that traditional IAM does not cover. A connector can be correctly authenticated and still ingest adversarial content that later drives model behaviour. That means NHI governance must extend beyond who can authenticate to what a principal is allowed to write into the knowledge layer. Practitioners should map this to OWASP-NHI and NIST-CSF because provenance and write authority are now part of the control surface.

Context assembly is the named concept teams need to operationalise. Context-assembly blindness is the specific failure mode this article exposes: the system can retrieve safely and still assemble unsafely. The reranker, chunker, and framework middleware become part of the trust boundary once they decide which material reaches the model. Practitioners should instrument the pre-inference path as a governed security layer, not an application detail.

Generic AI baselines collapse distinct RAG signals into one noisy bucket. Treating retrieval as ordinary data access hides the difference between index population, query shape, result pattern, and assembly behaviour. That flattening is why teams miss low-and-slow poisoning and multi-stage abuse. Practitioners should adopt a RAG-specific signal model that preserves those distinctions so investigations route to the right control owner on the first pass.

Agent and workload identity still matter even in an AI security story. The article’s technical evidence shows that connector identity, index write authority, and sanctioned integrations are the hinges that determine whether poisoned content enters the system. That is exactly where IAM, PAM, and NHI governance intersect with AI security. Practitioners should use identity controls to limit who can populate the knowledge substrate, then use posture telemetry to verify it stays that way.

What this signals

Context-assembly blindness: this is the operational gap most RAG programmes will need to close next. If the security stack cannot see what chunks are merged, reordered, or filtered before inference, then adversarial content can pass every upstream check and still alter outcomes. The practical response is to treat framework-level tracing and connector governance as part of the security baseline, not as optional observability.

The programme-level implication is that IAM and NHI controls now affect AI quality and AI safety at the same time. Source principals, service accounts, and third-party connectors determine what enters the index, while runtime posture determines whether the assembled context stays trustworthy. Teams that already track identity lifecycle and access scope should extend those controls to RAG ingestion paths, then align them with MITRE ATT&CK Enterprise Matrix thinking for attack-chain visibility.

For practitioners

• Instrument index writes as a governed control point Capture connector identity, source provenance, and vector database write cadence so poisoned content can be detected before retrieval ever happens.
• Separate retrieval shapes from normal data access Build per-agent baselines for top-k depth, similarity score floor, and retrieval frequency so abnormal query behaviour is visible in context.
• Trace the pre-inference assembly path Add framework callbacks or in-process tracing around reranking, chunk ordering, and context composition so the assembled prompt is observable.
• Govern the identities that can write into the knowledge layer Review which service accounts, API keys, and connector principals can populate RAG sources, then reduce standing access and verify provenance regularly.

Key takeaways

• RAG threat detection fails when teams watch retrieval but ignore indexing and context assembly.
• The strongest signals are provenance, retrieval shape, and pre-inference assembly behaviour, not generic alert volume.
• Identity governance matters because service accounts and connector principals decide what the model can be taught to trust.

Key terms

• Context assembly: The act of combining live data from multiple systems into a single working session or tool context. This matters because it expands the effective trust boundary at runtime, making the session itself the unit of governance rather than any one connector or endpoint.
• Index-Time Poisoning: The insertion of malicious or misleading content into a retrieval index before a query ever occurs. In RAG systems, this can happen through compromised sources, connector abuse, or manipulated embeddings, creating a dormant risk that only appears later at runtime.
• Access Provenance: Access provenance is the record of how an identity was created, approved, used, and withdrawn. In NHI governance, it is the evidence trail that lets teams prove an account is legitimate, explainable, and still within its intended access boundary.
• Retrieval Shape: The pattern of documents a RAG system returns for a given agent or query, including frequency, top-k depth, similarity distribution, and source selection. It is useful because abuse often shows up as a change in pattern, not as an obvious blocked request.

What's in the full article

ARMO's full blog covers the operational detail this post intentionally leaves for the source:

• Framework-level instrumentation examples for LangChain and LlamaIndex pre-inference paths
• A deeper breakdown of the five-signal RAG baseline and how to map it to your own telemetry stack
• Deployment-pattern differences between in-process, sidecar, and external reranking architectures
• Implementation notes on correlating vector database audit logs with framework callbacks and eBPF telemetry

👉 The full ARMO post covers the three-window model, signal categories, and deployment-pattern differences in more depth.

Deepen your knowledge

NHI Foundation Level course, the industry's only accredited NHI security programme, covers NHI governance, workload identity, secrets management, and identity lifecycle control. It helps practitioners connect identity discipline to the broader security programmes they run every day.