They should do both, but if the training inputs are poorly understood, data lineage comes first because you cannot reliably defend what you cannot trace. Red teaming then tests whether harmful data has already created exploitable behaviour. The two controls work together, but lineage gives the investigation a starting point.
Why data lineage should come before red teaming when AI inputs are unclear
When training inputs are poorly understood, lineage is the first control that makes the rest of the AI risk work reliable. It tells you what data entered the system, where it came from, how it moved, and which downstream models or fine-tunes may carry that history. Without that traceability, red teaming can still find harmful behaviour, but it cannot explain which data path caused it or where to remove it.
That matters because AI risk is often cumulative: a dataset may be only partly documented, yet still influence multiple models, prompts, and retrieval layers. A tracing discipline gives you the evidence base for scope, ownership, and remediation. It also helps separate a one-off prompt failure from a structural data problem that needs reclassification, filtering, or retirement.
A practical starting point is to treat lineage as the investigation layer and red teaming as the behavioural validation layer. Lineage answers what should be tested and what might need to be excluded. Red teaming answers whether the system can still be induced to produce harmful, unsafe, or policy-breaking outcomes despite that knowledge.
What red teaming adds once the data path is visible
Red teaming is the faster way to test whether contaminated or poorly governed inputs have already turned into exploitable behaviour. A model can look well documented on paper and still leak sensitive content, follow malicious instructions, or amplify bias when probed in the right way. That is why red teaming is valuable even after lineage work starts: it checks whether the observed data history has become an operational weakness.
For AI systems, red teaming is most useful when it is tied to a concrete hypothesis from lineage. If the lineage shows noisy web ingestion, weak human review, or mixed-trust sources, the test design should reflect those conditions rather than use generic prompts. That makes findings more actionable, because the team can connect a failure to a specific data source, pipeline choice, or deployment pattern.
For a broader explanation of red team structure, scope, and remediation loops, see Red Teaming Guide. If the concern is specifically AI behaviour under adversarial testing, AI Red Teaming Guide is the more direct reference.
How to sequence the two controls without slowing delivery
The sequence should follow the quality of your input evidence. If the team cannot explain the training set, retrieval corpus, or fine-tuning history, start with lineage discovery and minimum viable scoping. If the data path is already well inventoried, run red teaming earlier so you can prioritise the most credible failure modes and avoid overinvesting in low-probability hypotheticals.
In practice, the controls should feed each other. Lineage tells you which sources deserve harsher testing, and red teaming tells you which sources are more likely to matter operationally. That combination is stronger than either alone: lineage narrows the search space, while red teaming proves whether the risk is theoretical or already exploitable.
For teams building a formal risk process around this work, the best external anchor is NIST AI Risk Management Framework. If you need a governance-oriented view of AI risk treatment and escalation, AI Risk Assessment Guide is the most relevant internal companion.
Risk and Threat Considerations
When data lineage is weak, the main risk is not only bad documentation, it is blind remediation. Organisations may test the wrong model, fix the wrong prompt, or miss the shared dataset that is causing repeated failures across multiple systems. That creates a false sense of assurance, especially when red team results are treated as proof that the underlying data problem has been understood.
Failure mechanism: Untraceable or partial data provenance breaks the link between observed model behaviour and the source material that shaped it, so harmful outputs can persist even after superficial testing.
Impact: Teams lose the ability to prioritise remediation, prove containment, or determine whether the issue is isolated to one deployment or present across a wider AI estate.
Standards & Framework Alignment
This section maps relevant standards and security frameworks to the operational risks and controls described in this guidance.
NIST AI RMF and NIST SP 800-53 Rev 5 set the technical controls, while ISO/IEC 42001:2023 defines the regulatory obligations.
| Framework | Control / Reference | Relevance |
|---|---|---|
| NIST AI RMF | GV.2 — AI Risk Management Culture | AI risk sequencing needs governance over traceability and testing priorities. |
| MAP.1 — AI Context and Purpose | Lineage clarifies what data and system context the AI use case actually includes. | |
| MEASURE.2 — AI Risk and Impact Assessment | Red teaming measures whether harmful behaviour emerges under realistic adversarial conditions. | |
| Recommendation — Establish a repeatable AI risk process that links data traceability to adversarial testing decisions. Define the AI system context and data flows before deciding what to test and remediate. Measure model behaviour under adversarial scenarios to validate risk assumptions. | ||
| ISO/IEC 42001:2023 | A.5.2 — AI risk assessment | The question is about sequencing AI risk controls and treatment decisions. |
| A.6.1 — Objectives for AI system management | Control priorities must reflect whether traceability or behavioural testing is the first gap. | |
| Recommendation — Assess AI risks using documented inputs, then decide whether lineage or testing must lead. Set AI control objectives that require traceability before adversarial validation where inputs are unclear. | ||
| NIST SP 800-53 Rev 5 | AU-3 — Content of Audit Records | Lineage depends on records that show what data entered and moved through the system. |
| SA-11 — Developer Testing and Evaluation | Red teaming is a form of adversarial evaluation used to validate security and safety claims. | |
| Recommendation — Record data provenance events and transformations so AI behaviour can be investigated later. Test AI systems adversarially before relying on them for material decisions. | ||
Practitioner Guidance
What to prioritise: If you cannot answer where the training or retrieval data came from, start with lineage before investing heavily in scenario testing. If lineage is already credible, move faster to red teaming and use the findings to target the highest-risk data sources or model behaviours.
What to verify: Confirm that the lineage record is good enough to support a remediation decision, not just a catalogue entry. You should be able to identify source class, ingestion path, ownership, and whether the same data can influence more than one model or application.
Practitioner takeaway: Red teaming finds exploitable behaviour, but lineage tells you what that behaviour came from and what to fix first. When the input history is uncertain, traceability is the control that makes every later AI risk decision more trustworthy.
Related resources from NHI Mgmt Group
- Guide: Red Teaming AI Agents for Identity Abuse
- Should organisations prioritise external exposure or internal credential governance first?
- Should organisations prioritise inline blocking or forensic visibility for AI data risk?
- When should organisations prioritise AI data lineage over more alerting?
- How do organisations decide whether to prioritise AI discovery, data governance, or broader compliance mapping first?