Topic starter
27/06/2026 2:05 pm
TL;DR: Keeping signal extraction consistent and reducing drift across scoring and aggregation now depends on running a Signals DAG across 3 production systems, including 2 online services at up to 35k QPS and 1 batch job processing 3TB daily, according to Abnormal AI. The governance lesson is that explicit data dependencies matter more than isolated model tuning when detection pipelines scale.
Editorial analysis by NHI Mgmt Group, based on content published by Abnormal AI: “Abnormal AI Innovation: High-Scale Aggregation Systems in Production”.
Key questions
Q: How should teams prevent drift between batch and realtime ML pipelines?
A: Teams should define a single canonical feature or signal model and reuse it across batch and realtime paths.
Q: Why do explicit dependencies matter in detection and risk scoring pipelines?
A: Explicit dependencies make it possible to see how a signal is derived and what upstream data it depends on.
Q: What breaks when aggregate features are computed in separate systems?
A: The main failure is semantic drift.
Practitioner guidance
- Define signal contracts before scaling pipelines Document each derived signal’s inputs, outputs, and dependency order so scoring and aggregation use the same semantics across environments.
- Unify realtime and batch feature logic Review where batch jobs and streaming services compute the same feature differently, then standardise the derivation logic across both paths.
- Treat drift as a production defect Add checks that compare realtime and offline feature outputs for the same entity, then investigate any mismatch as a governance failure.
Bottom line: The article shows that explicit dependency modelling is the mechanism that keeps complex ML pipelines from drifting as they split across execution environments.
Explore further
View Full Forum → | NHI Foundation Course → | Our Services → | Read the full analysis →
27/06/2026 3:42 pm
Explicit dependency modelling is becoming a governance control, not just an engineering convenience. When feature extraction logic is scattered across batch jobs, realtime services, and scoring code, teams lose the ability to reason about what a signal actually means. The Signals DAG pattern turns that hidden coupling into an auditable structure. For security and identity programmes, the lesson is that governed pipelines need declared data dependencies before they can be trusted at scale.
A question worth separating out:
Q: When should organisations unify batch and streaming feature processing?
A: They should unify the logic when the same derived signal influences both realtime decisions and longer-horizon analysis. If two implementations are required for performance reasons, the dependency model still needs to stay shared. Otherwise the organisation is optimising speed while losing consistency in the underlying control plane.
👉 Read our full editorial: Signals DAG architecture shows why ML pipelines need explicit dependencies