Data and AI lifecycle security is becoming an AppSec blind spot

At a glance

What this is: This is a lifecycle overview of the distinct data, model, deployment and runtime stages behind AI applications, with the key finding that AppSec controls do not fully cover that operational surface.

Why it matters: IAM, AppSec and platform teams need to treat AI data, notebooks, model registries and runtime endpoints as governed assets because access, lineage and operational control now shape application risk.

Context

The security gap here is not that AI systems are mysterious. It is that the Data and AI lifecycle now spans distinct assets, tools and workflows that sit alongside the SDLC rather than inside it, so traditional application security views miss how these systems are actually built and operated.

For AppSec and product security teams, the governance problem is broader than scanning code. Data preparation, model training, deployment and runtime each introduce different access paths, hand-offs and control points, which means AI applications need lifecycle governance across notebooks, pipelines, registries and serving layers.

Where AI models can reach directly into data, tools and runtime actions, the control model starts to resemble non-human identity governance as much as software assurance. That changes how teams think about inventory, access scope, review cadence and operational ownership.

Key questions

Q: How should security teams secure the Data & AI lifecycle without treating it like a separate island from application security?

A: Security teams should treat the Data & AI lifecycle as part of the application surface, not as a parallel world. The practical starting point is to map the stages, tools, handoffs, and runtime dependencies so controls can follow data, code, models, and prompts end to end. That gives AppSec and product security teams a way to apply governance without losing sight of where AI systems actually change.

Q: Why do AI pipelines and model registries create governance risk?

A: AI pipelines and model registries create governance risk because they control what data is used, what model version is deployed, and which jobs can execute automatically. If those systems are overprivileged or poorly owned, they can become durable machine access points. Security teams should treat them as privileged control planes and review them with the same discipline used for administrative access.

Q: What should teams do to secure AI runtime behaviour?

A: Teams should define controls for prompts, retrieval sources, model endpoints and telemetry, because runtime is where model behaviour becomes dynamic. Without those controls, a model can appear approved while the inputs and context shaping its output remain ungoverned.

Q: How can organisations tell whether their AI governance evidence is complete?

A: A complete evidence set can answer four questions without reconstruction: what the model received, what it returned, what version and policy were active, and who was accountable for the change or action. If any of those answers depend on Slack, screenshots, or memory, the evidence model is incomplete and should be redesigned.

Technical breakdown

Why the Data and AI lifecycle sits outside the SDLC

The Data and AI lifecycle is not just software development with a model added on top. It includes data preparation, model training, deployment and runtime operations, each with different artefacts and control points. In practice, that means notebooks, pipelines, registries and model servers create a second delivery path that application security tools often do not model well. The result is a governance gap between code-centric assurance and AI-specific operational reality. When teams only secure the application shell, they miss the assets that determine what the model sees, what it can use and what it can do at runtime.

Practical implication: Map AI assets to their own lifecycle stages instead of assuming SDLC controls cover them.

How notebooks, pipelines and registries change the trust model

Jupyter notebooks, data pipelines and model registries are not just tooling conveniences. They are sensitive control points because they concentrate access to data, training logic, model versions and lineage. A notebook can combine exploratory code with direct data access. A pipeline can transform and move data between systems. A registry can determine which model version is trusted for deployment. Together, these components create a chain of trust that is separate from source code management and needs its own inventory, access boundaries and review logic. If those layers are not governed, the model lifecycle becomes difficult to explain or defend.

Practical implication: Treat notebooks, pipelines and registries as governed identity-bearing assets, not informal developer workspaces.

Why AI runtime operations need continuous governance

AI runtime is where the model stops being a static artefact and becomes an operating function. At this stage, prompts, context, memory and inference pipelines shape the model’s outputs in real time. For GenAI, runtime may also include retrieval-augmented generation, where external data can be pulled into the response path. That creates a moving trust boundary: the model can appear stable while the data it consumes, the tools it reaches and the outputs it emits all change dynamically. Security teams therefore need visibility into runtime behaviour, not just build-time assurance.

Practical implication: Define runtime controls around prompts, retrieval sources, model endpoints and telemetry, not only around build artefacts.

Breaches seen in the wild

• Moltbook AI agent keys breach: Moltbook breach exposed 1.5M AI agent keys.
• DeepSeek database exposure 2025: An unauthenticated DeepSeek ClickHouse database exposed over a million log lines with plaintext chat history and API keys in 2025.

Read and download The State of NHI & AI Agent Breach Report 2026, covering 200+ breaches impacting Non-Human Identities including AI Agents.

NHI Mgmt Group analysis

Data and AI lifecycle security is becoming a separate governance domain, not a subtopic of AppSec. The lifecycle described in the source article has its own assets, workflows and trust boundaries, and those do not map cleanly onto conventional SDLC thinking. That is why application security leaders experience it as a black box: the security problem lives in data movement, model handling and runtime behaviour as much as in code. Practitioners should treat AI lifecycle governance as a distinct operating model.

Notebook, pipeline and registry access now function like identity infrastructure for AI work. These environments do not merely store artefacts. They determine who can shape training inputs, change model versions and move a model toward production. That makes least privilege, review cadence and lifecycle ownership central concerns even when the team does not describe the environment in IAM terms. The practical conclusion is that AI asset governance and identity governance are converging.

Runtime controls matter more than build-time confidence once models begin consuming live context. The article’s discussion of prompts, memory, inference pipelines and retrieval-augmented generation shows that model behaviour is shaped at execution time, not only during development. That means assurance based only on source, package or model provenance is incomplete. Practitioners need governance that follows the model into runtime, where risk actually manifests.

Data and AI lifecycle security creates a governance gap that conventional AppSec metrics do not expose. SCA, SAST, DAST and API security can each cover part of the stack, but none of them fully describe how data, model artefacts and serving layers interact across the lifecycle. The blind spot is structural, not accidental. Teams should expect AI assurance to require a broader control map than standard software security programmes normally maintain.

Model supply chains now behave like multi-stage trust chains. The article shows how third-party datasets, open-source models, notebooks, registries and model servers combine into one continuous delivery path. That path is harder to secure than a traditional application chain because each stage can alter both behaviour and exposure. The practitioner takeaway is that model lineage and operational hand-offs deserve the same scrutiny that source code and build provenance already receive.

From our research library:

• AI-related credential leaks surged 81.5% year-over-year in 2025, with the surrounding AI infrastructure leaking 5x faster than core LLM providers, according to the State of Secrets Sprawl 2026.
• Read next: Agentic AI Security Guide

What this signals

Data and AI lifecycle governance is becoming a programme design issue, not just a tooling issue. Teams that only extend AppSec checks into AI projects will keep missing the places where trust is actually created: data pipelines, notebooks, model registries and serving layers. The better model is lifecycle ownership, where each stage has explicit control points and an accountable team.

Model lineage, access scope and runtime visibility now matter together. If a team cannot trace where data came from, which model version was approved and what retrieval sources shape runtime output, it does not have meaningful assurance over the AI application. That is the governance standard practitioners should expect to mature toward.

Data and AI lifecycle security also changes how teams should think about shadow AI. Undocumented notebooks, unmanaged model endpoints and ad hoc retrieval sources can create hidden behaviour even when the surrounding application is well governed. Discovery and lifecycle inventory therefore become prerequisites for any credible AI risk programme.

For practitioners

• Map the AI lifecycle separately from the SDLC Document data preparation, training, deployment and runtime as distinct control domains with named owners, hand-offs and approvals.
• Inventory notebooks, pipelines and registries Capture every notebook environment, pipeline system and model registry that can read data, change models or trigger deployment.
• Define runtime governance for prompts and retrieval Set controls for model endpoints, retrieval-augmented generation sources, prompt handling and runtime telemetry so behaviour is observable.
• Review third-party models and data sources Track external datasets, open-source models and embedded model services as dependencies with explicit lineage and access rules.
• Align AppSec and data engineering ownership Assign security responsibilities across application security, product security and data or ML teams so gaps do not fall between functions.

Key takeaways

• AI applications now depend on a separate lifecycle of data, model and runtime assets that standard SDLC controls do not fully cover.
• Notebooks, pipelines, registries and model servers create distinct governance points that can change model behaviour without changing application code.
• Practitioners need lifecycle ownership, runtime visibility and cross-team accountability if AI security is to move beyond a black box.

Key terms

• Data and AI Lifecycle: The Data and AI Lifecycle is the end-to-end set of processes used to prepare data, train models, deploy AI systems, and run them in production. It includes tooling and runtime layers that sit alongside, but not inside, the conventional software lifecycle, which is why it creates separate governance and identity risks.
• Model Registry: A model registry is a system used to store, version, and track machine learning models and their metadata across development and deployment. It gives teams lineage and version control, but it also becomes a privileged control point because the wrong model version or metadata can reach production if access is not tightly governed.
• Data Pipeline Management: The process of collecting, transforming, enriching, and routing security data before it reaches analytics systems. In practice, it is a control layer that decides what evidence is retained, normalized, or forwarded, and it shapes both cost and visibility across SIEM, storage, and investigation workflows.
• Inference Pipeline: An inference pipeline is the part of an AI application that receives a user prompt, retrieves relevant context, and generates the final answer. In enterprise settings, it should also apply redaction and authorization checks so the system only uses data the requester is allowed to access.

Deepen your knowledge

NHI governance, agentic AI identity, and machine identity lifecycle are core topics in our NHI Foundation Level course, the industry's only accredited NHI security programme. If you are building or maturing an IAM programme, it is worth exploring.