Medallion Data Lifecycle

Expanded Definition

The Medallion data lifecycle is not just a naming convention for tables; it is a trust progression that defines how data quality, lineage, and access expectations change as data moves from Bronze to Silver to Gold. In practice, Bronze keeps data close to the source, Silver applies cleansing and enrichment, and Gold publishes curated outputs for analytics, reporting, and AI consumption. The model is widely adopted in lakehouse architectures, but definitions vary across vendors, and no single standard governs this yet. In NHI and agentic AI contexts, the distinction matters because each layer should carry different controls for secrets, service accounts, and downstream tool access. Mature implementations align Bronze with ingestion integrity, Silver with validation and transformation, and Gold with governed consumption, as described in the NHI Lifecycle Management Guide and the OWASP Non-Human Identity Top 10. The most common misapplication is treating Gold as “trusted by default,” which occurs when teams expose curated datasets without rechecking lineage, freshness, or access scope.

Examples and Use Cases

Implementing the medallion pattern rigorously often introduces governance overhead, requiring organisations to weigh faster reuse against stricter validation and access control.

• Bronze ingestion captures raw API events from SaaS platforms while preserving source identifiers so investigators can trace anomalies back to the originating NHI or token.
• Silver processing standardises timestamps, removes duplicates, and enriches records with account metadata, reducing the chance that flawed upstream data reaches automation.
• Gold datasets feed dashboards and agent workflows, but only after data owners confirm that permissions, retention, and schema stability are appropriate for business use.
• Teams that manage data products alongside NHIs often pair Bronze pipelines with secret hygiene reviews, referencing the Guide to the Secret Sprawl Challenge when credentials are embedded in ingestion jobs.
• Where service accounts call transformation jobs, practitioners commonly map access expectations to the Ultimate Guide to NHIs — Lifecycle Processes for Managing NHIs and validate design choices against the OWASP guidance.

Why It Matters in NHI Security

The medallion lifecycle becomes an NHI security issue because each layer often uses different identities, tokens, and transformation jobs, and those identities inherit the data exposure level of the layer they touch. If Bronze systems are over-permissioned, raw data and upstream secrets can spread into downstream jobs; if Silver logic is poorly governed, enrichment can mask malicious changes or duplicate compromised records; if Gold is overexposed, business-facing consumers may receive data they should not see. NHIMG research shows that 62% of all secrets are duplicated and stored in multiple locations, which makes pipeline sprawl especially dangerous when credentials are reused across stages. The Guide to the Secret Sprawl Challenge and the Guide to NHI Rotation Challenges show how quickly lifecycle weakness becomes operational risk. Practitioners should also treat Gold outputs as a governed interface, not a clean-room guarantee, consistent with identity governance principles in OWASP Non-Human Identity Top 10. Organisations typically encounter medallion lifecycle failures only after a bad dataset has already powered reporting or an agent has acted on poisoned data, at which point the lifecycle becomes operationally unavoidable to address.

Related resources from NHI Mgmt Group

• Lifecycle Governance
• Non-Human Identity Lifecycle Management
• Secrets Lifecycle
• What is the difference between SDLC security and Data and AI lifecycle security?