Secure-by-design breaks when security is treated as someone else’s job

At a glance

What this is: This podcast episode argues that secure-by-design fails when accountability is externalised, and that AI systems can become permission amplifiers when broad access outpaces governance.

Why it matters: It matters because IAM, PAM, and NHI programmes now have to govern not just who can access systems, but how software and AI systems turn access into outcomes across multiple business domains.

By the numbers:

• Only 44% of developers are reported to follow security best practices for secrets management, exposing a significant developer behaviour gap.

👉 Read 1Password's Chasing Entropy episode on secure-by-design and AI permission amplification

Context

Secure-by-design only works when the organisation treats security outcomes as part of product delivery, not as a downstream customer problem. In identity terms, that means access, auditability, and default behaviour have to be designed into the system before users, workloads, or AI systems start taking action.

The episode also points to a broader IAM and NHI problem: systems with broad internal access can turn into permission amplifiers if roles, rights, and logging do not constrain what they can retrieve or combine. That is especially relevant when AI systems can span email, chat, HR, and business applications faster than existing control models were built to supervise.

The core issue is not whether security teams know what to do. It is whether leadership is willing to own the operational cost of making secure outcomes normal, measurable, and difficult to ignore.

Key questions

Q: How should teams govern AI systems that can combine data across business apps?

A: Treat the AI system as a privilege amplifier, not just another authenticated workload. Define which data combinations are allowed, log every source the system touches, and review whether the combined output reveals more than any single human role should see. The goal is to govern the business effect of access, not just the token behind it.

Q: Why do secure-by-design programmes fail in practice?

A: They fail when organisations treat security as a feature, a compliance activity, or someone else’s job. In practice, unsafe defaults, weak disclosure handling, and slow remediation persist because no leader owns the customer security outcome end to end. Secure-by-design only works when accountability is built into product delivery and measured continuously.

Q: What breaks when access controls do not account for AI correlation risk?

A: Traditional controls assume access is evaluated one system at a time. That breaks when an AI layer can correlate data from HR, email, chat, and internal tools into a single sensitive answer. The result is a larger effective privilege than the original entitlement suggests, which means audit and segmentation have to move up a level.

Q: Who is accountable when a product ships with unsafe defaults or weak dependency hygiene?

A: The vendor that ships the product remains accountable, even when the issue came from upstream code or a third-party component. Customers do not experience supply chains, they experience shipped software. That means ownership must cover dependency visibility, remediation, and disclosure, not just the existence of an SBOM.

Technical breakdown

Secure-by-design as an accountability model

Secure-by-design is not a checklist of secure features. It is an operating model that places responsibility for customer security outcomes on the organisation that ships the product or platform. In practice, that means defaults, update paths, disclosure handling, and friction all matter as much as the underlying technical control. If customers must notice, interpret, and correct every unsafe choice, the design has already failed. The episode frames this as an incentives problem because accountability shifts only when leadership treats security as part of product quality, not a post-sale concern.

Practical implication: align security ownership to product and engineering leadership, not just the security team.

AI permission amplifiers and over-broad access paths

An AI system becomes a permission amplifier when it can combine access across systems in ways a human user normally cannot. The risk is not only data exposure. It is the ability to query, correlate, and return sensitive conclusions across email, HR, chat, and internal tools at machine speed. That changes the meaning of access control, because the concern is no longer just whether a token is valid. It is whether the combined rights create an outcome the organisation never intended to grant.

Practical implication: review every AI access path for cross-system correlation risk, not just authentication strength.

Software supply chain responsibility does not stop upstream

The supply chain discussion in the episode reinforces a common governance mistake: treating upstream dependencies as someone else’s risk. If insecure or unmaintained components ship inside customer-facing products, the vendor still owns the outcome because the product is what the customer receives. SBOMs help only if they are tied to visibility, remediation, and disclosure discipline. Otherwise, they become a record of known risk without a management response.

Practical implication: tie dependency visibility to remediation ownership and customer communication, not documentation alone.

Breaches seen in the wild

• Cisco DevHub NHI breach — IntelBroker exploited exposed Cisco credentials, API tokens and keys in DevHub.
• DeepSeek breach — DeepSeek breach exposed 1M+ log lines and sensitive secret keys.

Read our 52 NHI Breaches Analysis report for a comprehensive view of breaches impacting Non-Human Identities including AI Agents.

NHI Mgmt Group analysis

Secure-by-design fails when organisations outsource security outcomes. The episode’s central point is that product teams often treat security as a feature, while customers are left to absorb the operational risk. That is not a tooling gap. It is a governance failure where accountability is detached from delivery. The practitioner conclusion is that identity and product governance must be owned where decisions are made, not where incidents are handled.

AI systems create a permission amplification problem, not just an access problem. When a system can query HR, email, chat, and business applications in one flow, its effective privilege exceeds any single human role. That is a structural issue for IAM and NHI programmes because traditional role design assumes the actor is bounded by a person or a narrowly scoped service account. The practitioner conclusion is to govern combined access effects, not isolated entitlements.

Cross-domain access is the new blast-radius variable. The episode shows why broad internal connectivity matters more than whether a system is technically authenticated. If an AI or automation layer can fuse data from multiple sources, the security question becomes how far one request can travel before it becomes a business decision. The practitioner conclusion is that audit and segmentation must be designed around outcomes, not just per-system permissions.

Secure-by-design only works when leadership accepts measurable obligation. The article’s quality-and-safety comparison is the right frame because software security will not improve through aspiration alone. Leaders have to make the cost of safe defaults, transparency, and update adoption visible in the product lifecycle. The practitioner conclusion is that governance without executive accountability produces the same unsafe behaviour in new packaging.

Permission amplification is a useful named concept for modern identity governance. It describes the point where valid access becomes broader-than-intended decision power because systems can combine data and actions across domains. That concept applies to human workflows, service accounts, and AI-enabled systems, but it becomes most visible when software can act at machine speed. The practitioner conclusion is to evaluate what an identity can do after access is combined, not just what it can reach individually.

From our research:

• Companies are dedicating an average of 32.4% of their security budgets to secrets management and code security, with US organisations leading at 40.8%, according to The State of Secrets in AppSec.
• 43% of security professionals are concerned about AI systems learning and reproducing sensitive information patterns from codebases.
• Secure-by-design and AI governance both fail faster when access, secrets, and code controls are managed in separate programmes, as explored in Ultimate Guide to NHIs , Why NHI Security Matters Now.

What this signals

Permission amplification: the practical test for AI governance is no longer whether a system can authenticate, but whether it can combine access in ways that exceed the intent of any single entitlement. If your programme cannot explain the largest possible answer a system can assemble from legitimate access, it is not yet governing the real blast radius.

Organisations should expect product and platform teams to absorb more security accountability, because downstream fix-up is too slow for modern attack and misuse patterns. The governance shift is from patching defects after release to proving that safe defaults, logging, and update adoption were designed in from the start.

The broader NHI lesson is that access scope, correlation power, and customer outcomes are now linked. That makes identity governance a delivery discipline, not just an access review cycle, and it requires explicit ties to controls such as CISA cyber threat advisories and internal control ownership.

For practitioners

• Map permission amplification paths across systems Inventory where one authenticated actor can query or combine email, HR, chat, and business application data. Focus on the post-authentication path, not just login controls, and identify where a single request can produce a cross-domain answer that no human role should normally see.
• Tie secure-by-design to executive ownership Assign product and engineering leaders explicit responsibility for secure defaults, remediation friction, disclosure handling, and update adoption. Security teams should measure whether the organisation can prove customer outcomes, not just whether patches exist.
• Review AI access for combined-scope risk Assess every AI integration for rights that become dangerous only when combined across multiple systems. Treat correlated access as a governance issue and require logging that shows what data sources were touched before the answer was produced.
• Make upstream dependency risk visible to customers Maintain a current software bill of materials and connect it to remediation ownership, disclosure workflows, and release gates. Do not let upstream provenance become a substitute for product accountability.

Key takeaways

• Secure-by-design fails when organisations treat security as a downstream customer problem instead of a delivery obligation.
• AI systems can amplify legitimate access into cross-domain power, which makes correlation risk a governance issue for IAM and NHI teams.
• Executive ownership, logging, and dependency visibility are the controls that turn accountability from a slogan into an operational reality.

Key terms

• Permission Amplifier: An identity or system that turns legitimate access into broader decision power by combining data, tools, or workflows across domains. The access itself may be valid, but the resulting outcome exceeds what the original entitlement was meant to enable. In AI environments, this is often where governance breaks first.
• Secure by Design: A delivery model in which product teams build security outcomes into defaults, update paths, disclosure handling, and operational ownership from the start. It is not a feature list. It is a governance commitment that treats shipped risk as the responsibility of the organisation, not the customer.
• Cross-Domain Access: Access that spans multiple systems or business functions and can be combined into a higher-risk result than any one system reveals on its own. The issue is not only breadth of reach. It is the ability to fuse data and actions across domains in ways traditional per-system controls do not see.

Deepen your knowledge

Secure-by-design, AI permission amplification, and cross-domain access governance are core topics in our NHI Foundation Level course, the industry's only accredited NHI security programme. If you are building a programme that has to govern software, service accounts, and AI systems together, it is worth exploring.

This post draws on content published by 1Password: Chasing Entropy with Bob Lord on secure-by-design, AI systems, and software supply chains. Read the original.