Design-Time Threat Modelling

Expanded Definition

Design-time threat modelling is a pre-implementation discipline: teams analyse a proposed feature, architecture, or workflow before code is written so they can shape the design around realistic abuse paths, trust boundaries, and data handling choices. It is broader than a late-stage security review because it asks what could be introduced by the design itself, not only what the implementation might later break.

Its practical boundary is important. A design-time model is not a substitute for testing, code review, or production monitoring; it is the point where assumptions are still cheap to change. For identity-heavy systems, that includes service accounts, API tokens, delegated access, and human approval steps that may not yet be visible in diagrams. Guidance across the industry is broadly consistent on this early-analysis approach, but the exact method varies by team and architecture.

For a standards-oriented reference point, Microsoft’s threat modelling guidance is useful because it treats the exercise as part of secure design rather than a one-time audit. See Microsoft’s threat modelling guidance for a structured view of design-stage analysis.

Examples and Use Cases

Design-time threat modelling shows up whenever a team pauses to ask what the system will trust, what it will expose, and how an attacker or careless operator could misuse those choices before launch.

• A product team reviews a new file-upload workflow and spots that direct object access would bypass intended approval checks.
• An AI feature team maps prompt, retrieval, and tool-call boundaries before release to see where injected content could influence downstream actions.
• An identity team models a new service-to-service flow and decides whether short-lived credentials or brokered access are needed instead of long-lived secrets.
• A platform team examines a multi-tenant design and finds that logging, caching, or shared storage could cross data boundaries if isolation is weak.
• A fraud or abuse team uses the exercise to determine whether an onboarding flow creates an account-takeover path through weak recovery logic.

In practice, the main trade-off is speed versus fidelity: the earlier the exercise happens, the less complete the architecture is likely to be, but the more influence the team has over the final security posture.

Security Implications

When design-time threat modelling is skipped or treated as a paperwork exercise, the failure is usually structural rather than cosmetic. Weak trust boundaries, overbroad access, unclear ownership of sensitive data, and hidden dependency chains get baked into the system and become expensive to unwind later. The result is often not a single bug but a pattern of predictable exposures.

Common consequences include privilege creep, insecure service interactions, confused-deputy behaviour, data overexposure, and controls that are technically present but misplaced in the flow. A practitioner should watch for symptoms such as diagrams that omit secrets, external calls, fallback paths, human override steps, or cross-domain data movement. Those omissions often reveal where the true attack surface lives.

The value of the exercise is that it turns vague concern into a concrete design choice. A team that identifies the risk early can change the architecture, add explicit approval boundaries, or narrow trust assumptions before implementation locks them in.

Domain and Governance Relevance

In identity and agentic systems, design-time threat modelling matters because the most important security decisions are often hidden in orchestration rather than in a single application control. If a workflow will authenticate workloads, delegate actions, call tools, or pass secrets between components, the design itself determines who can act, on what authority, and with what blast radius.

That is why the term is especially relevant to non-human identity and AI-enabled environments. The question is not only whether a component is authenticated, but whether its identity, permissions, and delegation path are appropriate for the action it may trigger. In NHI governance, design-time modelling helps teams decide whether a process should use a reusable credential, a short-lived token, scoped delegation, or a brokered approval step. Those choices shape accountability before deployment, when they are still governable.

For NHIMG readers, the important point is simple: good design-time modelling does not merely find risks earlier. It defines safer identity, trust, and execution patterns before they become operational dependencies.

Risk and Threat Considerations

Design-time threat modelling fails when teams assume the first implementation will be secure enough to fix later. The material risk is that architectural weaknesses become durable exposure, especially in systems with external inputs, delegated access, or automation that can act faster than human review.

Failure mechanism: Attackers and abuse cases exploit the trust relationships the design failed to challenge, such as excessive privilege, weak separation of duties, unsafe fallback logic, or shared credentials across components. In AI and workflow systems, the same mechanism can appear as tool abuse, prompt injection into downstream actions, or escalation through overtrusted automation.

Impact: The result can be unauthorized access, data leakage, unsafe actions taken by trusted components, lateral movement across services, or a redesign effort so large that security constraints are applied inconsistently after release.

Practitioner Guidance

Why practitioners should care: Design-time threat modelling is where security teams still have leverage over architecture, identity boundaries, and trust decisions. Once a workflow or agentic path is implemented, many of those choices become difficult to change without rework.

Common misunderstanding: Teams often treat the exercise as a diagram review, but its real value is challenging assumptions about who or what is allowed to act, what data crosses boundaries, and which failures would create unsafe fallback behaviour.

Practitioner takeaway: Use the session to surface one design decision that materially changes trust or privilege, then force a documented choice before build work proceeds.

Related resources from NHI Mgmt Group

• What do organisations get wrong about AI agent safety at design time?
• How can organisations tell whether their threat modelling is actually improving security?
• Who is accountable when threat intelligence is not acted on in time?
• Why do digital credentials change privacy and IAM design at the same time?