Join our Newsletter — 33% off our NHI Course

Notifications
Clear all

Threat model mismatch in app security - what teams miss


(@nhi-mgmt-group)
Member Moderator
Joined: 1 year ago
Posts: 20026
Topic starter  

TL;DR: Application security fails when protections are designed for the wrong attacker, not when the cryptography itself is weak, according to Arxan Technologies. The IEEE 1735 example and the LinkedIn recruiting email both show that assumptions about environment, access, and automation now matter more than nominal compliance.

NHIMG editorial — based on content published by Arxan Technologies: How to Meet (and Exceed!) IEEE 1735

Questions worth separating out

Q: What breaks when application security controls assume a passive attacker?

A: Controls break when they are designed for a user who only reads protected content, but the attacker can probe the runtime, observe errors, or automate trust abuse.

Q: Why do personalised AI-generated lures increase security risk?

A: They reduce the signal gap between legitimate outreach and malicious outreach by using public data to create highly contextual messages at scale.

Q: Where do runtime protections fail in hostile execution environments?

A: They fail when the control protects data in storage but not the process that handles it.

Practitioner guidance

  • Document the attacker model explicitly Record what the control assumes the attacker cannot do, then test whether that assumption still holds when the attacker can automate, instrument, or control the runtime.
  • Review error handling as an attack surface Remove descriptive error output where it can be used as an oracle, and separate internal diagnostics from externally visible responses.
  • Test security controls against AI-assisted abuse Run exercises that simulate scraped identity data, personalised lures, and repeated query attempts so teams can see where trust fails under scale.

What's in the full article

Arxan Technologies' full post covers the operational detail this post intentionally leaves for the source:

  • The paper-like walkthrough of IEEE 1735 failure mechanics and why its assumptions broke under active probing.
  • The detailed explanation of how AI-generated recruiting lures were assembled from public identity data and routed through spoofed infrastructure.
  • The specific cryptographic and runtime integrity discussions around white-box cryptography in hostile execution environments.
  • The references and source material that support the article's argument about threat model failure and control design.

👉 Read Arxan Technologies' analysis of threat model failure in application security →

Threat model mismatch in app security - what teams miss?

Explore further

View Full Forum →  |  NHI Foundation Course →



   
Quote
(@mr-nhi)
Member Moderator
Joined: 4 months ago
Posts: 19617
 

Threat model mismatch is the hidden control failure behind many security incidents: the strongest-looking protection can fail if it was built for a passive user and the attacker is active, instrumented, and automated. IEEE 1735 is a clean example because the cryptography was not the problem. The control gap was the assumption about who could interact with the system and how. Practitioners should treat the threat model itself as a security control, not a design appendix.

A question worth separating out:

Q: Should security teams treat threat modelling as a governance control?

A: Yes. Threat modelling should be treated as a governance control because it determines whether every other control is aimed at the real attacker. If the model is wrong, compliance can still look good while the system remains vulnerable. Teams should require evidence that assumptions were tested against automation, runtime access, and adversarial behaviour.

👉 Read our full editorial: Threat model mismatch is the real failure in application security



   
ReplyQuote
Share: