AI model context matters more than attacker hype in appsec

At a glance

What this is: This is Aikido’s analysis arguing that the security debate around Mythos overstates attacker gains and understates the importance of context in AI-driven application security.

Why it matters: It matters because IAM, NHI, and security teams need to account for how access to source code, credentials, and runtime context shapes both offensive testing and defensive coverage.

By the numbers:

• Aikido says it has run 1,000 real-world AI penetration tests.

👉 Read Aikido's analysis of Mythos, attacker context, and appsec risk

Context

AI model context is the missing variable in much of the current debate about frontier model risk. Capability matters, but security outcomes depend on whether the system under assessment can see source code, runtime behaviour, internal logic, and the access paths that shape real exploitation.

For identity and access programmes, that same principle applies to NHI and agentic AI security. A model or testing system with limited context can miss the practical significance of credentials, tokens, and application boundaries, while a defender with full visibility can evaluate how those identities behave across code, infrastructure, and integrations.

Key questions

Q: What breaks when AI security testing lacks source code context?

A: Testing tools without source code context miss the relationships between control flow, dependency behaviour, and identity-linked access. That leads to shallow findings, more false negatives, and weaker prioritisation. Organisations should treat partial-context testing as useful for screening, but not sufficient for high-confidence security decisions or remediation planning.

Q: Why do service accounts matter so much in application security?

A: Service accounts often define how one component trusts another, which means they can open or close the attack path. If those identities are over-scoped, stale, or poorly monitored, an attacker can move from a single application foothold into adjacent systems. That makes machine identity governance a core appsec control, not an administrative detail.

Q: How do you know if AI-assisted testing is actually improving security coverage?

A: You know it is working when the findings change with added context, not just with model size. A useful programme shows deeper defects in whitebox or hybrid testing, ties results to runtime behaviour, and produces remediation items that map to real access paths rather than generic vulnerability lists.

Q: What is the difference between greybox and whitebox security testing?

A: Greybox testing uses limited internal knowledge, while whitebox testing has access to source code and deeper system detail. Whitebox is better for finding structural issues, trust boundary failures, and identity-linked weaknesses. Greybox remains useful for simulating less informed attackers, but it should not be the only basis for risk decisions.

Technical breakdown

Why model capability is not the same as security context

Frontier models can reason well, but security work depends on context-rich evidence. In application testing, context means source code, API behaviour, dependency relationships, and runtime signals that explain what changed and whether that change is legitimate. Without that, even a strong model is inferring from partial data, which limits both detection quality and exploit validity. The article’s core point is that attackers do not automatically gain the same visibility defenders already possess, especially in environments where application logic and identity controls are tightly coupled.

Practical implication: measure tools by the quality of context they can consume, not by model size alone.

Why whitebox testing produces deeper findings than greybox testing

Whitebox testing gives the model source code and structural knowledge, while greybox testing limits what it can inspect. That difference changes output quality because the model can connect code paths to execution paths and identify hidden weaknesses more accurately. The article says whitebox tests found 7x more critical and high-severity issues and ran at roughly twice the efficiency of greybox tests. That pattern shows security testing is constrained by observability, not just reasoning power.

Practical implication: prioritise test workflows that combine static and dynamic analysis across code and runtime.

How identity context changes the value of application security

The article’s mention of application credentials and tokens points to a broader identity lesson. Credentials are not just secrets, they are contextual keys to behaviour, privilege, and trust boundaries. If defenders can trace where tokens are used, how they propagate, and which services they unlock, they can see changes that a scanner or model without that context will miss. That is where NHI governance intersects with appsec: access is only meaningful when it is tied to system behaviour and lifecycle control.

Practical implication: extend identity governance into application runtime so secrets, tokens, and service accounts are visible in context.

Threat narrative

Attacker objective: The objective is to uncover exploitable weaknesses or modify trusted dependencies without being detected by controls that lack full system context.

1. Entry occurs when attackers or testing systems work from limited external visibility rather than complete source and runtime context.
2. Escalation happens when incomplete context leads to missed weaknesses in code paths, dependencies, or identity-linked access flows.
3. Impact follows when the defender cannot distinguish legitimate behaviour from malicious change because observability is too shallow to support effective review.

NHI Mgmt Group analysis

Context is the real security control. The article correctly rejects the idea that model capability alone determines attacker advantage. In practice, security outcomes depend on whether the defender has source code, runtime telemetry, and identity context that attackers do not. That is why application security, NHI governance, and agentic AI controls increasingly converge around visibility rather than raw detection breadth. Practitioners should treat context as a first-class control surface.

Whitebox advantage is a governance signal, not just a testing preference. If whitebox AI tests produce materially deeper findings than greybox tests, the programme is telling you where your blind spots sit. That is especially relevant for secrets, service accounts, and application tokens that only become visible when code and runtime are analysed together. The practical lesson is that governance should follow observability, because hidden identity paths are where abuse survives.

Service credentials are part of the application attack path. The article’s reference to application credentials and tokens is a reminder that NHI security is not separate from appsec. When secrets, tokens, and internal business logic are evaluated as a single system, organisations get closer to the real blast radius of compromise. That makes lifecycle control, credential scope, and runtime visibility the meaningful governance levers.

AI will compress attacker effort, but not erase defender context. The market narrative often assumes that frontier models flip the balance in favour of the attacker. NHIMG’s view is that this overstates the near-term change and undervalues the defender’s structural advantage in knowing how systems actually work. The challenge is not model hype, it is whether organisations can operationalise the context they already own into better identity and application controls.

What this signals

AI-assisted testing will increasingly reward organisations that can supply full context to their tooling, because partial visibility produces partial security answers. That shifts programme value away from isolated scans and toward linked code, runtime, and identity observability, especially where service accounts and tokens shape execution paths.

Context-rich attack surfaces: the next maturity gap is not whether teams use AI in security, but whether their AI can see the system the way a defender should. Where code, dependency data, and identity telemetry remain fragmented, the programme will keep producing shallow findings that are hard to operationalise.

For identity teams, this is a reminder that NHI visibility cannot stop at secret storage. Runtime use, privilege scope, and dependency change history need to be visible together if teams want to reduce blind spots and support faster triage across application and access controls.

For practitioners

• Map security testing to context depth Classify your testing coverage by whether each workflow has source code, runtime telemetry, dependency data, and identity context. Where only partial context exists, treat the findings as incomplete and schedule deeper review of the affected application paths.
• Extend NHI visibility into application runtime Track where API keys, tokens, and service account credentials are used in production rather than only where they are stored. This helps security teams link identity artefacts to behaviour and spot changes that would otherwise look legitimate.
• Use whitebox results to prioritise control fixes Move the deepest findings from whitebox testing into remediation queues for access boundaries, dependency trust, and privilege scope. The goal is to fix structural weaknesses that greybox scans are unlikely to expose.
• Review trusted dependency change controls Require extra scrutiny for package updates, maintainer-account changes, and dependency additions that alter runtime behaviour. That is where attackers can hide in plain sight when scanners lack the surrounding context.

Key takeaways

• The article’s central claim is that model hype is less important than system context in determining security outcomes.
• Evidence from 1,000 AI penetration tests shows whitebox visibility produces materially deeper findings than limited-context testing.
• Practitioners should connect application security, runtime telemetry, and NHI governance so secrets and tokens are judged in context, not isolation.

Key terms

• White Box Testing: A testing approach where the tester has access to source code and full system context. It supports deeper validation of logic, data flow, and authorization behaviour, and in agentic settings it produces the most precise findings because the agent reasons from implementation truth.
• Context Depth: The amount of system knowledge available to a security tool or analyst when evaluating risk. In application security, deeper context means code, runtime behaviour, dependency relationships, and identity data are all visible together, which improves the quality and precision of findings.
• Runtime Identity Visibility: Runtime identity visibility is the ability to see which non-human identity accessed which dataset, through which tool, and at what time. It connects identity governance to privacy control by making machine behaviour auditable instead of inferred from process documentation.

What's in the full article

Aikido's full blog post covers the operational detail this post intentionally leaves for the source:

• The Mythos-Ready checklist that translates the article’s argument into a practical security workflow.
• The examples Aikido uses to compare whitebox and greybox AI penetration testing in more detail.
• The operational guidance on using static and dynamic analysis together to improve security findings.
• The source article’s specific framing around attacker context, defender visibility, and application security priorities.

👉 The full Aikido post covers the whitebox testing results, dependency example, and defender-context argument in more detail.

Deepen your knowledge

The NHI Foundation Level course, the industry's only accredited NHI security programme, covers NHI governance, machine identity security, and secrets management. It is designed for practitioners who need to connect identity controls to real-world security operations.