AI model context matters more than attacker hype in appsec

At a glance

What this is: Aikido argues that AI model capability alone does not flip the attacker-defender balance in appsec, because effective testing still depends on source code, runtime behaviour, and business logic context.

Why it matters: For IAM, NHI, and application security practitioners, the key implication is that visibility and contextual understanding matter more than raw automation when evaluating access paths, dependencies, and attack surfaces.

By the numbers:

• Whitebox tests surfaced 7x more critical and high-severity issues than greybox tests in Aikido's analysis of 1,000 real-world AI penetration tests.

Context

The core governance gap here is a false equivalence between model capability and security effectiveness. In application security, an agent or model can only reason well when it has enough context to understand code, runtime behaviour, dependencies, and the business logic that ties them together. Without that context, speed increases output volume, but not necessarily finding quality or decision quality.

That matters for IAM and NHI programmes because application security failures often hinge on access paths, tokens, API credentials, and service-level trust relationships that are only visible when static and dynamic signals are combined. The article's point is not that AI is harmless. It is that defenders usually hold the richer operational context, and that advantage changes how AI-assisted testing should be assessed.

Key questions

Q: How should security teams use AI models for vulnerability detection without overestimating their coverage?

A: Use AI models as high-signal assistants, not as complete scanners. In benchmarked code reviews, precision can be strong while recall stays low, which means the model may be right when it flags a flaw but still miss most real issues. Security teams should pair model output with deterministic checks, human review, and clear thresholds for what counts as actionable evidence.

Q: Why do whitebox tests produce better results than greybox tests for AI-assisted appsec?

A: Whitebox tests expose the implementation details that let an AI reason about hidden branches, vulnerable flows, and unsafe assumptions. Greybox tests can observe behaviour, but they cannot fully explain why a condition exists or how multiple components interact. In practice, richer context improves finding quality more than raw model strength alone.

Q: What signs show that an AI security workflow is missing enough context?

A: The main signs are shallow findings, inconsistent severity ratings, and weak explanations for why a weakness matters. If the workflow cannot connect code changes to runtime behaviour or dependency updates to actual trust impact, it is probably operating with partial visibility. That is a context problem, not just a tuning problem.

Q: How do NHI credentials change the way AI should be used in appsec?

A: NHI credentials add a trust layer that AI cannot infer reliably from static code alone. Service accounts, tokens, and API keys shape which actions are actually possible, so the analysis has to include issuance, scope, and runtime use. If those identities are invisible, AI testing will miss the real governance boundary.

Technical breakdown

Why context changes AI pentest quality

AI-assisted testing is not just a function of model scale. The quality of findings depends on whether the system can see source code, runtime behaviour, dependency relationships, and business logic at the same time. When those inputs are available, a testing agent can connect a vulnerable call path to how the application actually executes, which improves both precision and severity ranking. Without that context, the same model mostly produces broader but weaker inferences. In practice, context is the control surface that determines whether AI helps identify real attack paths or merely accelerates superficial scanning.

Practical implication: prioritise data access and telemetry quality before assuming model choice will improve application security outcomes.

Whitebox versus greybox testing in AI workflows

Whitebox testing gives the agent source code, which lets it reason about implementation details, hidden control flow, and unsafe assumptions. Greybox testing provides partial visibility, usually enough to observe behaviour but not enough to explain why the behaviour exists or where the weakness originates. That difference matters because vulnerability discovery is often a combinatorial problem across code, configuration, and runtime state. The article's 1,000-test sample shows that fuller visibility materially changes result quality, not just testing speed. This is why context-aware AI security work should be treated as a visibility problem, not a capability race.

Practical implication: use whitebox-style workflows for high-value applications where source and runtime access can be combined safely.

Why dependency and execution context matter in appsec

A scanner that sees a package name but not the dependency tree, transitive changes, or live behaviour cannot reliably distinguish legitimate updates from malicious ones. The Axios example in the article shows that the failure was not missing malware signatures alone, but missing context about what changed and how it should have behaved. That same pattern applies to modern appsec more broadly: attack paths are often created by trusted updates, credentialed access, or subtle logic changes rather than obvious payloads. Context makes those changes legible.

Practical implication: pair dependency analysis with runtime and provenance checks so trust decisions are based on behaviour as well as code.

Threat narrative

Attacker objective: The objective is to introduce or exploit a weakness that passes superficial security checks because the defender lacks enough system context to recognise it.

1. Entry occurs through ordinary application changes or trusted component updates that do not look obviously malicious in isolation.
2. The attacker relies on the defender's lack of code, dependency, or runtime context to hide the meaningful change inside a legitimate-looking workflow.
3. Impact comes from a weakness that only becomes visible when the full system state is understood, which allows the malicious change to survive shallow scanning.

Breaches seen in the wild

• Laravel-Lang Composer compromise 2026: A stolen GitHub PAT let an attacker rewrite every Laravel-Lang release tag in May 2026, serving a PHP stealer of cloud, CI and SSH credentials.

Read and download The State of NHI & AI Agent Breach Report 2026, covering 200+ breaches impacting Non-Human Identities including AI Agents.

NHI Mgmt Group analysis

Context is now the controlling variable in AI-assisted application security: model capability does not produce reliable security outcomes unless the testing system can see code, runtime behaviour, and internal logic together. That is why the same model can look impressive in lab conditions and underperform in constrained environments. For security teams, the real question is not how strong the model sounds, but how much of the system it can actually reason over.

The attacker advantage narrative is overstated when defenders already hold structural visibility: organisations usually know more about their own code paths, credentials, API interactions, and deployment behaviour than any external attacker or model prompt can infer. That visibility is especially relevant where NHI, tokens, and service credentials shape application behaviour. The governance problem is not model hype. It is whether defenders can convert existing system knowledge into usable security control.

AI will lower the cost of discovery, but it will not erase context asymmetry overnight: speed helps both sides, yet defenders start with richer internal truth and more complete telemetry. This creates a durable advantage if it is operationalised through testing, triage, and dependency governance. The field should therefore treat AI-assisted appsec as a context-management problem, not a race to the strongest model.

Application security is moving toward context-rich detection, not context-free automation: the most useful AI work combines static analysis, runtime signals, and dependency intelligence into a single decision loop. That is the named shift here, and it changes how teams should judge both tools and test results. The practitioner takeaway is clear: measure whether your security stack can reconstruct system behaviour, not just produce findings.

For NHI and IAM teams, the intersection is direct: application context includes credentials, service identities, and API trust relationships, and those are often the real control points attackers target. A model that cannot see how a token is used, where a service account is authorised, or what a legitimate dependency update looks like will miss the governance issue. Practitioners should therefore treat identity and runtime context as part of the same security problem.

What this signals

Context-rich application security is becoming the differentiator: the organisations that can join source code, runtime behaviour, and dependency intelligence will get better results from AI-assisted testing than those that treat model choice as the main decision. That shift also matters for NHI governance, because tokens and service identities are part of the same trust graph as code and APIs.

Defenders should expect AI to accelerate discovery, but not to eliminate the need for operational visibility, provenance checks, and identity-aware analysis. Where application logic, service credentials, and dependency changes intersect, the security programme needs evidence about behaviour rather than assumptions about intent.

For practitioners

• Combine source and runtime visibility Correlate source code review, runtime telemetry, and dependency metadata before trusting AI-assisted findings on production apps.
• Prioritise whitebox testing for high-value applications Use fuller access for applications where business logic, internal APIs, and credential flows materially affect risk.
• Govern dependency changes as security events Treat unexpected package updates, maintainer-account compromise, and transitive dependency changes as review triggers, not routine hygiene.
• Map NHI and API trust paths explicitly Inventory service credentials, tokens, and API call chains so AI-assisted analysis can follow the actual trust relationships in the application.

Key takeaways

• AI model capability does not automatically create attacker advantage in application security because context remains the limiting factor.
• The strongest findings come when source code, runtime behaviour, and dependency relationships are analysed together, not in isolation.
• Security teams should treat AI-assisted appsec as a visibility and governance problem, especially where NHI credentials and API trust paths are involved.

Key terms

• Context-aware security: Context-aware security is a security approach that changes decisions based on the situation around a user, device, workload, or request. It uses signals such as identity, location, device health, behavior, data sensitivity, and time to decide access, monitoring, or response. This helps security controls adapt to real risk instead of relying on static rules.
• White Box Testing: A testing approach where the tester has access to source code and full system context. It supports deeper validation of logic, data flow, and authorization behaviour, and in agentic settings it produces the most precise findings because the agent reasons from implementation truth.
• Gray Box Testing: A testing approach that gives the tester partial internal context such as credentials, documentation, or workflow information without full source code. It improves coverage and efficiency, but still leaves implementation details and some trust assumptions invisible.
• Dependency context: The information that shows how packages, transitive dependencies, maintainer trust, and updates relate to one another inside a software system. Security value comes from understanding what changed, why it changed, and whether the change matches legitimate behaviour, not just from spotting a package name.

Deepen your knowledge

NHI Mgmt Group covers identity security, NHI governance, and agentic AI through independent research, practitioner guides, and the NHI Foundation Level course, the industry's only accredited NHI security programme. It is designed for practitioners who need to govern machine identities, secrets, and access lifecycles with practical discipline.