Agent system design, not prompting, drives CyberGym performance

At a glance

What this is: This is an analysis of why agentic AI performance in vulnerability reproduction improved sharply when the surrounding system was redesigned, not just the model.

Why it matters: It matters because the same lesson applies to AI agents used in security workflows, where identity, tooling, and runtime guardrails determine whether autonomy stays governable.

👉 Read depthfirst's analysis of how agent system design changed CyberGym performance

Context

CyberGym is a benchmark for how well an agent can reproduce software vulnerabilities, but the article’s real subject is the system around the model. In plain terms, it argues that agentic AI capability is not only a model question, because environment, tooling, and feedback loops can either unlock or suppress performance. That makes the topic relevant to AI governance and to identity teams watching how agents receive access to code, tools, and execution contexts.

For security and identity practitioners, the lesson is that control of the agent runtime matters as much as model selection. If an AI system can edit files, recompile code, submit tests, or call tools, then its identity, privilege scope, and auditing model become part of the security design, not an afterthought. That is typical of early agent deployments, and it is exactly why governance often lags capability.

Key questions

Q: How should security teams govern AI agents that can choose tools at runtime?

A: Security teams should govern runtime agent choice as an access event, not as a simple application action. That means scoping permissions to the task, limiting token lifetime, logging every tool decision, and blocking the agent from reaching systems outside its approved context. Static roles alone are not enough when the execution path changes on each run.

Q: Why do AI agents need more than better prompting to perform reliably?

A: Because performance depends on the surrounding environment, not just the model’s language ability. Agents need task context, runtime feedback, and architecture that matches the job. Without those controls, they may misread failures, stop iterating, or use the wrong workflow, which turns capability into noise instead of repeatable output.

Q: What breaks when AI agents are given broad standing access?

A: Broad standing access breaks governance because the agent can move from one task to another without a fresh authorization check. That creates a control gap between intended scope and actual runtime behaviour. The result is weak accountability, limited containment, and audit trails that show activity without explaining why the activity was allowed.

Q: How do teams reduce risk when deploying modular agent architectures?

A: Use separate identities for the main agent, support agents, and any tooling that touches execution environments. Give each component only the permissions needed for its stage, then review their logs independently. Modular design reduces context overload, but it only improves security if the identity model is modular too.

Technical breakdown

Why agent scaffolding changes benchmark outcomes

CyberGym measures whether a model can turn a vulnerability description into a working proof of concept. The article shows that baseline prompting leaves a lot of capability hidden, because the agent lacks the right execution context, feedback, and task framing. Once the system includes clearer instructions, runtime visibility, and targeted tooling, the same underlying model can iterate more effectively. In other words, agent performance emerges from the interaction between model reasoning and the surrounding control plane, not from the model alone.

Practical implication: Treat agent scaffolding as part of the security boundary, because the identity and permissions of supporting tools shape what the agent can actually do.

Runtime feedback versus static analysis

The article contrasts static-only evaluation with an environment where the agent can inspect runtime behaviour during testing. That matters because many failure modes are not obvious from source code alone. When the agent can instrument the program, rebuild it, and observe data flow, it can test hypotheses instead of guessing. This is especially important in security use cases, where a single blind step can hide the real constraint or the real exploit path. The mechanism is iterative debugging, but at machine speed.

Practical implication: Give agentic security workflows controlled runtime feedback, but restrict the tools and data paths so observability does not become uncontrolled access.

Modular agent architectures reduce context overload

A key design choice in the article is splitting work between a main agent, an instrumentation agent, and subagents. That modularity keeps the main context cleaner while pushing narrow investigative tasks into smaller components. The benefit is not just performance. It also reduces the chance that one agent becomes a catch-all runtime identity with sprawling privileges and muddled responsibility. For security teams, that is the same structural issue seen in over-broad service accounts and overly capable automation identities.

Practical implication: Design agent roles as separate identities with narrow scopes, so each toolchain component has only the access needed for its part of the workflow.

NHI Mgmt Group analysis

Agent capability is a system design problem, not a prompt engineering problem. The article’s core evidence is that a redesigned environment delivered a major jump in benchmark performance without changing the basic task. That means organisations should stop assuming that a better model alone will fix agent risk or agent usefulness. The real control surface includes runtime access, orchestration, tool permissions, and observability. For practitioners, the conclusion is simple: govern the system around the agent, not just the model inside it.

AI agents create an identity governance problem as soon as they receive tools. Once an agent can edit files, execute commands, or trigger recompiled code, it behaves like a non-human actor with operational privileges. That makes access scope, delegation, logging, and revocation central governance questions. The identity bridge here is direct: every useful agent is also an identity-bearing workload, and unmanaged tool access becomes the route from capability to risk. Practitioners should assign explicit ownership and lifecycle controls before scaling agent autonomy.

Context is now a security control, not just a usability detail. The article shows that a missing description of the execution environment caused the agent to form the wrong mental model and stop iterating. In security operations, that maps to poor task framing, ambiguous runbooks, and brittle automation. The named concept here is meta-context drift: when an agent’s internal assumptions about the environment diverge from the actual control surface, performance drops and governance becomes unreliable. Practitioners should treat environment context as a controlled input, not informal guidance.

Modular agent architecture is the safer path to scaling autonomy. Splitting work into specialised subagents improved performance because each component had a narrower job and a cleaner context window. That same design pattern supports better accountability, because distinct identities can be monitored, constrained, and revoked separately. For AI governance, this is a stronger pattern than building one highly privileged generalist agent. Practitioners should prefer small, purpose-built agent identities over monolithic automation with broad access.

Agentic AI will expose the gap between theoretical governance and actual runtime control. The article’s results reinforce a broader market shift: organisations can no longer rely on policy statements that are disconnected from tool execution. Standards such as the NIST AI Risk Management Framework matter here because they push teams toward measurable governance, not aspirational policy. Practitioners should expect autonomy programmes to be judged by the quality of runtime controls, not the sophistication of the model branding.

What this signals

The governance signal is that AI agents are becoming a production identity class faster than most programmes can classify them. The right response is not to treat autonomy as a future-state exception, but to define access boundaries, auditability, and lifecycle controls now. For teams already mapping agent risk, the OWASP Agentic AI Top 10 provides a useful threat-model lens.

Meta-context drift: when an agent’s internal assumptions about the environment do not match the actual execution boundary, failure looks like model weakness even when the real issue is governance. That means programme owners need explicit control of task framing, runtime instrumentation, and delegated tool scope. The more an agent can touch code, data, or infrastructure, the more its identity model must look like a governed workload, not a casual automation script.

For practitioners

• Define each agent as a governed identity Assign every agent, subagent, and automation component its own identity, ownership, and access boundary. Avoid shared credentials or catch-all runtime accounts so that tool use, logs, and revocation stay traceable.
• Limit tool permissions to the minimum viable workflow Separate file editing, recompilation, submission, and data retrieval into distinct tool permissions. Keep the agent from moving between stages without explicit need, because broad tool reach becomes broad operational risk.
• Instrument runtime feedback with guardrails Let agents observe execution state, but constrain what they can inspect, change, and retain. Runtime visibility should improve debugging without creating unrestricted access to production-like environments or sensitive code paths.
• Split exploratory work into narrow subagents Use specialist subagents for instrumentation, call graph tracing, or targeted investigation. This reduces context overload in the main agent and makes privilege assignment and audit review easier.

Key takeaways

• The article’s central finding is that agent capability scales with system design, not prompting alone.
• Once an AI system can edit, execute, and iterate, it behaves like a non-human identity that needs explicit governance.
• Security teams should design agent identities, tool permissions, and runtime feedback together, because capability and control now rise or fail as one system.

Key terms

• Agentic AI: Autonomous AI systems capable of planning, deciding, and taking actions — including calling APIs, writing code, and orchestrating other agents — with minimal human oversight. Agentic AI introduces new NHI risks as agents must authenticate to external services.
• Runtime Instrumentation: Runtime instrumentation inserts observation points into a running app or operating system so investigators can see behaviour that encrypted traffic or obfuscation would otherwise hide. In mobile research, it helps reveal command-and-control activity, data theft, and environment checks during execution.
• Meta-Context Drift: A mismatch between what an agent believes about its operating environment and what the environment actually is. This can cause bad decisions, false conclusions, or unnecessary task abandonment, especially when the system lacks explicit execution context or clear control boundaries.
• Identity-bearing workload: An identity-bearing workload is any non-human system that can authenticate, hold credentials, or reach other systems with meaningful access. This includes service accounts, tokens, integrations, and AI-connected systems. The key governance point is that these workloads need ownership, lifecycle handling, and review, not just technical connectivity.

What's in the full article

depthfirst's full blog post covers the implementation detail this post intentionally leaves for the source:

• The exact agent scaffolding changes that produced the performance jump across the development set and final test set.
• The instrumentation workflow used to let the agent inspect runtime behaviour and refine proofs of concept.
• The rationale for using specialised subagents and how the architecture divided responsibilities across tasks.
• The benchmark caveats around dataset quality and why some failures may reflect benchmark construction rather than pure model limits.

👉 depthfirst's full post covers the architecture choices, runtime instrumentation, and benchmark caveats in more detail.

Deepen your knowledge

The NHI Foundation Level course, the industry's only accredited NHI security programme, covers NHI governance, machine identity security, secrets management, and agentic AI identity. It is designed for practitioners who need to bring identity discipline to automated systems and privileged workloads.