Self-improving coding agents still fail at sustained vigilance

At a glance

What this is: This is ngrok’s analysis of a self-improving coding agent that built tools and learned patterns but still struggled to maintain attention and act on recurring lessons across sessions.

Why it matters: It matters because AI agents that can modify their own behaviour still need governance at the harness and runtime layers, not just better prompts or stronger model reasoning, to avoid drift and deferred risk.

Context

Self-improving coding agents sit at the intersection of software automation, operational governance, and non-human identity control. The core problem is not whether an agent can reason about its own mistakes, but whether it can reliably turn that reasoning into action while it is still in the middle of a task.

In this article, ngrok shows that a coding agent can accumulate tools, skills, and reflections, yet still defer work and miss repeated patterns when the right trigger is not present. That is typical of agentic systems: introspection is cheap, but sustained execution discipline is hard to enforce without strong runtime structure.

Key questions

Q: Why do self-improving coding agents still miss the lessons they already learned?

A: Because learning and execution use different control paths. An agent can record a correction, but if the live workflow does not force that lesson into the current turn, the model will often continue with the most probable next action. The fix is not more memory alone. It is stricter runtime structure that turns learning into an enforced behaviour change.

Q: When do self-improving coding agents become a governance problem instead of a productivity gain?

A: They become a governance problem when deferred work, tool creation, and self-modification can happen without clear phase boundaries. At that point, the system can expand its own operational surface area while leaving today’s friction unresolved. Practitioners should treat that as an authorisation and lifecycle issue, not just a model-quality issue.

Q: What do teams get wrong about agent reflections and learning loops?

A: They often assume the agent will notice when a correction matters and will apply it consistently. In practice, recognition tasks are much weaker than structured tasks, so open-ended self-diagnosis fails unless the trigger is obvious and repeatable. Fixed reflection checkpoints are more reliable than asking the model to infer when to improve itself.

Q: What should security teams do when a coding agent starts rewriting its own harness?

A: Treat the harness as part of the control plane. Review which tools, skills, and memory updates the agent can modify, and separate live task permissions from maintenance permissions. That prevents a self-modifying agent from silently increasing its own scope while appearing to stay within normal workflow limits.

Technical breakdown

Why self-improving coding agents still defer work

A self-improving coding agent can log observations, update skills, and even build new tools, but those capabilities do not create persistent vigilance. The article’s key mechanism is the difference between a prompt-level instruction and a runtime trigger. If the agent is asked to notice friction only implicitly, it tends to defer the problem to a later maintenance pass because that path is already encoded in the workflow. In other words, the control failure is not lack of intelligence. It is lack of an enforced decision boundary that makes immediate action the default.

Practical implication: Design agent harnesses so that friction creates an immediate execution branch, not a deferred note for later review.

How reflection templates outperform open-ended learning capture

The article contrasts a vague learning-event-capture skill with a reflection template that always fires at session end. That difference matters because agent behaviour is shaped less by abstract capability than by clear invocation conditions. When the agent must decide whether a lesson counts as a learning event, it often skips the step. When the trigger is time and place bound, the workflow becomes reliable. This is a classic control-design lesson for agent systems: the more judgment you delegate to the model, the more likely you are to lose consistency.

Practical implication: Use structured checkpoints and fixed triggers for agent learning, rather than asking the model to infer when reflection is appropriate.

Why harness design matters more than model introspection

The article argues that the agent could analyze itself well, but not while simultaneously doing the primary job. That exposes a broader architecture problem for coding agents and other AI agents: parallel self-monitoring is fragile when the same context must support task execution. A harness that separates execution, reflection, and maintenance creates better behavioural reliability than a single monolithic loop that expects the model to police itself continuously. For NHI governance, this is the same pattern seen in service accounts and automation tokens: behaviour becomes safer when the environment constrains what the identity can do in each phase.

Practical implication: Separate task execution from self-review in the agent architecture, and govern each phase with different controls.

NHI Mgmt Group analysis

Self-improvement in coding agents is mostly an orchestration problem, not a cognition problem. The article shows that the agent could generate tools, capture feedback, and analyse its own sessions, yet still fail to act on that insight when the work was live. That is a governance signal: capability does not equal control. For practitioner teams, the lesson is to treat harness design as the primary security and reliability boundary.

Deferral buckets create a new class of agentic governance debt. Once deferred work is given a named place, the agent has a ready-made path to postpone action rather than resolve friction. That pattern matters for AI agent operations because the workflow itself can become a control failure. The named concept here is deferral drift, where the system learns to route present risk into future maintenance instead of closing it now.

Agent introspection has value only when it is bounded by explicit runtime conditions. The reflection template worked because it was triggered predictably, while open-ended self-diagnosis did not. That is consistent with modern agent governance thinking: the identity of the agent is less important than the authority model around its tools, memory, and refresh cycles. For teams running coding agents, the practical conclusion is to constrain when learning, tool creation, and execution can occur.

For NHI and agentic AI governance, this is an authorisation story as much as a learning story. A self-modifying coding agent is effectively a non-human actor that can change its own operational surface area over time. That means tool access, maintenance privileges, and session boundaries need policy, not optimism. The field should read this as evidence that runtime guardrails, not better self-talk, determine whether agent behaviour stays inside acceptable bounds.

Human oversight still sets the ceiling on agent autonomy. The article’s most important moment is not when the agent built something new, but when the operator recognised the gap between knowing and doing. That is the real control point for agentic systems: humans must define the conditions under which the system is allowed to learn, defer, or self-repair. The practitioner conclusion is to govern the harness first and the model second.

From our research library:

• Claude Code-assisted commits leaked secrets at a rate of 3.2%, more than double the human-only baseline of 1.5%, with peaks reaching 31 secrets per 1,000 commits in August 2025, according to the State of Secrets Sprawl 2026.
• Read next: AI Agent Observability, Audit and Incident Response Guide

What this signals

The pattern to watch is not whether a coding agent can self-improve in theory, but whether the harness forces improvement to happen at the moment friction appears. Once deferral becomes an available path, the system starts treating today’s problem as tomorrow’s maintenance work, which is a governance failure before it is a technical one.

Deferral drift: when an agent learns to route present friction into future cleanup instead of resolving it in the active session. That dynamic is especially dangerous in coding workflows because the same runtime that creates value can also create unattended risk, so the control objective must shift toward immediate action boundaries.

AI agents that modify their own tools and prompts need the same kind of lifecycle discipline we expect from other non-human identities. Access scope, maintenance authority, and reflection triggers should be treated as distinct policy domains, not blended into a single prompt that hopes for consistent judgement.

For practitioners

• Define hard triggers for agent reflection Require end-of-session or event-driven reflection steps that fire automatically, rather than leaving self-review to model discretion.
• Separate live task execution from maintenance loops Move tool creation, backlog processing, and harness updates into a distinct maintenance phase so the agent does not defer live friction into the next session.
• Limit agent tool authority during active work Give the agent only the minimum tools needed for the current task, and elevate privileges only for explicit maintenance or repair actions.
• Instrument deferred work as a control signal Track every postponed action as an observable governance metric so repeated deferral becomes a measurable runtime failure instead of a hidden habit.
• Review self-modification boundaries for coding agents Approve which tools, skills, and memory updates an agent may change on its own, and block any runtime rewrite that expands authority without review.

Key takeaways

• Self-improving coding agents can accumulate capabilities without reliably changing their runtime behaviour, which leaves the real control problem unresolved.
• The article shows that structured triggers and fixed reflection points outperform open-ended self-diagnosis when an agent needs to learn while working.
• For practitioners, the main lesson is to govern the harness, permissions, and maintenance boundaries around the agent rather than relying on better prompts alone.

Key terms

• Self-improving Coding Agent: A coding agent that can change its own tools, prompts, or operating workflow based on experience. The important governance question is not whether it learns, but whether those changes stay bounded by clear approval, scope, and runtime controls.
• Agent Harness: The agent harness is the runtime layer that wraps a model and turns it into an acting system. It usually includes the loop, tools, context handling, permissions, hooks, and logs. In security terms, it is often the real place where privilege sits and where identity evidence must be governed.
• Deferral Drift: The tendency for an agent to postpone present friction into a later maintenance phase instead of resolving it in the active session. This is a control failure because the system learns a convenient delay path that reduces immediate reliability and hides operational risk.
• Learning Event Capture: A structured method for recording corrections, preferences, and repeated patterns while a session is still active or immediately after it ends. It turns informal feedback into durable memory, but it only works when the trigger is explicit enough that the system does not have to infer whether learning occurred.

Deepen your knowledge

NHI Mgmt Group covers identity security, NHI governance, and agentic AI through independent research, practitioner guides, and the NHI Foundation Level course. Explore nhimg.org for resources that connect identity governance to the broader security disciplines your programme depends on.