Self-evolving agent

An AI system that can change some part of its own operational behaviour during execution or through an improvement loop. In practice, this may involve memory, prompts, skills, or code. The governance challenge is that the system being managed can also influence the rules that manage it.

Expanded Definition

A self-evolving agent is an AI system that can alter some part of its own operating behaviour while it is running or across repeated improvement cycles. That change may affect memory, prompts, tool selection, policies, skills, or even generated code. The defining feature is not simple automation, but the presence of a feedback loop that lets the system adapt without a full human redesign.

In security terms, the risk is that the system is no longer static enough for one-time approval. A self-evolving agent may learn from outcomes, rewrite parts of its own workflow, or update the way it invokes tools. That makes governance harder because the controls must cover both the current state and the next state. Guidance varies across vendors and research groups, but the industry increasingly treats this as an OWASP Agentic AI Top 10 concern, especially where autonomous change can expand attack surface or weaken oversight.

The most common misapplication is treating a self-evolving agent as a fixed application, which occurs when organisations approve its behaviour once and ignore later changes introduced by memory updates, tool chaining, or self-modification.

Examples and Use Cases

Implementing self-evolving behaviour rigorously often introduces control overhead, requiring organisations to weigh improved task performance against the cost of tighter monitoring, versioning, and rollback readiness.

• A support agent updates its memory after each resolved ticket so it can adapt tone, routing, or escalation choices on later interactions.
• An operations agent revises its own prompt or tool policy after observing which actions succeed, creating a moving target for NIST AI Risk Management Framework governance.
• A coding agent generates patches, tests them, and then changes its internal repair strategy based on test outcomes, which can accelerate delivery but also propagate flawed assumptions.
• A security analyst agent refines detection logic or investigation steps after each incident review, improving response quality while increasing the need for traceability.
• A research agent updates retrieval preferences or task decomposition rules after each run, which can improve accuracy but also shift what data it trusts over time.

These use cases overlap with autonomous tool use, but they become self-evolving only when the system changes itself rather than simply executing a prewritten workflow. For adversarial framing, the MITRE ATLAS adversarial AI threat matrix is useful for thinking about how those changes can be manipulated by attackers.

Why It Matters for Security Teams

Security teams need to know whether an agent can alter its own behaviour because every adaptive step can create a new trust decision. A system that rewrites prompts, changes memory, or edits code may also bypass assumptions embedded in approval processes, logging, or segregation of duties. That is why self-evolving agents sit at the intersection of AI security, software assurance, and identity governance when tool access or privileged actions are involved.

The operational issue is not just model quality. It is whether the organisation can prove what the agent was allowed to do before the change, what it changed, and whether the new behaviour still fits policy. Frameworks such as the NIST AI Risk Management Framework and CSA MAESTRO agentic AI threat modeling framework help teams structure those questions, while the OWASP Top 10 for Agentic Applications 2026 highlights how autonomy can be abused if change controls are weak.

Organisations typically encounter the real impact only after an agent has silently drifted from approved behaviour, at which point self-evolving control becomes operationally unavoidable to address.

Related resources from NHI Mgmt Group

• How do you know whether an agent’s self-map is actually useful?
• How should security teams govern AI agent self-service in ITSM?
• Who should own the decision to self-host agent governance infrastructure?
• Agent Self-Documentation