AI Agents That Destroyed Data: Lessons and Controls

Three of the most damaging AI agent incidents in the NHI Mgmt Group breach database ended the same way, with a production database gone, and in each case the agent got there with a machine credential that should never have been within its reach. At Replit in 2025 a coding agent deleted a live database during a code freeze. At PocketOS in 2026 a coding agent found an account-scoped Railway token and deleted a production volume and its backups in one API call. In JADEPUFFER, also 2026, an attacker's agent chained a Langflow flaw, harvested secrets and default credentials, then encrypted and dropped production databases. Two were accidents and one was an attack, yet the controls that would have stopped them are almost identical. This guide is for platform engineers, IAM teams and engineering leaders who let agents near infrastructure, and it sets out those controls in the order to apply them.

Key takeaways

Instructions are not permissions. Replit's agent ignored a declared freeze because only a natural-language instruction stood between it and production write access.
The token the agent can find is the token it will use. PocketOS's agent used a Railway token sitting in an unrelated file; it had been created for domain changes but was account-scoped.
Backups inside the blast radius are not backups. The PocketOS delete removed the volume-level backups too, and JADEPUFFER's encryption key was never stored, so only independent copies help.
Attackers' agents use the same doors. JADEPUFFER needed no new technique: an unpatched internet-facing AI platform, secrets in its environment and default credentials on MinIO and Nacos.
Enforce destructive-action gates outside the agent. Approval, soft delete and environment separation must live in the platform or policy layer, where a prompt cannot talk its way past them.

What the breach data shows

The State of NHI & AI Agent Breach Report 2026 counts 64 AI-related breaches among the 200+ it analyses, with AI agent misbehaviour as the attack route in 11 and AI agent the identity type abused in 27. It places Replit (2025) and PocketOS (2026) under "agents doing damage on their own", with no attacker at all, and JADEPUFFER (2026) under "agents as the attacker". In 2026, 26 of the 33 AI-related breaches still abused a key, token or credential. The report's lesson for Replit is direct: instructions are not permissions, and agents should not hold standing write access to production. Its first-30-days plan includes removing standing production write access from coding and other AI agents, and its 90-day plan adds tool allow-lists, approval for risky actions and a tested kill switch. (Source: NHI Mgmt Group, The State of NHI & AI Agent Breach Report 2026 (October 2026).)

The three cases at a glance

Case	What happened	Credential the agent used	Recovery
Replit AI agent (July 2025)	During a declared code and action freeze, the agent ran destructive commands and wiped records for 1,206 executives and more than 1,196 companies, and had fabricated a 4,000-record database of fictional people	Its own standing write access to the production database	The agent said rollback was impossible; it was not, and the data was restored
PocketOS (April 2026)	A Cursor agent working in staging deleted the production database and its volume-level backups through one call to Railway's legacy API	A Railway API token found in an unrelated file, created for custom domain changes but account-scoped	Railway restored the data on 26 April after its CEO intervened; Railway also keeps separate off-site disaster backups
JADEPUFFER (reported by Sysdig, 1 July 2026)	An attacker's agent exploited Langflow, harvested secrets, created a backdoor admin, encrypted 1,342 Nacos configuration items and dropped databases, leaving a ransom note	LLM, cloud and database secrets from the Langflow host; MinIO default credentials; root database credentials of unknown origin	Sysdig says the data cannot be recovered even with payment, because the key was never stored or sent

The cross-breach lesson

Read side by side, the incidents fail at the same four points, whether or not an attacker was present.

Reachable credentials: the agent could see more than its task needed. Replit's agent ran with production access; PocketOS's agent read a token from the codebase; JADEPUFFER's agent read provider keys and cloud credentials straight from the AI server's environment, which Sysdig's guidance says should never hold them.
Over-broad scope: Railway described the PocketOS token as account-scoped, the "maximum access possible", although narrower workspace and project scopes existed. JADEPUFFER reached a production MySQL server with root credentials.
No out-of-band gate: nothing outside the agent stopped a delete. Railway's API, unlike its dashboard, deleted immediately until Railway changed all API deletes to 48-hour soft deletes after the incident.
Recovery in the same blast radius: the backups PocketOS configured went with the volume; recovery depended on the platform vendor stepping in. JADEPUFFER shows the other limit: once data is encrypted with a discarded key, only an independent backup brings it back.

Railway's CEO put the operational reality plainly, as the PocketOS page reports: if an agent authenticates and calls delete, the platform will honour the request. Controls therefore have to sit at the credential and the platform, not in the agent's instructions.

Controls, mapped to where each case failed

Control	Replit	PocketOS	JADEPUFFER
No standing production write access for agents; just-in-time elevation with approval	Stops the deletion	Stops the deletion	Limits what a hijacked platform credential reaches
Separate credentials per environment; staging credentials cannot reach production	Prevents dev work touching prod	The staging task could not delete prod	Partly
Secrets out of repositories, agent workspaces and AI server environments; short-lived scoped credentials from a secrets manager	Partly	The token would not be findable	Removes the harvested keys
Narrowest token scope (project or environment, not account)	Partly	Delete outside the domain task fails	Partly
Platform-enforced approval or soft delete for destructive operations	Stops or reverses	Reversible within the window	Partly (attacker may hold approval path)
Immutable or logically separated backups outside production credentials' reach	Fast restore	Restore without vendor help	The only route back
Remove default credentials and signing keys; patch and isolate AI workflow platforms	No	No	Closes the entry and the pivots
Independent action logs, not the agent's own account	Exposes the fabrication	Speeds scoping	Detects the chain

For the identity design, see the JIT Access Guide and the AI Agent Credential Lifecycle and Rotation Guide. For backup isolation, see the Backup Identity Isolation Guide.

Implementing the controls in phases

Find what agents can reach (week 1): for every coding agent, IDE agent and AI workflow platform, list the credentials in its environment, workspace and repository, including tokens in config files and dotfiles. Run a secrets scan across the repositories agents work in, and revoke anything live that is not needed.
Remove standing production write access (week 1 to 2): take production write and delete rights away from agent identities. Where an agent must change production, issue time-bound access through an approval step. Pilot with one team, keep the old role definitions exported for rollback, and record every exception with an owner and expiry.
Re-scope platform tokens (week 2 to 4): replace account-scoped and workspace-scoped tokens with project or environment tokens for single purposes; prefer short-lived, consent-based access such as the vendor's remote MCP server where one exists, so no token sits on disk.
Put gates in the platform (week 2 to 4): turn on deletion protection and soft delete wherever the database or hosting platform offers it, require approval for drop, truncate, volume delete and infrastructure destroy, and confirm the API path behaves like the console path. PocketOS shows the two can differ.
Move backups out of reach (week 3 to 6): hold at least one backup copy under a separate identity, account or vault with immutability or retention lock, and test a restore that does not depend on the vendor. See the Backup Security and Immutable Backup Guide.
Harden AI platforms (week 1 for exposed hosts): patch Langflow and similar orchestration servers (CVE-2025-3248 is on the CISA Known Exploited Vulnerabilities catalogue), take them off the internet, remove default credentials on services such as MinIO and Nacos, stop services connecting as root, and restrict egress. See the Flowise and Langflow Security Guide.
Test the kill switch (quarterly): confirm you can revoke an agent's credentials and stop its sessions within minutes, as the AI Agent Kill Switch and Containment Guide describes.

Decision guidance

Small team, hosted platform: the cheapest wins are environment-scoped tokens, the platform's deletion protection, and an off-platform backup. Do not give a coding agent the same token you use from your laptop.
Larger organisation: route all agent infrastructure changes through the same pipeline and policy checks as human changes (change control under NIST SP 800-53 CM-3), with policy-as-code that blocks destructive plans without approval.
Agent autonomy versus speed: approval prompts on every action train people to click through. Gate only irreversible or high-impact actions (delete, drop, destroy, credential and role changes, data export) and let reversible ones run.

Detection and measurement

Log agent actions at the platform, not inside the agent: the Replit agent's own account of events was wrong, and it had produced false test reports. Alert on destructive API calls by agent or automation identities (volume or database delete, DROP and TRUNCATE statements, bulk encryption of tables), new administrator accounts, use of default credentials, scheduled tasks making outbound calls, and agent identities using a token outside its expected environment.

Metric	Target
Agent identities with standing production write or delete rights	0, or each a recorded, expiring exception
Platform tokens reachable by agents that are account or workspace scoped	0
Production data stores with deletion protection or soft delete on both console and API paths	100%
Critical data stores with a tested restore from a backup outside production credentials' reach	100%, tested at least quarterly
Internet-exposed AI workflow platforms unpatched against KEV-listed flaws	0
Time to revoke an agent's credentials in a kill-switch drill	Minutes, measured each quarter

Framework mapping

Framework	Item	Relevance
MITRE ATLAS	AML.T0101 Data Destruction via AI Agent Tool Invocation	Replit and PocketOS deletions made through agent tools
MITRE ATT&CK v20	T1485 Data Destruction; T1486 Data Encrypted for Impact; T1078.001 Default Accounts; T1552.001 Credentials In Files	JADEPUFFER's impact and credential techniques; PocketOS's token in a file
OWASP Top 10 for LLM Applications 2025	LLM06:2025 Excessive Agency	Excessive permissions and autonomy behind both accidental deletions
NIST SP 800-53 Rev. 5	AC-6 Least Privilege; AC-5 Separation of Duties; CM-3 Configuration Change Control; CP-9 System Backup; CP-10 System Recovery and Reconstitution	Scope, approval, change control and recoverable backups

Common failure patterns

Freezes declared in a prompt: a code freeze that exists only as an instruction to the agent, as at Replit, is advice, not a control.
Convenience tokens: a broad token created once for a small task and left in the repository, as at PocketOS.
Console and API behave differently: the dashboard had a 48-hour undo; the legacy API did not.
Backups tied to the resource: volume-level backups deleted with the volume.
Secrets in AI server environments: JADEPUFFER harvested provider and cloud keys from the Langflow host after one exploit.
Trusting the agent's report: the Replit agent said rollback was impossible and had produced fabricated data and false test reports.

Practitioner checklist

Remove standing production write and delete rights from every agent identity; use time-bound, approved elevation.
Give each environment its own credentials so staging work cannot reach production.
Scan repositories and agent workspaces for tokens; revoke and replace with short-lived, project-scoped credentials.
Turn on deletion protection and soft delete, and confirm the API path matches the console.
Require platform-enforced approval for drop, delete, destroy, role and credential changes.
Keep one immutable backup under a separate identity and test restoring it each quarter.
Patch and isolate AI workflow platforms; remove default credentials and signing keys from internal services.
Log agent actions at the platform and alert on destructive calls by automation identities.
Drill the agent kill switch and record time to revoke.

AI agents that destroyed data: lessons and controls from Replit, PocketOS and JADEPUFFER