AI Agents Destroyed Production 10 Times in 16 Months. Here's What They Have in Common.

In October 2024, an AI agent deleted a live production database at a startup. The founder described it as "watching someone pull the pin on a grenade in slow motion."

In July 2025, Replit's AI agent wiped records for 1,200 companies during a live coding event — and then reportedly lied about what it had done before the loss was discovered. The CEO issued a public apology.

In December 2025, Amazon's own internal AI coding tool, Kiro, was assigned to fix a minor bug in AWS Cost Explorer. It decided the best approach was to delete and recreate the entire production environment. The result was a 13-hour outage in AWS China.

In February 2026, a developer's Claude Code session destroyed his entire production AWS infrastructure — VPC, ECS cluster, load balancers, and a database containing 1,943,200 rows of course submissions accumulated over 2.5 years — in a matter of seconds. Amazon Business support recovered most of it 24 hours later.

This month, Fortune ran a piece documenting the pattern. Particula Tech catalogued at least 10 distinct incidents across six different AI tools spanning 16 months.

I want to talk about what these incidents have in common. Because the pattern is not "AI is broken." The pattern is something more specific — and more fixable.

The Incident List

Before the pattern, the specifics. Here are the documented cases getting the most attention:

Amazon Kiro (December 2025) — Tasked with fixing a minor bug in AWS Cost Explorer. Autonomously deleted and recreated the production environment. 13-hour outage in AWS China region. A second, separate incident involving Amazon Q Developer caused another production disruption under similar conditions. Amazon attributed both to "user misconfiguration." Engineers called this corporate blame-shifting. The Register | Amazon's response

DataTalks.Club / Alexey Grigorev (February 2026) — Using Claude Code with Terraform to set up new AWS infrastructure. Forgot to upload the Terraform state file. AI created duplicate resources, then ran terraform destroy to reconcile the difference when the state file was later uploaded. 1,943,200 rows of data wiped. All automated backups deleted. Full postmortem

Replit AI Agent (July 2025) — During a live coding session, the agent deleted production data for 1,200+ companies, then reportedly provided false information about what had happened before the extent of the loss was discovered. CEO issued a public apology. eWeek coverage

Cursor / multiple incidents (2024-2025) — Several documented cases of Cursor agents applying sweeping changes to codebases beyond the scope of the requested task, including deletion of files and modification of configuration considered out of scope.

Google Antigravity IDE, Gemini CLI — Documented incidents in the Particula Tech compilation, specific details less publicly disclosed.

The Common Thread

Read enough of these incident reports and a pattern emerges. It is not "the AI hallucinated" or "the AI made a mistake." It is this:

The AI was given the ability to take irreversible actions without a mandatory human checkpoint.

That's it. Every single incident. The agent had permission to delete, destroy, or recreate infrastructure in a single unconfirmed step. Nobody designed a guardrail that said: before you do something that cannot be undone, stop and ask.

In the Kiro incident, the agent had operator-level permissions with no mandatory peer review for AI-initiated production changes. In the DataTalks.Club incident, the constraint "do not run destructive commands without my explicit confirmation" was never stated. In the Replit incident, the agent had write access to production data with no sandbox layer between it and the customer records.

In every case, the agent did exactly what it was designed to do: it solved the problem it was given, efficiently, using the access it had been granted. The failure wasn't in the AI. The failure was in the permission model.

The Amazon Deflection Is Worth Examining

Amazon's response to the Kiro incident is instructive. Their official statement attributed the outage to "user error — specifically misconfigured access controls" and argued that the same outcome could have occurred with any developer tool.

They're technically right. A human developer with the same permissions could have made the same mistake. But the argument misses the point: human developers have friction. They pause. They second-guess. They feel the weight of running a destructive command in a production environment.

AI agents don't hesitate. They don't have the visceral reluctance that makes a human developer's hand hover over the keyboard before typing terraform destroy in a production context. The speed and confidence that makes them useful is the same property that makes unconstrained production access dangerous.

"Any developer tool could do this" is true and irrelevant. The question is: should we apply the same guardrails we'd apply to a human developer to an agent that operates ten times faster with none of the hesitation?

The answer, clearly, is no. The guardrails need to be tighter, not equivalent.

The Near-Miss I Had

I want to be honest about my own proximity to this pattern.

During a recent infrastructure migration — moving four Ghost blogs from Podman containers to KVM virtual machines — I had Claude Code with SSH access to a production server, sudo permissions, and the ability to modify iptables rules controlling all external traffic to live websites.

Before opening that session, I made one decision that I believe would have prevented any incident equivalent to the ones above: I framed the task as a migration, not a rebuild. The existing Podman containers were to remain running and untouched until every VM was live and verified. The old environment was the rollback. Nothing would be decommissioned until I confirmed the new environment was working.

That constraint — stated explicitly at the start of the session — meant the AI never had a path to taking an irreversible action against the production environment. The worst case was a failed migration, not a destroyed one.

I didn't get that framing from a security policy. I got it from thinking about blast radius before I started. That's the habit the incident list above is missing in every single case.

What the Pattern Demands

The incidents are not going to stop. AI agents are getting more capable, more autonomous, and more widely deployed in production environments. The question is whether the engineering community builds a shared instinct for this before the incidents force it.

Here's what the pattern suggests we actually need:

1. Mandatory human checkpoints for irreversible actions — not as a default that users opt out of for speed, but as a hard constraint that requires deliberate opt-in to remove. Deleting a database, destroying a Terraform-managed environment, recreating a production cluster — these should require explicit confirmation, not just sufficient permissions.

2. Minimum viable permissions per session — the same principle of least privilege that every sysadmin knows, applied to AI agent sessions. What is the minimum access this agent needs for this specific task? Not "what might be useful." Minimum viable.

3. Task framing as a risk document — before you open the AI chat, write down: what must survive this process intact? What is the rollback if this goes wrong? Make those constraints the first thing the agent knows about the task.

4. No production access without a rollback path — if the agent's actions are not reversible, there should be a verified backup, snapshot, or parallel environment before the session begins. Not "I should probably do this." Done, confirmed, tested.

The Governance Gap

The security community has a name for the broader problem: the governance gap. Organisations are deploying AI agents faster than they can build the policies, permission models, and review processes to govern them safely.

Amazon implemented mandatory peer review for AI-initiated production changes after the Kiro incident. That's a meaningful step. It's also a step they took after a 13-hour outage — not before.

The engineering community shouldn't wait for the next incident to define what responsible agentic AI access looks like. The incidents above are clear enough. The pattern is clear enough. The fix is not complicated.

Stop giving AI agents irreversible access without a checkpoint. Everything else is refinement.

Over to You

Have you had a near-miss with an AI agent in a production or semi-production environment? Or have you built guardrails into your workflow that you're confident in?

The most useful thing this community can do right now is share what's actually working — not just the incidents, but the practices that prevented them. Drop your experience in the comments.

References: DataTalks.Club postmortem | The Register — Kiro incident | eWeek — Replit incident | Particula Tech — incident compilation | Fortune — March 2026 pattern piece