The Irony of AI SREs: Automating Incidents Erodes the Skills Needed for the Hard Ones

AI incident-response tools—so-called “AI SREs”—can now triage alerts, form hypotheses, query telemetry, tie failures to recent deploys, and even ship fixes on their own. Sylvain Kalache, an AI Labs lead at Rootly and former LinkedIn SRE, argues this competence comes with a hidden cost: routine incidents are exactly how on-call engineers build intuition for how their systems fail. As automation absorbs the easy cases, humans get less practice while remaining on the hook for the rare, ambiguous, high-severity events that automation can’t crack. He predicts average MTTR will fall even as resolution time for genuinely complex incidents climbs, because responders will have lost touch with their systems.

The framing draws directly on Lisanne Bainbridge’s 1983 paper “The Ironies of Automation,” which observed that automating routine work leaves operators responsible for abnormal situations they’re now less equipped to handle—implying they need more training, not less. Kalache leans on aviation as the model: in-flight engine shutdowns are vanishingly rare, yet pilots drill emergencies in simulators and face FAA proficiency checks every six months. He cites TransAsia Flight 235, where a crew misdiagnosed an autofeathered propeller and crashed within two minutes, as the cost of untrained response to a survivable failure.

His prescription is deliberate practice. Software teams risk accumulating “comprehension debt”—a widening gap between how systems actually work and how well responders understand them—and should counter it with realistic incident simulation as a standard part of on-call readiness. He points to Rootly’s work with Uptime Labs, putting engineers in the incident-commander seat during simulated outages with LLM-driven stakeholders in Slack, and notes that while AI can explain its own reasoning, watching isn’t the same as doing. Tabletop exercises and chaos engineering aren’t new, but he contends they matter more now that LLMs are doing so much of the work.