AI‑Assisted Incident Response Risks Skill Decay for SREs

TL;DR

AI‑powered incident response tools now resolve most routine alerts automatically, but the reduction in hands‑on practice threatens engineers’ intuition and ability to handle rare, high‑severity failures.

AI SREs Deliver Faster Routine Fixes

  • Modern AI agents can inspect alerts, hypothesize causes, query telemetry, correlate deployments, and apply fixes without human intervention.
  • The author, a former LinkedIn SRE, notes that a self‑healing prototype built in 2012 is now a production reality.
  • Average mean‑time‑to‑resolution (MTTR) for common incidents is expected to drop sharply as AI handles the bulk of the workload.

"These tools do it all: inspect alerts, form hypotheses, query telemetry, correlate recent deployments, and even implement the fix themselves." – Sylvain Kalache

The Ironies of Automation: Skill Erosion

  • Lisanne Bainbridge’s 1983 The Ironies of Automation paper warns that automation reduces routine practice while keeping operators responsible for novel situations.
  • When an AI‑agent encounters a novel, complex incident, engineers may lack the mental models needed for rapid diagnosis.
  • Several commenters echo this concern, describing “ego‑inflation feedback loops” and a loss of “intuitive knowledge” of the systems they built.

"I gave the team an exact solution on a silver platter and they still failed to identify how to go about it after 3 days slamming it into Claude." – @bob1029

"The more I use AI, the more I have to rely on it… I end up feeling empty; no intuitive knowledge of the system I built or fixed." – @krtkush

Aviation as a Cautionary Analogy

  • Pilots rely on automation for routine flight phases but must train constantly on rare emergencies (engine failures, stalls, etc.).
  • Regulatory mandates (e.g., FAA recurrent simulator checks) enforce this training; no such mandates exist for SREs.
  • The analogy highlights that skill decay is a known risk when automation handles the majority of tasks.

Simulations as a Remedy

  • Rootly and Uptime Labs have built LLM‑driven incident simulations where engineers act as incident commanders in realistic e‑commerce outages.
  • Simulations let engineers practice:
    • Interpreting incomplete telemetry
    • Coordinating stakeholders (CEO, support, etc.)
    • Making rapid decisions under pressure
  • The author argues that explanations from AI are not a substitute for hands‑on practice—learning by doing remains essential.

"You might pick up a few things from watching Serena Williams play, but you only learn tennis by getting on the court." – Sylvain Kalache

Community Perspectives on Training and Policy

  • Policy proposals: Some commenters suggest professional licensure for SREs to enforce regular training, akin to pilot certification.

    "If you want SREs to spend time training for disasters, make it a condition of holding a license." – @solatic

  • Skepticism about feasibility: Others note that many companies already neglect basic disaster‑recovery drills, making widespread simulation adoption unlikely.

    "Very few companies spend time practicing restoring backups or rotating secrets… Ops organizations push for flashy work instead." – @solatic

  • Alternative viewpoints: A few participants argue that AI already improves productivity and that skill loss is overstated.

    "LLMs regularly go above and beyond when troubleshooting… this is progress, not doom.” – @spicyusername

Practical Recommendations for Teams

  1. Schedule regular hands‑on drills – tabletop exercises, chaos‑monkey attacks, or full‑stack simulations at least quarterly.
  2. Pair AI agents with human explanations – after an AI‑driven fix, require the model to surface its reasoning and evidence; then have engineers review and discuss.
  3. Maintain a “human‑only” incident quota – deliberately resolve a subset of low‑severity alerts without AI to keep intuition sharp.
  4. Invest in observability literacy – ensure engineers can query logs, metrics, and traces independently of AI suggestions.
  5. Consider industry standards or certifications – analogous to pilot recurrent training, a formal SRE competency framework could mandate periodic skill assessments.

The Long‑Term Outlook

  • If AI capability growth continues, routine incidents will be almost entirely automated, shrinking human MTTR but potentially inflating resolution time for the rare, complex failures.
  • The trade‑off mirrors aviation: safety gains from automation are offset by the need for rigorous human training.
  • Organizations that ignore skill decay risk a future where critical incidents become catastrophically slow to resolve, despite faster average fixes.

This post synthesizes the original blog entry by Sylvain Kalache (Sept 4 2026) and the top‑voted Hacker News comments, highlighting the core argument, supporting evidence, and community reactions.

Sources

Related