Project Bathtub: A Reliability-Engineering Approach to Failure-Mode Taxonomy for AI Coding Agents
Year: 2026
Type: preprint
Abstract: Autonomous coding agents fail in ways that are numerous, unevenly costly, and unevenly hard to detect, yet most public discussion of agent reliability still centers on a single failure family: hallucinated APIs and factual errors. This report introduces Project Bathtub, an effort to import a decades-old industrial reliability discipline — Failure Mode and Effects Analysis (FMEA) — into the domain of LLM-based coding agents. We present Phase 0 of the project: an 80-item failure-mode taxonomy spanning ten categories, each scored on Severity, Occurrence, and Detectability (1–10). Ranking by RPN surfaces a finding at odds with the prevailing narrative: verification and self-assessment failures and specification/communication failures score as high or higher than any hallucination-related failure mode.
Author: Satpathy, Snehasish
Language: English
Subfield: Information Systems
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W7171301459
Cites:
Cited by:
Related to: