Counterfactual World Models for Software Defect Prediction: Long-Horizon Planning and Decision Quality
Year: 2026
Type: conference-paper
Abstract: Software defect prediction (SDP) identifies modules that are likely to contain defects before maintenance. Conventional machine learning (ML) predictors, however, do not directly recommend changes to software metrics. This study evaluates Random Forest (RF), Extreme Gradient Boosting (XGBoost), and Logistic Regression (LR) on the National Aeronautics and Space Administration (NASA) PROMISE JM1 dataset and uses XGBoost as a probabilistic surrogate model for exploratory counterfactual planning. RF achieved the highest F1-score (0.4335), whereas XGBoost achieved the highest accuracy (0.8478) but low defective-class recall (0.1710). For the ten test modules with the largest XGBoost risk estimates, a five-step greedy planner evaluated bounded reductions in complexity, size, effort, and operator metrics. The mean module-level relative reduction in predicted risk was 40.38%; this quantity measures change in model output, not a demonstrated causal reduction in real defects. The aggregate predicted probability decreased from 0.9126 to 0.5518. The results therefore establish proof-of-concept model behavior rather than validated software-maintenance prescriptions. Practical action mappings, plausibility constraints, repeated-seed analysis, cross-project validation, and runtime benchmarking are identified as necessary validation stages before deployment. The implementation code is publicly available at the following GitHub repository: https://github.com/mldlprojects/Rakesh01
Institutions: Goldman Sachs (United States), CVS Health (United States)
Language: English
Subfield: Information Systems
Field: Computer Science
Domain: Physical Sciences
Open Access status: gold
OpenAlex ID: W7224917422