Work

The Threshold, Not the Algorithm: Decomposing the Monetary Value of Model Selection versus Decision-Threshold Calibration in Credit Scoring

Year: 2026
Type: preprint
Abstract: Cost-sensitive re-evaluations of credit-scoring models typically report savings as a percentage of exposure, a form that is scientifically sound but difficult for a risk or finance team to act on directly. This paper converts the same class of evaluation into currency and asks a more operational question: of the total money a lender leaves on the table, how much is recoverable by choosing a better algorithm, and how much by simply setting a better decision threshold. Using a paired, Optuna-tuned, repeated stratified cross-validation re-evaluation of four classifiers (Logistic Regression, Random Forest, XGBoost, LightGBM) on the UCI Default of Credit Card Clients benchmark (6,000-account holdout, NT$1.01 billion in credit exposure, all figures in New Taiwan Dollar, NT$, the currency of the source dataset), current practice - selecting the AUC-best model (LightGBM) and applying the conventional 0.5 decision threshold - carries an expected loss of NT$99.48 million. A cost-aware alternative - selecting the cost-best model (Random Forest) at a nested, cost-optimized threshold - carries an expected loss of NT$12.16 million: a saving of NT$87.32 million (87.8%), or NT$14,554 per account. A two-path, order-independent decomposition of this saving attributes 99.83% of it to threshold calibration alone and just 0.17% to the choice of algorithm - an issuer that only recalibrates its decision threshold, without ever touching its model, captures essentially all of the available saving, a result consistent with our finding that the four models are not statistically distinguishable in cost terms once each is given its own optimal threshold. We illustrate the scale of this effect for a hypothetical 100,000-account portfolio (linear extrapolation, NT$1.46 billion in projected savings) and discuss the implication for how risk and data-science teams should prioritize engineering effort: threshold governance, not model replacement, is where the money is.
Language: English
Subfield: Accounting
Open Access status: green
OpenAlex ID: W7234949350
Cites:
Cited by:
Related to: 10