Trust-Aware Multi-Axis Reliability Auditing for NLP Classifiers: A Theoretical and Empirical Framework
Year: 2026
Type: conference-paper
Abstract: Accuracy remains the dominant criterion for selecting NLP classifiers, yet it fails to characterize the operational reliability required for deployment under class imbalance, distribution shift, and demographic heterogeneity.We introduce TNMM++ (Trustworthy NLP Multi-axis Monitoring, extended), a theoretically grounded reliability audit that evaluates classifiers along seven complementary axes, utility, calibration, robustness, safety, fairness, selective prediction, and uncertainty quantification, and formalises model selection as constrained multi-objective optimization.Beyond metric aggregation, TNMM++ contributes six algorithmic results: a reliability tensor, a Pareto convex-hull theorem, varianceadaptive weighting, a boundary-predictor criterion with exclusion rule for majority-class collapse, a Bayesian risk decomposition, and the TAMS algorithm with optimality bound.Evaluating eight lightweight Transformer encoders (MiniLM, ELECTRA, mBERT, DistilBERT, MobileBERT, ALBERT, RoBERTa, XLM-R) and excluding models that failed to learn the task, we find that properly trained mid-tier models exhibit balanced utility and calibration superior to collapsed high-accuracy counterparts.TrustScore identifies mBERT (0.6685), ELEC-TRA (0.6630), and DistilBERT (0.6627) as deployment-reliable.Sensitivity analysis under 1,000 Dirichlet weight perturbations confirms ranking stability in 92.4% of configurations, and multi-seed evaluation (5 seeds) demonstrates conclusions are not seed artifacts.Our findings establish that single-metric selection is statistically unreliable and motivate multi-axis reliability auditing as the default protocol for trustworthy NLP deployment.
Author: Noor Islam S. Mohammad
Language: English
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: gold
OpenAlex ID: W7224252831