Healthcare-Centric Confidence Assessment of Machine Learning Models via Average Precision and Feature-Subset Stability
Year: 2026
Type: article
Abstract: The paper presents a confidence-based assessment system of machine learning models which uses the Average Precision (AP) which is the result of the precision-recall curve as a threshold-agnostic ranking metric and indicator of operational reliability. This method is not based on the accuracy of the class-labels of the model like the traditional metrics, but the consistency of the model performance at the entire range of decision thresholds. To act as an experimental test bed, a one-dimensional convolutional neural network (1D CNN) was trained using the Breast Cancer Wisconsin (Diagnostic) dataset. Permutation based feature importance was done after full-feature baseline AP to measure the AP degradation on a feature-by-feature basis and an even higher-order subset, such as pairs, triplets and quadruplets. Empirical analysis revealed worst concave points, worst concavity and mean texture as the main factors of individual AP drops. Within higher-order subsets, worst concavity, worst radius and mean texture always prevailed in the AP-based reliability profile. It is interesting to note that several four-feature combinations produced an AP score of the same size as or greater than the baseline of 30 features, which indicates a high level of informational redundancy. AP functions as a stable threshold-agnostic ranking and operational-performance measure, not as direct probabilistic confidence, as evidenced by its consistently high AP (>0.95) across operating thresholds. The paper finds that high AP-based operational reliability does not necessitate a complete feature set, but instead the consistency of AP-based reliability in small feature sets increases the interpretability, trust and deployability of machine learning systems in the studied medical diagnostic setting and in related safety-critical applications pending external validation.
Language: English
Subfield: Health Information Management
Field: Health Professions
Domain: Health Sciences
Open Access status: diamond
OpenAlex ID: W7166707653