Year: 2025
Type: article
Abstract: Knowledge of the domain of applicability of a machine learning model is essential to ensuring accurate and reliable model predictions. In this work, we develop a new and general approach of assessing model domain and demonstrate that our approach provides accurate and meaningful domain designation across multiple model types and material property data sets. Our approach assesses the distance between data in feature space using kernel density estimation, where this distance provides an effective tool for domain determination. We show that chemical groups considered unrelated based on chemical knowledge exhibit significant dissimilarities by our measure. We also show that high measures of dissimilarity are associated with poor model performance (i.e., high residual magnitudes) and poor estimates of model uncertainty (i.e., unreliable uncertainty estimation). Automated tools are provided to enable researchers to establish acceptable dissimilarity thresholds to identify whether new predictions of their own machine learning models are in-domain versus out-of-domain.
Source: npj Computational Materials
Institutions: University of Wisconsin–Madison, Carnegie Mellon University
Language: English
Subfield: Materials Chemistry
Field: Materials Science
Domain: Physical Sciences
Open Access status: gold
Estimated APC paid: $3,490
Awards: G2306240780, G3226045097, XSEDE 2.0: Integrating, Enabling and Enhancing National Cyberinfrastructure with Expanding Community Involvement, Collaborative Research: Framework: Machine Learning Materials Innovation Infrastructure, Bridge to the Doctorate: Wisconsin Louis Stokes Alliance for Minority Participation
OpenAlex ID: W4409175124