The Treatment of Absence and Null Results in Materials Machine Learning Literature: A Review Study
Year: 2022
Type: article
Study design: Systematic Review
Abstract: This review systematically examines the treatment of absence and null results in the materials machine learning literature spanning 2017-2022, drawing exclusively on a curated set of 30 peer-reviewed publications and foundational works that address publication bias, negative findings, and reproducibility challenges in data-driven materials discovery.Through a targeted search strategy across databases such as Web of Science, Scopus, and arXiv using terms including "null result," "negative result," "publication bias," "file drawer," "failed synthesis," and "reproducibility" combined with materials informatics keywords, the analysis reveals a persistent imbalance: while successful predictions and syntheses dominate published outputs, systematic documentation of failed predictions, unsuccessful syntheses, null correlations, and abandoned model architectures remains exceedingly rare.What is currently reported tends to be limited to negative outcomes that coincidentally reveal mechanistic insights or contradict high-profile hypotheses, whereas what is systematically unreported encompasses the vast majority of unsuccessful hyperparameter searches, negative active learning campaigns, and non-discoveries that yield no novel materials meeting target criteria.The typology of absence and null results developed here identifies six distinct categoriesnegative predictive outcomes, null hypothesis non-rejection, failed synthesis, non-discovery, failed replication, and abandoned architecture-each carrying unique implications for scientific progress.The consequences of this nonreporting include severe overestimation of model performance, widespread redundant experimental effort, a false sense of methodological consensus across the field, and slowed overall discovery rates as potentially informative negative signals remain invisible.Ultimately, this review offers concrete recommendations for authors, journals, and the broader community to shift incentives toward transparent reporting of absence, thereby restoring balance to the materials AI literature and accelerating reliable data-driven discovery.1 2The materials machine learning literature from 2017 to 2022 presents a striking paradox: while the field has celebrated rapid advances in property prediction, synthesis recommendation, and discovery pipelines, the overwhelming majority of published studies report only positive outcomes.Failed predictions, unsuccessful syntheses prompted by AI models, null correlations between descriptors and properties, and outright negative experimental results are almost absent from the record.This systematic omission distorts the perceived reliability of computational methods and creates a feedback loop in which researchers unknowingly repeat the same unproductive approaches.The present review, therefore, surveys the
DOI: 10.68159/c772411799
Authors: Nguyen Thanh Huy, Pham Quang Minh, Le Thi Bich
Subfield: Materials Chemistry
Field: Materials Science
Domain: Physical Sciences
Open Access status: hybrid
OpenAlex ID: W7211870757