Shortcomings of deep learning for distributional predictors: a note
Year: 2025
Type: article
Abstract: A number of domains in biomedical research use data with a large number of predictors all representing the same type of measurement. Often, an important summary is the within-person distribution of these predictors. Here we focus on settings where the mean relationship between outcome and predictors is fully captured by this distribution and, more generally, on problems where the goal is to learn a mapping that is invariant under permutations of the input vector. We compare unstructured neural networks, which do not explicitly incorporate the permutation invariance property, versus networks that we call ordered predictors neural networks. We show in simulations that the unstructured deep learning approach can yield higher prediction errors, compared to the approach that explicitly leverages the invariance to simplify the learning task. Additionally, in the context of neural Bayes estimation, in which neural networks are used to construct point estimators, we show that ordered predictors neural networks can yield substantially more precise estimators. We therefore recommend that, when permutation invariance is known or suspected to hold, investigators use a learning or statistical modeling approach that can leverage the invariance, rather than an unstructured deep learning approach.
Source: Biostatistics
Authors: Bonnie B Smith, Abhirup Datta, Brian Caffo
Institution: Johns Hopkins University
Language: English
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
Funder: National Institutes of Health
Award: G9000268679
OpenAlex ID: W7124867442