Predictive Multiplicity in Leakage-Safe EEG-Based Alzheimer Classification: A Secondary Analysis of Subject-Level Predictions on OpenNeuro ds004504
Year: 2026
Type: preprint
Abstract: This preprint examines predictive multiplicity in leakage-safe EEG-based Alzheimer disease classification using openly released participant-level leave-one-subject-out predictions from OpenNeuro ds004504.A secondary analysis was performed on four EEG pipelines evaluated in 65 participants (36 Alzheimer disease, 29 cognitively normal controls). The analyzed pipelines used relative band power, spectral/aperiodic features, multidomain features, and Riemannian tangent-space representations with logistic regression. Performance and model agreement were independently recomputed from the frozen public prediction files. An absolute balanced-accuracy tolerance of 0.02 defined a descriptive near-best set, and uncertainty in multiplicity statistics was estimated with 10,000 participant-level bootstrap resamples.Subject-level balanced accuracy ranged from 0.785 to 0.854. The two near-best pipelines differed by only 0.7 percentage points in balanced accuracy but assigned different labels to 8 of 65 participants (12.3%; bootstrap 95% CI, 4.6-20.0%). Across all four pipelines, 17 participants (26.2%; 95% CI, 15.4-36.9%) received non-unanimous labels.The results show that leakage-safe models with similar aggregate performance are not necessarily interchangeable at the participant level. Reporting participant-level agreement and predictive multiplicity can therefore complement conventional performance metrics in biomedical machine learning.Preprint, Version 1.0. This manuscript has not undergone peer review.
Author: Spina, Felipe Sabino
Institution: Instituto de Tecnologia e Liderança
Language: English
Subfield: Cognitive Neuroscience
Field: Neuroscience
Domain: Life Sciences
Open Access status: green
OpenAlex ID: W7222669183
Cites:
Cited by:
Related to: