Get started
Work

Dimension-Calibrated Unexplained Mass: An Interpretable Drift Statistic for Contamination Monitoring in Data Streams

Year: 2026
Type: preprint
Abstract: Drift detectors that work tend not to explain themselves, and drift detectors that explain themselves tend to fail in high dimension. We close that gap for Gaussian mixture models (GMMs): each fitted component is a named "regime," and the fraction of a stream window matching no regime -- its unexplained mass -- is a drift signal that is simultaneously its own explanation. We identify why this statistic collapses in high dimension and repair it. Under a correct component a normal point in d dimensions lies about sqrt(d) sigma from the mean, so once d exceeds 9 essentially every point exceeds a fixed 3-sigma radius: window-level ROC-AUC is exactly 0.50 on Satellite (d=36) and Optdigits (d=64). Calibrating the radius to sqrt(chi-squared_d(0.99)) removes the collapse -- AUC 1.00 and 0.89 -- while leaving low dimensions unchanged. Across seven public benchmarks, five seeds, and eight model-free detectors spanning the kernel, classifier, projection, density-difference, transport, likelihood and partition families, the repaired statistic is best or tied-best on five of seven datasets at 10% window contamination (its two losses are Pendigits, where the whole field beats it, and Optdigits), and as contamination becomes sparse the sample-level detectors fade toward chance while it degrades most gracefully: at 2% its mean AUC across the benchmarks is 0.86 against at most 0.73 for any model-free detector (1.00 vs. MMD's 0.72 on KDD-http) -- while alone among them reporting which regime the data left and how far outside it the window lies. We delimit its scope honestly: unexplained mass detects and explains novel-regime drift but is blind by construction to in-support re-weighting of known regimes, where distribution-level tests are required and explain nothing; and the underlying density model's EVT-calibrated false-alarm rates degrade above d of about 36. All code and experiments are released.
DOI: 10.48550/arxiv.2607.16811
Open Access status: green
OpenAlex ID: W7169842996
Cites:
Cited by:
Related to: