Year: 2026
Type: preprint
Abstract: There exist tests calibrated under a null narrower than the one implied by their test statistic -- the detection-null set. The part of the detection-null set not in the null is the test's blind spot. A framework for assessing detection coherence -- whether a test's blind spot is empty -- is introduced, built on new concepts of calibration statistic, detector, detection functional, detection-null set, and blind spot. It is demonstrated that the Wilcoxon--Mann--Whitney, Kruskal--Wallis, Friedman, and logrank tests are detection incoherent as unrestricted tests. Detection incoherent tests may become coherent under restrictions that make the blind spot empty, though domain restriction is not a reliable route to coherence in practice, since the boundary of the coherence-restoring domain is unknown and unverifiable. The Kolmogorov--Smirnov and Zaremba tests, and the Maximum Mean Discrepancy test with a characteristic kernel, are shown to be universally detection coherent. Due to the blind spot, a detection incoherent test misses discoveries when non-rejecting and yields spurious discoveries when rejecting, in both cases regardless of sample size. Detection incoherent tests should be abandoned.
Source: arXiv (Cornell University)
Author: Grendar, Marian
Subfield: Statistics and Probability
Field: Mathematics
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W7202268296
Cites:
Cited by:
Related to: