Work

Prediction is not selection: a pre-registered study of design-time mechanism choice in author name disambiguation

Year: 2026
Type: preprint
Abstract: Can a designer measure a property of their own data before building anything and learn which classification mechanism to build? On seven public author name disambiguation datasets the answer is mostly no, for two separate reasons. The first is structural. A rule assigning on an exact surface match is a strict sub-case of a retriever assigning from the single nearest surface form whenever the similarity function gives its maximum only to identical strings, because a test string present in the history is then its own nearest neighbour. In the comparison most practitioners would run, the rule therefore cannot win at any cutoff, and we measured zero rule wins on 199,713 held-out author mentions in two metric families before we understood why. A wider form of the argument says where a precision-first rule cannot win. The second reason is empirical. On a redesigned pair in which neither mechanism contains the other, an exact-evidence rule against a k-nearest vote, neither pre-registered signal is actionable at the margin we fixed in advance once the seven datasets are pooled: the mechanisms sit half an accuracy point apart, 0.8244 against 0.8294, no band on either axis clears the margin, and both selection hypotheses return Inconclusive. The pooled result is mostly inspire's, which holds four fifths of the blocks, and it is not uniform. Two of the five datasets large enough to judge are decided at that margin in opposite directions: kisti for the rule by 0.1131, through better matching, and zbmath for the vote by 0.0631, through the rule's abstentions. One unregistered signal, block size, does carry a decided pooled verdict, and it is composition and abstention rather than matching quality. The consistency statistic, by contrast, predicts each mechanism's own accuracy at Spearman 0.51 and 0.83. Prediction is not selection.
Language: English
Open Access status: green
OpenAlex ID: W7226031977
Cites:
Cited by:
Related to: 10