An Exact-Bayes Account of the Few-Label Protocol Gap in Tabular Foundation Models
Year: 2026
Type: preprint
Abstract: Few-label benchmarks for tabular foundation models often label equally many rows per class. Compared with random labelling, this stratified labelling raises balanced accuracy, lowers accuracy and usually raises macro-F1, a difference we call the protocol gap. An exact Bayesian classifier, which many of these models approximate, has the same gap. A stratified labelled set says nothing about the class priors, which we call class shares, so the classifier favours the common classes only under random labelling. The gap splits exactly into a class-learning part and a class-share part. With weak features, decoding for balanced accuracy or macro-F1 removes almost all of that metric's gain. A network trained on such tasks reproduces the exact gap. On 50 new real tables, 43 of them binary, dividing four models' outputs by the exact Bayesian's class-share estimate removes 52% to 79% of the balanced-accuracy gain of three. Decoding for macro-F1 leaves no model a significantly positive gain, and the protocol alone reverses most pairwise macro-F1 orders. A further 18 multiclass tables, analysed as exploratory, show the same signs. Logistic regression shows the same signature, so the account does not imply that these models are Bayesian. Evaluations should report the labelling protocol and the decision rule. Preprint, version 3 (revised manuscript). Adds the stored results, the code that produces the paper's numbers from them, and redacted copies of the analysis plans (with the SHA-256 digests of the originals) for the analyses added since version 2: decoding of the stored outputs, removal of the feature signal within tables, a fresh multiclass frame, a second training seed and a signal-hidden version of the in-family network, and held-out comparisons. The analysis plans were written and frozen before the runs they govern and were not publicly registered. Files: p5_preprint_v3.pdf (manuscript) and p5_supplement_v3.zip (code, derived data and redacted analysis plans; README.txt inside).
Author: Coutinho, Felipe Provezano
Language: English
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W7225316155