Year: 2018
Type: preprint
Abstract: In this entry we review the generalization error for classification and single-stage decision problems. We distinguish three alternative definitions of the generalization error which have, at times, been conflated in the statistics literature and show that these definitions need not be equivalent even asymptotically. Because the generalization error is a non-smooth functional of the underlying generative model, standard asymptotic approximations, e.g., the bootstrap or normal approximations, cannot guarantee correct frequentist operating characteristics without modification. We provide simple data-adaptive procedures that can be used to construct asymptotically valid confidence sets for the generalization error. We conclude the entry with a discussion of extensions and related problems.
Source: arXiv (Cornell University)
Authors: Laber, Eric B., Qian, Min
Institutions: North Carolina State University, Columbia University
Language: English
Subfield: Statistics and Probability
Field: Mathematics
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W2904042674