Work

How good is PAC-Bayes at explaining generalisation?

Year: 2025

Type: preprint

Abstract: We discuss necessary conditions for a PAC-Bayes bound to provide a meaningful generalisation guarantee. Our analysis reveals that the optimal generalisation guarantee depends solely on the distribution of the risk induced by the prior distribution. In particular, achieving a target generalisation level is only achievable if the prior places sufficient mass on high-performing predictors. We relate these requirements to the prevalent practice of using data-dependent priors in deep learning PAC-Bayes applications, and discuss the implications for the claim that PAC-Bayes ``explains'' generalisation.

Source: arXiv (Cornell University)

DOI: 10.48550/arxiv.2503.08231

Authors: Picard-Weibel, Antoine, Clerico, Eugenio, Moscoviz, Roman, Guedj, Benjamin

Institutions: Suez (France), Centre Inria de l'Université de Lille, MODAL: MOdel for Data Analysis and Learning, Universitat Pompeu Fabra, University College London

Language: English

Topic: Machine Learning and Algorithms

Subfield: Artificial Intelligence

Field: Computer Science

Domain: Physical Sciences

Open Access status: green

Funder: Agence Nationale de la Recherche

Award: Sharp Theoretical and Algorithmic Principles for frugal ML

OpenAlex ID: W4414577676

Cites:

Cited by:

Related to: