Work

Tighter Generalisation Bounds via Interpolation

Year: 2024

Type: preprint

Abstract: This paper contains a recipe for deriving new PAC-Bayes generalisation bounds based on the $(f, Γ)$-divergence, and, in addition, presents PAC-Bayes generalisation bounds where we interpolate between a series of probability divergences (including but not limited to KL, Wasserstein, and total variation), making the best out of many worlds depending on the posterior distributions properties. We explore the tightness of these bounds and connect them to earlier results from statistical learning, which are specific cases. We also instantiate our bounds as training objectives, yielding non-trivial guarantees and practical performances.

Source: HAL (Le Centre pour la Communication Scientifique Directe)

DOI: 10.48550/arxiv.2402.05101

Authors: Viallard, Paul, Haddouche, Maxime, Şimşekli, Umut, Guedj, Benjamin

Institutions: Centre National de la Recherche Scientifique, Institut national de recherche en sciences et technologies du numérique, Institut de Recherche en Informatique et Systèmes Aléatoires, Centre Inria de Paris, Université de Rennes, Université de Lille, University College London

Language: English

Topic: Machine Learning and Algorithms

Subfield: Artificial Intelligence

Field: Computer Science

Domain: Physical Sciences

Open Access status: green

Funders: Agence Nationale de la Recherche, HORIZON EUROPE European Research Council

Awards: G1190173965, G3010527862, PaRis Artificial Intelligence Research InstitutE, Dynamics-Aware Theory of Deep Learning, Sharp Theoretical and Algorithmic Principles for frugal ML

OpenAlex ID: W4391670451

Cites:

Cited by:

Related to: 10