Year: 2023
Type: preprint
Abstract: We rigorously quantify the improvement in the sample complexity of variational divergence estimations for group-invariant distributions. In the cases of the Wasserstein-1 metric and the Lipschitz-regularized $α$-divergences, the reduction of sample complexity is proportional to the group size if the group is finite. In addition to the published version at ICML 2023, our proof indeed has included the case when the group is infinite such as compact Lie groups, the convergence rate can be further improved and depends on the intrinsic dimension of the fundamental domain characterized by the scaling of its covering number. Our approach is different from that in [Tahmasebi & Jegelka, ICML 2024] and our work also applies to asymmetric divergences, such as the Lipschitz-regularized $α$-divergences. For the maximum mean discrepancy (MMD), the improvement of sample complexity is more nuanced, as it depends on not only the group size but also the choice of kernel. Numerical simulations verify our theories.
Source: arXiv (Cornell University)
Institution: University of Massachusetts Amherst
Language: English
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
Awards: G1523888516, HDR TRIPODS: Institute for Integrated Data Science: A Transdisciplinary Approach to Understanding Fundamental Trade-offs and Theoretical Foundations, G4516736450, CDS&E: Applied Geometry and Harmonic Analysis in Deep Learning Regularization: Theory and Applications, Robust Uncertainty Quantification and Statistical Learning for Heavy Tails and Rare Events, G6518258739, G8616775632, EAGER: CDS&E: Applied geometry and harmonic analysis in deep learning regularization: theory and applications
OpenAlex ID: W4319323840