Work

A Dual Optimization View to Empirical Risk Minimization with f-Divergence Regularization

Year: 2025

Type: conference-paper

Abstract: The dual formulation of empirical risk minimization with f-divergence regularization (ERM-fDR) is introduced. The solution of the dual optimization problem to the ERM-fDR is connected to the notion of normalization function introduced as an implicit function. This dual approach leverages the Legendre-Fenchel transform and the implicit function theorem to provide a nonlinear ODE expression to the normalization function. Furthermore, the nonlinear ODE expression and its properties provide a computationally efficient method to calculate the normalization function of the ERM-fDR solution under a mild condition.

Source: IEEE Information Theory Workshop (ITW)

DOI: 10.1109/itw62417.2025.11240444

Authors: Francisco Daunas, Iñaki Esnaola, Samir M. Perlaza

Institutions: NEO: Ingénierie et Opérations des Réseaux, University of Sheffield, Institut national de recherche en sciences et technologies du numérique, Centre Inria d'Université Côte d'Azur

Language: English

Topic: Stochastic Gradient Optimization Techniques

Subfield: Artificial Intelligence

Field: Computer Science

Domain: Physical Sciences

Open Access status: green