Work

prunAdag: an adaptive pruning-aware gradient method

Year: 2025

Type: article

Abstract: A pruning-aware adaptive gradient method is proposed which classifies the variables in two sets before updating them using different strategies. This technique extends the “relevant/irrelevant" approach of Ding et al. (Adv Neural Inf Process Syst 32, 2019) and Zimmer et al. (Mathematical optimization for machine learning: proceedings of the MATH+ thematic Einstein semester 2023, 2025) and allows a posteriori sparsification of the solution of model parameter fitting problems. The new method is proved to be convergent with a global rate of decrease of the averaged gradient’s norm of the form $$\mathcal{O}(\log (k)/\sqrt{k+1})$$ . Numerical experiments on several applications show that it is competitive with existing pruning-aware Frank-Wolfe algorithms, see e.g. Zimmer et al. (Mathematical optimization for machine learning: proceedings of the MATH+ thematic Einstein semester 2023, 2025).

Source: Computational Optimization and Applications

DOI: 10.1007/s10589-025-00723-7

Authors: Margherita Porcelli, Giovanni Seraghiti, Philippe L. Toint

Institutions: Istituto di Scienza e Tecnologie dell'Informazione "Alessandro Faedo", University of Florence, University of Mons, University of Namur, Namur Institute for Complex Systems

Language: English

Topic: Stochastic Gradient Optimization Techniques

Subfield: Artificial Intelligence

Field: Computer Science

Domain: Physical Sciences

Open Access status: hybrid

Estimated APC paid: $3,190

Funders: Istituto Nazionale di Alta Matematica "Francesco Severi", European Commission, Università degli Studi di Firenze, Gruppo Nazionale per il Calcolo Scientifico