Work

Stability-based Generalization Analysis of Randomized Coordinate Descent for Pairwise Learning

Year: 2025

Type: conference-paper

Abstract: Pairwise learning includes various machine learning tasks, with ranking and metric learning serving as the primary representatives. While randomized coordinate descent (RCD) is popular in various problems, there is much less theoretical analysis on the generalization behavior of models trained by RCD, especially under the pairwise learning framework. In this paper, we consider the generalization of RCD for pairwise learning. We measure the on-average argument stability for both convex and strongly convex objective functions, based on which we develop generalization bounds in expectation. The early-stopping strategy is adopted to quantify the balance between estimation and optimization. Our analysis further incorporates the low-noise setting into the excess risk bounds to achieve the optimistic bound as O(1/n), where n is the sample size.

Source: Proceedings of the AAAI Conference on Artificial Intelligence

DOI: 10.1609/aaai.v39i20.35457

Authors: Liang Wu, Ruixi Hu, Yunwen Lei

Institutions: Southwestern University of Finance and Economics, University of Hong Kong

Language: English

Topic: Stochastic Gradient Optimization Techniques

Subfield: Artificial Intelligence

Field: Computer Science

Domain: Physical Sciences

Open Access status: diamond

Funders: National Science Foundation, National Natural Science Foundation of China

Awards: G1640885283, G2184711172, G8170088271