Work

Ridgeless Regression with Random Features

Year: 2022

Type: conference-paper

Abstract: Recent theoretical studies illustrated that kernel ridgeless regression can guarantee good generalization ability without an explicit regularization. In this paper, we investigate the statistical properties of ridgeless regression with random features and stochastic gradient descent. We explore the effect of factors in the stochastic gradient and random features, respectively. Specifically, random features error exhibits the double-descent curve. Motivated by the theoretical findings, we propose a tunable kernel algorithm that optimizes the spectral density of kernel during training. Our work bridges the interpolation theory and practical algorithm.

Source: International Joint Conference on Artificial Intelligence

DOI: 10.24963/ijcai.2022/445

Authors: Jian Li, Yong Liu, Yingying Zhang

Institutions: Chinese Academy of Sciences, Institute of Information Engineering, Renmin University of China, Yantai University

Language: English

Topic: Stochastic Gradient Optimization Techniques

Subfield: Artificial Intelligence

Field: Computer Science

Domain: Physical Sciences

Open Access status: bronze

Funder: National Natural Science Foundation of China

Awards: G3842386738, G4873187559, G6070446019