Work

Performance of empirical risk minimization in linear aggregation

Year: 2016

Type: article

Abstract: We study conditions under which, given a dictionary $F=\{f_{1},\ldots,f_{M}\}$ and an i.i.d. sample $(X_{i},Y_{i})_{i=1}^{N}$, the empirical minimizer in $\operatorname{span}(F)$ relative to the squared loss, satisfies that with high probability \[R(\tilde{f}^{\mathrm{ERM}})\leq\inf_{f\in\operatorname{span}(F)}R(f)+r_{N}(M),\] where $R(\cdot)$ is the squared risk and $r_{N}(M)$ is of the order of $M/N$. Among other results, we prove that a uniform small-ball estimate for functions in $\operatorname{span}(F)$ is enough to achieve that goal when the noise is independent of the design.

Source: Bernoulli

DOI: 10.3150/15-bej701

Authors: Guillaume Lecué, Shahar Mendelson

Institutions: Centre National de la Recherche Scientifique, École Polytechnique, Centre de Mathématiques Appliquées de l'École polytechnique, Technion – Israel Institute of Technology

Language: English

Topic: Statistical Methods and Inference

Subfield: Statistics and Probability

Field: Mathematics

Domain: Physical Sciences

Open Access status: bronze

Funders: Australian National University, Israel Science Foundation, Division of Mathematical Sciences

Award: G4454725197