Year: 2025
Type: article
Abstract: In online learning, a learner receives data in rounds and, at each round, predicts a label that is then compared to the true label, incurring a loss. The total loss overTrounds, when compared to the loss of the best expert from a class of experts or forecasters, is called the regret. In this paper, we focus on logarithmic loss for logistic-like experts withunbounded d-dimensional weights, a scenario that has been largely unexplored. To address the irregularities introduced by the unbounded weight norm, we introduce aregularizedversion of the average (fixed design) minimax regret by imposing asoft constrainton the weight norm. We demonstrate that the regularized minimax regret is fully characterized by a complexity measure we term the regularized Shtarkov sum. We also show how the behavior of the standard regret can be inferred from the regularized regret. Our main results provide aprecisecharacterization of the regularized Shtarkov sum and, consequently, the regularized regret with unbounded weights up to second-order asymptotics. Notably, unlike thed/2 logTregret growth known for bounded weights, our results imply that the regularized regret grows as (1/2+α/4)dlogTwhen the regularization parameter is of order Θ(T−α) for α ≤ 1/2. We achieve this using tools from analytic combinatorics, including multidimensional Fourier analysis, the saddle point method, and the Mellin transform.
Language: English
Field: Decision Sciences
Domain: Social Sciences
Open Access status: green
Funder: National Science Foundation
OpenAlex ID: W4399318252