Work

Generalized adaptive bridge regression: a unified framework for high-dimensional architecture discovery

Year: 2026

Type: article

Abstract: In high-dimensional regression, the choice of regularization penalty typically forces a rigid assumption upon the underlying signal structure, dichotomizing data into strictly sparse (Lasso, $$q=1$$ q = 1 ) or entirely dense (Ridge, $$q=2$$ q = 2 ) regimes. However, real-world data generating mechanisms frequently exist on a continuum between these extremes, requiring flexible geometries to handle varying degrees of sparsity and considerable multicollinearity. In this work, we propose a data-driven framework to learn the optimal regularization norm by elevating the $$L_q$$ L q exponent ( $$q \in (0, 2]$$ q ∈ ( 0 , 2 ] ) from a discrete choice to a strictly continuous, learnable hyper-parameter. To overcome the computational bottleneck of evaluating non-convex and non-smooth penalty landscapes, we develop a universal proximal coordinate descent solver that utilizes a safeguarded jumping threshold operator and a novel empirical Karush-Kuhn-Tucker (KKT) verification strategy. This solver is coupled with a stochastic Tree-structured Parzen Estimator (TPE) utilizing randomized internal validation splits, enabling the rapid discovery of optimal penalty geometries without over-fitting. We evaluate the framework on simulated architectures, demonstrating its dynamic adaptivity to structural sparsity, collinearity, and varying signal-to-noise ratios. Applied to four high-dimensional genomic datasets (scaling up to $$P \approx 50,000$$ P ≈ 50 , 000 features), our generalized adaptive bridge regression (GABR) framework successfully identifies optimal, off-grid grouping architectures ( $$q \approx 1.63$$ q ≈ 1.63 to 1.80), outperforming purely sparse and purely dense alternatives. These results demonstrate that the exact regression geometry can be efficiently learned from the data, enabling a unified approach to high-dimensional inference without the computational restrictions of exhaustive discrete grid searches.

Source: Statistics and Computing

DOI: 10.1007/s11222-026-10938-1

Author: Patrik Waldmann

Institution: University of Oulu

Language: English

Topic: Statistical Methods and Inference

Subfield: Statistics and Probability

Field: Mathematics

Domain: Physical Sciences

Open Access status: hybrid

Estimated APC paid: $2,890

Funder: Academy of Finland