Surrogate-Based Autotuning for Randomized Sketching Algorithms in Regression Problems

Year: 2025

Type: article

Abstract: . Algorithms from Randomized Numerical Linear Algebra (RandNLA) are known to be effective in handling high-dimensional computational problems, providing high-quality empirical performance as well as strong probabilistic guarantees. However, their practical application is complicated by the fact that the user needs to set various algorithm-specific tuning parameters which are different from those used in traditional NLA. This paper demonstrates how a surrogate-based autotuning approach can be used to address fundamental problems of parameter selection in RandNLA algorithms. In particular, we provide a detailed investigation of surrogate-based autotuning for sketch-and-precondition (SAP)–based randomized least squares methods, which have been one of the great success stories in modern RandNLA. Empirical results show that our surrogate-based autotuning approach can achieve near-optimal performance with much less tuning cost than a random search (up to about 7.6x fewer trials of different parameter configurations). Moreover, while our experiments focus on least squares, our results demonstrate a general-purpose autotuning pipeline applicable to any kind of RandNLA algorithm. Reproducibility of computational results. This paper has been awarded the “SIAM Reproducibility Badge: code and data available”, as a recognition that the authors have followed reproducibility principles valued by SISC and the scientific computing community. Code and data that allow readers to reproduce the results in this paper are available at https://gptune.lbl.gov/ . [Formula: see text]

Source: SIAM Journal on Matrix Analysis and Applications

DOI: 10.1137/23m1597526

Authors: Younghyun Cho, James Demmel, Michał Dereziński, Haoyun Li, Hengrui Luo, Michael Mahoney, Riley Murray

Institutions: Santa Clara University, University of California, Berkeley, University of Michigan, Georgia Institute of Technology, Lawrence Berkeley National Laboratory, Rice University, International Computer Science Institute, Sandia National Laboratories California

Language: English

Topic: Matrix Theory and Algorithms

Subfield: Computational Theory and Mathematics

Field: Computer Science

Domain: Physical Sciences

Open Access status: green

Funders: National Science Foundation, U.S. Department of Energy, Intel Corporation, Advanced Micro Devices, Qualcomm, Georgia Institute of Technology, Google, National Energy Research Scientific Computing Center, Apple, Office of Science, National Nuclear Security Administration, Natural Sciences and Engineering Research Council of Canada

OpenAlex ID: W4409820333