Work

Mixed Precision Iterative Refinement for Low-Rank Matrix and Tensor Approximations

Year: 2025

Type: article

Abstract: . We present a new mixed precision algorithm to compute low-rank matrix and tensor approximations, a fundamental task in numerous applications in scientific computing and data analysis. Our algorithm is reminiscent of the iterative refinement framework for linear systems: we first compute a low-rank approximation in low precision and then refine its accuracy by iteratively updating it. We carry out an error analysis of our algorithm which proves that we can reach a high accuracy while performing most of the operations in low precision. We measure the computational cost of the algorithm, which depends on the numerical rank of the input (matrix or tensor) as well as the speed ratio between low and high precision arithmetic. We identify two situations where our method has a strong potential: when the hardware provides fast low precision matrix multiply–accumulate units, and when the numerical rank of the input is small at low accuracy levels. We confirm experimentally the potential of our algorithm for computing various low-rank matrix and tensor decompositions such as SVD, QR, Tucker, hierarchical Tucker, and tensor-train.

Source: SIAM Journal on Scientific Computing

DOI: 10.1137/23m1576992

Authors: Marc Baboulin, Oguz Kaya, Theo Mary, Matthieu Robeyns

Institutions: Centre National de la Recherche Scientifique, Université Paris-Saclay, Laboratoire Interdisciplinaire des Sciences du Numérique, Sorbonne Université, LIP6

Language: English

Topic: Matrix Theory and Algorithms

Subfield: Computational Theory and Mathematics

Field: Computer Science

Domain: Physical Sciences

Open Access status: green

Funder: Agence Nationale de la Recherche

Award: Scalable and Efficient Linear and Eigenvalue Solvers Using Tensor Decompositions