Next Generation Linear Algebra: One Singular Value Decomposition to Catch Them All across Data Size, Precision, and Hardware
Year: 2025
Type: article
Abstract: The Singular Value Decomposition (SVD) is a foundational building block in many applications, including low-rank adaptation (LoRA) for large language models (LLMs). Historically, separate SVD implementations have been designed for each data precision, for each hardware vendor, and for each hardware type (personal computer and HPC). This divergence leads to increased development time, the need to redevelop entire libraries when new architectures or data types emerge, and significant complexity for the end user. In this abstract, we discuss a work in progress to develop an alternative: a unified SVD, enabled by abstraction layers. We demonstrate that state-of-the-art performance across the board can be reached using abstraction frameworks, and investigate the performance engineering process and the characteristics that enable adaptable performance.
Authors: Ringoot, Evelyne, Edelman, Alan
Language: English
Topic: Matrix Theory and Algorithms
Subfield: Computational Theory and Mathematics
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W7104519439
FWCI:
Cites:
Cited by:
Related to: