Work

Fast Differentiable SVD on GPU via Polar Decomposition

Year: 2026

Type: preprint

Abstract: We present a fully GPU-oriented SVD pipeline based on polar decomposition, motivated by iterative methods that rely solely on matrix multiplications, such as the Newton-Schulz iteration. We show that this approach enables up to a $2\times$ speedup compared to standard implementations. Furthermore, we derive a numerically stable backward pass for the polar decomposition and leverage it to obtain a fully differentiable SVD. Our methods are released as open-source implementations in both PyTorch and JAX: https://github.com/fallnlove/cans_svd.

Source: arXiv (Cornell University)

DOI: 10.48550/arxiv.2609.32505

Authors: Parkina, Uliana, Tsyganov, Askar, Kudriashov, Sergei, Samsonov, Sergey, Rakhuba, Maxim

Institution: National Research University Higher School of Economics

Language: English

Topic: Matrix Theory and Algorithms

Subfield: Computational Theory and Mathematics

Field: Computer Science

Domain: Physical Sciences

Open Access status: green

OpenAlex ID: W7214804283

Cites: 8

Cited by:

Related to: 10