Low-Rank Approximation via Sparse Frequent Directions
Year: 2017
Type: preprint
Abstract: In this paper, we establish a new randomized algorithm named Sparse Frequent Directions (SpFD) for low-rank approximation problems. It combines techniques of frequent directions (FD) and sparse subspace embedding (SpEmb). Given a matrix $A\in\mathbb{R}^{n\times d}$ ($n\geq d$) and let $A_k$ denote the best rank-$k$ approximation of $A$. Our algorithm is able to find a sketch $B\in\mathbb{R}^{\ell\times d}$ with $\ell = O(k/(\varepsilon\delta))$, and a matrix $V\in\mathbb{R}^{d\times \ell}$ formed by the right singular vectors of $B$, such that with probability at least $1-\delta$, \[ \norm{A- \Pi_{B, k}^F(A)}_F^2 \left( = \norm{A-[AV]_kV^T}_F^2\right) \leq \norm{A-A_k}_F^2 + \varepsilon\norm{A}_F^2 \] with running time \[O(nnz(A)+dk^3/(\varepsilon^3\delta^2)). \]
In comparison with FD and SpEmb, SpFD is more efficient in terms of time complexity and it is able to reach high accuracy with relatively small sketch size. SpFD can be seen as an integration of FD and SpEmb in favor of FD's accuracy and SpEmb's efficiency which is supported by theoretical analysis and experimental results.
Source: arXiv (Cornell University)
Language: English
Topic: Matrix Theory and Algorithms
Subfield: Computational Theory and Mathematics
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W2618459411