When Lempel-Ziv-Welch Meets Machine Learning: A Case Study of Accelerating Machine Learning using Coding.
Year: 2017
Type: preprint
Abstract: In this paper we study the use of coding techniques to accelerate machinelearning (ML). Coding techniques, such as prefix codes, have been extensively
studied and used to accelerate low-level data processing primitives such as
scans in a relational database system. However, there is little work on how to
exploit them to accelerate ML algorithms. In fact, applying coding techniques
for faster ML faces a unique challenge: one needs to consider both how the
codes fit into the optimization algorithm used to train a model, and the
interplay between the model structure and the coding scheme. Surprisingly and
intriguingly, our study demonstrates that a slight variant of the classical
Lempel-Ziv-Welch (LZW) coding scheme is a good fit for several popular ML
algorithms, resulting in substantial runtime savings. Comprehensive experiments
on several real-world datasets show that our LZW-based ML algorithms exhibit
speedups of up to 31x compared to a popular and state-of-the-art ML library,
with no changes to ML accuracy, even though the implementations of our LZW
variants are not heavily tuned. Thus, our study reveals a new avenue for
accelerating ML algorithms using coding techniques and we hope this opens up a
new direction for more research.
Source: arXiv (Cornell University)
Language: English
Subfield: Theoretical Computer Science
Field: Mathematics
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W2591908351