Work

QuIP#: Even Better LLM Quantization with Hadamard Incoherence and Lattice Codebooks

Year: 2024
Type: article
Abstract: lattice, which achieves the optimal 8-dimension unit ball packing. Third, QuIP# uses fine-tuning to improve fidelity to the original model. Our experiments show that QuIP# outperforms existing PTQ methods, enables new behaviors in PTQ scaling, and supports fast inference. Our code can be found at https://github.com/Cornell-RelaxML/quip-sharp.
Source: PubMed
Authors: Tseng, Albert, Chee, Jerry, Sun, Qingyao, Kuleshov, Volodymyr, De Sa, Christopher
Language: English
Open Access status: green
OpenAlex ID: W7079896301
FWCI: 1.12
Cites:
Cited by: 8
Related to: