Work

CompactifAI: Extreme Compression of Large Language Models using Quantum-Inspired Tensor Networks

Year: 2025

Type: conference-paper

Abstract: Large Language Models (LLMs) like ChatGPT and LlaMA offer immense opportunities but face challenges due to their vast size, leading to high training/inference costs and energy demands.Traditional compression methods focus on reducing neurons or precision.We introduce CompactifAI, a novel LLM compression using quantum-inspired Tensor Networks to compress the model's correlation space.Testing on LlaMA-2 7B showed a 93% memory and 70% parameter reduction, with minimal accuracy loss (2-3%) and significant training (50%) and inference (25%) speedups.

Source: European Symposium on Artificial Neural Networks, Computational Intelligence and Machine Learning (ESANN)

DOI: 10.14428/esann/2025.es2025-8

Authors: Saeed Jahromi, Uygar Kurt, Sukhi Singh, David Montero, Borja Aizpurua Altuna, Román Orús Lacort

Institutions: Multiverse Computing (Spain), Donostia International Physics Center, Centre for Social Innovation

Language: English

Topic: Advanced Neural Network Applications

Subfield: Computer Vision and Pattern Recognition

Field: Computer Science

Domain: Physical Sciences

Open Access status: bronze

OpenAlex ID: W4409480075

FWCI:

Cites:

Cited by:

Related to: 10