SharVeT: Similarity-aware Parameter Sharing with Vector-based Tuning for Efficient LLM Compression
Year: 2026
Type: conference-paper
Abstract: How can we share parameters within large language models to significantly reduce memory costs while preserving accuracy?While parameter sharing is a promising solution to the memory overhead of large language models, existing methods rely on naive grouping and fail to correct sharing-induced discrepancies.We propose an accurate and efficient parameter sharing framework, SharVeT (Similarity-aware sharing with Vector-based Tuning), which performs similarity-based grouping to ensure accurate sharing, allocates parameters adaptively to preserve diversity within each group, and applies lightweight refinement with knowledge distillation to correct sharing-induced discrepancies.Experiments show that SharVeT outperforms existing sharing methods, achieving up to 32.1% lower perplexity and 21.2% higher few-shot reasoning accuracy.
Institution: Seoul National University
Language: English
Subfield: Computer Vision and Pattern Recognition
Field: Computer Science
Domain: Physical Sciences
Open Access status: hybrid
Funder: Samsung
OpenAlex ID: W7166800395