Aggregate based light incremental sharding for efficient embedding table management for recommender systems
Year: 2025
Type: article
Abstract: This work develops a novel method for efficient dynamic embedding table sharding during recommendation inference. Although existing works have developed various sharding methods to reduce network overhead, they often focus on training-centric approaches and overlook inference-specific challenges such as evolving co-occurrence patterns and latency sensitivity. As such, these methods can be suboptimal for real-time recommendation inference scenarios. In this paper, we propose a new method named ALIS, short for Aggregate based Light Incremental Sharding, to address these challenges. ALIS features two key mechanisms: aggregate based sharding, which enhances sharding stability and quality by integrating historical and current sharding results, and light incremental sharding, which reduces dynamic sharding costs by combining iterative and statistics-based approaches. The evaluation of ALIS demonstrates its superior performance over the state-of-the-art method in reducing network overhead and improving inference efficiency. Both quantitative results and qualitative analysis verify the superiority and rationality of our ALIS method.
Source: Scientific Reports
Authors: Chao Kong, Jiahui Chen, Dan Meng, Haibei Zhu, Yi Zhang
Institutions: Anhui Polytechnic University, Georgia Institute of Technology
Language: English
Subfield: Information Systems
Field: Computer Science
Domain: Physical Sciences
Open Access status: gold
Estimated APC paid: $2,690
Funder: Anhui Polytechnic University
OpenAlex ID: W4413786763