EKD4Rec: Ensemble Knowledge Distillation from LLM-based Models to Traditional Sequential Recommenders
Year: 2025
Type: conference-paper
Abstract: In this paper, we propose a new ensemble knowledge distillation method for distilling knowledge from LLM-based recommendation (teacher) models to traditional light-weight sequential recommendation (student) models. In particular, instead of using one single teacher model, the averaged prediction from multiple teachers is employed as the soft targets of knowledge distillation. Further, only the top K soft labels of teachers' output distribution are sampled for distillation to make it more focused on the corresponding high-ranked items. Extensive experiments on three public datasets show the effectiveness of the proposed ensemble knowledge distillation for sequential recommendation (EKD4Rec).
Authors: Yue Wang, Dingyi Zhang, Haoyu Wenren, Yue Wang, Yingming Li
Institution: Zhejiang University
Language: English
Subfield: Information Systems
Field: Computer Science
Domain: Physical Sciences
Open Access status: gold
Award: G2503274796
OpenAlex ID: W4410638153