Work

Beyond Semantic Understanding: Preserving Collaborative Frequency Components in LLM-based Recommendation

Year: 2025

Type: preprint

Abstract: Recommender systems in concert with Large Language Models (LLMs) present promising avenues for generating semantically-informed recommendations. However, LLM-based recommenders exhibit a tendency to overemphasize semantic correlations within users' interaction history. When taking pretrained collaborative ID embeddings as input, LLM-based recommenders progressively weaken the inherent collaborative signals as the embeddings propagate through LLM backbones layer by layer, as opposed to traditional Transformer-based sequential models in which collaborative signals are typically preserved or even enhanced for state-of-the-art performance. To address this limitation, we introduce FreLLM4Rec, an approach designed to balance semantic and collaborative information from a spectral perspective. Item embeddings that incorporate both semantic and collaborative information are first purified using a Global Graph Low-Pass Filter (G-LPF) to preliminarily remove irrelevant high-frequency noise. Temporal Frequency Modulation (TFM) then actively preserves collaborative signal layer by layer. Note that the collaborative preservation capability of TFM is theoretically guaranteed by establishing a connection between the optimal but hard-to-implement local graph fourier filters and the suboptimal yet computationally efficient frequency-domain filters. Extensive experiments on four benchmark datasets demonstrate that FreLLM4Rec successfully mitigates collaborative signal attenuation and achieves competitive performance, with improvements of up to 8.00\% in NDCG@10 over the best baseline. Our findings provide insights into how LLMs process collaborative information and offer a principled approach for improving LLM-based recommendation systems.

Source: arXiv (Cornell University)

Authors: Wang, Minhao, He, Yunhang, Xu, Cong, Zhu, Zhangchi, Hao, Shuang, Liu, Ning, Zhang, Wei

Institutions: Shanghai Normal University, Beijing Jiaotong University, Shandong University, Shanghai Innovation Institute, East China Normal University

Language: English

Topic: Recommender Systems and Techniques

Subfield: Information Systems

Field: Computer Science

Domain: Physical Sciences

Open Access status: green

Funder: National Natural Science Foundation of China

Awards: G3927590844, G6020845085

OpenAlex ID: W6979300143

Cites:

Cited by:

Related to: