Year: 2026
Type: conference-paper
Abstract: While LLMs have demonstrated remarkable potential in Question Answering (QA), evaluating personalization remains a critical bottleneck.Existing paradigms predominantly rely on lexical-level similarity or manual heuristics, often lacking sufficient data-driven validation.We address this by mining Community-Individual Preference Divergence (CIPD), where individual choices override consensus, to distill six key personalization factors as evaluative dimensions.Accordingly, we introduce CoPA, a benchmark with 1,985 user profiles for fine-grained, factor-level assessment.By quantifying the alignment between model outputs and user-specific cognitive preferences inferred from interaction patterns, CoPA provides a more comprehensive and discriminative standard for evaluating personalized QA than generic metrics.The code is available at https://github.com/bjzgcai/CoPA.
Authors: Hang Su, Zequn Liu, Chen Hu, Xuesong Lu, Yingce Xia, Liu Zhen
Institution: East China Normal University
Language: English
Subfield: Information Systems
Field: Computer Science
Domain: Physical Sciences
Open Access status: hybrid
Awards: G3750304191, G4466500611
OpenAlex ID: W7166807524