Work

Relevance-aware Individual Item Fairness Measures for Recommender Systems: Limitations and Usage Guidelines

Year: 2025

Type: article

Abstract: Recommender Systems (RSs) aim at providing relevant items to users, with a recent emphasis on improving recommendation fairness. Quantifying fairness of the recommended items can be done with two types of evaluation measures: measures that are purely based on item exposure ( exposure-based ) and measures that account for both item exposure and item relevance ( relevance-aware ). While exposure-based measures have been thoroughly analysed, relevance-aware measures have not been examined in such detail yet. We gather all existing relevance-aware individual item fairness measures for RSs and study their theoretical properties. We find that all measures suffer from one or more limitations, which may cause issues in their computation, interpretability, or expressiveness. To address this, we correct the affected measures or explain why a limitation is unresolvable. Further, we empirically investigate the extent of the limitations on the measures and compare the original measures to our reformulations under common and extreme evaluation scenarios across real-world and synthetic data. Our experiments show that our reformulated measures successfully resolve the issues in the original measures. We conclude by providing practical guidelines on how to select measures for a range of use cases.

Source: ACM Transactions on Recommender Systems

DOI: 10.1145/3765624

Authors: Theresia Veronika Rampisela, Maria Maistro, Tuukka Ruotsalo, Falk Scholer, Christina Lioma

Institutions: University of Copenhagen, Lappeenranta-Lahti University of Technology, RMIT University

Language: English

Topic: Recommender Systems and Techniques

Subfield: Information Systems

Field: Computer Science

Domain: Physical Sciences

Open Access status: hybrid

Funder: Australian Research Council

Award: New approaches to interactive sessional search for complex tasks