When Fixing Imbalance Creates New Bias: Category-Sentiment Confounding from Non-Uniform Class Rebalancing in Multi-Category Review Datasets
Year: 2026
Type: preprint
Abstract: Class rebalancing is a standard step in preparing sentiment datasets for training, typically applied to correct an overall skew toward one class. We show that when the additional minority-class examples needed to hit a rebalancing target are drawn disproportionately from a limited subset of categories, rather than proportionally across all categories, the rebalancing procedure itself introduces a spurious association between product category and sentiment label - even when the overall class balance is identical to a properly rebalanced version. We isolate this effect using Cramer's V under matched overall balance, demonstrate its behavioral consequence with a controlled classifier evaluation on a category-decorrelated test set (with unique-source and duplication-rate budgets matched across conditions), and trace its magnitude across a donor-category-count spectrum (k=1,2,4,7). Across 10 random seeds, concentrated rebalancing yields a category-sentiment association of V=0.57 versus V=0.10 for uniform rebalancing, and a macro-F1 gap of 0.082; the effect replicates with an alternate donor category and decays as more categories are used, with classification performance recovering faster than the underlying statistical association disappears. We discuss implications for dataset construction practice.
Author: Kapitsa, Vlad
Language: English
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W7208733785
Cites:
Cited by:
Related to: