Work

Class Imbalance and Batch Effects in LLM-Based Screening for Systematic Reviews

Year: 2026

Type: review

Abstract: This study analyses LLMs in imbalanced binary classification, using study screening in systematic reviews as the application domain. An experiment was conducted in five reviews, comparing individual and batch processing, with and without prevalence metadata. The results indicate a limited influence of the prevalence metadata, with no evidence that it improves performance. In contrast, batch processing produced larger behavioral changes that varied according to the prevalence of the class. The aggregate and item-level analyses did not always coincide. Therefore, batch processing should be evaluated not only in terms of cost, but also in relation to its effects on decision-making behavior.

Source: arXiv (Cornell University)

DOI: 10.48550/arxiv.2608.14737

Authors: Hida, Gilberto Sussumu, Ribeiro, Danilo Monteiro, Hida, Clayton Suguio

Institutions: Centro de Estudos e Sistemas Avançados do Recife, Universidade Estadual do Amapá

Language: English

Topic: Imbalanced Data Classification Techniques

Subfield: Artificial Intelligence

Field: Computer Science

Domain: Physical Sciences

Open Access status: green

OpenAlex ID: W7203709730

FWCI: 1.487

Cites: 5

Cited by: 3

Related to: 10