How Low Can You Go? The Data-Light SE Challenge
Year: 2026
Type: article
Abstract: Much of current Software Engineering (SE) research assumes that progress requires massive datasets and CPU-intensive optimizers. However, has this been rigorously tested? This work bears evidence that for over 100 SE optimization tasks, with only a few dozen labels, simple optimizers achieve ≈ 90% of the best reported results, matching complex optimizers such as SMAC, TPE, and DEHB while running orders of magnitude faster. We propose the data-light challenge: when will a handful of labels suffice for SE tasks? We contribute (1) a mathematical formalization of labeling costs, (2) a library of lightweight algorithms, and (3) empirical results across 100+ SE tasks showing when lightweight algorithms excel or fail.
Authors: Kishan Kumar Ganguly, Tim Menzies
Institution: North Carolina State University
Language: English
Subfield: Information Systems
Field: Computer Science
Domain: Physical Sciences
Open Access status: hybrid
OpenAlex ID: W4417459245