Recombination or New Methods: What Actually Drives State-of-the-Art Progress in AI Research
Type: article
Abstract: Machine learning treats novelty in the method employed (e.g., new architectures such as the Transformer) as the primary marker of scientific contribution. However, generative AI-based automated research systems such as AutoSOTA routinely achieve state-of-the-art (SOTA) benchmark results by recombining existing methods, without introducing new methods. This raises a question: What drives SOTA progress? Recombination or novel methods? We address this using a longitudinal dataset of 15,525 machine learning papers linked to method records and leaderboard outcomes from a public AI benchmarking platform. We distinguish component novelty (previously unseen methods) from recombinatory novelty (first-time pairings of existing methods). Our results show that component novelty is not significantly associated with SOTA achievement or improvement magnitude, whereas recombinatory novelty is positively associated with both outcomes. This paper extends IS research on knowledge innovation, providing the first large-scale empirical test of the recombination hypothesis within a machine learning context, with theoretical and practical implications.
Source: AIS Electronic Library (AISeL)
Authors: Chen, Dylan Junluo, Liu, Ben
Institution: City University of Hong Kong
Language: English
Subfield: Library and Information Sciences
Field: Social Sciences
Domain: Social Sciences
Open Access status: green
OpenAlex ID: W7222909883
