Year: 2022
Type: article
Abstract: Many papers are chasing state-of-the-art (SOTA) numbers, and more will do so in the future. SOTA-chasing comes with many costs. SOTA-chasing squeezes out more promising opportunities such as coopetition and interdisciplinary collaboration. In addition, there is a risk that too much SOTA-chasing could lead to claims of superhuman performance, unrealistic expectations, and the next AI winter. Two root causes for SOTA-chasing will be discussed: (1) lack of leadership and (2) iffy reviewing processes. SOTA-chasing may be similar to the replication crisis in the scientific literature. The replication crisis is yet another example, like evaluation, of over-confidence in accepted practices and the scientific method, even when such practices lead to absurd consequences.
Source: Natural Language Engineering
Authors: Kenneth Ward Church, Valia Kordoni
Institutions: Baidu (China), Humboldt-Universität zu Berlin
Language: English
Subfield: Library and Information Sciences
Field: Social Sciences
Domain: Social Sciences
Open Access status: hybrid
Estimated APC paid: $3,255
OpenAlex ID: W4210913687