Get started
Work

Should We Redesign Forecasting Competitions

Year: 2001
Type: preprint
Abstract: The M3-Competition continues to improve the design of forecasting competitions: It examines more series than any previous competition, improves error analyses. and includes commercial forecasting programs as competitors. To judge where to go from here, I step back to look at the M-Competitions as a whole. I discuss the advantages of the M-Competitions in hopes that they will be retained, describe how to gain additional benefit from future competitions, and finally, describe a low-cost approach to competitions. 1. Favorable design aspects of the M-Competitions The M-Competitions provide a model for conducting scientific research. They employ at least five key aspects: empirical testing, multiple hypotheses, large samples, independent validation, and full disclosure. While these aspects might seem obvious, studies in management science seldom include all of them. 1.1. Empirical testing Empirical testing is necessary to test forecasting methods. Despite the resistance of timeseries researchers (Fildes & Makridakis, 1995), interest in empirical studies has been growing among forecasters. Forecasting journals now publish many empirical comparisons. The M-Competitions have led the way in such comparisons. 1.2. Multiple hypotheses Academic researchers rely heavily upon advocacy (Armstrong, Brodie, & Parsons, 2001b); they develop what they believe to be the beat method (hypothesis), then seek information to support it. It is uncommon in management science for a researcher to examine competing hypotheses. However, nearly 60 % of empirical papers in the Journal of Forecasting and the International Journal of Forecasting tested competing hypotheses (Armstrong, 1989). The M-Competitions have been exemplary in providing open calls, thus allowing those with different approaches to participate.
Language: English
Open Access status: green
OpenAlex ID: W2104065866
Cites: 9
Cited by: 7
Related to: 10