Year: 2016
Type: report
Abstract: Academic experts frequently recommend policies and treatments.But how well do they anticipate the impact of different treatments?And how do their predictions compare to the predictions of non-experts?We analyze how 208 experts forecast the results of 15 treatments involving monetary and non-monetary motivators in a real-effort task.We compare these forecasts to those made by PhD students and non-experts: undergraduates, MBAs, and an online sample.We document seven main results.First, the average forecast of experts predicts quite well the experimental results.Second, there is a strong wisdom-of-crowds effect: the average forecast outperforms 96 percent of individual forecasts.Third, correlates of expertise---citations, academic rank, field, and contextual experience--do not improve forecasting accuracy.Fourth, experts as a group do better than non-experts, but not if accuracy is defined as rank ordering treatments.Fifth, measures of effort, confidence, and revealed ability are predictive of forecast accuracy to some extent, especially for non-experts.Sixth, using these measures we identify `superforecasters' among the non-experts who outperform the experts out of sample.Seventh, we document that these results on forecasting accuracy surprise the forecasters themselves.We present a simple model that organizes several of these results and we stress the implications for the collection of forecasts of future experimental results.
DOI: 10.3386/w22566
Authors: Stefano DellaVigna, Devin Pope
Language: English
Field: Decision Sciences
Domain: Social Sciences
Open Access status: green
Funder: Alfred P. Sloan Foundation
OpenAlex ID: W2514261086
