Year: 2021
Type: preprint
Abstract: The standard experimental paradigm in the social, behavioral, and economic sciences is extremely limited. Although recent advances in digital technologies and crowdsourcing services allow individual experiments to be deployed and run faster than in traditional physical labs, a majority of experiments still focus on one-off results that do not generalize easily to real-world contexts or even to other variations of the same experiment. As a result, there exist few universally acknowledged findings, and even those are occasionally overturned by new data. We argue that to achieve replicable, generalizable, scalable and ultimately useful social and behavioral science, a fundamental rethinking of the model of virtual-laboratory style experiments is required. Not only is it possible to design and run experiments that are radically different in scale and scope than was possible in an era of physical labs; this ability allows us to ask fundamentally different types of questions than have been asked historically of lab studies. We posit, however, that taking full advantage of this new and exciting potential will require four major changes to the infrastructure, methodology, and culture of experimental science: (1) significant investments in software design and participant recruitment, (2) innovations in experimental design and analysis of experimental data, (3) adoption of new models of collaboration, and (4) a new understanding of the nature and role of theory in experimental social and behavioral science. We conclude that the path we outline, although ambitious, is well within the power of current technology and has the potential to facilitate a new class of scientific advances in social, behavioral and economic studies.This paper emerged from discussions at a workshop held by the Computational Social Science Lab at the University of Pennsylvania in January 2020. The work was supported by James and Jane Manzi Analytics Fund and the Alfred P. Sloan Foundation.
Authors: Abdullah Almaatouq, Joshua Aaron Becker, Michael Bernstein, Robert Botto, Eric Bradlow, Ekaterina Damer, Angela Lee Duckworth, Tom Griffiths, Joshua K. Hartshorne, Edith Law, David Lazer, Min Liu, J. Nathan Matias, David Gertler Rand, Matthew Salganik, Emma Satlof-Bedrick, Maurice Schweitzer, Hirokazu Shirado, Jordan W Suchow, Siddharth Suri, Milena Tsvetkova, Duncan J Watts, Mark E Whiting, Ming Yin
Institutions: Massachusetts Institute of Technology, Princeton University
Language: English
Subfield: Safety Research
Field: Social Sciences
Domain: Social Sciences
Open Access status: green
Funder: Alfred P. Sloan Foundation
OpenAlex ID: W4411209086