Investigating the replicability of the social and behavioural sciences
Year: 2026
Type: preprint
Abstract: We attempted replications of 274 claims of positive results from 164 quantitative papers published from 2009 to 2018 in 54 journals in the social and behavioral sciences. Replications were high-powered on average to detect the original effect size (Median = 99.6%), used original materials when relevant and available, and were peer-reviewed in advance through a standardized internal protocol. Replications showed statistically significant results in the original pattern for 151 of 274 claims (55.1% [95% CI 49.2 - 60.9%]) and for 80.8 of 164 papers (49.3% [95% CI 43.8 - 54.7%]) weighed for replicating multiple claims per paper. We observed modest variation in replication rates across disciplines (42.5% to 63.1%) though some estimates had high uncertainty. For claims with Pearson’s r effect sizes, the median was 0.25 [95% CI 0.21 - 0.27] for original studies and 0.10 [95% CI 0.09 - 0.13] for replication studies, a 82.4% [95% CI 67.8 - 88.2%] reduction in shared variance. Thirteen methods for evaluating replication success provided estimates ranging from 28.6% to 74.8% (median = 49.3%). Some decline in effect size and significance is expected based on power to detect original effects and regression to the mean due to replicating only positive results. The conditions that promote or inhibit replicability are worthy of additional investigation.
Source: MetArXiv (OSF Preprints)
Authors: Chris N. H. Street, Laura Smalarz, Jule Oettinghaus, Sangsuk Yoon, Angela Jones, Johannes Michalak, Kevin M Esterling, Joel S. Snyder, Amanda Sargent, Ilka Helene Gleibs, Jonathan Wilbiks, Domna Tsaklakidou, Anna Szabelska, Kimberly A. Quinn, Rachel P. Ostrowski, Linda Geven, Cameron Brick, Vladimir Ponizovskiy, Daniel C. Molden, Kathleen Schmidt, Marie Isabelle Weißflog, Ernest Baskin, Kathryn Caldwell, Gideon Nave, Sanghyun Hong, Tom Hostler, Michelangelo Landgrave, Ignazio Ziano, Jason Thatcher, Niels van de Ven, Ellie Shackleton, Jamie Patrianakos, Florian Neubauer, C. Cozette Comer, Matthew Ryan Sisco, Susanne Täuber, Marco Ramljak, Sara Capitan, Yinxian Zhang, Michael Andreychik, Eli C. Awtrey, Ewa Szumowska, Elise Turkovich, Karolina Urbanska, Gordon Brown, Jesper N. Wulff, Ruben Van den Goorbergh, Margaret Ritzau, Michaela Ritchie, Jeewon Oh, Miroslav Sirota, Brinna Mawhinney, Shishir Shakya, Colin Smith, Amber Schroeder, Jacob Rothschild, Gabriel Briceño Jiménez, Katie Munt, Natalia Stanulewicz-Buckley, Josh Matacotta, Macie Daley, Mariah Purol, Dr Sofia Persson, Dimitra Kouroupaki, Martina Grunow, Bence Bago, Lara Warmelink, Pavol Kačmár, Balazs Aczel, Nathaniel D. Porter, Ambar G. Monjaras, Ernest O'Boyle, Laura Getz, Tom Coupe, Phuong Linh L. Nguyen, Rahul Chawdhary, Peter Allen, Michalis Michaelides, Elisabeth Julie Vargo, Cristina Greculescu, David Dobolyi, Jordan MacDonald, Nathan C. Higgins, James G. Field, Leigh Ann Vaughn, Daniel Jolles, Miles Bader, Frank Walter, Ignacio Pavez, Peter Mallik, Carolin Nast, Michael Inzlicht, Courtney K. Soderberg, Austin Lee Nichols, Cécile Emery, Aneta Czernatowicz-Kukuczka, Arathy Puthillam, Daniel J. Dunleavy, Eirik André Strømland, Priya Silverstein, Sophie M. Behr, Moin Syed, Soorya Sankaran, Kathryn Nason, Jesse J. Chandler, Finnian Wort, Lars Tummers, Richard Shafranek, Tina Sundelin, Arijit Nandi, Christopher R. Chartier, Joel Le Forestier, Kim Peters, Jared Leslie, Matus Adamkovic, Corinna Martarelli, Danka Purić, Christian Bokhove, George Christopher Banks, Laura Boeschoten, Fatih Sonmez, Anna Lou Abatayo, Constant Pieters, Jamal R. Williams, Diem Vo, Krystal Hahn, Cristóbal Moya, Angelo Christopher Limnios, Noah Haber, Andrew H. Tyner, Timothy M. Errington, Bojana Bodroža, Jedrzej Bialkowski, Flavio Souza Azevedo, Daniel J. Mallinson, Cristina Zogmaister, Stuti Thapa, Anna Fedor, Scott Braithwaite, Giulio Costantini, Miguel Fonseca, Patrick Durkee, Johannes Jarke-Neuert, John Wills Lloyd, Abiola OYEBANJO, Tabaré Capitán, Ahmed Abbasi, Gabriel Baník, Aaron Wichman, Hause Lin, Dermot Lynott, Samuel Field, Simine Vazire, Kamil Izydorczak, Phillip Jolly, Anna van 't Veer, Melissa Kline Struhl, Donna Pamella Gonzales, L. Robert Slevc, Matthew Jensen, Thomas Rhys Evans, Kelly Wolfe, Nathan Fiala, Nicolas Sommet, Annette Brown, Robbie C. M. van Aert, Erick Axxe, Sungmok Lee, Jan Philipp Röer, Gail McMillan, James Bailey, Brian A. Nosek, Samuel Shaki, Mitchell Metzger, Alexandra L.J. Freeman, Nax, Gustav Nilsonne, Sandra J. Geiger, Sam Harper, Marco Perugini, Andjela Keljanovic, Belén Fernández-Castilla, Jeremy K. Miller, Jentien Vermeulen, Timothy Brady, Esteban Méndez-Chacón, Miklos Bognar, Justine Saal, Michael J. Larson, XINDONG xue, Mary Sanchez, Biljana Jokic, Siobhan Griffin, Iris Zezelj, Marina Milyavskaya, Romina Rodela, Victoria E. Colvin, Nicholas Fox [SCORE], Deliah Wagner, Ivan Ropovik, Tessa Charles, Nicholas Gunby, Nathan Fong, Nihan Albayrak, Daniele Marinazzo, Louis Tay, Elliot Ludvig, Kent Jason Go Cheng, Eric-Jan Wagenmakers, Hansika Kapoor, Katelin E. Leahy, Joshua de Leeuw, Andrew McBride, Sean Patrick Roche, Paul H.P. Hanel, Robin Bouwman, Savannah C. Lewis, Bobbie Spellman, Kristin Jankowsky, Sebastian Schuetz, Lyonel Laulié, Janaki Gooty, William J. Chopik, Ryan Wright, Marjan Bakker, Audrey Morrow, Adoril Oshana, Lukasz Walasek, Eric L. Sevigny, Liuqing Wei, Olivia Miske, Bastian Jaeger, Jason Samaha, Diane B. V. Bonfiglio, Samjhana Koirala, Laetitia Mulder, An-Chiao Liu, Ulrich Kühnen, Rolf Zwaan, George Stock, Hani Safadi, Melba Tutor, Traci Bricka, Veronica Yan, Cillian McHugh, W. Robert Reed, Theresa Stankov, Zachariah Berry, Svenja Hippel, Roman Briker, Nicholas Weller, Barnabas Szaszi, Radoslaw Panczak, Lusine Grigoryan, Marta Kołczyńska, Sven Hoeppner, Jeremy Freese, Landon Schnabel, Zorana Zupan, Liad Mudrik, Braeden Hall, Alice Charlotte Emily Lawrence, Anirudh Tagat, Bruce Clark, Jing Geng, Astrid Schütz, Marcel A.L.M. van Assen, Marie Juanchich, Thomas Ostermann, Erin Hannon, Victor Volkman, Leticia Micheli, Jamie Cummins, Nicholas Berente, David Santos, Jane duan, Maksim Sitnikov, Marin Dujmović, Marian Sauter, Michael Schulte-Mecklenbeck, Yuzhi Yang, Yuri Pavlov, James Druckman, Chris Aberson, Amélie Gourdon-Kanhukamwe, Marco Juergen Held, David Moreau, Louis Hickman
Language: English
Subfield: Statistics, Probability and Uncertainty
Field: Decision Sciences
Domain: Social Sciences
Open Access status: green
OpenAlex ID: W7170130759