Year: 2023
Type: dissertation
Abstract: The replication crisis revealed high levels of bias and questionable research practices (QRPs) in psychology research. Registered Reports (RRs) have been increasingly adopted as a possible solution, but there has been relatively little evidence of whether this novel publishing format appears to be working as intended to reduce bias and QRPs. This project sought to build a detailed database of RRs and closely matched standard reports (SRs), to investigate whether RRs perform better than SRs on indicators of quality, rigour, and transparency. 170 RRs were gathered, representing psychology, health, and related disciplines. Each RR was matched to 2 SRs and their characteristics were coded and compared between the two article types. Some brief descriptive analyses were also undertaken on a larger total sample of RRs (n = 359) that did not have a comparison sample, and a smaller sample of 12 RRs and 12 SRs was examined for signs of HARKing. Six key findings were observed. First, RRs exhibit lower rates of supported hypotheses and higher rates of unsupported hypotheses compared with SRs. Second, rates of open practices are higher among RRs than SRs. Third, RRs also appear to be more strongly associated with some methodological practices indicative of greater rigour and transparency. Fourth, author demographics and article citation rates revealed few differences between the article types. Fifth, higher citation rates were associated with more positive findings and fewer negative findings within both the SRs and RRs, but there was no statistically significant relationship between the journal impact factor and whether hypotheses were supported, for either article type. Finally, HARKing appeared to be non-existent in RRs, while some evidence of HARKing was observed in SRs, however, replication is needed. Overall, the evidence presented in this work demonstrates that, while there are still areas for improvement, RRs do appear to be working as intended in being associated with improved research practices, although further research is needed to determine causal impacts.
Author: O'Mahony, Aoife
Institution: Cardiff University
Language: English
Subfield: Experimental and Cognitive Psychology
Field: Psychology
Domain: Social Sciences
Open Access status: green
OpenAlex ID: W7055450649