Wellbeing web survey
Year: 2022
Type: study-registration
Abstract: As in previous Registrations, the foremost intended output from the Baseline survey (and related surveys) is a paper which is tentatively titled, “Adjusting for Scale-Use Heterogeneity in Self-Reported Well-Being.” The Baseline survey consists of four types of questions: aspect ratings, aspect tradeoffs, calibration questions, and “other” questions. The “other” questions fall in five general categories: demographic questions, questions to learn about respondents’ scale-use tendencies, questions to explore respondents’ interpretation of specific aspects, questions to assess respondent effort and experience, and exploratory questions. We fielded an exploratory “N=500” sample of the Baseline survey on Amazon MTurk in June-August 2020. We have also fielded follow-up surveys (Baseline2 and BottomlessA) on that sample. We have used these data to further refine our methodology for using calibration questions to correct self-reported well-being ratings for interpersonal differences in scale use. We made minor edits to the Baseline survey and fielded a larger “N=4000” sample on MTurk in June-July 2022. We implemented our planned quality-control procedure (using open-ended text box questions) and found, unfortunately, that the survey responses were of lower quality than in 2020. Of 3,309 respondents who completed the survey, only 976 (29.5%) passed quality control and qualified for the primary analysis sample and follow-up survey invitations. In the “N=500” sample from 2020, the counterpart figures had been 292 and 497, respectively, for a qualification rate of 58.8%. In light of the lower response quality, we have made two changes to the survey design. Registration 5 describes these changes. 1) We added a brief 20-question “Prescreening” survey before the Baseline, which includes the text box questions, so that we pay less compensation to respondents who don’t pass quality control. 2) We will recruit survey participants on Prolific (a different crowdsourcing labor platform) in addition to Amazon's Mechanical Turk (MTurk). We are hoping that Prolific respondents pass quality control at a higher rate (based on other researchers’ anecdotal experiences and some Prolific features—they require a photo ID for registration and restrict IP addresses from VPNs). Apart from quality issues, Prolific also should help us get to our target sample size, which is N=4000 after quality control. Our next step is to field the Prescreening survey on MTurk and Prolific with a target on N=1000 responses on each.
Source: Open Science Framework
Language: English
Subfield: Social Psychology
Field: Psychology
Domain: Social Sciences
Open Access status: green
OpenAlex ID: W7169566927