Explaining How to Interpret a Usability Survey May Enhance Internal Reliability Without Negatively Influencing Responses
Year: 2025
Type: article
Abstract: A great challenge to employing questionnaires is ensuring that respondents understand them. One possible strategy is to explain the meaning of items but there is a risk in biasing respondents’ answers. In this study, we assessed how explaining the meaning of items to respondents might affect their responses and the validity/reliability of the Positive System Usability Scale (PSUS). We employed straightforward, general explanations that we felt matched the intended meaning of the items. We found a statistically significant difference between the Explanations and No Explanations groups, but we judged this effect as not practically significant. We found the validity and reliability of the questionnaires from both groups to be acceptable. Finally, the reliability of the PSUS questionnaires given to the Explanations group was systematically higher than the No Explanations group, suggesting that correctly explaining items may enhance the reliability of questionnaires, perhaps by enhancing respondents’ understanding of the questionnaire items.
Institution: Rice University
Language: English
Subfield: Human-Computer Interaction
Field: Computer Science
Domain: Physical Sciences
Open Access status: hybrid
Estimated APC paid: $1,500
OpenAlex ID: W4414100392