Applications of the ErrVarNorm index in case 1 best-worst scaling and data quality insights
Year: 2025
Type: article
Abstract: The present research extended Llobell et al. (2025) [Fd. Qual. Pref., 10533] who introduced normalised error variance (ErrVarNorm; EVN) as an index of participant response consistency in Case 1 Best-Worst Scaling (BWS). As EVN increases through its 0 to 1 range, response consistency and, in turn, the BWS data quality increase. This easy-to-calculate index was applied to online data collected from 4587 adults from the United Kingdom. The key findings included: 1) Researchers can establish a tolerance for post hoc exclusions (%) and calculate a corresponding EVN cut-off value; 2) Men aged 18–19 years old were most at risk of exclusion based on lower EVN values; 3) Excluding participants with lower EVN value changed BWS object scores and study conclusions; 4) EVN values were not strongly correlated with self-reported BWS task perceptions; and 5) EVN can be used as a general indicator of individual-level survey data quality. EVN = 0.5 was used as the value for participant exclusions, but the results showed that this cut-off may need to be study-specific. Beyond further characterising the EVN index, the present research contributes to the discourse on data quality in online consumer research and advocates for the perspective that data quality exists on a continuum and that researchers are the ones who are primarily responsible for defining needed data quality levels in a focal study and then transparently disclosing steps taken to achieve this.
Source: Food Quality and Preference
Authors: S.R. Jaeger, S.L. Chheang, F. Llobell
Institutions: Aarhus University, Plant & Food Research
Language: English
Subfield: Sociology and Political Science
Field: Social Sciences
Domain: Social Sciences
Open Access status: hybrid
Estimated APC paid: $4,420
Award: G1426966229
OpenAlex ID: W4414974408