When “Excellent” Reliability Is a Warning Sign: A Diagnostic Pilot Study of Five Translated Instruments among Indonesian Civil-Servant Teachers
Year: 2026
Type: article
Abstract: Pilot studies of translated instruments are usually reported as pass-or-fail checks in which high reliability is read as evidence of quality. This study shows why that practice can mislead. Indonesian versions of five instruments the Multifactor Leadership Questionnaire (MLQ) transformational scales, the Organizational Trust Index (OTI), the Work and Meaning Inventory (WAMI), the Organizational Commitment Questionnaire (OCQ), and the School Culture Survey (SCS) were adapted through back-translation and completed by 70 civil-servant teachers in Padang, West Sumatra. By conventional criteria, all 118 items passed: corrected item–total correlations were.756–.985, alpha was.977–.994, and each instrument yielded one factor explaining 75.4%–84.0% of the variance. Diagnostic indicators, however, revealed four warning signs: mean inter-item correlations of.75–.83, unidimensional solutions for multidimensional instruments, negatively worded items behaving like positively worded items, and item-content flaws, including a transactional item in the MLQ and two mistranslated OCQ items. These signals proved accurate: in a subsequent survey of 523 teachers, the flagged items were among those that had to be removed. To our knowledge, this is the first joint pilot evaluation of these five instruments among Indonesian civil-servant teachers and the first to verify pilot diagnostics against main-survey outcomes. A four-step diagnostic checklist is proposed to help researchers detect such problems before the main study.
Source: F1000Research
Institutions: Universitas Adzkia, National University of Malaysia
Language: English
Field: Decision Sciences
Domain: Social Sciences
Open Access status: gold
Estimated APC paid: $1,350
OpenAlex ID: W7225741660