User Experience and Usability Evaluation Identifies Learnability Barriers in an Industry Linked Vocational Learning Management System
Year: 2026
Type: preprint
Study design: Observational Study
Abstract: Learning management systems used in industry-linked vocational education must do more than deliver content: they must enable clear and equitable participation in pathways leading to industry certification. This study evaluates UpMySkill (UMS), the learning platform used in the Axioo Class Program, by combining the User Experience Questionnaire (UEQ) and System Usability Scale (SUS). A cross-sectional online survey involved 232 users from Indonesian vocational education settings (212 students and 20 teachers). All UEQ means were positive: attractiveness 1.541, perspicuity 1.190, efficiency 1.489, dependability 1.438, stimulation 1.649, and novelty 0.941. Stimulation was benchmarked as good; four scales were above average; and perspicuity was below average. In contrast, the mean SUS score was 59.43 (SD = 16.09; 95% CI 57.36-61.50), with 73.3% of respondents below the reference value of 68. Five UEQ scales showed acceptable-to-good internal consistency (alpha = 0.73-0.86), whereas novelty was weaker (alpha = 0.59); SUS reliability was borderline acceptable (alpha = 0.70). Item-level and sensitivity analyses consistently identified learning effort, conventionality, and initial support as the main frictions. The study advances LMS research by treating UX as quality-assurance evidence in an industry-linked, certification-oriented vocational pathway and by showing that favourable multidimensional experience can coexist with inadequate global usability. This UX-usability gap supports redesign priorities centred on guided onboarding, visible learning and certification pathways, simplified navigation, and task-based usability testing. The findings indicate enabling conditions for quality digital education rather than direct evidence of learning, certification, employability, or SDG attainment. Clinical trial registration Clinical trial number: not applicable.
Source: Research Square
Language: English
Subfield: Human-Computer Interaction
Field: Computer Science
Domain: Physical Sciences
SDG: Quality education
Open Access status: green
Funder: Universitas Pendidikan Indonesia
OpenAlex ID: W7214702972
