Work

Assessing artificial intelligence literacy in higher education: A systematic review of measurement tools, current trends and future directions

Year: 2026
Type: article
Study design: Systematic Review
Abstract: Artificial intelligence (AI) literacy is increasingly recognised as a core capability in higher education, but its assessment remains fragmented across generic, generative-AI and discipline-specific instruments. Differences in construct definitions, response formats and intended uses make it difficult to compare students’ and educators’ literacy or to identify instruments that are fit for educational decisions. This systematic review examines available tools, current empirical patterns and future directions for assessment. Following PRISMA-informed procedures, searches of Scopus, Web of Science and ERIC yielded 852 records, of which 679 remained after deduplication. The review synthesises 39 primary studies: 16 tool-development, validation or adaptation studies and 23 studies of status, correlates or educational interventions. Data extraction covered target constructs, assessment formats, samples, reported measurement evidence and principal findings; methodological quality was appraised using an adapted Mixed Methods Appraisal Tool framework. Most instruments assess understanding, application or use, evaluation and ethics; newer generative-AI and discipline-specific measures also address prompting, autonomous learning, pedagogical integration, critical writing and self-regulation. Evidence for factor structure and internal consistency is common, whereas evidence for content validity, test–retest reliability, responsiveness and cross-cultural comparability is less developed. Across the reviewed studies, profiles varied by discipline, study stage and learner group, with recurring weaknesses in technical understanding and socio-critical judgement. Course interventions yielded promising learning signals, but the available evidence was largely derived from small, uncontrolled pre–post studies. Heterogeneity in constructs, samples and outcomes precluded meta-analysis, and most empirical studies provided moderate rather than high-quality evidence. Taken together, these findings indicate that universities should move beyond self-reported confidence and frequency of use. Assessment should combine self-report with performance-based, context-sensitive and equity-aware approaches to support meaningful capability development.
Institution: UCSI University
Language: English
Subfield: Education
Open Access status: diamond
OpenAlex ID: W7223195246
FWCI:
Cites:
Cited by:
Related to: 10