Towards AI-assisted validation of university course profiles: A work-in-progress framework for policy compliance
Year: 2026
Type: article
Abstract: Reviewing university course profiles against institutional policies remains a largely manual process that is time-consuming and repetitive. This paper presents the design of an AI-assisted framework for course profile university policy compliance checking that combines Large Language Model (LLM)-based semantic analysis with deterministic rule-based validation, together with the initial implementation and evaluation of its LLM-based validation component. A prototype using Claude Sonnet 4.5 was developed and evaluated though an exploratory case study on a published course profile, using a baseline scenario and three modified versions with deliberately introduced policy violations. Although the LLM was able to recognise a stated policy exception occasionally, it produced false positives on the compliant baseline course profile, failed to detect a numerical error, and returned inconsistent validation outcomes for the same input. These findings show that LLM-only validation is not sufficiently reliable, motivating a combined approach that integrates deterministic rule-based validation with an LLM-based agentic workflow. This work represents the first stage of a broader study investigating hybrid AI-assisted approaches to course profile policy validation in higher education.
Source: ASCILITE Publications
Authors: Xintong Tan, Aneesha Bakharia, Lizzie Li, Hui Zhou
Institution: The University of Queensland
Language: English
Subfield: Education
Field: Social Sciences
Domain: Social Sciences
SDG: Quality education
Open Access status: diamond
OpenAlex ID: W7234056464
