Work

A Three-Tier LLM Framework for Forecasting Student Engagement from Qualitative Longitudinal Data

Year: 2025

Type: conference-paper

Abstract: Forecasting nuanced shifts in student engagement from longitudinal experiential (LE) data-multi-modal, qualitative trajectories of academic experiences over time-remains challenging due to high dimensionality and missingness.We propose a natural language processing (NLP)-driven framework using large language models (LLMs) to forecast binary engagement levels across four dimensions: Lecture Engagement Disposition, Academic Self-Efficacy, Performance Self-Evaluation, and Academic Identity and Value Perception.Evaluated on 960 trajectories from 96 first-year STEM students, our three-tier approach-LLM-informed imputation to generate textual descriptors for missing-not-at-random (MNAR) patterns, zeroshot feature selection via ensemble voting, and fine-tuned LLMs-processes textual noncognitive responses.LLMs substantially outperform numeric baselines (e.g., Random Forest, LSTM) by capturing contextual nuances in student responses.Encoder-only LLMs surpass decoder-only variants, highlighting architectural strengths for sparse, qualitative LE data.Our framework advances NLP solutions for modeling student engagement from complex LE data, excelling where traditional methods struggle.

Source: Conference on Computational Natural Language Learning

DOI: 10.18653/v1/2025.conll-1.22

Authors: Ahatsham Hayat, Helen Martinez, Bilal Khan, Mohammad Rashedul Hasan

Institutions: University of Nebraska–Lincoln, Lehigh University

Language: English

Topic: Online Learning and Analytics

Subfield: Computer Science Applications

Field: Computer Science

Domain: Physical Sciences

SDG: Quality education

Open Access status: hybrid