Work

Comparing Behavioral Patterns of LLM and Human Tutors: A Population-level Analysis with the CIMA Dataset

Year: 2025

Type: conference-paper

Abstract: Large Language Models (LLMs) offer exciting potential as educational tutors, and much research explores this potential.Unfortunately, there's little research in understanding the baseline behavioral pattern differences that LLM tutors exhibit, in contrast to human tutors.We conduct a preliminary study of these differences with the CIMA dataset and three stateof-the-art LLMs (GPT-4o, Gemini Pro 1.5, and LLaMA 3.1 450B).Our results reveal systematic deviations in these baseline patterns, particulary in the tutoring actions selected, complexity of responses, and even within different LLMs.This research brings forward some early results in understanding how LLMs when deployed as tutors exhibit systematic differences, which has implications for educational technology design and deployment.We note that while LLMs enable more powerful and fluid interaction than previous systems, they simultaneously develop characteristic patterns distinct from human teaching.Understanding these differences can inform better integration of AI in educational settings.

Source: Workshop on Innovative Use of NLP for Building Educational Applications

DOI: 10.18653/v1/2025.bea-1.64

Authors: Aayush Kucheria, Nitin Sawhney, Arto Hellas

Institution: Aalto University

Language: English

Topic: Intelligent Tutoring Systems and Adaptive Learning

Subfield: Artificial Intelligence

Field: Computer Science

Domain: Physical Sciences

SDG: Quality education

Open Access status: hybrid

OpenAlex ID: W4412889245