Get started
Work

Level-specific feedback generation for scene descriptions via fine-tuning multimodal large language models

Year: 2025
Type: article
Abstract: Scene description tasks effectively enhance students' English writing skills in contextual settings, facilitating the establishment of authentic situational connections. However, evaluating descriptive quality and providing accurate, level-appropriate feedback present significant challenges. Although Multimodal Large Language Models (MLLMs) have demonstrated strong capabilities in vision-language tasks, their generated feedback for scene description tasks often remains generic. It fails to account for students' educational stages. To address this limitation, we construct a novel level-specific feedback dataset for scene description tasks. This dataset is constructed using GPT-4o with Retrieval-Augmented Generation (RAG), guided by the Hong Kong primary and secondary school English word lists, which categorize vocabulary into four educational stages (key stages 1 to 4). We fine-tuned a designed MLLM on this dataset and evaluated its performance against open-source and closed-source baselines. Experimental results demonstrate that the proposed fine-tuned MLLM significantly enhances educational stage relevance in feedback generation while reducing hallucinated content. These findings substantiate the efficacy of fine-tuned MLLM in providing level-specific feedback for scene description tasks, advancing the potential for more adaptive AI-assisted writing support in educational contexts. • Highlights (3-5) • Fine-tuning MLLMs to generate level-specific feedbacks for scene descriptions. • Fine-tuned MLLM improves educational relevance and reduces hallucinated content. • Demonstrated significant performance enhancement over baseline models in feedback generation.
DOI: 10.1016/j.caeai.2025.100510
Language: English
Open Access status: gold
Estimated APC paid: $2,500
OpenAlex ID: W4416701740
FWCI:
Cites: 38
Cited by:
Related to: