Year: 2026
Type: conference-paper
Abstract: Large Language Models are increasingly used as automated evaluators of model-generated outputs, forming the basis of the LLMs-as-judges paradigm. AlignLLM adopts this perspective and reframes evaluation as a semantic alignment problem inspired by Case-Based Reasoning, measuring the consistency between reconstructed problem and solution representations across multiple judges. In this study, we investigate how answer length influences the stability and reliability of alignment-based evaluation within the AlignLLM framework. Using the SQuAD 1.1 dataset, we analyse alignment behaviour across answers of varying lengths using three alignment metrics: ILRAlign, WILRAlign, and Tuned-WILRAlign. Our findings show that very short answers produce comparatively lower and more unstable alignment scores due to insufficient semantic grounding during the reconstruction process. To address this limitation, we introduce a lightweight answer expansion mechanism that slightly enriches short responses while aiming to preserve their original meaning. Experimental results demonstrate that this approach improves alignment score stability within the AlignLLM framework, yielding an average improvement of 32.16% across evaluation settings and increasing sensitivity to correctness by 5.88%. These findings highlight the importance of response richness in alignment-based evaluation and suggest that modest answer expansion improves the robustness of LLM-based automated assessment.
Institution: Robert Gordon University
Language: English
Topic: Topic Modeling
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W7223159971
