Work

How Do Large Language Models Evaluate Lexical Complexity?

Year: 2025
Type: conference-paper
Abstract: In this work, we explore the prediction of lexical complexity by combining supervised approaches and the use of large language models (LLMs).We first evaluate the impact of different prompting strategies (zero-shot, oneshot, and chain-of-thought) on the quality of the predictions, comparing the results with human annotations from the CompLex 2.0 corpus.Our results indicate that LLMs, and in particular gpt-4o, benefit from explicit instructions to better approximate human judgments, although some discrepancies remain.Moreover, a calibration approach to better align LLMs predictions and human judgements based on few manually annotated data appears as a promising solution to improve the reliability of the annotations in a supervised scenario.
Language: English
Open Access status: hybrid
OpenAlex ID: W4416033751
FWCI:
Cites: 11
Cited by:
Related to: 10