Evaluating GPT -5 Literary Metaphor Translation from English into Arabic through Recognition, Interpretation, and Rendering
Year: 2026
Type: preprint
Abstract: This study introduces a stage-aware framework for evaluating GPT-5's Arabic translation of literary metaphor by separately assessing metaphor recognition, interpretation, and rendering. Using 202 metaphorical expressions identified through MIP/MIPVU in the first two chapters of Kazuo Ishiguro's The Buried Giant , the study compared GPT-5's recognition and interpretation outputs with its Arabic renderings and evaluated rendering quality against DeepL and Kholoud Amro's published translation. Each dimension was scored on a 0–3 ordinal scale, with high inter-rater reliability (weighted Cohen's κ = .89–.93). GPT-5 demonstrated strong metaphor recognition and interpretation (M = 2.30 and 2.29), but substantially weaker rendering performance (M = 1.37), with 37.6% of renderings scoring zero. DeepL achieved moderately higher rendering scores, while the published translation significantly outperformed both AI systems. Qualitative analysis identified four factors influencing rendering success: lexical equivalence, discourse-level consistency, contextual disambiguation, and rhetorical recreation. The findings reveal a substantial recognition–rendering gap, suggesting that conventional fluency- or comprehension-based evaluations may overestimate AI performance in literary metaphor translation. The study offers a replicable methodology for diagnosing AI limitations in figurative language processing.
Source: Research Square
Authors: Hani Hamad M Albelihi, Arwa Alsaleh
Institution: Qassim University
Language: English
Subfield: Language and Linguistics
Field: Arts and Humanities
Domain: Social Sciences
Open Access status: green
OpenAlex ID: W7225736576