A Systematic Evaluation of Automatic Post-editing using Large Language Models on Neural Machine Translation Output
Year: 2025
Type: dissertation
Abstract: This master’s thesis evaluated automatic post-editing using large language models on neural machine translation outputs. The evaluation was designed to understand the results provided by different APE settings and in response to the research tasks: 1/ Evaluate and compare different APE settings (NMT+LLM) based on the quality improvements (BLEU) for machine translation; 2/ Evaluate and compare different APE settings’ effort and performance in post-editing, with a novel performance evaluation approach that assesses APE efficiency in achieving Task 1. The APE settings were from a combination of two LLMs and three NMTs. Both automatic metrics and human expert evaluation were adopted. We also proposed dividing the delta BLEU by the TER to evaluate a given APE setting’s efficiency. Although the results were consistent in both test and control groups, this metric still needs a more solid literature review, particularly in understanding the relationship between the APE’s effort and its energy consumption.
Author: Yang, Xiaoyan
Language: English
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W7112809463