Machine translationese of large language models: Dependency triplets, text classification, and SHAP analysis
Year: 2026
Type: article
Abstract: This study addresses the challenge of distinguishing human translations from those generated by Large Language Models (LLMs) by utilizing dependency triplet features and evaluating 16 machine learning classifiers. Using 10-fold cross-validation, the SVM model achieves the highest mean F1-score of 93%, while all other classifiers consistently differentiate between human and machine translations. SHAP analysis helps identify key dependency features that distinguish human and machine translations, improving our understanding of how LLMs produce translationese. The findings provide practical insights for enhancing translation quality assessment and refining translation models across various languages and text genres, contributing to the advancement of natural language processing techniques. The dataset and implementation code of our study are available at: https://github.com/KiemaG5/LLM-translationese.
Source: PLoS ONE
Authors: Shukang Zhang, Chaoyong Zhao
Institution: East China Normal University
Language: English
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: gold
Estimated APC paid: $2,382
OpenAlex ID: W7119868656