Year: 2026
Type: conference-paper
Abstract: Ancient and endangered languages pose a unique challenge for NLP: their datasets are inherently scarce, difficult to expand, and built from formulaic corpora-making data-quality issues especially consequential yet rarely audited.Motivated by the need to understand what current NMT can realistically achieve for such languages, we investigate hieroglyphicto-German translation, where a recent study reported 61.5 BLEU using fine-tuned M2M-100.Our reproduction yields only 37.0 BLEU with the released model.Investigating this gap, we find 32% of test targets appear identically in training (16/50; 50% under 8gram overlap at 70% threshold).This contamination inflates scores dramatically: contaminated samples achieve up to 83.8 BLEU / 0.924 COMET-22 versus 30.9-39.2BLEU / 0.622-0.676COMET-22 on clean samples across five model configurations spanning two architectures.Document-level decontamination reduces contaminated BLEU by only 4.6 points because 8/16 targets persist via other source documents-target-level deduplication is required.We release a decontaminated 34sample test set and establish corrected baselines (30.9-39.2BLEU), providing a realistic assessment of NMT capability for this endangered writing system.
Language: English
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: hybrid
OpenAlex ID: W7166799010