Work

Marian: Cost-effective High-Quality Neural Machine Translation in C++

Year: 2018

Type: preprint

Abstract: This paper describes the submissions of the "Marian" team to the WNMT 2018 shared task. We investigate combinations of teacher-student training, low-precision matrix products, auto-tuning and other methods to optimize the Transformer model on GPU and CPU. By further integrating these methods with the new averaging attention networks, a recently introduced faster Transformer variant, we create a number of high-quality, high-performance models on the GPU and CPU, dominating the Pareto frontier for this shared task.

Source: arXiv (Cornell University)

DOI: 10.48550/arxiv.1805.12096

Authors: Junczys-Dowmunt, Marcin, Heafield, Kenneth, Hoang, Hieu, Grundkiewicz, Roman, Aue, Anthony

Institutions: Adam Mickiewicz University in Poznań, University of Edinburgh, Microsoft (United States)

Language: English

Topic: Natural Language Processing Techniques

Subfield: Artificial Intelligence

Field: Computer Science

Domain: Physical Sciences

Open Access status: green

OpenAlex ID: W2949118787

Cites: 5

Cited by: 1

Related to: 20