Work

Learning attention for historical text normalization by learning to pronounce

Year: 2017

Type: conference-paper

Abstract: Automated processing of historical texts often relies on pre-normalization to modern word forms.Training encoder-decoder architectures to solve such problems typically requires a lot of training data, which is not available for the named task.We address this problem by using several novel encoder-decoder architectures, including a multi-task learning (MTL) architecture using a grapheme-to-phoneme dictionary as auxiliary data, pushing the state-of-theart by an absolute 2% increase in performance.We analyze the induced models across 44 different texts from Early New High German.Interestingly, we observe that, as previously conjectured, multi-task learning can learn to focus attention during decoding, in ways remarkably similar to recently proposed attention mechanisms.This, we believe, is an important step toward understanding how MTL works.

Source: Annual Meeting of the Association for Computational Linguistics (ACL)

DOI: 10.18653/v1/p17-1031

Authors: Marcel Bollmann, Joachim Bingel, Anders Søgaard

Institutions: Ruhr University Bochum, University of Copenhagen

Language: English

Topic: Natural Language Processing Techniques

Subfield: Artificial Intelligence

Field: Computer Science

Domain: Physical Sciences

Open Access status: hybrid

Funders: Deutsche Forschungsgemeinschaft, TrygFonden, FP7 Ideas: European Research Council

Award: Parsing low-resource languages and domains