Work

Surprisingly Easy Hard-Attention for Sequence to Sequence Learning

Year: 2018

Type: conference-paper

Abstract: In this paper we show that a simple beam approximation of the joint distribution between attention and output is an easy, accurate, and efficient attention mechanism for sequence to sequence learning.The method combines the advantage of sharp focus in hard attention and the implementation ease of soft attention.On five translation and two morphological inflection tasks we show effortless and consistent gains in BLEU compared to existing attention mechanisms.

Source: Conference on Empirical Methods in Natural Language Processing (EMNLP)

DOI: 10.18653/v1/d18-1065

Authors: Shiv Shankar, Siddhant Garg, Sunita Sarawagi

Institution: Indian Institute of Technology Bombay

Language: English

Topic: Natural Language Processing Techniques

Subfield: Artificial Intelligence

Field: Computer Science

Domain: Physical Sciences

Open Access status: hybrid

Funder: Nvidia

OpenAlex ID: W2890909908