Work

A Visual Attention Grounding Neural Model for Multimodal Machine Translation

Year: 2018
Type: conference-paper
Abstract: We introduce a novel multimodal machine translation model that utilizes parallel visual and textual information.Our model jointly optimizes the learning of a shared visuallanguage embedding and a translator.The model leverages a visual attention grounding mechanism that links the visual semantics with the corresponding textual semantics.Our approach achieves competitive state-of-the-art results on the Multi30K and the Ambiguous COCO datasets.We also collected a new multilingual multimodal product description dataset to simulate a real-world international online shopping scenario.On this dataset, our visual attention grounding model outperforms other methods by a large margin.
Language: English
Open Access status: hybrid
OpenAlex ID: W2888070626
FWCI: 2.729
Cites: 40
Cited by: 99
Related to: 10