Hindi Visual Genome: A Dataset for Multimodal English-to-Hindi Machine Translation
TL
TL;DR
The dataset is the first for multimodal English-Hindi machine translation, freely available for non-commercial research purposes and allows to create Hindi image labelers or other practical tools.
Abstract
Visual Genome is a dataset connecting structured image information with English language. We present ``Hindi Visual Genome'', a multimodal dataset consisting of text and images suitable for English-Hindi multimodal machine translation task and multimodal research. We have selected short English segments (captions) from Visual Genome along with associated images and automatically translated them to Hindi with manual post-editing which took the associated images into account. We prepared a set of 31525 segments, accompanied by a challenge test set of 1400 segments. This challenge test set was created by searching for (particularly) ambiguous English words based on the embedding similarity and manually selecting those where the image helps to resolve the ambiguity. Our dataset is the first for multimodal English-Hindi machine translation, freely available for non-commercial research purposes. Our Hindi version of Visual Genome also allows to create Hindi image labelers or other practical tools. Hindi Visual Genome also serves in Workshop on Asian Translation (WAT) 2019 Multi-Modal Translation Task.
References26 items
1
Findings of the Third Shared Task on Multimodal Machine Translation
2
A Visual Attention Grounding Neural Model for Multimodal Machine Translation
3
Multimodal Neural Machine Translation for Low-resource Language Pairs using Synthetic Data
4
Tensor2Tensor for Neural Machine Translation
5