A Corpus of Natural Language for Visual Reasoning
TL
TL;DR
A method of crowdsourcing linguistically-diverse data, and an analysis of the data demonstrates a broad set of linguistic phenomena, requiring visual and set-theoretic reasoning.
Abstract
We present a new visual reasoning language dataset, containing 92,244 pairs of examples of natural statements grounded in synthetic images with 3,962 unique sentences. We describe a method of crowdsourcing linguistically-diverse data, and present an analysis of our data. The data demonstrates a broad set of linguistic phenomena, requiring visual and set-theoretic reasoning. We experiment with various models, and show the data presents a strong challenge for future research.
References29 items
1
CLEVR: A Diagnostic Dataset for Compositional Language and Elementary Visual Reasoning
2
Areas of Attention for Image Captioning
3
Bootstrap, Review, Decode: Using Out-of-Domain Textual Data to Improve Image Captioning
4
Picture it in your mind: generating high level visual representations from textual descriptions
5