Think Visually: Question Answering through Virtual Imagery
TL
TL;DR
Dynamic Spatial Memory Network (DSMN), a new deep network architecture that specializes in answering questions that admit latent visual representations, and learns to generate and reason over such representations, is introduced.
Abstract
In this paper, we study the problem of geometric reasoning (a form of visual reasoning) in the context of question-answering. We introduce Dynamic Spatial Memory Network (DSMN), a new deep network architecture that specializes in answering questions that admit latent visual representations, and learns to generate and reason over such representations. Further, we propose two synthetic benchmarks, FloorPlanQA and ShapeIntersection, to evaluate the geometric reasoning capability of QA systems. Experimental results validate the effectiveness of our proposed DSMN for visual thinking tasks.
References34 items
1
Attention is All you Need
2
Cognitive Mapping and Planning for Visual Navigation
3
CLEVR: A Diagnostic Dataset for Compositional Language and Elementary Visual Reasoning
4
Tracking the World State with Recurrent Entity Networks
5