Exploring Models and Data for Image Question Answering
Published in
Neural Information Processing Systems(2015)External Links:
TL
TL;DR
This work proposes to use neural networks and visual semantic embeddings, without intermediate stages such as object detection and image segmentation, to predict answers to simple questions about images, and presents a question generation algorithm that converts image descriptions into QA form.
Abstract
This work aims to address the problem of image-based question-answering (QA) with new models and datasets. In our work, we propose to use neural networks and visual semantic embeddings, without intermediate stages such as object detection and image segmentation, to predict answers to simple questions about images. Our model performs 1.8 times better than the only published results on an existing image QA dataset. We also present a question generation algorithm that converts image descriptions, which are widely available, into QA form. We used this algorithm to produce an order-of-magnitude larger dataset, with more evenly distributed answers. A suite of baseline results on this new dataset are also presented.
References49 items
1
Learning to Answer Questions from Image Using Convolutional Neural Network
2
Are You Talking to a Machine? Dataset and Methods for Multilingual Image Question
3
Exploring Nearest Neighbor Approaches for Image Captioning
4
Ask Your Neurons: A Neural-Based Approach to Answering Questions about Images
5