Grounding Answers for Visual Questions Asked by Visually Impaired People
TL
TL;DR
The VizWiz-VQA-Grounding dataset is introduced, the first dataset that visually grounds answers to visual questions asked by people with visual impairments, and it is demonstrated that current SOTA algorithms often fail to identify the correct visual evidence where the answer is located.
Abstract
Visual question answering is the task of answering questions about images. We introduce the VizWiz-VQA-Grounding dataset, the first dataset that visually grounds answers to visual questions asked by people with visual impairments. We analyze our dataset and compare it with five VQA-Grounding datasets to demonstrate what makes it similar and different. We then evaluate the SOTA VQA and VQA-Grounding models and demonstrate that current SOTA algorithms often fail to identify the correct visual evidence where the answer is located. These models regularly struggle when the visual evidence occupies a small fraction of the image, for images that are higher quality, as well as for visual questions that require skills in text recognition. The dataset, evaluation server, and leader-board all can be found at the following link: https://vizwiz.org/tasks-and-datasets/answer-grounding-for-vqa/.
References42 items
1
Privacy Concerns for Visual Assistance Technologies
2
VinVL: Revisiting Visual Representations in Vision-Language Models
3
Found a Reason for me? Weakly-supervised Grounded Visual Question Answering using Capsules
4
A First Look: Towards Explainable TextVQA Models via Visual and Textual Explanations
5