Answerer in Questioner's Mind: Information Theoretic Approach to Goal-Oriented Visual Dialog (2018-02-12T00:00:00.000000Z)

TL;DR

Answerer in Questioner's Mind (AQM) is proposed, a novel information theoretic algorithm for goal-oriented dialog that a questioner asks and infers based on an approximated probabilistic model of the answerer.

Abstract

Goal-oriented dialog has been given attention due to its numerous applications in artificial intelligence. Goal-oriented dialogue tasks occur when a questioner asks an action-oriented question and an answerer responds with the intent of letting the questioner know a correct action to take. To ask the adequate question, deep learning and reinforcement learning have been recently applied. However, these approaches struggle to find a competent recurrent neural questioner, owing to the complexity of learning a series of sentences. Motivated by theory of mind, we propose "Answerer in Questioner's Mind" (AQM), a novel information theoretic algorithm for goal-oriented dialog. With AQM, a questioner asks and infers based on an approximated probabilistic model of the answerer. The questioner figures out the answerer’s intention via selecting a plausible question by explicitly calculating the information gain of the candidate intentions and possible answers to each question. We test our framework on two goal-oriented visual dialog tasks: "MNIST Counting Dialog" and "GuessWhat?!". In our experiments, AQM outperforms comparative algorithms by a large margin.

Authors

Sang-Woo Lee

6 papers

Byoung-Tak Zhang

7 papers

Y. Heo

2 papers

TL;DR

Abstract

Authors

References50 items

Emergence of Linguistic Communication from Referential Games with Symbolic and Pixel Input

CoDraw: Collaborative Drawing as a Testbed for Grounded Goal-driven Communication

CoDraw: Visual Dialog for Collaborative Drawing

Unified Pragmatic Models for Generating and Following Instructions

Question Asking as Program Generation

Mastering the game of Go without human knowledge

Visual Reference Resolution using Attention Memory for Visual Dialog

Learning with Opponent-Learning Awareness

Evaluating Visual Conversational Agents via Cooperative Human-AI Games

Learning against sequential opponents in repeated stochastic games

Natural Language Does Not Emerge ‘Naturally’ in Multi-Agent Dialog

Emergent Language in a Multi-Modal, Multi-Step Referential Game

It Takes Two to Tango: Towards Theory of AI's Mind

Colors in Context: A Pragmatic Neural Model for Grounded Language Understanding

Learning Cooperative Visual Dialog Agents with Deep Reinforcement Learning

Emergence of Grounded Compositional Language in Multi-Agent Populations

End-to-end optimization of goal-driven and visually grounded dialogue systems

What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?

Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks

End-to-End Task-Completion Neural Dialogue Systems

A Joint Speaker-Listener-Reinforcer Model for Referring Expressions

YOLO9000: Better, Faster, Stronger

Visual Dialog

GuessWhat?! Visual Object Discovery through Multi-modal Dialogue

Towards End-to-End Learning for Dialog State Tracking and Management using Deep Reinforcement Learning

Learning End-to-End Goal-Oriented Dialog

A Network-based End-to-End Trainable Task-oriented Dialogue System

Reasoning about Pragmatics with Neural Listeners and Speakers

Generation and Comprehension of Unambiguous Object Descriptions

Building End-To-End Dialogue Systems Using Generative Hierarchical Neural Network Models

Hierarchical Neural Network Generative Models for Movie Dialogues

A Neural Conversational Model

VQA: Visual Question Answering

Show and tell: A neural image caption generator

Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation

Microsoft COCO: Common Objects in Context

Information-Theoretic Objective Functions for Lifelong Learning

Partially observable Markov decision processes for spoken dialog systems

Efficient Selectivity and Backup Operators in Monte-Carlo Tree Search

An ISU Dialogue System Exhibiting Reinforcement Learning of Dialogue Policies: Generic Slot-Filling in the TALK In-car System

On Discriminative vs. Generative Classifiers: A comparison of logistic regression and naive Bayes

Approaches to the Evolution of Language

Approaches To The Evolution Of Language: Social And Cognitive Bases

The Structure of Action and Interaction

Does the chimpanzee have a theory of mind?

Multi-agent compositional communication learning from raw visual input

Criteria for Human-Compatible AI in Two-Player Vision-Language Tasks

Learning Database Content for Spoken Dialogue System Design

Information Theory, Inference, and Learning Algorithms

When the answerer's answer distribution p(a t |c, q t , a 1:t−1 , q 1:t−1 ) is fixed, the questioner achieves an ideal performance when the likelihoodp is the same as p(a t |c

Field of Study

Venue Information

Name

Type

URL

Alternate Names