Text-based RL Agents with Commonsense Knowledge: New Challenges, Environments and Baselines (2020-10-08T00:00:00.000000Z)

TL;DR

This paper designs a new text-based gaming environment called TextWorld Commonsense (TWC) for training and evaluating RL agents with a specific kind of commonsense knowledge about objects, their attributes, and affordances, and shows that agents which incorporate Commonsense knowledge in TWC perform better, while acting more efficiently.

Abstract

Text-based games have emerged as an important test-bed for Reinforcement Learning (RL) research, requiring RL agents to combine grounded language understanding with sequential decision making. In this paper, we examine the problem of infusing RL agents with commonsense knowledge. Such knowledge would allow agents to efficiently act in the world by pruning out implausible actions, and to perform look-ahead planning to determine how current actions might affect future world states. We design a new text-based gaming environment called TextWorld Commonsense (TWC) for training and evaluating RL agents with a specific kind of commonsense knowledge about objects, their attributes, and affordances. We also introduce several baseline RL agents which track the sequential context and dynamically retrieve the relevant commonsense knowledge from ConceptNet. We show that agents which incorporate commonsense knowledge in TWC perform better, while acting more efficiently. We conduct user-studies to estimate human performance on TWC and show that there is ample room for future improvement.

Authors

Kartik Talamadupula

6 papers

Mrinmaya Sachan

6 papers

K. Murugesan

2 papers

TL;DR

Abstract

Authors

References44 items

Enhancing Text-based Reinforcement Learning Agents with Commonsense Knowledge

Learning Dynamic Knowledge Graphs to Generalize on Text-Based Games

Learning Dynamic Belief Graphs to Generalize on Text-Based Games

Graph Constrained Reinforcement Learning for Natural Language Action Spaces

Infusing Knowledge into the Textual Entailment Task Using Graph Convolutional Networks

LeDeepChef: Deep Reinforcement Learning Agent for Families of Text-Based Games

KagNet: Knowledge-Aware Graph Networks for Commonsense Reasoning

Playing Text-Adventure Games with Graph-Based Deep Reinforcement Learning

TextWorld: A Learning Environment for Text-based Games

Learn What Not to Learn: Action Elimination with Deep Reinforcement Learning

QANet: Combining Local Convolution with Global Self-Attention for Reading Comprehension

Neural Natural Language Inference Models Enhanced with External Knowledge

Graph Attention Networks

ConceptNet 5.5: An Open Multilingual Graph of General Knowledge

Knowing When to Look: Adaptive Attention via a Visual Sentinel for Image Captioning

Bidirectional Attention Flow for Machine Comprehension

Sample Efficient Actor-Critic with Experience Replay

Towards Deep Symbolic Reinforcement Learning

Asynchronous Methods for Deep Reinforcement Learning

Deep Reinforcement Learning with a Natural Language Action Space

Transferring knowledge as heuristics in reinforcement learning: A case-based approach

Commonsense reasoning and commonsense knowledge in artificial intelligence

Effective Approaches to Attention-based Neural Machine Translation

Language Understanding for Text-based Games using Deep Reinforcement Learning

GloVe: Global Vectors for Word Representation

Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation

Learning to Win by Reading Manuals in a Monte-Carlo Framework

Freebase: a collaboratively created graph database for structuring human knowledge

Planning and Acting in Partially Observable Stochastic Domains

Self-improving reactive agents based on reinforcement learning, planning and teaching

The Ecological Approach to the Visual Perception of Pictures

Programs with common sense

BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding

Atomic: An atlas of machine commonsense for if-then reasoning

CommonsenseQA: A Question Answering Challenge Targeting Commonsense Knowledge

Language Models are Unsupervised Multitask Learners

Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACLHLT

Dungeons and DQNs: Toward Reinforcement Learning Agents that Play Tabletop Roleplaying Games

What Can You Do with a Rock?

Integrated Common Sense Learning and Planning in POMDPs

Reasoning

Reinforcement Learning for Robots Using Neural Networks

Content Analysis: An Introduction to Its Methodology

Understanding natural language

Field of Study

Venue Information

Name

Type

URL

Alternate Names