The NetHack Learning Environment (2020-06-01T00:00:00.000000Z)

TL;DR

It is argued that NetHack is sufficiently complex to drive long-term research on problems such as exploration, planning, skill acquisition, and language-conditioned RL, while dramatically reducing the computational resources required to gather a large amount of experience.

Abstract

Progress in Reinforcement Learning (RL) algorithms goes hand-in-hand with the development of challenging environments that test the limits of current methods. While existing RL environments are either sufficiently complex or based on fast simulation, they are rarely both. Here, we present the NetHack Learning Environment (NLE), a scalable, procedurally generated, stochastic, rich, and challenging environment for RL research based on the popular single-player terminal-based roguelike game, NetHack. We argue that NetHack is sufficiently complex to drive long-term research on problems such as exploration, planning, skill acquisition, and language-conditioned RL, while dramatically reducing the computational resources required to gather a large amount of experience. We compare NLE and its task suite to existing alternatives, and discuss why it is an ideal medium for testing the robustness and systematic generalization of RL agents. We demonstrate empirical success for early stages of the game using a distributed Deep RL baseline and Random Network Distillation exploration, alongside qualitative analysis of various agents trained in the environment. NLE is open source at this https URL.

Authors

Nantas Nardelli

3 papers

Edward Grefenstette

7 papers

Heinrich Kuttler

2 papers

TL;DR

Abstract

Authors

References78 items

Learning Combat in NetHack

Crawling in Rogue's Dungeons With Deep Reinforcement Techniques

RTFM: Generalising to New Environment Dynamics via Reading

First return, then explore

RIDE: Rewarding Impact-Driven Exploration for Procedurally-Generated Environments

Rotation, Translation, and Cropping for Zero-Shot Generalization

Dota 2 with Large Scale Deep Reinforcement Learning

Leveraging Procedural Generation to Benchmark Reinforcement Learning

Procedural Content Generation: From Automatically Generating Game Levels to Increasing Generality in Machine Learning

Grandmaster level in StarCraft II using multi-agent reinforcement learning

TorchBeast: A PyTorch Platform for Distributed RL

Emergent Systematic Generalization in a Situated Agent

Growing Up Together: Structured Exploration for Large Action Spaces

MazeExplorer: A Customisable 3D Benchmark for Assessing Generalisation in Reinforcement Learning

Language as an Abstraction for Hierarchical Deep Reinforcement Learning

A Survey of Reinforcement Learning Informed by Natural Language

The MineRL Competition on Sample Efficient Reinforcement Learning using Human Priors

CraftAssist Instruction Parsing: Semantic Parsing for a Minecraft Assistant

Rogue-Gym: A New Challenge for Generalization in Reinforcement Learning

The StarCraft Multi-Agent Challenge

Dungeon Crawl Stone Soup as an Evaluation Domain for Artificial Intelligence

Obstacle Tower: A Generalization Challenge in Vision, Control, and Planning

Go-Explore: a New Approach for Hard-Exploration Problems

Learning Montezuma's Revenge from a Single Demonstration

Quantifying Generalization in Reinforcement Learning

Experience Replay for Continual Learning

BabyAI: First Steps Towards Grounded Language Learning With a Human In the Loop

Exploration by Random Network Distillation

BabyAI: A Platform to Study the Sample Efficiency of Grounded Language Learning

Large-Scale Study of Curiosity-Driven Learning

Illuminating Generalization in Deep Reinforcement Learning through Procedural Level Generation

Playing hard exploration games by watching YouTube

Gotta Learn Fast: A New Benchmark for Generalization in RL

Unicorn: Continual Learning with a Universal, Off-policy Agent

Continual Lifelong Learning with Neural Networks: A Review

Value Propagation Networks

IMPALA: Scalable Distributed Deep-RL with Importance Weighted Actor-Learner Architectures

Exploration in NetHack With Secret Discovery

A survey of robot learning from demonstrations for Human-Robot Collaboration

Deep Reinforcement Learning: A Brief Survey

StarCraft II: A New Challenge for Reinforcement Learning

Gradient Episodic Memory for Continual Learning

Curiosity-Driven Exploration by Self-Supervised Prediction

Deep Q-learning From Demonstrations

Count-Based Exploration with Neural Density Models

FeUdal Networks for Hierarchical Reinforcement Learning

Stabilising Experience Replay for Deep Multi-Agent Reinforcement Learning

DeepMind Lab

#Exploration: A Study of Count-Based Exploration for Deep Reinforcement Learning

TorchCraft: a Library for Machine Learning Research on Real-Time Strategy Games

The Malmo Platform for Artificial Intelligence Experimentation

Unifying Count-Based Exploration and Intrinsic Motivation

ViZDoom: A Doom-based AI research platform for visual reinforcement learning

A Survey of Real-Time Strategy Game AI Research and Competition in StarCraft

The Arcade Learning Environment: An Evaluation Platform for General Agents

Learning to Win by Reading Manuals in a Monte-Carlo Framework

Transfer Learning for Reinforcement Learning Domains: A Survey

Apprenticeship learning via inverse reinforcement learning

Reinforcement Learning with Hierarchies of Machines

Long Short-Term Memory

Reinforcement Learning: A Survey

Feudal Reinforcement Learning

Robotics and autonomous systems

NetHack Wiki

NetHack Home Page

Kenneth Lorber

A Guide to the Mazes of Menace: Guidebook for NetHack. NetHack DevTeam

Public NetHack server at alt.org (NAO)

List of Nethack spoilers

2016) and CraftAssist (Jernite et al., 2019) provide visual and symbolic interfaces to the popular Minecraft game, focusing on single-agent and multi-agent interactions

Aransentin

Minimalistic Gridworld Environment for OpenAI Gym

URL http://arxiv.org/ abs/1606.01540

A comprehensive survey on safe reinforcement learning

TAEB Documentation: Other Bots

Reinforcement Learning for roguelike type games (eliteMod v0.9)