MiniHack the Planet: A Sandbox for Open-Ended Reinforcement Learning Research (2021-09-27T00:00:00.000000Z)

TL;DR

MiniHack is a one-stop shop for RL experiments with environments ranging from small rooms to complex, procedurally generated worlds, and can wrap existing RL benchmarks and provide ways to seamlessly add additional complexity.

Abstract

Progress in deep reinforcement learning (RL) is heavily driven by the availability of challenging benchmarks used for training agents. However, benchmarks that are widely adopted by the community are not explicitly designed for evaluating specific capabilities of RL methods. While there exist environments for assessing particular open problems in RL (such as exploration, transfer learning, unsupervised environment design, or even language-assisted RL), it is generally difficult to extend these to richer, more complex environments once research goes beyond proof-of-concept results. We present MiniHack, a powerful sandbox framework for easily designing novel RL environments. MiniHack is a one-stop shop for RL experiments with environments ranging from small rooms to complex, procedurally generated worlds. By leveraging the full set of entities and environment dynamics from NetHack, one of the richest grid-based video games, MiniHack allows designing custom RL testbeds that are fast and convenient to use. With this sandbox framework, novel environments can be designed easily, either using a human-readable description language or a simple Python interface. In addition to a variety of RL tasks and baselines, MiniHack can wrap existing RL benchmarks and provide ways to seamlessly add additional complexity.

Authors

Mikayel Samvelyan

7 papers

F. Petroni

8 papers

Edward Grefenstette

14 papers

TL;DR

Abstract

Authors

References68 items

Learning Combat in NetHack

On the Importance of Hyperparameter Optimization for Model-based Reinforcement Learning

Adversarially Guided Actor-Critic

How to train your robot with deep reinforcement learning: lessons we have learned

The MineRL 2020 Competition on Sample Efficient Reinforcement Learning using Human Priors

Introduction to Reinforcement Learning

BeBold: Exploration Beyond the Boundary of Explored Regions

Emergent Complexity and Zero-shot Transfer via Unsupervised Environment Design

Griddly: A platform for AI research in games

The NetHack Learning Environment

Agent57: Outperforming the Atari Human Benchmark

An empirical investigation of the challenges of real-world reinforcement learning

RIDE: Rewarding Impact-Driven Exploration for Procedurally-Generated Environments

Dota 2 with Large Scale Deep Reinforcement Learning

Leveraging Procedural Generation to Benchmark Reinforcement Learning

Meta-World: A Benchmark and Evaluation for Multi-Task and Meta Reinforcement Learning

TorchBeast: A PyTorch Platform for Distributed RL

Environmental drivers of systematicity and generalization in a situated agent

Behaviour Suite for Reinforcement Learning

Reinforcement learning

Rogue-Gym: A New Challenge for Generalization in Reinforcement Learning

Obstacle Tower: A Generalization Challenge in Vision, Control, and Planning

Go-Explore: a New Approach for Hard-Exploration Problems

An investigation of model-free planning

Exploration by Random Network Distillation

Gotta Learn Fast: A New Benchmark for Generalization in RL

General Video Game AI: A Multitrack Framework for Evaluating Agents, Games, and Content Generation Algorithms

IMPALA: Scalable Distributed Deep-RL with Importance Weighted Actor-Learner Architectures

RLlib: Abstractions for Distributed Reinforcement Learning

Ray: A Distributed Framework for Emerging AI Applications

Exploration in NetHack With Secret Discovery

Rogueinabox: an Environment for Roguelike Learning

Deep Reinforcement Learning that Matters

StarCraft II: A New Challenge for Reinforcement Learning

Proximal Policy Optimization Algorithms

The Malmo Platform for Artificial Intelligence Experimentation

Unifying Count-Based Exploration and Intrinsic Motivation

ViZDoom: A Doom-based AI research platform for visual reinforcement learning

Deep Exploration via Bootstrapped DQN

Asynchronous Methods for Deep Reinforcement Learning

Dueling Network Architectures for Deep Reinforcement Learning

Prioritized Experience Replay

Deep Reinforcement Learning with Double Q-Learning

Character-level Convolutional Networks for Text Classification

Human-level control through deep reinforcement learning

An Extensible Description Language for Video Games

Towards a Video Game Description Language

A video game description language for model-based or interactive learning

MuJoCo: A physics engine for model-based control

The Arcade Learning Environment: An Evaluation Platform for General Agents

Long Short-Term Memory

NetHack Wiki

What Matters for On-Policy Deep Actor-Critic Methods? A Large-Scale Study

Alchemy: A structured task distribution for meta-reinforcement learning

The Telepraph

Memory Based Trajectory-conditioned Policies for Learning from Sparse Rewards

A Guide to the Mazes of Menace: Guidebook for NetHack. NetHack DevTeam

Minimalistic Gridworld Environment for OpenAI Gym

} 24 SHUFFLE : $monster 25 $place = {

Third, an MLP 999 is used to encode the agent's statistic. These three vectors are concatenated and passed to another 1000 MLP which produces the final representation o t of the observation

The training on MiniHack's Room-5x5 task for two million timesteps using our IMPALA baseline 1034 takes approximately 4:30 minutes (roughly 7667 steps per second)

Maxpooling is applied after the first, second, and sixth layers. The convolutional layers are followed 1017 by an MLP

These settings were determined during a set of small-scale experiments

The characters within the in-game messages are encoded using an embedding of size 32 and passed 1015 to a 6-layer CNN architecture, each comprised of a 1D convolution of size 64 and ReLU nonlinearity

Three dense representations are produced. First, all visible entity embeddings are passed to a CNN

The 25 hardest video games ever

MAZE : " mylevel

For results on skill acquisition tasks, we use a learning rate of 5 * 10 −5

Field of Study

Journal Information

Name

Volume