ELF: An Extensive, Lightweight and Flexible Research Platform for Real-time Strategy Games (2017-07-01T00:00:00.000000Z)

TL;DR

ELF, an Extensive, Lightweight and Flexible platform for fundamental reinforcement learning research, is proposed and it is shown that a network with Leaky ReLU and Batch Normalization coupled with long-horizon training and progressive curriculum beats the rule-based built-in AI more than $70\% of the time in the full game of Mini-RTS.

Abstract

In this paper, we propose ELF, an Extensive, Lightweight and Flexible platform for fundamental reinforcement learning research. Using ELF, we implement a highly customizable real-time strategy (RTS) engine with three game environments (Mini-RTS, Capture the Flag and Tower Defense). Mini-RTS, as a miniature version of StarCraft, captures key game dynamics and runs at 40K frame-per-second (FPS) per core on a Macbook Pro notebook. When coupled with modern reinforcement learning methods, the system can train a full-game bot against built-in AIs end-to-end in one day with 6 CPUs and 1 GPU. In addition, our platform is flexible in terms of environment-agent communication topologies, choices of RL methods, changes in game parameters, and can host existing C/C++-based game environments like Arcade Learning Environment. Using ELF, we thoroughly explore training parameters and show that a network with Leaky ReLU and Batch Normalization coupled with long-horizon training and progressive curriculum beats the rule-based built-in AI more than $70\%$ of the time in the full game of Mini-RTS. Strong performance is also achieved on the other two games. In game replays, we show our agents learn interesting strategies. ELF, along with its RL platform, is open-sourced at this https URL.

Authors

C. L. Zitnick

19 papers

Yuxin Wu

5 papers

Yuandong Tian

7 papers

TL;DR

Abstract

Authors

References34 items

Multiagent Bidirectionally-Coordinated Nets for Learning to Play StarCraft Combat Games

Learning to Navigate in Complex Environments

Playing SNES in the Retro Learning Environment

Reinforcement Learning through Asynchronous Advantage Actor-Critic on a GPU

Training Agent for First-Person Shooter Game with Actor-Critic Curriculum Learning

TorchCraft: a Library for Machine Learning Research on Real-Time Strategy Games

Playing FPS Games with Deep Reinforcement Learning

Episodic Exploration for Deep Deterministic Policies: An Application to StarCraft Micromanagement Tasks

The Malmo Platform for Artificial Intelligence Experimentation

OpenAI Gym

ViZDoom: A Doom-based AI research platform for visual reinforcement learning

Asynchronous Methods for Deep Reinforcement Learning

Mastering the game of Go with deep neural networks and tree search

MazeBase: A Sandbox for Learning from Games

Better Computer Go Player with Neural Network and Long-term Prediction

Massively Parallel Methods for Deep Reinforcement Learning

Human-level control through deep reinforcement learning

Trust Region Policy Optimization

Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift

The Combinatorial Multi-Armed Bandit Problem and Its Application to Real-Time Strategy Games

The Arcade Learning Environment: An Evaluation Platform for General Agents

A Survey of Monte Carlo Tree Search Methods

Parallel Monte-Carlo Tree Search

Policy Gradient Methods for Reinforcement Learning with Function Approximation

Rectifier Nonlinearities Improve Neural Network Acoustic Models

Bullet physics engine

Spring: https://springrts.com

Johansson and Robin Westberg

OpenRA

ON THE DEVELOPMENT OF A FREE RTS GAME ENGINE

RoboCup simulation-league

BattleCode

Pumpkin Studios

mit's ai programming competition

Field of Study

Journal Information

Name

Volume

Venue Information

Name

Type

URL

Alternate Names