TLeague: A Framework for Competitive Self-Play based Distributed Multi-Agent Reinforcement Learning (2020-11-25T00:00:00.000000Z)

TL;DR

A framework, referred to as TLeague, that aims at large-scale training and implements several main-stream CSP-MARL algorithms, which achieves a high throughput and a reasonable scale-up when performing distributed training.

Abstract

Competitive Self-Play (CSP) based Multi-Agent Reinforcement Learning (MARL) has shown phenomenal breakthroughs recently. Strong AIs are achieved for several benchmarks, including Dota 2, Glory of Kings, Quake III, StarCraft II, to name a few. Despite the success, the MARL training is extremely data thirsty, requiring typically billions of (if not trillions of) frames be seen from the environment during training in order for learning a high performance agent. This poses non-trivial difficulties for researchers or engineers and prevents the application of MARL to a broader range of real-world problems. To address this issue, in this manuscript we describe a framework, referred to as TLeague, that aims at large-scale training and implements several main-stream CSP-MARL algorithms. The training can be deployed in either a single machine or a cluster of hybrid machines (CPUs and GPUs), where the standard Kubernetes is supported in a cloud native manner. TLeague achieves a high throughput and a reasonable scale-up when performing distributed training. Thanks to the modular design, it is also easy to extend for solving other multi-agent problems or implementing and verifying MARL algorithms. We present experiments over StarCraft II, ViZDoom and Pommerman to show the efficiency and effectiveness of TLeague. The code is open-sourced and available at https://github.com/tencent-ailab/tleague_projpage

Authors

Zhengyou Zhang

2 papers

Jiechao Xiong

3 papers

Peng Sun

2 papers

TL;DR

Abstract

Authors

References68 items

Introduction to Reinforcement Learning

TStarBot-X: An Open-Sourced and Comprehensive Study for Efficient League Training in StarCraft II Full Game

Acme: A Research Framework for Distributed Reinforcement Learning

Graphical Models

Mastering Complex Control in MOBA Games with Deep Reinforcement Learning

Dota 2 with Large Scale Deep Reinforcement Learning

Grandmaster level in StarCraft II using multi-agent reinforcement learning

Arena: a toolkit for Multi-Agent Reinforcement Learning

Multi-Agent Deep Reinforcement Learning for Liquidation Strategy Analysis

Arena: A General Evaluation Platform and Building Toolkit for Multi-Agent Intelligence

Open-ended Learning in Symmetric Zero-sum Games

Double Neural Counterfactual Regret Minimization

Deep Counterfactual Regret Minimization

Actor-Critic Policy Optimization in Partially Observable Multiagent Environments

Recurrent Experience Replay in Distributed Reinforcement Learning

Dopamine: A Research Framework for Deep Reinforcement Learning

Pommerman: A Multi-Agent Playground

Human-level performance in 3D multiplayer games with population-based reinforcement learning

Time and Space: Why Imperfect Information Games are Hard

Horovod: fast and easy distributed deep learning in TensorFlow

Distributed Prioritized Experience Replay

IMPALA: Scalable Distributed Deep-RL with Importance Weighted Actor-Learner Architectures

Kubernetes in Action

RLlib: Abstractions for Distributed Reinforcement Learning

Ray: A Distributed Framework for Emerging AI Applications

A Unified Game-Theoretic Approach to Multiagent Reinforcement Learning

Mastering the game of Go without human knowledge

Emergent Complexity via Multi-Agent Competition

Proximal Policy Optimization Algorithms

ELF: An Extensive, Lightweight and Flexible Research Platform for Real-time Strategy Games

Single Sample Fictitious Play

Efficient Parallel Methods for Deep Reinforcement Learning

Training Agent for First-Person Shooter Game with Actor-Critic Curriculum Learning

Reinforcement Learning through Asynchronous Advantage Actor-Critic on a GPU

Sampled Fictitious Play is Hannan Consistent

ViZDoom: A Doom-based AI research platform for visual reinforcement learning

Microservices Architecture Enables DevOps: Migration to a Cloud-Native Architecture

Deep Reinforcement Learning from Self-Play in Imperfect-Information Games

Asynchronous Methods for Deep Reinforcement Learning

Mastering the game of Go with deep neural networks and tree search

Massively Parallel Methods for Deep Reinforcement Learning

Fictitious Self-Play in Extensive-Form Games

Human-level control through deep reinforcement learning

A Unified View of Large-Scale Zero-Sum Equilibrium Computation

Programmable self-assembly in a thousand-robot swarm

Designing Collective Behavior in a Termite-Inspired Robot Construction Team

Cloud

Regret Minimization in Games with Incomplete Information

A Concise Introduction to Multiagent Systems and Distributed Artificial Intelligence

Long Short-Term Memory

Markov Games as a Framework for Multi-Agent Reinforcement Learning

AN ITERATIVE METHOD OF SOLVING A GAME

Drone racing

Welcome to airsim

Ømq - the guide

Reverb: An efficient data storage and transport system for ml research

Seed rl: Scalable and eﬃcient deep-rl with accelerated central inference

Open ai ﬁve

Programmable self-assembly in a thousand-robot swarm

Robust strategies and counter-strategies: Building a champion level computer poker player

Using MPI-2 : Advanced Features of the Message Passing Interface ( Scientific and Engineering Computation ) by Ewing L . Lusk

ViZDoom testing code

Command: Modern air naval operations

Vizdoom cig 2016 competition

kubeﬂow

A high-performance, open source universal rpc framework

Tencent kubernetes engine

Protocol buffers are a language-neutral, platform-neutral extensible mechanism for serializing structured data

Field of Study

Journal Information

Name

Volume

Venue Information

Name

Type

URL