ELF OpenGo: An Analysis and Open Reimplementation of AlphaZero

Published in

International Conference on Machine Learning(2019)

External Links:

Generate Graph

TL;DR

ELF OpenGo is the first open-source Go AI to convincingly demonstrate superhuman performance with a perfect (20:0) record against global top professionals and is proposed, anopen-source reimplementation of the AlphaZero algorithm.

Abstract

The AlphaGo, AlphaGo Zero, and AlphaZero series of algorithms are remarkable demonstrations of deep reinforcement learning's capabilities, achieving superhuman performance in the complex game of Go with progressively increasing autonomy. However, many obstacles remain in the understanding of and usability of these promising approaches by the research community. Toward elucidating unresolved mysteries and facilitating future research, we propose ELF OpenGo, an open-source reimplementation of the AlphaZero algorithm. ELF OpenGo is the first open-source Go AI to convincingly demonstrate superhuman performance with a perfect (20:0) record against global top professionals. We apply ELF OpenGo to conduct extensive ablation studies, and to identify and analyze numerous interesting phenomena in both the model training and in the gameplay inference procedures. Our code, models, selfplay datasets, and auxiliary data are publicly available.

Authors

Shubho Sengupta

5 papers

James Pinkerton

3 papers

C. L. Zitnick

19 papers

References29 items

AlphaX: eXploring Neural Architectures with Deep Neural Networks and Monte Carlo Tree Search

A general reinforcement learning algorithm that masters chess, shogi, and Go through self-play

Understanding & Generalizing AlphaGo Zero

Re-evaluating evaluation

From Gameplay to Symbolic Reasoning: Learning SAT Solver Heuristics in the Style of Alpha(Go) Zero

ELF OpenGo: An Analysis and Open Reimplementation of AlphaZero

Published in

International Conference on Machine Learning(2019)

External Links:

Generate Graph

TL;DR

Abstract

Authors

Shubho Sengupta

5 papers

James Pinkerton

3 papers

C. L. Zitnick

19 papers

References29 items

AlphaX: eXploring Neural Architectures with Deep Neural Networks and Monte Carlo Tree Search

A general reinforcement learning algorithm that masters chess, shogi, and Go through self-play

Understanding & Generalizing AlphaGo Zero

Re-evaluating evaluation

From Gameplay to Symbolic Reasoning: Learning SAT Solver Heuristics in the Style of Alpha(Go) Zero

Yuandong Tian

7 papers

Zhuoyuan Chen

4 papers

Qucheng Gong

2 papers

Demystifying AlphaGo Zero as AlphaGo GAN

Automatic differentiation in PyTorch

Mastering the game of Go without human knowledge

ELF: An Extensive, Lightweight and Flexible Research Platform for Real-time Strategy Games

Mastering the game of Go with deep neural networks and tree search

Deep Residual Learning for Image Recognition

Better Computer Go Player with Neural Network and Long-term Prediction

Training Deep Convolutional Neural Networks to Play Go

Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift

Move Evaluation in Go Using Deep Convolutional Neural Networks

A Survey of Monte Carlo Tree Search Methods

Multi-armed bandits with episode context

Fuego—An Open-Source Framework for Board Games and Go Engine Based on Monte Carlo Tree Search

Parallel Monte-Carlo Tree Search

Bandit Based Monte-Carlo Planning

The Solution for the Branching Factor of the Alpha-Beta Pruning Algorithm

DAWNBench : An End-to-End Deep Learning Benchmark and Competition

2015) and the ELF reinforcement learning platform (Tian et al., 2017)

Go engine We have integrated the DarkForest Go engine (Tian & Zhu, 2015) into ELF OpenGo, which provides efficient handling of game dynamics (approximately 1.2μs per move)

The rating of chess players, past and present

Field of Study

Computer ScienceMathematics

Venue Information

Name

International Conference on Machine Learning

Type

conference

URL

https://icml.cc/

Alternate Names

ICML
Int Conf Mach Learn

TL;DR

Abstract

Authors

References29 items

AlphaX: eXploring Neural Architectures with Deep Neural Networks and Monte Carlo Tree Search

A general reinforcement learning algorithm that masters chess, shogi, and Go through self-play

Understanding & Generalizing AlphaGo Zero

Re-evaluating evaluation

From Gameplay to Symbolic Reasoning: Learning SAT Solver Heuristics in the Style of Alpha(Go) Zero

TL;DR

Abstract

Authors

References29 items

AlphaX: eXploring Neural Architectures with Deep Neural Networks and Monte Carlo Tree Search

A general reinforcement learning algorithm that masters chess, shogi, and Go through self-play

Understanding & Generalizing AlphaGo Zero

Re-evaluating evaluation

From Gameplay to Symbolic Reasoning: Learning SAT Solver Heuristics in the Style of Alpha(Go) Zero

Demystifying AlphaGo Zero as AlphaGo GAN

Automatic differentiation in PyTorch

Mastering the game of Go without human knowledge

ELF: An Extensive, Lightweight and Flexible Research Platform for Real-time Strategy Games

OpenAI Gym

Mastering the game of Go with deep neural networks and tree search

Deep Residual Learning for Image Recognition

Better Computer Go Player with Neural Network and Long-term Prediction

Training Deep Convolutional Neural Networks to Play Go

Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift

Move Evaluation in Go Using Deep Convolutional Neural Networks

A Survey of Monte Carlo Tree Search Methods

Multi-armed bandits with episode context

Fuego—An Open-Source Framework for Board Games and Go Engine Based on Monte Carlo Tree Search

Parallel Monte-Carlo Tree Search

Bandit Based Monte-Carlo Planning

Deep Blue

The Solution for the Branching Factor of the Alpha-Beta Pruning Algorithm

MiniGo Team

DAWNBench : An End-to-End Deep Learning Benchmark and Competition

2015) and the ELF reinforcement learning platform (Tian et al., 2017)

Leela

Go engine We have integrated the DarkForest Go engine (Tian & Zhu, 2015) into ELF OpenGo, which provides efficient handling of game dynamics (approximately 1.2μs per move)

The rating of chess players, past and present

Field of Study

Venue Information

Name

Type

URL

Alternate Names