TStarBots: Defeating the Cheating Level Builtin AI in StarCraft II in the Full Game (2018-09-19T00:00:00.000000Z)

TL;DR

This is the first public work to investigate AI agents that can defeat the built-in AI in the StarCraft II full game, and the AI agent TStarBot1 is based on deep reinforcement learning over a flat action structure and theAI agent T starBot2 isbased on hard-coded rules over a hierarchical action structure.

Abstract

Starcraft II (SC2) is widely considered as the most challenging Real Time Strategy (RTS) game. The underlying challenges include a large observation space, a huge (continuous and infinite) action space, partial observations, simultaneous move for all players, and long horizon delayed rewards for local decisions. To push the frontier of AI research, Deepmind and Blizzard jointly developed the StarCraft II Learning Environment (SC2LE) as a testbench of complex decision making systems. SC2LE provides a few mini games such as MoveToBeacon, CollectMineralShards, and DefeatRoaches, where some AI agents have achieved the performance level of human professional players. However, for full games, the current AI agents are still far from achieving human professional level performance. To bridge this gap, we present two full game AI agents in this paper - the AI agent TStarBot1 is based on deep reinforcement learning over a flat action structure, and the AI agent TStarBot2 is based on hard-coded rules over a hierarchical action structure. Both TStarBot1 and TStarBot2 are able to defeat the built-in AI agents from level 1 to level 10 in a full game (1v1 Zerg-vs-Zerg game on the AbyssalReef map), noting that level 8, level 9, and level 10 are cheating agents with unfair advantages such as full vision on the whole map and resource harvest boosting. To the best of our knowledge, this is the first public work to investigate AI agents that can defeat the built-in AI in the StarCraft II full game.

Authors

Ji Liu

7 papers

T. Zhang

5 papers

Qing Wang

2 papers

TL;DR

Abstract

Authors

References44 items

Introduction to Reinforcement Learning

Global Rules

Relational Deep Reinforcement Learning

Meta Learning Shared Hierarchies

Mastering the game of Go without human knowledge

Autonomous Extracting a Hierarchical Structure of Tasks in Reinforcement Learning and Multi-task Reinforcement Learning

StarCraft II: A New Challenge for Reinforcement Learning

Proximal Policy Optimization Algorithms

ELF: An Extensive, Lightweight and Flexible Research Platform for Real-time Strategy Games

Hybrid Reward Architecture for Reinforcement Learning

Counterfactual Multi-Agent Policy Gradients

Multiagent Bidirectionally-Coordinated Nets: Emergence of Human-level Coordination in Learning to Play StarCraft Combat Games

FeUdal Networks for Hierarchical Reinforcement Learning

(CAD)$^2$RL: Real Single-Image Flight without a Single Real Image

Training Agent for First-Person Shooter Game with Actor-Critic Curriculum Learning

Target-driven visual navigation in indoor scenes using deep reinforcement learning

The Option-Critic Architecture

Episodic Exploration for Deep Deterministic Policies: An Application to StarCraft Micromanagement Tasks

Strategic Attentive Writer for Learning Macro-Actions

Unifying Count-Based Exploration and Intrinsic Motivation

Learning Multiagent Communication with Backpropagation

ViZDoom: A Doom-based AI research platform for visual reinforcement learning

Asynchronous Methods for Deep Reinforcement Learning

Mastering the game of Go with deep neural networks and tree search

Dueling Network Architectures for Deep Reinforcement Learning

Deep Reinforcement Learning with Double Q-Learning

High-Dimensional Continuous Control Using Generalized Advantage Estimation

End-to-End Training of Deep Visuomotor Policies

Human-level control through deep reinforcement learning

Trust Region Policy Optimization

Towards a unified behavior trees framework for robot control

A Survey of Real-Time Strategy Game AI Research and Competition in StarCraft

Evolving Behaviour Trees for the Mario AI Competition Using Grammatical Evolution

Between MDPs and Semi-MDPs: A Framework for Temporal Abstraction in Reinforcement Learning

Starcraft ii forum

Open ai ﬁve

Heuristic Search Techniques for Real-Time Strategy Games

Deep learning , volume 1

Artiﬁcial intelligence: a modern approach

Multi Agent Reinforcement Learning Independent vs Cooperative Agents

Combating ZoneA-Attack-ZoneB ZoneA-Attack-ZoneC

Explore Task

Liquipedia - ranking system

Tstarbots

Field of Study

Journal Information

Name

Volume

Venue Information

Name

Type

URL

Alternate Names