Counter-Strike Deathmatch with Large-Scale Behavioural Cloning (2021-04-09T00:00:00.000000Z)

TL;DR

An AI agent that plays the modern first-person-shooter video game ‘Counter-Strike; Global Offensive’ (CSGO) from pixel input, a deep neural network that matches the performance of a casual human gamer on the deathmatch game mode whilst adopting a humanlike play style.

Abstract

This paper describes an AI agent that plays the modern first-person-shooter (FPS) video game ‘Counter-Strike; Global Offensive’ (CSGO) from pixel input. The agent, a deep neural network, matches the performance of a casual human gamer on the deathmatch game mode whilst adopting a humanlike play style. Much previous research has focused on games with convenient APIs and low-resolution graphics, allowing them to be run cheaply at scale. This is not the case for CSGO, with system requirements orders of magnitude higher than previously studied FPS games. This limits the quantity of on-policy data that can be generated, precluding pure reward-driven reinforcement learning (RL) algorithms. Our solution uses a two-stage behavioural cloning methodology; 1) Pre-train on a large dataset scraped from human play on public servers (5.5 million frames or 95 hours) where actions are labelled in an automated way. 2) Fine-tune on a small dataset of clean expert demonstrations (190 thousand frames or 3 hours). This scale is an order of magnitude larger than prior work on imitation learning in FPS games, whilst being far more data efficient than pure RL algorithms. Video introduction: https://youtu.be/rnz3lmfSHv0 Code, model & datasets: https://github.com/TeaPearce

Authors

Jun Zhu

7 papers

Tim Pearce

1 papers

TL;DR

Abstract

Authors

References47 items

SCC: an efficient deep reinforcement learning agent mastering the game of StarCraft II

Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problems

Datasets for Data-Driven Reinforcement Learning

Benchmarking End-to-End Behavioural Cloning on Video Games

Dota 2 with Large Scale Deep Reinforcement Learning

Grandmaster level in StarCraft II using multi-agent reinforcement learning

Playing FPS Games With Environment-Aware Hierarchical Reinforcement Learning

MineRL: A Large-Scale Dataset of Minecraft Demonstrations

Combo-Action: Training Agent For FPS Game with Auxiliary Tasks

An Optimistic Perspective on Offline Reinforcement Learning

Causal Confusion in Imitation Learning

EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks

ViZDoom Competitions: Playing Doom From Pixels

Artificial Intelligence and Virtual Worlds – Toward Human-Level AI Agents

Human-level performance in 3D multiplayer games with population-based reinforcement learning

Imitation Learning with Concurrent Actions in 3D Games

Predicting Winning Team and Probabilistic Ratings in "Dota 2" and "Counter-Strike: Global Offensive" Video Games

Deep Q-learning From Demonstrations

Iterative Noise Injection for Scalable Imitation Learning

The Game Imitation: Deep Supervised Convolutional Networks for Quick Video Game AI

DeepMind Lab

Playing FPS Games with Deep Reinforcement Learning

ViZDoom: A Doom-based AI research platform for visual reinforcement learning

Mastering the game of Go with deep neural networks and tree search

Convolutional LSTM Network: A Machine Learning Approach for Precipitation Nowcasting

Human-level control through deep reinforcement learning

A Dynamic-Bayesian Network framework for modeling and evaluating learning from observation

Underlying factors of social capital acquisition in the context of online-gaming: Comparing World of Warcraft and Counter-Strike

A Reduction of Imitation Learning and Structured Prediction to No-Regret Online Learning

The 2K BotPrize

An evaluation of models for predicting opponent positions in first-person shooter video games

Believability Testing and Bayesian Imitation in Interactive Computer Games

Using a genetic algorithm to tune first-person shooter bots

Learning from Demonstration

Using Human Gameplay to Augment Reinforcement Learning Models for Crypt of the NecroDancer

Supplementary Material: Counter-Strike Deathmatch with Large-Scale Behavioural Cloning The supplementary material contains the following sections: • Section A

and accumulating errors can cause agents to stray from the input space for which expert collected [Ross et al., 2011, Laskey et al., 2017]

2019] considered a capture the flag mode, and trained agents able to outperform teams of humans familiar with FPS games – they used an actor-critic algorithm

Minecraft is ‘from pixels

"Skins" in the Game: Counter-Strike, Esports, and the Shady World of Online Gambling

Lecture Notes in Arti ﬁ cial Intelligence

Dynamic Business Process Formation for Instant Virtual Enterprises

IMITATIVE LEARNING OF COMBAT BEHAVIOURS IN FIRST-PERSON COMPUTER GAMES

Robot See, Robot Do : An Overview of Robot Imitation

Settings This section details the game settings we used to evaluate the agent (see https://github.com/ TeaPearce for full list)

Ablations and further results. • Section C, Comprehensive Related Work: Expanded related work section

Section B, Further Results: Ablations and further results

Field of Study

Journal Information

Name

Page