Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environments (2017-06-07T00:00:00.000000Z)

TL;DR

An adaptation of actor-critic methods that considers action policies of other agents and is able to successfully learn policies that require complex multi-agent coordination is presented.

Abstract

We explore deep reinforcement learning methods for multi-agent domains. We begin by analyzing the difficulty of traditional algorithms in the multi-agent case: Q-learning is challenged by an inherent non-stationarity of the environment, while policy gradient suffers from a variance that increases as the number of agents grows. We then present an adaptation of actor-critic methods that considers action policies of other agents and is able to successfully learn policies that require complex multi-agent coordination. Additionally, we introduce a training regimen utilizing an ensemble of policies for each agent that leads to more robust multi-agent policies. We show the strength of our approach compared to existing methods in cooperative as well as competitive scenarios, where agent populations are able to discover various physical and informational coordination strategies.

Authors

P. Abbeel

35 papers

Aviv Tamar

4 papers

J. Harb

2 papers

TL;DR

Abstract

Authors

References40 items

Counterfactual Multi-Agent Policy Gradients

Cooperative Multi-agent Control Using Deep Reinforcement Learning

Multiagent Bidirectionally-Coordinated Nets for Learning to Play StarCraft Combat Games

Deep Decentralized Multi-task Multi-Agent Reinforcement Learning under Partial Observability

Emergence of Grounded Compositional Language in Multi-Agent Populations

Intrinsic Motivation and Automatic Curricula via Asymmetric Self-Play

Stabilising Experience Replay for Deep Multi-Agent Reinforcement Learning

Multi-agent Reinforcement Learning in Sequential Social Dilemmas

Multi-Agent Cooperation and the Emergence of (Natural) Language

Categorical Reparameterization with Gumbel-Softmax

Learning to Protect Communications with Adversarial Neural Cryptography

Learning Multiagent Communication with Backpropagation

Learning to Communicate with Deep Multi-Agent Reinforcement Learning

Mastering the game of Go with deep neural networks and tree search

Multiagent cooperation and competition with deep reinforcement learning

Continuous control with deep reinforcement learning

End-to-End Training of Deep Visuomotor Policies

Human-level control through deep reinforcement learning

Deterministic Policy Gradient Algorithms

Coordinated Multi-Robot Exploration Under Communication Constraints Using Decentralized Markov Decision Processes

Predicting Pragmatic Reasoning in Language Games

Independent reinforcement learners in cooperative Markov games: a survey regarding coordination problems

Conjugate Markov Decision Processes

Et al

A Comprehensive Survey of Multiagent Reinforcement Learning

Hysteretic q-learning :an algorithm for decentralized reinforcement learning in cooperative multi-agent teams

Cooperative Multi-Agent Learning: The State of the Art

Extending Q-Learning to General Adaptive Multi-Agent Systems

Coordination in multiagent reinforcement learning: a Bayesian approach

An Algorithm for Distributed Reinforcement Learning in Cooperative Multi-Agent Systems

Policy Gradient Methods for Reinforcement Learning with Function Approximation

Online learning about other agents in a dynamic multiagent system

Learning Conventions in Multiagent Stochastic Domains using Likelihood Estimates

Markov Games as a Framework for Multi-Agent Reinforcement Learning

Feudal Reinforcement Learning

GENERATIVE ADVERSARIAL NETS

Simple Statistical Gradient-Following Algorithms for Connectionist Reinforcement Learning

Multi Agent Reinforcement Learning Independent vs Cooperative Agents

Reinforcement Learning: An Introduction

DeepMind AI reduces google data centre cooling bill by 40

Field of Study

Journal Information

Name

Volume

Venue Information

Name

Type

URL

Alternate Names