Gradient Surgery for Multi-Task Learning

Published in

Neural Information Processing Systems(2020)

External Links:

Generate Graph

TL;DR

This work identifies a set of three conditions of the multi-task optimization landscape that cause detrimental gradient interference, and develops a simple yet general approach for avoiding such interference between task gradients.

Abstract

While deep learning and deep reinforcement learning (RL) systems have demonstrated impressive results in domains such as image classification, game playing, and robotic control, data efficiency remains a major challenge. Multi-task learning has emerged as a promising approach for sharing structure across multiple tasks to enable more efficient learning. However, the multi-task setting presents a number of optimization challenges, making it difficult to realize large efficiency gains compared to learning tasks independently. The reasons why multi-task learning is so challenging compared to single-task learning are not fully understood. In this work, we identify a set of three conditions of the multi-task optimization landscape that cause detrimental gradient interference, and develop a simple yet general approach for avoiding such interference between task gradients. We propose a form of gradient surgery that projects a task's gradient onto the normal plane of the gradient of any other task that has a conflicting gradient. On a series of challenging multi-task supervised and multi-task RL problems, this approach leads to substantial gains in efficiency and performance. Further, it is model-agnostic and can be combined with previously-proposed multi-task architectures for enhanced performance.

Authors

Chelsea Finn

26 papers

S. Levine

59 papers

Abhishek Gupta

4 papers

References63 items

Meta-World: A Benchmark and Evaluation for Multi-Task and Meta Reinforcement Learning

Orthogonal Gradient Descent for Continual Learning

Improved Schemes for Episodic Memory-based Lifelong Learning

Regularizing Deep Multi-Task Networks using Orthogonal Gradients

Regularized Hierarchical Policies for Compositional Transfer in Robotics

Gradient Surgery for Multi-Task Learning

Published in

Neural Information Processing Systems(2020)

External Links:

Generate Graph

TL;DR

Abstract

Authors

Chelsea Finn

26 papers

S. Levine

59 papers

Abhishek Gupta

4 papers

References63 items

Meta-World: A Benchmark and Evaluation for Multi-Task and Meta Reinforcement Learning

Orthogonal Gradient Descent for Continual Learning

Improved Schemes for Episodic Memory-based Lifelong Learning

Regularizing Deep Multi-Task Networks using Orthogonal Gradients

Regularized Hierarchical Policies for Compositional Transfer in Robotics

Saurabh Kumar

1 papers

Karol Hausman

4 papers

Routing Networks and the Challenges of Modular and Compositional Computation

Ray Interference: a Source of Plateaus in Deep Reinforcement Learning

Attentive Single-Tasking of Multiple Tasks

Branched Multi-Task Networks: Deciding what layers to share

Distilling Policy Distillation

Multi-Task Learning as Multi-Objective Optimization

Efficient Lifelong Learning with A-GEM

Adapting Auxiliary Losses Using Gradient Similarity

Multi-task Deep Reinforcement Learning with PopArt

The Natural Language Decathlon: Multitask Learning as Question Answering

Taskonomy: Disentangling Task Transfer Learning

End-To-End Multi-Task Learning With Attention

Learning by Playing - Solving Sparse Reward Tasks from Scratch

Learning an Embedding Space for Transferable Robot Skills

IMPALA: Scalable Distributed Deep-RL with Importance Weighted Actor-Learner Architectures

Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor

Divide-and-Conquer Reinforcement Learning

GradNorm: Gradient Normalization for Adaptive Loss Balancing in Deep Multitask Networks

Routing Networks: Adaptive Selection of Non-linear Functions for Multi-Task Learning

End-to-End Driving Via Conditional Imitation Learning

Distral: Robust multitask reinforcement learning

An Overview of Multi-Task Learning in Deep Neural Networks

Gradient Episodic Memory for Continual Learning

Stabilizing Training of Generative Adversarial Networks through Regularization

Multi-task Learning Using Uncertainty to Weigh Losses for Scene Geometry and Semantics

PathNet: Evolution Channels Gradient Descent in Super Neural Networks

Learning modular neural network policies for multi-task and multi-robot transfer

UberNet: Training a Universal Convolutional Neural Network for Low-, Mid-, and High-Level Vision Using Diverse Datasets and Limited Memory

Progressive Neural Networks

Trace Norm Regularised Deep Multi-Task Learning

Integrated perception with recurrent multi-task neural networks

Cross-Stitch Networks for Multi-task Learning

The Cityscapes Dataset for Semantic Urban Scene Understanding

Actor-Mimic: Deep Multitask and Transfer Reinforcement Learning

Policy Distillation

SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation

Multi-Task Learning for Multiple Language Translation

Learning Multiple Tasks with Multilinear Relationship Networks

Learning Multiple Tasks with Deep Relationship Networks

End-to-End Training of Deep Visuomotor Policies

Distilling the Knowledge in a Neural Network

Adam: A Method for Stochastic Optimization

Qualitatively characterizing neural network optimization problems

Deep Learning Face Attributes in the Wild

Facial Landmark Detection by Deep Multi-task Learning

Convex optimization

Distributed Subgradient Methods for Multi-Agent Optimization

A unified architecture for natural language processing: deep neural networks with multitask learning

Multi-task reinforcement learning: a hierarchical Bayesian approach

Task Clustering and Gating for Bayesian Multitask Learning

Eigenvalues, invariant factors, highest weights, and Schubert calculus

Multitask Learning

Projected gradient methods for linearly constrained problems

Language Models are Unsupervised Multitask Learners

An Overview of Multi-task Learning

NOTES ON FIRST-ORDER METHODS FOR MINIMIZING SMOOTH FUNCTIONS

Some methods of speeding up the convergence of iteration methods

Field of Study

Computer ScienceMathematics

Journal Information

Name

ArXiv

Volume

abs/2001.06782

Venue Information

Name

Neural Information Processing Systems

Type

conference

URL

http://neurips.cc/

Alternate Names

Neural Inf Process Syst
NeurIPS
NIPS