Auxiliary Learning by Implicit Differentiation (2020-06-22T00:00:00.000000Z)

TL;DR

A novel framework, AuxiLearn, is proposed that targets both challenges of designing useful auxiliary tasks and combining auxiliary tasks into a single coherent loss, based on implicit differentiation.

Abstract

Training with multiple auxiliary tasks is a common practice used in deep learning for improving the performance on the main task of interest. Two main challenges arise in this multi-task learning setting: (i) Designing useful auxiliary tasks; and (ii) Combining auxiliary tasks into a single coherent loss. We propose a novel framework, \textit{AuxiLearn}, that targets both challenges, based on implicit differentiation. First, when useful auxiliaries are known, we propose learning a network that combines all losses into a single coherent objective function. This network can learn \textit{non-linear} interactions between auxiliary tasks. Second, when no useful auxiliary task is known, we describe how to learn a network that generates a meaningful, novel auxiliary task. We evaluate AuxiLearn in a series of tasks and domains, including image segmentation and learning with attributes. We find that AuxiLearn consistently improves accuracy compared with competing methods.

Authors

Haggai Maron

4 papers

Aviv Navon

3 papers

Ethan Fetaya

5 papers

TL;DR

Abstract

Authors

References63 items

Self-Supervised Learning for Domain Adaptation on Point Clouds

Improving Semantic Analysis on Point Clouds via Auxiliary Supervision of Local Geometric Priors

Optimizing Millions of Hyperparameters by Implicit Differentiation

Learning to Navigate

Unsupervised Multi-Task Feature Learning on Point Clouds

Meta-Learning with Implicit Gradients

Which Tasks Should Be Learned Together in Multi-task Learning?

Scaling and Benchmarking Self-Supervised Visual Representation Learning

Self-Supervised Visual Feature Learning With Deep Neural Networks: A Survey

Self-Supervised Generalisation with Meta Auxiliary Learning

Self-Supervised Deep Learning on Point Clouds by Reconstructing Space

Multi-Task Learning as Multi-Objective Optimization

Adapting Auxiliary Losses Using Gradient Similarity

End-To-End Multi-Task Learning With Attention

Reviving and Improving Recurrent Back-Propagation

On the Optimization of Deep Networks: Implicit Acceleration by Overparameterization

Unsupervised Representation Learning by Predicting Image Rotations

Learning Longer-term Dependencies in RNNs with Auxiliary Losses

Dynamic Graph CNN for Learning on Point Clouds

GradNorm: Gradient Normalization for Adaptive Loss Balancing in Deep Multitask Networks

Multi-task Self-Supervised Visual Learning

A Survey on Multi-Task Learning

An Overview of Multi-Task Learning in Deep Neural Networks

Transfer Learning for Neural Semantic Parsing

Multi-task Learning Using Uncertainty to Weigh Losses for Scene Geometry and Semantics

Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks

PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation

A scalable active framework for region annotation in 3D shape collections

Reinforcement Learning with Unsupervised Auxiliary Tasks

Matching Networks for One Shot Learning

Joint Semantic Segmentation and Depth Estimation with Deep Convolutional Networks

The Cityscapes Dataset for Semantic Urban Scene Understanding

Unsupervised Learning of Visual Representations by Solving Jigsaw Puzzles

Weight Normalization: A Simple Reparameterization to Accelerate Training of Deep Neural Networks

Hyperparameter optimization with approximate gradient

Deep Residual Learning for Image Recognition

ShapeNet: An Information-Rich 3D Model Repository

Scalable Gradient-Based Tuning of Continuous Regularization Hyperparameters

SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation

Unsupervised Visual Representation Learning by Context Prediction

Adam: A Method for Stochastic Optimization

Predicting Depth, Surface Normals and Semantic Labels with a Common Multi-scale Convolutional Architecture

Unsupervised Domain Adaptation by Backpropagation

Facial Landmark Detection by Deep Multi-task Learning

Very Deep Convolutional Networks for Large-Scale Image Recognition

Exact solutions to the nonlinear dynamics of learning in deep linear neural networks

3D Object Representations for Fine-Grained Categorization

Indoor Semantic Segmentation using depth information

Indoor Segmentation and Support Inference from RGBD Images

The conference paper

The Caltech-UCSD Birds-200-2011 Dataset

Caltech-UCSD Birds 200

ImageNet: A large-scale hierarchical image database

Efficient multiple hyperparameter learning for log-linear models

Gradient-Based Optimization of Hyperparameters

Design and regularization of neural networks: the optimal use of a validation set

Care of aged doctors

Adaptive Auxiliary Task Weighting for Reinforcement Learning

An Overview of Multi-task Learning

Deep Variational Information Bottleneck

Reading Digits in Natural Images with Unsupervised Feature Learning

Learning Multiple Layers of Features from Tiny Images

Gradient-based learning applied to document recognition

Field of Study

Journal Information

Name

Volume

Venue Information

Name

Type

URL

Alternate Names