What is the Effect of Importance Weighting in Deep Learning? (2018-12-08T00:00:00.000000Z)

TL;DR

The surprising finding that while importance weighting impacts models early in training, its effect diminishes over successive epochs is presented.

Abstract

Importance-weighted risk minimization is a key ingredient in many machine learning algorithms for causal inference, domain adaptation, class imbalance, and off-policy reinforcement learning. While the effect of importance weighting is well-characterized for low-capacity misspecified models, little is known about how it impacts over-parameterized, deep neural networks. This work is inspired by recent theoretical results showing that on (linearly) separable data, deep linear networks optimized by SGD learn weight-agnostic solutions, prompting us to ask, for realistic deep networks, for which many practical datasets are separable, what is the effect of importance weighting? We present the surprising finding that while importance weighting impacts models early in training, its effect diminishes over successive epochs. Moreover, while L2 regularization and batch normalization (but not dropout), restore some of the impact of importance weighting, they express the effect via (seemingly) the wrong abstraction: why should practitioners tweak the L2 regularization, and by how much, to produce the correct weighting effect? Our experiments confirm these findings across a range of architectures and datasets.

Authors

Zachary Chase Lipton

13 papers

Jonathon Byrd

2 papers

TL;DR

Abstract

Authors

References38 items

Regularized Learning for Domain Adaptation under Label Shifts

Addressing Sample Inefficiency and Reward Bias in Inverse Reinforcement Learning

Implicit Bias of Gradient Descent on Linear Convolutional Networks

Deep Learning with Logged Bandit Feedback

Detecting and Correcting for Label Shift with Black Box Predictors

Learning From Noisy Singly-labeled Data

The Implicit Bias of Gradient Descent on Separable Data

Teacher–Student Curriculum Learning

Understanding deep learning requires rethinking generalization

Estimating individual treatment effect: generalization bounds and algorithms

TSC-DL: Unsupervised trajectory segmentation of multi-modal surgical demonstrations with Deep Learning

Deep Residual Learning for Image Recognition

Prioritized Experience Replay

Importance Weighted Autoencoders

Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift

Self-Paced Curriculum Learning

Adam: A Method for Stochastic Optimization

Weighted importance sampling for off-policy learning with linear function approximation

Probabilistic Graphical Models - Principles and Techniques

Curriculum learning

Covariate Shift by Kernel Mean Matching

Importance weighted active learning

The Foundations of Cost-Sensitive Learning

Improving predictive inference under covariate shift by weighting the log-likelihood function

Eligibility Traces for Off-Policy Policy Evaluation

The central role of the propensity score in observational studies for causal effects

Simulation and the Monte Carlo method

Understanding

Methods of Reducing Sample Size in Monte Carlo Computations

A Generalization of Sampling Without Replacement from a Finite Universe

BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding

Pytorch pretrained bert

Counterfactual risk minimization: Learning from logged bandit feedback

Dropout: a simple way to prevent neural networks from overfitting

Active Learning Literature Survey

Automatically Constructing a Corpus of Sentential Paraphrases

Random images. (d) Classification ratios after training on cat/dog images

We replicate our results across a variety of networks, tasks, and datasets

Field of Study

Venue Information

Name

Type

URL

Alternate Names