Toward Causal Representation Learning (2021-05-01T00:00:00.000000Z)

TL;DR

Fundamental concepts of causal inference are reviewed and related to crucial open problems of machine learning, including transfer and generalization, thereby assaying how causality can contribute to modern machine learning research.

Abstract

The two fields of machine learning and graphical causality arose and are developed separately. However, there is, now, cross-pollination and increasing interest in both fields to benefit from the advances of the other. In this article, we review fundamental concepts of causal inference and relate them to crucial open problems of machine learning, including transfer and generalization, thereby assaying how causality can contribute to modern machine learning research. This also applies in the opposite direction: we note that most work in causality starts from the premise that the causal variables are given. A central problem for AI and causality is, thus, causal representation learning, that is, the discovery of high-level causal variables from low-level observations. Finally, we delineate some implications of causality for machine learning and propose key research areas at the intersection of both communities.

Authors

Bernhard Schölkopf

1 papers

Francesco Locatello

5 papers

Stefan Bauer

3 papers

TL;DR

Abstract

Authors

References287 items

The Bottleneck

How we learn: why brains learn better than any machine … for now

Towards Causal Representation Learning

Representations

Sample-Efficient Reinforcement Learning via Counterfactual-Based Data Augmentation

On the Binding Problem in Artificial Neural Networks

On the Transfer of Disentangled Representations in Realistic Settings

On the Fairness of Causal Algorithmic Recourse

CausalWorld: A Robotic Manipulation Benchmark for Causal Structure and Transfer Learning

Learning explanations that are hard to vary

Improving the accuracy of medical diagnosis with causal machine learning

On Robustness and Transferability of Convolutional Neural Networks

Generative Pretraining From Pixels

Object Files and Schemata: Factorizing Declarative and Procedural Knowledge in Dynamical Systems

Object-Centric Learning with Slot Attention

Hidden Markov Nonlinear ICA: Unsupervised Learning from Nonstationary Time Series

Is Independence all you need? On the Generalization of Representations Learned from Correlated Data

Structural Autoencoders Improve Representations for Generation and Transfer

Bootstrap Your Own Latent: A New Approach to Self-Supervised Learning

Algorithmic recourse under imperfect causal knowledge: a probabilistic approach

Language Models are Few-Shot Learners

An Analysis of the Adaptation Speed of Causal Models

Simpson's Paradox in COVID-19 Case Fatality Rates: A Mediation Analysis of Age-Related Causal Effects

Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problems

Jukebox: A Generative Model for Music

Towards causal generative scene models via competition of experts

A Human-Centered Evaluation of a Deep Learning System Deployed in Clinics for the Detection of Diabetic Retinopathy

A theory of independent mechanisms for extrapolation in generative models

Counterfactual Multi-Agent Reinforcement Learning with Graph Convolution Communication

Learning to Simulate Complex Physics with Graph Networks

A Simple Framework for Contrastive Learning of Visual Representations

Weakly-Supervised Disentanglement Without Compromises

Causally Correct Partial Models for Reinforcement Learning

Causal Models for Dynamical Systems

SPACE: Unsupervised Object-Oriented Scene Representation via Spatial Attention and Decomposition

Big Transfer (BiT): General Visual Representation Learning

Learning Representations by Maximizing Mutual Information in Variational Autoencoders

Dota 2 with Large Scale Deep Reinforcement Learning

Self-Supervised Learning of Video-Induced Visual Invariances

Causality for Machine Learning

Mastering Atari, Go, chess and shogi by planning with a learned model

Momentum Contrast for Unsupervised Visual Representation Learning

Grandmaster level in StarCraft II using multi-agent reinforcement learning

Semi-Supervised Learning

Weakly Supervised Disentanglement with Guarantees

Solving Rubik's Cube with a Robot Hand

CLEVRER: CoLlision Events for Video REpresentation and Reasoning

Learning Neural Causal Models from Unknown Interventions

Recurrent Independent Mechanisms

Learning First-Order Symbolic Representations for Planning from the Structure of the State Space

Water vapour in the atmosphere of the habitable-zone eight-Earth-mass planet K2-18 b

Water Vapor on the Habitable-Zone Exoplanet K2-18b

Benchmarking Robustness in Object Detection: Autonomous Driving when Winter is Coming

Invariant Risk Minimization

Causal Regularization

Shaping Belief States with Generative Environment Models for RL

Unsupervised Learning of Object Keypoints for Perception and Control

On the Transfer of Inductive Bias from Simulation to the Real World: a New Disentanglement Dataset

Do Image Classifiers Generalize Across Time?

AutoAugment: Learning Augmentation Strategies From Data

On the Fairness of Disentangled Representations

Are Disentangled Representations Helpful for Abstract Visual Reasoning?

Semi-supervised learning, causality, and the conditional cluster assumption

COBRA: Data-Efficient Model-Based RL through Unsupervised Object Discovery and Curiosity-Driven Exploration

The Incomplete Rosetta Stone problem: Identifiability results for Multi-view Nonlinear ICA

Fast AutoAugment

Making Convolutional Networks Shift-Invariant Again

Using Videos to Evaluate Image Model Robustness

Structured agents for physical construction

Benchmarking Neural Network Robustness to Common Corruptions and Perturbations

Causal Discovery from Heterogeneous/Nonstationary Data

Multi-Object Representation Learning with Iterative Variational Inference

Stochastic Prediction of Multi-Agent Interactions from Partial Observations

Time Series Deconfounder: Estimating Treatment Effects over Time in the Presence of Hidden Confounders

A Meta-Transfer Objective for Learning to Disentangle Causal Mechanisms

Causal Reasoning from Meta-reinforcement Learning