The Causal-Neural Connection: Expressiveness, Learnability, and Inference (2021-07-02T00:00:00.000000Z)

TL;DR

Leveraging the neural toolbox, an algorithm is developed that is both sufficient and necessary to determine whether a causal effect can be learned from data and then estimates the effect whenever identifiability holds (causal estimation).

Abstract

One of the central elements of any causal inference is an object called structural causal model (SCM), which represents a collection of mechanisms and exogenous sources of random variation of the system under investigation (Pearl, 2000). An important property of many kinds of neural networks is universal approximability: the ability to approximate any function to arbitrary precision. Given this property, one may be tempted to surmise that a collection of neural nets is capable of learning any SCM by training on data generated by that SCM. In this paper, we show this is not the case by disentangling the notions of expressivity and learnability. Specifically, we show that the causal hierarchy theorem (Thm. 1, Bareinboim et al., 2020), which describes the limits of what can be learned from data, still holds for neural models. For instance, an arbitrarily complex and expressive neural net is unable to predict the effects of interventions given observational data alone. Given this result, we introduce a special type of SCM called a neural causal model (NCM), and formalize a new type of inductive bias to encode structural constraints necessary for performing causal inferences. Building on this new class of models, we focus on solving two canonical tasks found in the literature known as causal identification and estimation. Leveraging the neural toolbox, we develop an algorithm that is both sufficient and necessary to determine whether a causal effect can be learned from data (i.e., causal identifiability); it then estimates the effect whenever identifiability holds (causal estimation). Simulations corroborate the proposed approach.

Authors

Yoshua Bengio

69 papers

K. Xia

1 papers

Kai-Zhan Lee

1 papers

TL;DR

Abstract

Authors

References75 items

On Pearl’s Hierarchy and the Foundations of Causal Inference

Generative Adversarial Networks

Estimating Identifiable Causal Effects through Double Machine Learning

Semiparametric Counterfactual Density Estimation

A Framework For Contrastive Self-Supervised Learning And Designing A New Approach

Differentiable Causal Discovery from Interventional Data

Estimating Causal Effects Using Weighting-Based Estimators

Generalization Bounds and Representation Learning for Estimation of Potential Outcomes and Causal Effects

Adapting Neural Networks for the Estimation of Treatment Effects

Causal Identification under Markov Equivalence: Completeness Results

Adversarial balancing-based representation learning for causal effect inference with observational data

A Meta-Transfer Objective for Learning to Disentangle Causal Mechanisms

Structural Causal Bandits: Where to Intervene?

A Survey of Learning Causality with Data

Causal Identification under Markov Equivalence

DeepMatch: Balancing Deep Covariate Representations for Causal Inference Using Adversarial Training

GANITE: Estimation of Individualized Treatment Effects using Generative Adversarial Nets

Elements of Causal Inference: Foundations and Learning Algorithms

Decoupled Weight Decay Regularization

Automatic differentiation in PyTorch

The Expressive Power of Neural Networks: A View from the Width

CausalGAN: Learning Causal Implicit Generative Models with Adversarial Training

Counterfactual Data-Fusion for Online Reinforcement Learners

Causal Effect Inference with Deep Latent-Variable Models

SGDR: Stochastic Gradient Descent with Warm Restarts

Causal inference and the data-fusion problem

Complete Graphical Characterization and Construction of Adjustment Sets in Markov Equivalence Classes of Ancestral Graphs

Estimating individual treatment effect: generalization bounds and algorithms

Learning Representations for Counterfactual Inference

XGBoost: A Scalable Tree Boosting System

A Kernelized Stein Discrepancy for Goodness-of-fit Tests

Variational Inference: A Review for Statisticians

Bandits with Unobserved Confounders: A Causal Approach

Variational Inference with Normalizing Flows

MADE: Masked Autoencoder for Distribution Estimation

Adam: A Method for Stochastic Optimization

Towards End-To-End Speech Recognition with Recurrent Neural Networks

Auto-Encoding Variational Bayes

Playing Atari with Deep Reinforcement Learning

ImageNet classification with deep convolutional neural networks

Local Characterizations of Causal Bayesian Networks

On the completeness of orientation rules for causal discovery in the presence of latent confounders and selection bias

A Kernel Method for the Two-Sample-Problem

Causation, Prediction, and Search

A general identification condition for causal effects

Causal diagrams for empirical research

The Probability Integral Transform and Related Results

Counterfactual Probabilities: Computational Methods, Bounds and Applications

Original Contribution: Multilayer feedforward networks with a nonpolynomial activation function can approximate any function

Approximation capabilities of multilayer feedforward networks

Approximation by superpositions of a sigmoidal function

The Book of Why

Partial Identification of Counterfactual Distributions

The Causal-Neural Connection: Expressiveness, Learnability, Inference

Non-Parametric Methods for Partial Identiﬁcation of Causal Effects

Characterizing Optimal Mixed Policies: Where to Intervene and What to Observe

Causal Discovery from Soft Interventions with Unknown Targets: Characterization and Learning

General Transportability of Soft Interventions: Completeness Results

Learning Causal Effects via Weighted Empirical Risk Minimization

Characterization and Learning of Causal Graphs with Latent Variables from Soft Interventions

General Identifiability with Arbitrary Surrogate Experiments

Representation Learning for Treatment Effect Estimation from Observational Data

pytorch-made. https://github.com/karpathy/pytorch-made [Source Code

Bareinboim

Matching on Balanced Nonlinear Representations for Treatment Effects Estimation

Experimental Design for Learning Causal Graphs with Latent Variables

Graph inference datasets. Replication Data for: "Learning Functional Causal Models with Generative Neural Networks"

Deep Learning

Causality : Models , Reasoning , and Inference

Statistical Inference, pages 54–55

Reinforcement Learning: An Introduction

A clinical trial of the effects of dietary patterns on blood pressure. DASH Collaborative Research Group.

Probabilistic reasoning in intelligent systems

Statistical Theory of Extreme Values and Some Practical Applications : A Series of Lectures

= 1)) = p 0 + p 3 . In other words, since P M (Y | do(X)) matches for all models from this class that are L 1 and G-consistent

Field of Study