Unsupervised Learning of Neurosymbolic Encoders (2021-07-28T00:00:00.000000Z)

TL;DR

This work integrates modern program synthesis techniques with the variational autoencoding (VAE) framework, in order to learn a neurosymbolic encoder in conjunction with a standard decoder, and shows that this approach offers significantly better separation of meaningful categories than standard VAEs and leads to practical gains on downstream analysis tasks.

Abstract

We present a framework for the unsupervised learning of neurosymbolic encoders, which are encoders obtained by composing neural networks with symbolic programs from a domain-specific language. Our framework naturally incorporates symbolic expert knowledge into the learning process, which leads to more interpretable and factorized latent representations compared to fully neural encoders. We integrate modern program synthesis techniques with the variational autoencoding (VAE) framework, in order to learn a neurosymbolic encoder in conjunction with a standard decoder. The programmatic descriptions from our encoders can benefit many analysis workflows, such as in behavior modeling where interpreting agent actions and movements is important. We evaluate our method on learning latent representations for real-world trajectory data from animal biology and sports analytics. We show that our approach offers significantly better separation of meaningful categories than standard VAEs and leads to practical gains on downstream analysis tasks, such as for behavior classification.

Authors

Jennifer J. Sun

4 papers

A. Kennedy

2 papers

Eric Zhan

1 papers

TL;DR

Abstract

Authors

References79 items

Neurosymbolic Programming for Science

Learning to Synthesize Programs as Interpretable and Generalizable Policies

Mining naturalistic human behaviors in long-term video and neural recordings

The Multi-Agent Behavior Dataset: Mouse Dyadic Social Interactions

A Survey on Neural Network Interpretability

Task Programming: Learning Data Efficient Behavior Representations

The Mouse Action Recognition System (MARS) software pipeline for automated analysis of social behaviors in mice

Learning Lane Graph Representations for Motion Forecasting

Learning Differentiable Programs with Admissible Neural Heuristics

NVAE: A Deep Hierarchical Variational Autoencoder

Disentangled Self-Supervision in Sequential Recommenders

Learning Task-General Representations with Generative Neuro-Symbolic Modeling

DreamCoder: growing generalizable, interpretable knowledge with wake–sleep Bayesian program learning

Identifying behavioral structure from deep variational embeddings of animal motion

VectorNet: Encoding HD Maps and Agent Dynamics From Vectorized Representation

Synthesizing Programmatic Policies that Inductively Generalize

Polarized-VAE: Proximity Based Disentangled Representation Learning for Text Generation

Guided Variational Autoencoder for Disentanglement Learning

Learning Calibratable Policies using Programmatic Style-Consistency

B-SOiD: An Open Source Unsupervised Algorithm for Discovery of Spontaneous Behaviors

Imitation-Projected Programmatic Reinforcement Learning

Unsupervised identification of the internal states that shape natural behavior

Learning to Infer and Execute 3D Shape Programs

Self-Consistent Trajectory Autoencoder: Hierarchical Reinforcement Learning with Trajectory Embeddings

Verifiable Reinforcement Learning via Policy Extraction

Understanding disentangling in β-VAE

Programmatically Interpretable Reinforcement Learning

Joint-VAE: Learning Disentangled Joint Continuous and Discrete Representations

HOUDINI: Lifelong Learning as Program Synthesis

Fast Decoding in Sequence Models using Discrete Latent Variables

Disentangled Sequential Autoencoder

Disentangling by Factorising

Isolating Sources of Disentanglement in VAEs

UMAP: Uniform Manifold Approximation and Projection for Dimension Reduction

Adversarial Information Factorization

Selecting Representative Examples for Program Synthesis

Neural Discrete Representation Learning

Program synthesis using abstraction refinement

Learning to Infer Graphics Programs from Hand-Drawn Images

Factorized Variational Autoencoders for Modeling Audience Reactions to Movies

Towards Synthesizing Complex Programs From Input-Output Examples

Toward Controlled Generation of Text

Learning Hierarchical Features from Generative Models

Variational Lossy Autoencoder

beta-VAE: Learning Basic Visual Concepts with a Constrained Variational Framework

Categorical Reparameterization with Gumbel-Softmax

Learning recurrent representations for hierarchical behavior modeling

Discrete Variational Autoencoders

TerpreT: A Probabilistic Programming Language for Program Induction

InfoGAN: Interpretable Representation Learning by Information Maximizing Generative Adversarial Nets

Adaptive Neural Compilation

Composing graphical models with neural networks for structured representations and fast inference

Mapping Sub-Second Structure in Mouse Behavior

Generating Sentences from a Continuous Space

Adam: A Method for Stochastic Optimization

Toward a Science of Computational Ethology

Semi-supervised Learning with Deep Generative Models

Neural Variational Inference and Learning in Belief Networks

Auto-Encoding Variational Bayes

Distributed Representations of Words and Phrases and their Compositionality

Mapping the stereotyped behaviour of freely moving fruit flies

Social behavior recognition in continuous video

Automating string processing in spreadsheets using input-output examples

Least squares quantization in PCM

Natural Scales in Geographical Patterns

Differentiable Synthesis of Program Architectures

2021b). Here, the neural encoder qφ and decoder pθ are instantiated with recurrent neural networks (RNN)

Neurosymbolic Programming

We use a variation of the VAE where the inputs are trajectory data, called a TVAE Co-Reyes et al

The DSL for basketball contains library functions that compute the speed, acceleration, ﬁnal positions, and distance-to-basket of players and take the maximum, minimum, or average over the players

Learning Representations for Time Series Clustering

Arka Pal, Loic Matthey, Nick Watters, Guillaume Desjardins, and Alexander Lerchner

2017) combines vector quantization with VAEs

An Introduction to Information Retrieval

Visualizing Data using t-SNE

Unsupervised Feature Extraction for Time Series Clustering Using Orthogonal Wavelet Transform