Deep Sets (2017-03-10T00:00:00.000000Z)

TL;DR

The main theorem characterizes the permutation invariant objective functions and provides a family of functions to which any permutation invincible objective function must belong, which enables the design of a deep network architecture that can operate on sets and which can be deployed on a variety of scenarios including both unsupervised and supervised learning tasks.

Abstract

In this paper, we study the problem of designing objective functions for machine learning problems defined on finite \emph{sets}. In contrast to traditional objective functions defined for machine learning problems operating on finite dimensional vectors, the new objective functions we propose are operating on finite sets and are invariant to permutations. Such problems are widespread, ranging from estimation of population statistics \citep{poczos13aistats}, via anomaly detection in piezometer data of embankment dams \citep{Jung15Exploration}, to cosmology \citep{Ntampaka16Dynamical,Ravanbakhsh16ICML1}. Our main theorem characterizes the permutation invariant objective functions and provides a family of functions to which any permutation invariant objective function must belong. This family of functions has a special structure which enables us to design a deep network architecture that can operate on sets and which can be deployed on a variety of scenarios including both unsupervised and supervised learning tasks. We demonstrate the applicability of our method on population statistic estimation, point cloud classification, set expansion, and image tagging.

Authors

M. Zaheer

8 papers

Alex Smola

9 papers

R. Salakhutdinov

32 papers

TL;DR

Abstract

Authors

References54 items

Learning a Probabilistic Latent Space of Object Shapes via 3D Generative-Adversarial Modeling

Generative and Discriminative Voxel Modeling with Convolutional Neural Networks

Estimating Cosmological Parameters from the Dark Matter Distribution

Group Equivariant Convolutional Networks

ShapeNet: An Information-Rich 3D Model Repository

Fast and Accurate Deep Network Learning by Exponential Linear Units (ELUs)

Exploration and evaluation of AR, MPCA and KL anomaly detection techniques to embankment dam piezometer data

Convolutional Networks on Graphs for Learning Molecular Fingerprints

DeepPano: Deep Panoramic Representation for 3-D Shape Recognition

DYNAMICAL MASS MEASUREMENTS OF CONTAMINATED GALAXY CLUSTERS USING MACHINE LEARNING

VoxNet: A 3D Convolutional Neural Network for real-time object recognition

Machine Learning Energies of 2 Million Elpasolite (ABC_{2}D_{6}) Crystals.

Multi-view Convolutional Neural Networks for 3D Shape Recognition

Adam: A Method for Stochastic Optimization

Deep Symmetry Networks

Deep Recursive Neural Networks for Compositionality in Language

Deep Learning Face Attributes in the Wild

Learning Theory for Distribution Regression

3D ShapeNets: A deep representation for volumetric shapes

Microsoft COCO: Common Objects in Context

Spectral Networks and Locally Connected Networks on Graphs

Distributed Representations of Words and Phrases and their Compositionality

Recursive Deep Models for Semantic Compositionality Over a Sentiment Treebank

Fast Image Tagging

Distribution to Distribution Regression

redMaPPer II: X-RAY AND SZ PERFORMANCE BENCHMARKS FOR THE SDSS CATALOG

Distribution-Free Distribution Regression

Domain Generalization via Invariant Feature Representation

Tensor decompositions for learning latent variable models

Learning from Distributions via Support Measure Machines

Support Distribution Machines

3D is here: Point Cloud Library (PCL)

TagProp: Discriminative metric learning in nearest neighbor models for image auto-annotation

A New Baseline for Image Annotation

Inference of population structure using multilocus genotype data: dominant markers and null alleles

Bayesian Sets

Multiple Bernoulli relevance models for image and video annotation

Labeling images with a computer game

Max-Margin Markov Networks

Inference of population structure using multilocus genotype data.

Slicing Through Multicolor Space: Galaxy Redshifts from Broadband Photometry

Interpolation of scattered data: Distance matrices and conditionally positive definite functions

Deep Sets

Deep Learning

2015) can be interpreted as set-pooling where the class-label is invariant to permutation of different

Groups and group convolu-tions

Latent Dirichlet Allocation

Analysis and evaluation of visual information systems performance

Training Invariant Support Vector Machines using Selective Sampling

A mesh reconstruction algorithm driven by an intrinsic property of a point cloud

ensembles, chapitres 1 à 4, volume

El´ements de math´ematiques: th´eorie des ensembles, chapitres 1 `a 4 , volume 1

Galactic astronomy.

IEEE Computer Society. URL

Field of Study