Markov-Lipschitz Deep Learning (2020-06-15T00:00:00.000000Z)

TL;DR

This work proposes a novel framework, called Markov-Lipschitz deep learning (MLDL), to tackle geometric deterioration caused by collapse, twisting, or crossing in vector-based neural network transformations for manifold-based representation learning and manifold data generation.

Abstract

We propose a novel framework, called Markov-Lipschitz deep learning (MLDL), to tackle geometric deterioration caused by collapse, twisting, or crossing in vector-based neural network transformations for manifold-based representation learning and manifold data generation. A prior constraint, called locally isometric smoothness (LIS), is imposed across-layers and encoded into a Markov random field (MRF)-Gibbs distribution. This leads to the best possible solutions for local geometry preservation and robustness as measured by locally geometric distortion and locally bi-Lipschitz continuity. Consequently, the layer-wise vector transformations are enhanced into well-behaved, LIS-constrained metric homeomorphisms. Extensive experiments, comparisons, and ablation study demonstrate significant advantages of MLDL for manifold learning and manifold data generation. MLDL is general enough to enhance any vector transformation-based networks. The code is available at this https URL.

Authors

S. Li

18 papers

Lirong Wu

2 papers

Zelin Zhang

1 papers

TL;DR

Abstract

Authors

References43 items

Lipschitz constant estimation of Neural Networks via sparse polynomial optimization

Graph Convolutional Networks Meet Markov Random Fields: Semi-Supervised Community Detection in Attribute Networks

Topological Autoencoders

GMNN: Graph Markov Neural Networks

Lipschitz Generative Adversarial Nets

Certified Adversarial Robustness via Randomized Smoothing

Efficient Neural Network Robustness Certification with General Activation Functions

Sorting out Lipschitz function approximation

TopoReg: A Topological Regularizer for Classifiers

Lipschitz regularity of deep neural networks: analysis and efficient estimation

Deep Generative Markov State Models

Markov Chain Neural Networks

Spectral Normalization for Generative Adversarial Networks

UMAP: Uniform Manifold Approximation and Projection for Dimension Reduction

Evaluating the Robustness of Neural Networks: An Extreme Value Theory Approach

Deep Learning with Topological Signatures

Spectrally-normalized margin bounds for neural networks

Inductive Representation Learning on Large Graphs

Improved Training of Wasserstein GANs

Loss-Sensitive Generative Adversarial Networks on Lipschitz Densities

Geometric Deep Learning: Going beyond Euclidean data

Auto-Encoding Variational Bayes

Testing the Manifold Hypothesis

Topological Data Analysis

Scikit-learn: Machine Learning in Python

Local Multidimensional Scaling for Nonlinear Dimension Reduction, Graph Drawing, and Proximity Analysis

Iterative Non-linear Dimensionality Reduction with Manifold Sculpting

MLLE: Modified Locally Linear Embedding Using Multiple Weights

Laplacian Eigenmaps for Dimensionality Reduction and Data Representation

Hessian eigenmaps: Locally linear embedding techniques for high-dimensional data

Nonlinear dimensionality reduction by locally linear embedding.

A global geometric framework for nonlinear dimensionality reduction.

Markov Random Field Modeling in Computer Vision

Stochastic Relaxation, Gibbs Distributions, and the Bayesian Restoration of Images

Principles of Riemannian Geometry in Neural Networks

Nearly Isometric Embedding by Relaxation

This PDF file includes: Materials and Methods

Visualizing Data using t-SNE

Reducing the Dimensionality of Data with Neural

Spatial Interaction and the Statistical Analysis of Lattice Systems

Markov fields on finite graphs and lattices

Kaiming He, and Saining Xie 2

Trust (Trustworthiness) measures how well the k nearest neighbors of a point are preserved when going from space X (l) to space X (l

Field of Study

Journal Information

Name

Volume

Venue Information

Name

Type

URL

Alternate Names