Input Convex Neural Networks (2016-09-01T00:00:00.000000Z)

TL;DR

This paper presents the input convex neural network architecture, which are scalar-valued (potentially deep) neural networks with constraints on the network parameters such that the output of the network is a convex function of (some of) the inputs.

Abstract

This paper presents the input convex neural network architecture. These are scalar-valued (potentially deep) neural networks with constraints on the network parameters such that the output of the network is a convex function of (some of) the inputs. The networks allow for efficient inference via optimization over some inputs to the network given others, and can be applied to settings including structured prediction, data imputation, reinforcement learning, and others. In this paper we lay the basic groundwork for these models, proposing methods for inference, optimization and learning, and analyze their representational power. We show that many existing neural network architectures can be made input-convex with a minor modification, and develop specialized optimization algorithms tailored to this setting. Finally, we highlight the performance of the methods on multi-label prediction, image completion, and reinforcement learning problems, where we show improvement over the existing state of the art in many cases.

Authors

J. Z. Kolter

11 papers

Lei Xu

3 papers

Brandon Amos

6 papers

TL;DR

Abstract

Authors

References49 items

Densely Connected Convolutional Networks

TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems

Continuous Deep Q-Learning with Model-based Acceleration

Deep Residual Learning for Image Recognition

Structured Prediction Energy Networks

Guide to NumPy

Continuous control with deep reinforcement learning

Human-level control through deep reinforcement learning

Conditional Random Fields as Recurrent Neural Networks

Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift

Adam: A Method for Stochastic Optimization

Going deeper with convolutions

Very Deep Convolutional Networks for Large-Scale Image Recognition

Learning Deep Structured Models

Playing Atari with Deep Reinforcement Learning

MuJoCo: A physics engine for model-based control

ImageNet classification with deep convolutional neural networks

An extensive experimental comparison of methods for multi-label learning

Generic Methods for Optimization-Based Modeling

Sum-product networks: A new deep architecture

Adaptive Subgradient Methods for Online Learning and Stochastic Optimization

Scikit-learn: Machine Learning in Python

Rectified Linear Units Improve Restricted Boltzmann Machines

Conditional Neural Fields

Probabilistic Graphical Models: Principles and Techniques - Adaptive Computation and Machine Learning

Convex piecewise-linear fitting

Bundle Methods for Machine Learning

(Approximate) Subgradient Methods for Structured Prediction

Large Margin Methods for Structured and Interdependent Output Variables

Learning structured prediction models: a large margin approach

Python Reference Manual

Nonmonotone Spectral Projected Gradient Methods on Convex Sets

Primal-Dual Interior-Point Methods

Parameterisation of a stochastic model for human face identification

Globally Trained Handwritten Word Recognizer Using Spatial Representation, Convolutional Neural Networks, and Hidden Markov Models

Reverse TDNN: An Architecture For Trajectory Generation

Learning representations by back-propagating errors

Projected Newton methods for optimization problems with simple constraints

On the Theory of the Brownian Motion

GENERATIVE ADVERSARIAL NETS

Jie Tang, and Wojciech Zaremba. Openai gym

M ULAN : A Java Library for Multi-Label Learning

Multilabel Text Classification for Automated Tag Suggestion

A Tutorial on Energy-Based Learning

Parameterisation of a Stochastic Model for Human Face Identi cation

Reinforcement Learning for Robots Using Neural Networks

Two-Point Step Size Gradient Methods

On the theory of brownian motion

Some methods of speeding up the convergence of iteration methods

Field of Study

Venue Information

Name

Type

URL

Alternate Names