Implicit Neural Representations with Periodic Activation Functions (2020-06-17T00:00:00.000000Z)

TL;DR

This work proposes to leverage periodic activation functions for implicit neural representations and demonstrates that these networks, dubbed sinusoidal representation networks or Sirens, are ideally suited for representing complex natural signals and their derivatives.

Abstract

Implicitly defined, continuous, differentiable signal representations parameterized by neural networks have emerged as a powerful paradigm, offering many possible benefits over conventional representations. However, current network architectures for such implicit neural representations are incapable of modeling signals with fine detail, and fail to represent a signal's spatial and temporal derivatives, despite the fact that these are essential to many physical signals defined implicitly as the solution to partial differential equations. We propose to leverage periodic activation functions for implicit neural representations and demonstrate that these networks, dubbed sinusoidal representation networks or Sirens, are ideally suited for representing complex natural signals and their derivatives. We analyze Siren activation statistics to propose a principled initialization scheme and demonstrate the representation of images, wavefields, video, sound, and their derivatives. Further, we show how Sirens can be leveraged to solve challenging boundary value problems, such as particular Eikonal equations (yielding signed distance functions), the Poisson equation, and the Helmholtz and wave equations. Lastly, we combine Sirens with hypernetworks to learn priors over the space of Siren functions.

Authors

V. Sitzmann

5 papers

Gordon Wetzstein

8 papers

Julien N. P. Martel

2 papers

TL;DR

Abstract

Authors

References103 items

Semantic Implicit Neural Scene Representations With Semi-Supervised Training

Fourier Features Let Networks Learn High Frequency Functions in Low Dimensional Domains

State of the Art on Neural Rendering

Inferring Semantic Information with 3D Neural Scene Representations

Deep Local Shapes: Learning Local SDF Priors for Detailed 3D Reconstruction

Local Implicit Grid Representations for 3D Scenes

Convolutional Occupancy Networks

Implicit Geometric Regularization for Learning Shapes

Differentiable Volumetric Rendering: Learning Implicit 3D Representations Without 3D Supervision

Deep Structured Implicit Functions

SAL: Sign Agnostic Learning of Shapes From Raw Data

Implicit Surface Representations As Layers in Neural Networks

Occupancy Flow: 4D Reconstruction by Learning Particle Dynamics

StructureFlow: Image Inpainting via Structure-Aware Appearance Flow

Scene Representation Networks: Continuous 3D-Structure-Aware Neural Scene Representations

Coherent Semantic Attention for Image Inpainting

Texture Fields: Learning Texture Representations in Function Space

PIFu: Pixel-Aligned Implicit Function for High-Resolution Clothed Human Digitization

Learning Shape Templates With Structured Implicit Functions

Unsupervised Deep Learning Algorithm for PDE-based Forward and Inverse Problems

Hypernetwork Functional Image Representation

Fourier neural networks: A comparative study

Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations

Real Variables: An Introduction To The Theory of Functions

Attentive Neural Processes

DeepSDF: Learning Continuous Signed Distance Functions for Shape Representation

Occupancy Networks: Learning 3D Reconstruction in Function Space

Learning Implicit Fields for Generative Shape Modeling

Improving full-waveform inversion by wavefield reconstruction with the alternating direction method of multipliers

Differentiable Image Parameterizations

Conditional Neural Processes

Neural Tangent Kernel: Convergence and Generalization in Neural Networks

Neural Ordinary Differential Equations

Learning Equations for Extrapolation and Control

Neural scene representation and rendering

Free-Form Image Inpainting With Gated Convolution

Image Inpainting for Irregular Holes Using Partial Convolutions

Adversarial Audio Synthesis

Deep Hidden Physics Models: Deep Learning of Nonlinear Partial Differential Equations

Natural TTS Synthesis by Conditioning Wavenet on MEL Spectrogram Predictions

Deep Image Prior

A unified deep artificial neural network approach to partial differential equations in complex geometries

Audio Set Classification with Attention Model: A Probabilistic Perspective

Automatic differentiation in PyTorch

DGM: A deep learning algorithm for solving partial differential equations

Globally and locally consistent image completion

Taming the waves: sine as activation function in deep neural networks

SampleRNN: An Unconditional End-to-End Neural Audio Generation Model

CNN architectures for large-scale audio classification

HyperNetworks

WaveNet: A Generative Model for Raw Audio

Multistability of Recurrent Neural Networks With Nonmonotonic Activation Functions and Mixed Time Delays

Neural Network with Unbounded Activation Functions is Universal Approximator

Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification

Deep Learning Face Attributes in the Wild

A benchmark for RGB-D visual odometry, 3D reconstruction and SLAM

Approximation by neural networks with scattered data

Mitigating local minima in full-waveform inversion by expanding the search space

Rudin-Osher-Fatemi Total Variation Denoising using Split Bregman

An optimal 25-point finite difference scheme for the Helmholtz equation with PML

Distributed Optimization and Statistical Learning via the Alternating Direction Method of Multipliers

Mathematical problems in image processing - partial differential equations and the calculus of variations

Understanding the difficulty of training deep feedforward neural networks

k-Wave: MATLAB toolbox for the simulation and reconstruction of photoacoustic wave fields.

Unsupervised feature learning for audio classification using convolutional deep belief networks

PatchMatch: a randomized correspondence algorithm for structural image editing

A Logistic Approximation to The Cumulative Normal Distribution

Unsupervised learning of auditory filter banks using non-negative matrix factorisation

Random Features for Large-Scale Kernel Machines

Compositional pattern producing networks: A novel abstraction of development

Texture optimization for example-based synthesis

Modeling Very Oscillating Signals. Application to Image Processing

Noise removal using fourth-order partial differential equation with applications to medical magnetic resonance images in space and time

Poisson image editing

Approximation of function and its derivatives using radial basis function networks

Object recognition supported by user interaction for service robots