TT-NF: Tensor Train Neural Fields (2022-09-30T00:00:00.000000Z)

TL;DR

This paper introduces a novel low-rank representation termed Tensor Train Neural Fields (TT-NF), a TT parameterization of the neural field, trained with backpropagation to minimize a non-convex objective for learning neural fields on dense regular grids and efficient methods for sampling from them.

Abstract

Learning neural fields has been an active topic in deep learning research, focusing, among other issues, on finding more compact and easy-to-fit representations. In this paper, we introduce a novel low-rank representation termed Tensor Train Neural Fields (TT-NF) for learning neural fields on dense regular grids and efficient methods for sampling from them. Our representation is a TT parameterization of the neural field, trained with backpropagation to minimize a non-convex objective. We analyze the effect of low-rank compression on the downstream task quality metrics in two settings. First, we demonstrate the efficiency of our method in a sandbox task of tensor denoising, which admits comparison with SVD-based schemes designed to minimize reconstruction error. Furthermore, we apply the proposed approach to Neural Radiance Fields, where the low-rank structure of the field corresponding to the best quality can be discovered only through learning.

Authors

L. Gool

68 papers

K. Schindler

19 papers

Anton Obukhov

3 papers

TL;DR

Abstract

Authors

References72 items

K-Planes: Explicit Radiance Fields in Space, Time, and Appearance

HexPlane: A Fast Representation for Dynamic Scenes

tntorch: Tensor Network Learning with PyTorch

TensoRF: Tensorial Radiance Fields

Pix2NeRF: Unsupervised Conditional $\pi$-GAN for Single Image to Neural Radiance Fields Translation

Instant neural graphics primitives with a multiresolution hash encoding

Plenoxels: Radiance Fields without Neural Networks

Implicit Neural Representations for Image Compression

Neural Fields in Visual Computing and Beyond

Direct Voxel Grid Optimization: Super-fast Convergence for Radiance Fields Reconstruction

NeRS: Neural Reflectance Surfaces for Sparse-view 3D Reconstruction in the Wild

Learning Signed Distance Field for Multi-view Surface Reconstruction

QTTNet: Quantized tensor train neural networks for 3D object and video recognition

Cherry-Picking Gradients: Learning Low-Rank Embeddings of Visual Data via Differentiable Cross-Approximation

Tensor Methods in Computer Vision and Deep Learning

Baking Neural Radiance Fields for Real-Time View Synthesis

Mip-NeRF: A Multiscale Representation for Anti-Aliasing Neural Radiance Fields

Spectral Tensor Train Parameterization of Deep Learning Layers

Iterative Power Algorithm for Global Optimization with Quantics Tensor Trains.

TT-TSDF: Memory-Efficient TSDF with Low-Rank Tensor Train Decomposition

Optimal High-Order Tensor SVD via Tensor-Train Orthogonal Iteration

Reparameterizing Convolutions for Incremental Multi-Task Learning without Task Interference

T-Basis: a Compact Representation for Neural Networks

PyTorch: An Imperative Style, High-Performance Deep Learning Library

AI Benchmark: All About Deep Learning on Smartphones in 2019

Learning Filter Basis for Convolutional Neural Network Compression

Fast BEM Solution for 2-D Scattering Problems Using Quantized Tensor-Train Format

T-Net: Parametrizing Fully Convolutional Nets With a Single High-Order Tensor

DeepSDF: Learning Continuous Signed Distance Functions for Shape Representation

opt\_einsum - A Python package for optimizing contraction order for einsum-like expressions

TTHRESH: Tensor Compression for Multidimensional Visual Data

Wide Compression: Tensor Ring Nets

Tensor Train decomposition on TensorFlow (T3F)

QTT-finite-element approximation for multiscale problems I: model problems in one dimension

QTT-finite-element approximation for multiscale problems I: model problems in one dimension

Ultimate tensorization: compressing convolutional and FC layers alike

TensorLy: Tensor Learning in Python

Tensor Contractions with Extended BLAS Kernels on CPU and GPU

Tensorizing Neural Networks

Analysis of tensor approximation for compression-domain volume visualization

Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift

Adam: A Method for Stochastic Optimization

Speeding-up Convolutional Neural Networks Using Fine-tuned CP-Decomposition

State‐of‐the‐Art in Compressed GPU‐Based Direct Volume Rendering

Putting MRFs on a Tensor Train

Speeding up Convolutional Neural Networks with Low Rank Expansions

Computation of extreme eigenvalues in higher dimensions using block tensor train format

Fast Solution of Parabolic Problems in the Tensor Train/Quantized Tensor Train Format with Initial Application to the Fokker-Planck Equation

On manifolds of tensors of fixed TT-rank

Tensor-Train Decomposition

Algebraic Wavelet Transform via Quantics Tensor Train Decomposition

O(dlog N)-Quantics Approximation of N-d Tensors in High-Dimensional Numerical Modeling

Rectified Linear Units Improve Restricted Boltzmann Machines

Approximation of matrices with logarithmic number of parameters

Tensor Decompositions and Applications

Backpropagation Applied to Handwritten Zip Code Recognition

Marching cubes: A high resolution 3D surface construction algorithm

Einops: Clear and Reliable Tensor Manipulations with Einstein-like Notation

descriptor and viewing direction can be learned with a dedicated “shading

Preprint. Under review

Cutlass

“NeRF: Representing scenes as neural radiance ﬁelds for view synthesis,”

Fast linear algebra in cuda

2016) propose a StridedBatchedGemm

Software available from tensorflow.org

“TensorFlow: Large-scale machine learning on heterogeneous systems,”

from the National Technical University of Athens, Athens, Greece

TT-cross approximation for multidimensional arrays

is one of the first functions undergoing heavy optimization on any new-generation computational device, along with memory operations such as permutations themselves

The maximum-volume concept in approximation by low-rank matrices

An Updated Set of Basic Linear Algebra Subprograms Blas

A computer oriented geodetic data base and a new technique in ﬁle sequencing

Field of Study

Journal Information

Name

Page