Home Research Papers Datasets State of the Art Pricing

Discover, visualize, and connect AI research papers. Explore the latest trends and insights in artificial intelligence research.

Product

Home
Research Papers
About

Support

Contact
Terms of Service
Privacy Policy

computer-code-1

Low-rank compression

3260 papers • 126 benchmarks • 313 datasets

This task has no description! Would you like to contribute one?

(Image credit: Papersgraph)

Benchmarks

These leaderboards are used to track progress in low-rank-compression-1

Trend

Dataset

Best Model

Actions

No benchmarks available.

Libraries

Use these libraries to find low-rank-compression-1 models and implementations

UCMerced-ML/LC-model-compression

3 papers 62

Datasets

No datasets available.

Subtasks

No subtasks available.

Most implemented papers

Domain-Adaptive Deep Network Compression

Joost van de Weijer, Andrew D. Bagdanov, J. Álvarez, Marc Masana, Luis Herranz•Sun Sep 03 2017

The Domain Adaptive Low Rank (DALR) method significantly outperforms existing low-rank compression techniques and takes into account the target domain, so it can more optimally remove the redundancy in the weights.

Content

Introduction Benchmarks Datasets Subtasks Libraries Papers

Paper Graph

Compressing Neural Networks: Towards Determining the Optimal Layer-wise Decomposition

Lucas Liebenwein, Alaa Maalouf, O. Gal, Dan Feldman, Daniela Rus•Thu Jul 22 2021

We present a novel global compression framework for deep neural networks that automatically analyzes each layer to identify the optimal per-layer compression ratio, while simultaneously achieving the desired overall compression. Our algorithm hinges on the idea of compressing each convolutional (or fully-connected) layer by slicing its channels into multiple groups and decomposing each group via low-rank decomposition. At the core of our algorithm is the derivation of layer-wise error bounds from the Eckart Young Mirsky theorem. We then leverage these bounds to frame the compression problem as an optimization problem where we wish to minimize the maximum compression error across layers and propose an efficient algorithm towards a solution. Our experiments indicate that our method outperforms existing low-rank compression approaches across a wide range of networks and data sets. We believe that our results open up new avenues for future research into the global performance-size trade-offs of modern neural networks. Our code is available at https://github.com/lucaslie/torchprune.

54 0

Paper Graph

Decomposable-Net: Scalable Low-Rank Compression for Neural Networks

Taiji Suzuki, A. Yaguchi, Shuhei Nitta, Y. Sakata, A. Tanizawa•Mon Oct 28 2019

A novel method, Decomposable-Net (the network decomposable in any size), which allows flexible changes to model size without retraining and introduces a simple criterion for rank selection that effectively suppresses approximation error.

10 0

Paper Graph

A flexible, extensible software framework for model compression based on the LC algorithm

Yerlan Idelbayev, Miguel 'A. Carreira-Perpin'an•Thu May 14 2020

A software framework that allows a user to compress a neural network or other machine learning model using different compression schemes with minimal effort, and the compressed model is competitive in terms of prediction accuracy and compression ratio with other algorithms.

10 0

Paper Graph

Low-Rank Compression of Neural Nets: Learning the Rank of Each Layer

Yerlan Idelbayev, M. A. Carreira-Perpiñán•Sun May 31 2020

Neural net compression can be achieved by approximating each layer's weight matrix by a low-rank matrix. The real difficulty in doing this is not in training the resulting neural net (made up of one low-rank matrix per layer), but in determining what the optimal rank of each layer is—effectively, an architecture search problem with one hyperparameter per layer. We show that, with a suitable formulation, this problem is amenable to a mixed discrete-continuous optimization jointly over the ranks and over the matrix elements, and give a corresponding algorithm. We show that this indeed can select ranks much better than existing approaches, making low-rank compression much more attractive than previously thought. For example, we can make a VGG network faster than a ResNet and with nearly the same classification error.

181 0

Paper Graph

Model compression as constrained optimization, with application to neural nets. Part V: combining compressions

Yerlan Idelbayev, Miguel 'A. Carreira-Perpin'an•Thu Jul 08 2021

experimentally with deep neural nets, it is observed that 1) the authors can find significantly better models in the error-compression space, indicating that different compression types have complementary benefits, and 2) the best type of combination depends exquisitely on the type of neural net.

6 0

Paper Graph

Compact Model Training by Low-Rank Projection With Energy Transfer

Xiaofen Xing, Xiangmin Xu, K. Guo, Zhenquan Lin, Canyang Chen, Fang Liu•Mon Apr 11 2022

A new training method, low-rank projection with energy transfer (LRPET), that trains low-rank compressed networks from scratch and achieves competitive performance and combines LRPET with quantization and hashing methods and achieves even better compression than the original single method.

8 0

Paper Graph

TT-NF: Tensor Train Neural Fields

L. Gool, K. Schindler, Anton Obukhov, Christos Sakaridis, Mikhail (Misha) Usvyatsov•Thu Sep 29 2022

This paper introduces a novel low-rank representation termed Tensor Train Neural Fields (TT-NF), a TT parameterization of the neural field, trained with backpropagation to minimize a non-convex objective for learning neural fields on dense regular grids and efficient methods for sampling from them.

7 0

Paper Graph

Adding a benchmark result helps the community track progress.

Low-rank compression | State-of-the-Art