methodology-3

Neural Network Compression

3260 papers • 126 benchmarks • 313 datasets

This task has no description! Would you like to contribute one?

(Image credit: Papersgraph)

Benchmarks

These leaderboards are used to track progress in neural-network-compression-15

Trend

Dataset

Best Model

Actions

CIFAR-10

Libraries

i

Use these libraries to find neural-network-compression-15 models and implementations

yoshitomo-matsubara/torchdistill

4 papers 1,254

Datasets

CIFAR-10

Subtasks

No subtasks available.

Most implemented papers

Soft Weight-Sharing for Neural Network Compression

M. Welling, Karen Ullrich, Edward Meeds•Sun Feb 12 2017

This paper shows that competitive compression rates can be achieved by using a version of “soft weight-sharing” (Nowlan & Hinton, 1992) and achieves both quantization and pruning in one simple (re-)training procedure, exposing the relation between compression and the minimum description length (MDL) principle.

437

Content

0

Paper Graph

Improving Neural Network Quantization without Retraining using Outlier Channel Splitting

Ritchie Zhao, Yuwei Hu, Jordan Dotzel, Christopher De Sa, Zhiru Zhang•Sun Jan 27 2019

This work proposes outlier channel splitting (OCS), which duplicates channels containing outliers, then halves the channel values, and shows that OCS can outperform state-of-the-art clipping techniques with only minor overhead.

334 0

Paper Graph

MUSCO: Multi-Stage Compression of neural networks

L. Markeeva, I. Oseledets, Julia Gusak, Maksym Kholiavchenko, E. Ponomarev, A. Cichocki•Sat Mar 23 2019

A new simple and efficient iterative approach, which alternates low-rank factorization with a smart rank selection and fine-tuning, which improves the compression rate while maintaining the accuracy for a variety of tasks.

10 0

Paper Graph

Data-Free Learning of Student Networks

Yunhe Wang, Hanting Chen, Chunjing Xu, Zhaohui Yang, Chao Xu, Boxin Shi, Chuanjian Liu, Qi Tian•Mon Apr 01 2019

A novel framework for training efficient deep neural networks by exploiting generative adversarial networks (GANs) is proposed, where the pre-trained teacher networks are regarded as a fixed discriminator and the generator is utilized for derivating training samples which can obtain the maximum response on the discriminator.

418 0

Paper Graph

ZeroQ: A Novel Zero Shot Quantization Framework

Z. Yao, A. Gholami, K. Keutzer, Michael W. Mahoney, Zhen Dong, Yaohui Cai•Wed Jan 01 2020

THE AUTHORS' enables mixed-precision quantization without any access to the training or validation data, and it can finish the entire quantization process in less than 30s, which is very low computational overhead.

462 0

Paper Graph

Learning Filter Basis for Convolutional Neural Network Compression

L. Gool, Shuhang Gu, Radu Timofte, Yawei Li•Thu Aug 22 2019

This paper tries to reduce the number of parameters of CNNs by learning a basis of the filters in convolutional layers, and validate the proposed solution for multiple CNN architectures on image classification and image super-resolution benchmarks.

103 0

Paper Graph

NeRV: Neural Representations for Videos

Abhinav Shrivastava, Ser-Nam Lim, Bo He, Hao Chen, Hanyu Wang, Yixuan Ren•Mon Oct 25 2021

A novel neural representation for videos (NeRV) which encodes videos in neural networks taking frame index as input, which can be used as a proxy for video compression, and achieve comparable performance to traditional frame-based video compression approaches.

336 0

Paper Graph

Weightless: Lossy Weight Encoding For Deep Neural Network Compression

Udit Gupta, Alexander M. Rush, Brandon Reagen, Bob Adolf, M. Mitzenmacher, Gu-Yeon Wei, D. Brooks•Sun Nov 12 2017

A novel scheme for lossy weight encoding co-designed with weight simplification techniques that can compress weights by up to 496x without loss of model accuracy, resulting in up to a 1.51x improvement over the state-of-the-art.

41 0

Paper Graph

Minimal Random Code Learning: Getting Bits Back from Compressed Model Parameters

José Miguel Hernández-Lobato, Marton Havasi, Robert Peharz•Wed Sep 26 2018

This paper sets new state-of-the-art in neural network compression, as it strictly dominates previous approaches in a Pareto sense: on the benchmarks LeNet-5/MNIST and VGG-16/CIFAR-10, the approach yields the best test performance for a fixed memory budget, and vice versa, it achieves the highest compression rates for aFixed test performance.

97 0

Paper Graph

Adding a benchmark result helps the community track progress.