MUSCO: Multi-Stage Compression of neural networks (2019-03-24T00:00:00.000000Z)

TL;DR

A new simple and efficient iterative approach, which alternates low-rank factorization with a smart rank selection and fine-tuning, which improves the compression rate while maintaining the accuracy for a variety of tasks.

Abstract

The low-rank tensor approximation is very promising for the compression of deep neural networks. We propose a new simple and efficient iterative approach, which alternates low-rank factorization with a smart rank selection and fine-tuning. We demonstrate the efficiency of our method comparing to non-iterative ones. Our approach improves the compression rate while maintaining the accuracy for a variety of tasks.

Authors

L. Markeeva

4 papers

I. Oseledets

5 papers

Julia Gusak

1 papers

MUSCO: Multi-Stage Compression of neural networks

TL;DR

Abstract

Authors

References23 items

AMC: AutoML for Model Compression and Acceleration on Mobile Devices

Tensor Train decomposition on TensorFlow (T3F)

Channel Pruning for Accelerating Very Deep Neural Networks

Weighted-Entropy-Based Quantization for Deep Neural Networks

Feature Pyramid Networks for Object Detection

XNOR-Net: ImageNet Classification Using Binary Convolutional Neural Networks

Convolutional Neural Networks using Logarithmic Data Representation

Compression of Deep Convolutional Neural Networks for Fast and Low Power Mobile Applications

Fixed Point Quantization of Deep Convolutional Networks

Deep Compression: Compressing Deep Neural Network with Pruning, Trained Quantization and Huffman Coding

Subdominant Dense Clusters Allow for Simple Learning and High Computational Performance in Neural Networks with Discrete Synapses.

Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks

Accelerating Very Deep Convolutional Networks for Classification and Detection

Speeding-up Convolutional Neural Networks Using Fine-tuned CP-Decomposition

The Pascal Visual Object Classes Challenge: A Retrospective

Speeding up Convolutional Neural Networks with Low Rank Expansions

Microsoft COCO: Common Objects in Context

Exploiting Linear Structure Within Convolutional Networks for Efficient Evaluation

Perfect Dimensionality Recovery by Variational Bayesian PCA

Most Tensor Problems Are NP-Hard

Some mathematical notes on three-mode factor analysis

Foundations of the PARAFAC procedure: Models and conditions for an "explanatory" multi-model factor analysis

Implications of factor analysis of three-way matrices for measurement of changes, problems of measuring change

Field of Study

Journal Information

Name