Searching for MobileNetV3 (2019-05-06T00:00:00.000000Z)

TL;DR

This paper starts the exploration of how automated search algorithms and network design can work together to harness complementary approaches improving the overall state of the art of MobileNets.

Abstract

We present the next generation of MobileNets based on a combination of complementary search techniques as well as a novel architecture design. MobileNetV3 is tuned to mobile phone CPUs through a combination of hardware-aware network architecture search (NAS) complemented by the NetAdapt algorithm and then subsequently improved through novel architecture advances. This paper starts the exploration of how automated search algorithms and network design can work together to harness complementary approaches improving the overall state of the art. Through this process we create two new MobileNet models for release: MobileNetV3-Large and MobileNetV3-Small which are targeted for high and low resource use cases. These models are then adapted and applied to the tasks of object detection and semantic segmentation. For the task of semantic segmentation (or any dense pixel prediction), we propose a new efficient segmentation decoder Lite Reduced Atrous Spatial Pyramid Pooling (LR-ASPP). We achieve new state of the art results for mobile classification, detection and segmentation. MobileNetV3-Large is 3.2% more accurate on ImageNet classification while reducing latency by 20% compared to MobileNetV2. MobileNetV3-Small is 6.6% more accurate compared to a MobileNetV2 model with comparable latency. MobileNetV3-Large detection is over 25% faster at roughly the same accuracy as MobileNetV2 on COCO detection. MobileNetV3-Large LR-ASPP is 34% faster than MobileNetV2 R-ASPP at similar accuracy for Cityscapes segmentation.

Authors

Quoc V. Le

42 papers

Liang-Chieh Chen

14 papers

Andrew G. Howard

7 papers

TL;DR

Abstract

Authors

References57 items

Semantic Segmentation of Satellite Images using a Modified CNN with Hard-Swish Activation Function

ChamNet: Towards Efficient Network Design Through Platform-Aware Model Adaptation

Concentrated-Comprehensive Convolutions for lightweight semantic segmentation

FBNet: Hardware-Aware Efficient ConvNet Design via Differentiable Neural Architecture Search

ESPNetv2: A Light-Weight, Power Efficient, and General Purpose Convolutional Neural Network

ProxylessNAS: Direct Neural Architecture Search on Target Task and Hardware

MnasNet: Platform-Aware Neural Architecture Search for Mobile

DARTS: Differentiable Architecture Search

Quantizing deep convolutional networks for efficient inference: A whitepaper

NetAdapt: Platform-Aware Neural Network Adaptation for Mobile Applications

ESPNet: Efficient Spatial Pyramid of Dilated Convolutions for Semantic Segmentation

ADC: Automated Deep Compression and Acceleration with Reinforcement Learning

Efficient Neural Architecture Search via Parameter Sharing

Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation

MobileNetV2: Inverted Residuals and Linear Bottlenecks

Inverted Residuals and Linear Bottlenecks: Mobile Networks for Classification, Detection and Segmentation

Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inference

Progressive Neural Architecture Search

CondenseNet: An Efficient DenseNet Using Learned Group Convolutions

Shift: A Zero FLOP, Zero Parameter Alternative to Spatial Convolutions

Squeeze-and-Excitation Networks

Learning Transferable Architectures for Scalable Image Recognition

ShuffleNet: An Extremely Efficient Convolutional Neural Network for Mobile Devices

Rethinking Atrous Convolution for Semantic Image Segmentation

MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications

In-datacenter performance analysis of a tensor processing unit

Sigmoid-Weighted Linear Units for Neural Network Function Approximation in Reinforcement Learning

Incremental Network Quantization: Towards Lossless CNNs with Low-Precision Weights

Pyramid Scene Parsing Network

Designing Neural Network Architectures using Reinforcement Learning

Neural Architecture Search with Reinforcement Learning

Bridging Nonlinearities and Stochastic Regularizers with Gaussian Error Linear Units

DoReFa-Net: Training Low Bitwidth Convolutional Neural Networks with Low Bitwidth Gradients

DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs

The Cityscapes Dataset for Semantic Urban Scene Understanding

XNOR-Net: ImageNet Classification Using Binary Convolutional Neural Networks

SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and <1MB model size

Inception-v4, Inception-ResNet and the Impact of Residual Connections on Learning

Quantized Convolutional Neural Networks for Mobile Devices

BinaryConnect: Training Deep Neural Networks with binary weights during propagations

ParseNet: Looking Wider to See Better

Modeling local and global deformations in Deep Learning: Epitomic convolution, Multiple Instance Learning, and sliding window detection

Distilling the Knowledge in a Neural Network

Semantic Image Segmentation with Deep Convolutional Nets and Fully Connected CRFs

Expectation Backpropagation: Parameter-Free Training of Multilayer Neural Networks with Continuous or Discrete Weights

Fully convolutional networks for semantic segmentation

ImageNet Large Scale Visual Recognition Challenge

The Pascal Visual Object Classes Challenge: A Retrospective

Microsoft COCO: Common Objects in Context

OverFeat: Integrated Recognition, Localization and Detection using Convolutional Networks

Model compression

Curves and Surfaces in Computer Vision and Graphics

Searching for Activation Functions

Software available from tensorflow.org

Tensor-Flow: Large-scale machine learning on heterogeneous systems

A real-time algorithm for signal analysis with the help of the wavelet transform

Chamnet: Towards efﬁcient network de-sign

Field of Study

Journal Information

Name

Page

Venue Information

Name

Type

URL

Alternate Names