Searching Efficient 3D Architectures with Sparse Point-Voxel Convolution (2020-07-31T00:00:00.000000Z)

TL;DR

This work proposes Sparse Point-Voxel Convolution (SPVConv), a lightweight 3D module that equips the vanilla Sparse Convolution with the high-resolution point-based branch, and presents 3D Neural Architecture Search (3D-NAS) to search the optimal network architecture over this diverse design space efficiently and effectively.

Abstract

Self-driving cars need to understand 3D scenes efficiently and accurately in order to drive safely. Given the limited hardware resources, existing 3D perception models are not able to recognize small instances (e.g., pedestrians, cyclists) very well due to the low-resolution voxelization and aggressive downsampling. To this end, we propose Sparse Point-Voxel Convolution (SPVConv), a lightweight 3D module that equips the vanilla Sparse Convolution with the high-resolution point-based branch. With negligible overhead, this point-based branch is able to preserve the fine details even from large outdoor scenes. To explore the spectrum of efficient 3D models, we first define a flexible architecture design space based on SPVConv, and we then present 3D Neural Architecture Search (3D-NAS) to search the optimal network architecture over this diverse design space efficiently and effectively. Experimental results validate that the resulting SPVNAS model is fast and accurate: it outperforms the state-of-the-art MinkowskiNet by 3.3%, ranking 1st on the competitive SemanticKITTI leaderboard. It also achieves 8x computation reduction and 3x measured speedup over MinkowskiNet with higher accuracy. Finally, we transfer our method to 3D object detection, and it achieves consistent improvements over the one-stage detection baseline on KITTI.

Authors

Haotian Tang

2 papers

Yujun Lin

4 papers

Zhijian Liu

1 papers

TL;DR

Abstract

Authors

References78 items

Hardware-Centric AutoML for Mixed-Precision Quantization

APQ: Joint Search for Network Architecture, Pruning and Quantization Policy

HAT: Hardware-Aware Transformers for Efficient Natural Language Processing

SqueezeSegV3: Spatially-Adaptive Convolution for Efficient Point-Cloud Segmentation

PointGroup: Dual-Set Point Grouping for 3D Instance Segmentation

PolarNet: An Improved Grid Representation for Online LiDAR Point Clouds Semantic Segmentation

GAN Compression: Efficient Architectures for Interactive Conditional GANs

OccuSeg: Occupancy-Aware 3D Instance Segmentation

SalsaNext: Fast, Uncertainty-Aware Semantic Segmentation of LiDAR Point Clouds

3D-MiniNet: Learning a 2D Representation From Point Clouds for Fast and Efficient 3D LIDAR Semantic Segmentation

ImVoteNet: Boosting 3D Object Detection in Point Clouds With Image Votes

AutoML for Architecting Efficient and Specialized Neural Networks

PV-RCNN: Point-Voxel Feature Set Abstraction for 3D Object Detection

C2FNAS: Coarse-to-Fine Neural Architecture Search for 3D Medical Image Segmentation

SGAS: Sequential Greedy Architecture Search

RandLA-Net: Efficient Semantic Segmentation of Large-Scale Point Clouds

Searching Learning Strategy with Reinforcement Learning for 3D Medical Image Segmentation

Resource Optimized Neural Architecture Search for 3D Medical Image Segmentation

Once for All: Train One Network and Specialize it for Efficient Deployment

Interpolated Convolutional Networks for 3D Point Cloud Understanding

STD: Sparse-to-Dense 3D Object Detector for Point Cloud

Point-Voxel CNN for Efficient 3D Deep Learning

3D Instance Segmentation via Multi-Task Metric Learning

Scalable Neural Architecture Search for 3D Medical Image Segmentation

V-NAS: Neural Architecture Search for Volumetric Medical Image Segmentation

Energy and Policy Considerations for Deep Learning in NLP

Learning Object Bounding Boxes for 3D Instance Segmentation on Point Clouds

On Network Design Spaces for Visual Recognition

EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks

Deep Hough Voting for 3D Object Detection in Point Clouds

4D Spatio-Temporal ConvNets: Minkowski Convolutional Neural Networks

KPConv: Flexible and Deformable Convolution for Point Clouds

Single-Path NAS: Designing Hardware-Efficient ConvNets in less than 4 Hours

SemanticKITTI: A Dataset for Semantic Scene Understanding of LiDAR Sequences

Single Path One-Shot Neural Architecture Search with Uniform Sampling

DetNAS: Backbone Search for Object Detection

MetaPruning: Meta Learning for Automatic Neural Network Channel Pruning

Octree Guided CNN With Spherical Kernels for 3D Point Clouds

PointRCNN: 3D Object Proposal Generation and Detection From Point Cloud

FBNet: Hardware-Aware Efficient ConvNet Design via Differentiable Neural Architecture Search

HAQ: Hardware-Aware Automated Quantization With Mixed Precision

PointConv: Deep Convolutional Networks on 3D Point Clouds

SECOND: Sparsely Embedded Convolutional Detection

ProxylessNAS: Direct Neural Architecture Search on Target Task and Hardware

Adaptive O-CNN

VoxSegNet: Volumetric CNNs for Semantic Part Segmentation of 3D Shapes

MnasNet: Platform-Aware Neural Architecture Search for Mobile

ShuffleNet V2: Practical Guidelines for Efficient CNN Architecture Design

DARTS: Differentiable Architecture Search

Tangent Convolutions for Dense Prediction in 3D

SpiderCNN: Deep Learning on Point Sets with Parameterized Convolutional Filters

SPLATNet: Sparse Lattice Networks for Point Cloud Processing

AMC: AutoML for Model Compression and Acceleration on Mobile Devices

Dynamic Graph CNN for Learning on Point Clouds

PointCNN: Convolution On X-Transformed Points

MobileNetV2: Inverted Residuals and Linear Bottlenecks

Progressive Neural Architecture Search

3D Semantic Segmentation with Submanifold Sparse Convolutional Networks

Large-Scale Point Cloud Semantic Segmentation with Superpoint Graphs

Frustum PointNets for 3D Object Detection from RGB-D Data

VoxelNet: End-to-End Learning for Point Cloud Based 3D Object Detection

Learning Transferable Architectures for Scalable Image Recognition

Phase-functioned neural networks for character control

ShuffleNet: An Extremely Efficient Convolutional Neural Network for Mobile Devices

PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space

MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications

PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation

OctNet: Learning Deep 3D Representations at High Resolutions

Neural Architecture Search with Reinforcement Learning

Volumetric and Multi-view CNNs for Object Classification on 3D Data

SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and <1MB model size

ShapeNet: An Information-Rich 3D Model Repository

VoxNet: A 3D Convolutional Neural Network for real-time object recognition

Vision meets robotics: The KITTI dataset

Are we ready for autonomous driving? The KITTI vision benchmark suite

ImageNet: A large-scale hierarchical image database