LMSCNet: Lightweight Multiscale 3D Semantic Completion (2020-08-24T00:00:00.000000Z)

TL;DR

The ablation studies demonstrate the method is robust to lower density inputs, and that it enables very high speed semantic completion at the coarsest level, and provides a great performance/speed trade-off for mobile-robotics applications.

Abstract

We introduce a new approach for multiscale 3Dsemantic scene completion from voxelized sparse 3D LiDAR scans. As opposed to the literature, we use a 2D UNet backbone with comprehensive multiscale skip connections to enhance feature flow, along with 3D segmentation heads. On the SemanticKITTI benchmark, our method performs on par on semantic completion and better on occupancy completion than all other published methods – while being significantly lighter and faster. As suchit provides a great performance/speed trade-off for mobile-robotics applications. The ablation studies demonstrate our method is robust to lower density inputs, and that it enables very high speed semantic completion at the coarsest level. Our code is available at https://github.com/cv-rits/LMSCNet.

Authors

Raoul de Charette

7 papers

Luis Roldão

1 papers

Anne Verroust-Blondet

1 papers

TL;DR

Abstract

Authors

References46 items

3D Gated Recurrent Fusion for Semantic Scene Completion

SG-NN: Sparse Generative Neural Networks for Self-Supervised Scene Completion of RGB-D Scans

RangeNet ++: Fast and Accurate LiDAR Semantic Segmentation

EdgeNet: Semantic Scene Completion from RGB-D images

Cascaded Context Pyramid for Full-Resolution 3D Semantic Scene Completion

Point-Voxel CNN for Efficient 3D Deep Learning

3D Surface Reconstruction from Voxel-based Lidar Data

3D Semantic Scene Completion from a Single Depth Image Using Adversarial Training

SemanticKITTI: A Dataset for Semantic Scene Understanding of LiDAR Sequences

nuScenes: A Multimodal Dataset for Autonomous Driving

RGBD Based Dimensional Decomposition Residual Network for 3D Semantic Scene Completion

DeepSDF: Learning Continuous Signed Distance Functions for Shape Representation

Efficient Semantic Scene Completion Network with Spatial Group Convolution

Adversarial Semantic Scene Completion from a Single Depth Image

Sparse and Dense Data with CNNs: Depth Completion and Semantic Segmentation

PCN: Point Completion Network

View-Volume Network for Semantic Scene Completion from a Single Depth Image

Two Stream 3D Semantic Scene Completion

ScanComplete: Large-Scale Scene Completion and Semantic Segmentation for 3D Scans

3D Semantic Segmentation with Submanifold Sparse Convolutional Networks

Matterport3D: Learning from RGB-D Data in Indoor Environments

Learning for Active 3D Mapping

PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space

OctNetFusion: Learning Depth Fusion from Data

ScanNet: Richly-Annotated 3D Reconstructions of Indoor Scenes

A Survey of Surface Reconstruction from Point Clouds

Shape Completion Using 3D-Encoder-Predictor CNNs and Shape Synthesis

Semantic Scene Completion from a Single Depth Image

Multi-view 3D Object Detection Network for Autonomous Driving

DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs

Structured Prediction of Unobserved Voxels from a Single Depth Image

Joint 3D Object and Layout Inference from a Single RGB-D Image

3D Scene Understanding by Voxel-CRF

3DNN: Viewpoint Invariant 3D Geometry Matching for Scene Understanding

Reinforcement learning in robotics: A survey

Are we ready for autonomous driving? The KITTI vision benchmark suite

KinectFusion: Real-time dense surface mapping and tracking

Towards total scene understanding: Classification, annotation and segmentation in an automatic framework

Poisson surface reconstruction

Shape from symmetry

A volumetric method for building complex models from range images

See and Think: Disentangling Semantic Scene Completion

A Survey of Augmented Reality Technologies, Applications and Limitations

A.2. Architecture comparison

provide additional details on the implementation of the baselines listed in main article Tab. 1. TS3D baselines. We compare our method with 3 variants of the Two Stream 3D (TS3D) network as reported

[ (N+1) × H × W × D input tensor

Field of Study

Journal Information

Name

Page

Venue Information

Name

Type

URL

Alternate Names