Indoor Scene Recognition in 3D (2020-02-28T00:00:00.000000Z)

TL;DR

This work studies scene recognition from 3D point cloud (or voxel) data, and shows that it greatly outperforms methods based on 2D birds-eye views, and advocates multi-task learning as a way to improve scene recognition.

Abstract

Recognising in what type of environment one is located is an important perception task. For instance, for a robot operating indoors it is helpful to be aware whether it is in a kitchen, a hallway or a bedroom. Existing approaches attempt to classify the scene based on 2D images or 2.5D range images. Here, we study scene recognition from 3D point cloud (or voxel) data, and show that it greatly outperforms methods based on 2D birds-eye views. Moreover, we advocate multi-task learning as a way to improve scene recognition, building on the fact that the scene type is highly correlated with the objects in the scene, and therefore with its semantic segmentation into different object classes. In a series of ablation studies, we show that successful scene recognition is not just the recognition of individual objects unique to some scene type (such as a bathtub), but depends on several different cues, including coarse 3D geometry, colour, and the (implicit) distribution of object categories. Moreover, we demonstrate that surprisingly sparse 3D data is sufficient to classify indoor scenes with good accuracy.

Authors

K. Schindler

19 papers

Shengyu Huang

2 papers

Mikhail (Misha) Usvyatsov

3 papers

TL;DR

Abstract

Authors

References46 items

3D Instance Segmentation via Multi-Task Metric Learning

Translate-to-Recognize Networks for RGB-D Scene Recognition

4D Spatio-Temporal ConvNets: Minkowski Convolutional Neural Networks

JSIS3D: Joint Semantic-Instance Segmentation of 3D Point Clouds With Multi-Task Pointwise Networks and Multi-Value Conditional Random Fields

Associatively Segmenting Instances and Semantics in Point Clouds

Learning Effective RGB-D Representations for Scene Recognition

Self-Supervised Model Adaptation for Multimodal Semantic Segmentation

Scene recognition with objectness

Inference, Learning and Attention Mechanisms that Exploit and Preserve Sparsity in CNNs

Dynamic Graph CNN for Learning on Point Clouds

Improved Regularization of Convolutional Neural Networks with Cutout

A Survey on Multi-Task Learning

PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space

Submanifold Sparse Convolutional Networks

The Kinetics Human Action Video Dataset

ScanNet: Richly-Annotated 3D Reconstructions of Indoor Scenes

3D Shape Segmentation with Projective Convolutional Networks

PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation

Convolutional Neural Networks on Graphs with Fast Localized Spectral Filtering

Modality and Component Aware Feature Fusion for RGB-D Scene Classification

Discriminative Multi-modal Feature Fusion for RGBD Indoor Scene Recognition

Scene Recognition with CNNs: Objects, Scales and Dataset Bias

Volumetric and Multi-view CNNs for Object Classification on 3D Data

Deep Residual Learning for Image Recognition

VoxNet: A 3D Convolutional Neural Network for real-time object recognition

Second-order constrained parametric proposals and sequential search-based structured prediction for semantic segmentation in RGB-D images

SUN RGB-D: A RGB-D scene understanding benchmark suite

Multi-view Convolutional Neural Networks for 3D Shape Recognition

Indoor Scene Understanding with RGB-D Images: Bottom-up Segmentation, Object Detection and Semantic Segmentation

Adam: A Method for Stochastic Optimization

Learning Deep Features for Scene Recognition using Places Database

Spatially-sparse convolutional neural networks

SUN3D: A Database of Big Spaces Reconstructed Using SfM and Object Labels

Indoor Segmentation and Support Inference from RGBD Images

Object Bank: A High-Level Image Representation for Scene Classification & Semantic Feature Sparsification

SUN database: Large-scale scene recognition from abbey to zoo

Recognizing indoor scenes

ImageNet: A large-scale hierarchical image database

Scene Classification Using a Hybrid Generative/Discriminative Approach

Scene Classification Using Bag-of-Regions Representations

Beyond Bags of Features: Spatial Pyramid Matching for Recognizing Natural Scene Categories

Modeling the Shape of the Scene: A Holistic Representation of the Spatial Envelope

Classification of scene photographs from local orientations features

Multitask Learning

Mesh R-CNN

Scene Classication Using a Hybrid Generative/Discriminative Approach

Field of Study

Journal Information

Name

Page

Venue Information

Name

Type

URL

Alternate Names