Stratified Transformer for 3D Point Cloud Segmentation (2022-03-28T00:00:00.000000Z)

TL;DR

Stratified Transformer is proposed that is able to capture long-range contexts and demonstrates strong generalization ability and high performance and first-layer point embedding to aggregate local information, which facilitates convergence and boosts performance.

Abstract

3D point cloud segmentation has made tremendous progress in recent years. Most current methods focus on aggregating local features, but fail to directly model long-range dependencies. In this paper, we propose Stratified Transformer that is able to capture long-range contexts and demonstrates strong generalization ability and high performance. Specifically, we first put forward a novel key sampling strategy. For each query point, we sample nearby points densely and distant points sparsely as its keys in a stratified way, which enables the model to enlarge the effective receptive field and enjoy long-range contexts at a low computational cost. Also, to combat the challenges posed by irregular point arrangements, we propose first-layer point embedding to aggregate local information, which facilitates convergence and boosts performance. Besides, we adopt contextual relative position encoding to adaptively capture position information. Finally, a memory-efficient implementation is introduced to overcome the issue of varying point numbers in each window. Extensive experiments demonstrate the effectiveness and superiority of our method on S3DIS, ScanNetv2 and ShapeNetPart datasets. Code is available at https://github.com/dvlab-research/Stratified-Transformer.

Authors

Jiaya Jia

23 papers

Xiaojuan Qi

13 papers

Hengshuang Zhao

10 papers

TL;DR

Abstract

Authors

References62 items

An End-to-End Transformer Model for 3D Object Detection

Voxel Transformer for 3D Object Detection

ICM-3D: Instantiated Category Modeling for 3D Instance Segmentation

Rethinking and Improving Relative Position Encoding for Vision Transformer

Visual Parser: Representing Part-whole Hierarchies with Transformers

CSWin Transformer: A General Vision Transformer Backbone with Cross-Shaped Windows

Focal Self-attention for Local-Global Interactions in Vision Transformers

PVT v2: Improved baselines with Pyramid Vision Transformer

Twins: Revisiting the Design of Spatial Attention in Vision Transformers

Group-Free 3D Object Detection via Transformers

Going deeper with Image Transformers

PAConv: Position Adaptive Convolution with Dynamic Kernel Assembling on Point Clouds

Pyramid Vision Transformer: A Versatile Backbone for Dense Prediction without Convolutions

Training data-efficient image transformers & distillation through attention

Point Transformer

An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale

Deformable DETR: Deformable Transformers for End-to-End Object Detection

JSENet: Joint Semantic Segmentation and Edge Detection Network for 3D Point Clouds

SegGCN: Efficient 3D Point Cloud Segmentation With Fuzzy Spherical Kernel

End-to-End Object Detection with Transformers

Exploring Self-Attention for Image Recognition

PointASNL: Robust Point Clouds Processing Using Nonlocal Neural Networks With Adaptive Sampling

On Layer Normalization in the Transformer Architecture

RandLA-Net: Efficient Semantic Segmentation of Large-Scale Point Clouds

Hierarchical Point-Edge Interaction Network for Point Cloud Semantic Segmentation

DensePoint: Learning Densely Contextual Representation for Efficient Point Cloud Processing

Interpolated Convolutional Networks for 3D Point Cloud Understanding

A Unified Point-Based Framework for 3D Segmentation

Dilated Point Convolutions: On the Receptive Field Size of Point Convolutions on 3D Point Clouds

Point-Voxel CNN for Efficient 3D Deep Learning

PointWeb: Enhancing Local Neighborhood Features for Point Cloud Processing

KPConv: Flexible and Deformable Convolution for Point Clouds

4D Spatio-Temporal ConvNets: Minkowski Convolutional Neural Networks

Relation-Shape Convolutional Neural Network for Point Cloud Analysis

Modeling Point Clouds With Self-Attention and Gumbel Subset Sampling

PanopticFusion: Online Volumetric Semantic Mapping at the Level of Stuff and Things

PointConv: Deep Convolutional Networks on 3D Point Clouds

Point2Sequence: Learning the Shape Representation of 3D Point Clouds with an Attention-based Sequence to Sequence Network

Deep Parametric Continuous Convolutional Neural Networks

Tangent Convolutions for Dense Prediction in 3D

SpiderCNN: Deep Learning on Point Sets with Parameterized Convolutional Filters

3DMV: Joint 3D-Multi-View Prediction for 3D Semantic Scene Segmentation

Point convolutional neural networks by extension operators

SPLATNet: Sparse Lattice Networks for Point Cloud Processing

Dynamic Graph CNN for Learning on Point Clouds

PointCNN: Convolution On X-Transformed Points

3D Semantic Segmentation with Submanifold Sparse Convolutional Networks

Large-Scale Point Cloud Semantic Segmentation with Superpoint Graphs

Non-local Neural Networks

SEGCloud: Semantic Segmentation of 3D Point Clouds

Attention is All you Need

PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space

Submanifold Sparse Convolutional Networks

ScanNet: Richly-Annotated 3D Reconstructions of Indoor Scenes

Understanding the Effective Receptive Field in Deep Convolutional Neural Networks

PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation

Layer Normalization

3D Semantic Parsing of Large-Scale Indoor Spaces

ShapeNet: An Information-Rich 3D Model Repository

U-Net: Convolutional Networks for Biomedical Image Segmentation

Swin Transformer: Hierarchical Vision Transformer using Shifted Windows

Deep FusionNet for Point Cloud Semantic Segmentation

Field of Study

Journal Information

Name

Page

Venue Information

Name

Type

URL

Alternate Names