PointMixer: MLP-Mixer for Point Cloud Understanding (2021-11-22T00:00:00.000000Z)

TL;DR

This paper proposes PointMixer, a universal point set operator that facilitates information sharing among unstructured 3D points by simply replacing token-mixing MLPs with a softmax function, which can be broadly used in the network as inter-set mixing, intra- set mixing, and pyramid mixing.

Abstract

MLP-Mixer has newly appeared as a new challenger against the realm of CNNs and transformer. Despite its simplicity compared to transformer, the concept of channel-mixing MLPs and token-mixing MLPs achieves noticeable performance in visual recognition tasks. Unlike images, point clouds are inherently sparse, unordered and irregular, which limits the direct use of MLP-Mixer for point cloud understanding. In this paper, we propose PointMixer, a universal point set operator that facilitates information sharing among unstructured 3D points. By simply replacing token-mixing MLPs with a softmax function, PointMixer can"mix"features within/between point sets. By doing so, PointMixer can be broadly used in the network as inter-set mixing, intra-set mixing, and pyramid mixing. Extensive experiments show the competitive or superior performance of PointMixer in semantic segmentation, classification, and point reconstruction against transformer-based methods.

Authors

François Rameau

2 papers

In-So Kweon

7 papers

Jaesik Park

6 papers

TL;DR

Abstract

Authors

References104 items

Rethinking Network Design and Local Geometry in Point Cloud: A Simple Residual MLP Framework

Patches Are All You Need?

Fast Point Transformer

Putting 3D Spatially Sparse Networks on a Diet

Deep Point Cloud Reconstruction

MetaFormer is Actually What You Need for Vision

ASSANet: An Anisotropic Separable Set Abstraction for Efficient Point Cloud Representation Learning

ConvMLP: Hierarchical Convolutional MLPs for Vision

Voxel Transformer for 3D Object Detection

Hire-MLP: Vision MLP via Hierarchical Rearrangement

S2-MLPv2: Improved Spatial-Shift MLP Architecture for Vision

CycleMLP: A MLP-like Architecture for Dense Prediction

AS-MLP: An Axial Shifted MLP Architecture for Vision

Vision Permutator: A Permutable MLP-Like Architecture for Visual Recognition

S2-MLP: Spatial-Shift MLP Architecture for Vision

Point Cloud Upsampling via Disentangled Refinement

SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformers

Pay Attention to MLPs

ResMLP: Feedforward Networks for Image Classification With Data-Efficient Training

RepMLP: Re-parameterizing Convolutions into Fully-connected Layers for Image Recognition

MLP-Mixer: An all-MLP Architecture for Vision

CrossViT: Cross-Attention Multi-Scale Vision Transformer for Image Classification

PAConv: Position Adaptive Convolution with Dynamic Kernel Assembling on Point Clouds

Volumetric Propagation Network: Stereo-LiDAR Fusion for Long-Range Depth Estimation

Scaling Local Self-Attention for Parameter Efficient Visual Backbones

ConViT: improving vision transformers with soft convolutional inductive biases

Transformer in Transformer

Pyramid Vision Transformer: A Versatile Backbone for Dense Prediction without Convolutions

LambdaNetworks: Modeling Long-Range Interactions Without Attention

Tokens-to-Token ViT: Training Vision Transformers from Scratch on ImageNet

Transformers in Vision: A Survey

Training data-efficient image transformers & distillation through attention

PCT: Point cloud transformer

Point Transformer

An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale

Efficient Transformers: A Survey

Searching Efficient 3D Architectures with Sparse Point-Voxel Convolution

Cloud Transformers: A Universal Approach To Point Cloud Processing Tasks

Synthesizer: Rethinking Self-Attention for Transformer Models

Synthesizer: Rethinking Self-Attention in Transformer Models

Exploring Self-Attention for Image Recognition

FPConv: Learning Local Flattening for Point Convolution

5分で分かる!? 有名論文ナナメ読み：Jacob Devlin et al. : BERT : Pre-training of Deep Bidirectional Transformers for Language Understanding

Deep Learning for 3D Point Clouds: A Survey

Geometry Sharing Network for 3D Point Cloud Classification and Segmentation

Grid-GCN for Fast and Scalable Point Cloud Learning

RandLA-Net: Efficient Semantic Segmentation of Large-Scale Point Clouds

Hierarchical Point-Edge Interaction Network for Point Cloud Semantic Segmentation

DensePoint: Learning Densely Contextual Representation for Efficient Point Cloud Processing

Interpolated Convolutional Networks for 3D Point Cloud Understanding

Point-Voxel CNN for Efficient 3D Deep Learning

Stand-Alone Self-Attention in Vision Models

PointWeb: Enhancing Local Neighborhood Features for Point Cloud Processing

Local Relation Networks for Image Recognition

4D Spatio-Temporal ConvNets: Minkowski Convolutional Neural Networks

KPConv: Flexible and Deformable Convolution for Point Clouds

A-CNN: Annularly Convolutional Neural Networks on Point Clouds

Relation-Shape Convolutional Neural Network for Point Cloud Analysis

PointConv: Deep Convolutional Networks on 3D Point Clouds

On the Spectral Bias of Neural Networks

Deep Parametric Continuous Convolutional Neural Networks

MVSNet: Depth Inference for Unstructured Multi-view Stereo

Local Spectral Graph Convolution for Point Set Feature Learning

SO-Net: Self-Organizing Network for Point Cloud Analysis

SPLATNet: Sparse Lattice Networks for Point Cloud Processing

Recurrent Slice Networks for 3D Segmentation of Point Clouds

Open3D: A Modern Library for 3D Data Processing

Dynamic Graph CNN for Learning on Point Clouds

PointCNN: Convolution On X-Transformed Points

3D Semantic Segmentation with Submanifold Sparse Convolutional Networks

VoxelNet: End-to-End Learning for Point Cloud Based 3D Object Detection

3D Graph Neural Networks for RGBD Semantic Segmentation

Attention is All you Need

PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space

Submanifold Sparse Convolutional Networks

Dynamic Edge-Conditioned Filters in Convolutional Neural Networks on Graphs