MVT: Multi-view Vision Transformer for 3D Object Recognition (2021-10-25T00:00:00.000000Z)

TL;DR

This work proposes a Multi-view Vision Transformer (MVT) for 3D object recognition, and develops a global-local structure for the MVT that takes much less inductive bias compared with its CNN counterparts.

Abstract

Inspired by the great success achieved by CNN in image recognition, view-based methods applied CNNs to model the projected views for 3D object understanding and achieved excellent performance. Nevertheless, multi-view CNN models cannot model the communications between patches from different views, limiting its effectiveness in 3D object recognition. Inspired by the recent success gained by vision Transformer in image recognition, we propose a Multi-view Vision Transformer (MVT) for 3D object recognition. Since each patch feature in a Transformer block has a global reception field, it naturally achieves communications between patches from different views. Meanwhile, it takes much less inductive bias compared with its CNN counterparts. Considering both effectiveness and efficiency, we develop a global-local structure for our MVT. Our experiments on two public benchmarks, ModelNet40 and ModelNet10, demonstrate the competitive performance of our MVT.

Authors

Shuo Chen

4 papers

Tan Yu

2 papers

Ping Li

2 papers

TL;DR

Abstract

Authors

References40 items

Container: Context Aggregation Network

Multi-View 3D Shape Recognition via Correspondence-Aware Deep Learning

Twins: Revisiting the Design of Spatial Attention in Vision Transformers

Rethinking Spatial Dimensions of Vision Transformers

CvT: Introducing Convolutions to Vision Transformers

Transformer in Transformer

Pyramid Vision Transformer: A Versatile Backbone for Dense Prediction without Convolutions

Tokens-to-Token ViT: Training Vision Transformers from Scratch on ImageNet

3D Object Representation Learning: A Set-to-Set Matching Perspective

Training data-efficient image transformers & distillation through attention

An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale

View-GCN: View-Based Graph Convolutional Network for 3D Shape Analysis

Learning Relationships for Multi-View 3D Object Recognition

DeepCCFV: Camera Constraint-Free Multi-View Convolutional Neural Network for 3D Object Retrieval

Dominant Set Clustering and Pooling for Multi-View 3D Object Recognition

LP-3DCNN: Unveiling Local Phase in 3D Convolutional Neural Networks

3D2SeqViews: Aggregating Sequential Views for 3D Global Feature Learning by CNN With Hierarchical Attention Aggregation

SeqViews2SeqLabels: Learning 3D Global Features via Aggregating Sequential Views by RNN With Attention

VV-Net: Voxel VAE Net With Group Convolutions for Point Cloud Segmentation

3DmFV: Three-Dimensional Point Cloud Classification in Real-Time Using Convolutional Neural Networks

GVCNN: Group-View Convolutional Neural Networks for 3D Shape Recognition

Multi-view Harmonized Bilinear Network for 3D Object Recognition

3D-A-Nets: 3D Deep Dense Descriptor for Volumetric Shapes with Adversarial Networks

Decoupled Weight Decay Regularization

Attention is All you Need

PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space

PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation

Layer Normalization

Pairwise Decomposition of Image Sequences for Active Multi-view Recognition

Volumetric and Multi-view CNNs for Object Classification on 3D Data

GIFT: A Real-Time and Scalable 3D Shape Search Engine

RotationNet: Joint Object Categorization and Pose Estimation Using Multiviews from Unsupervised Viewpoints

Deep Residual Learning for Image Recognition

VoxNet: A 3D Convolutional Neural Network for real-time object recognition

Multi-view Convolutional Neural Networks for 3D Shape Recognition

3D ShapeNets: A deep representation for volumetric shapes

ImageNet classification with deep convolutional neural networks

ImageNet: A large-scale hierarchical image database

Swin Transformer: Hierarchical Vision Transformer using Shifted Windows

BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding

Field of Study

Journal Information

Name

Volume

Venue Information

Name

Type

URL

Alternate Names