Parameter is Not All You Need: Starting from Non-Parametric Networks for 3D Point Cloud Analysis (2023-03-14T00:00:00.000000Z)

Abstract

We present a Non-parametric Network for 3D point cloud analysis, Point-NN, which consists of purely non-learnable components: farthest point sampling (FPS), k-nearest neighbors (k-NN), and pooling operations, with trigonometric functions. Surprisingly, it performs well on various 3D tasks, requiring no parameters or training, and even surpasses existing fully trained models. Starting from this basic non-parametric model, we propose two extensions. First, Point-NN can serve as a base architectural framework to construct Parametric Networks by simply inserting linear layers on top. Given the superior non-parametric foundation, the derived Point-PN exhibits a high performance-efficiency trade-off with only a few learnable parameters. Second, Point-NN can be regarded as a plug-and-play module for the already trained 3D models during inference. Point-NN captures the complementary geometric knowledge and enhances existing methods for different 3D benchmarks without re-training. We hope our work may cast a light on the community for understanding 3D point clouds with non-parametric methods. Code is available at https://github.com/ZrrSkywalker/Point-NN.

Authors

Hongsheng Li

26 papers

Yali Wang

11 papers

Jianbo Shi

5 papers

Abstract

Authors

References100 items

Revisiting Multimodal Representation in Contrastive Learning: From Patch and Token Embeddings to Finite Discrete Tokens

PiMAE: Point Cloud and Image Interactive Masked Autoencoders for 3D Object Detection

HiCLIP: Contrastive Language-Image Pretraining with Hierarchy-aware Attention

Prompt, Generate, Then Cache: Cascade of Foundation Models Makes Strong Few-Shot Learners

Joint-MAE: 2D-3D Joint Masked Autoencoders for 3D Point Cloud Pre-training

Nearest Neighbors Meet Deep Neural Networks for Point Cloud Analysis

TiG-BEV: Multi-view BEV 3D Object Detection via Target Inner-Geometry Learning

Learning 3D Representations from 2D Pre-Trained Models via Image-to-Point Masked Autoencoders

CLIP2Point: Transfer CLIP to Point Cloud Classification with Image-Depth Pre-Training

CALIP: Zero-Shot Enhancement of CLIP with Parameter-free Attention

Frozen CLIP Models are Efficient Video Learners

Can Language Understand Depth?

PointNeXt: Revisiting PointNet++ with Improved Training and Scaling Strategies

Point-M2AE: Multi-scale Masked Autoencoders for Hierarchical Point Cloud Pre-training

POS-BERT: Point Cloud One-Stage BERT Pre-Training

Stratified Transformer for 3D Point Cloud Segmentation

Rethinking Network Design and Local Geometry in Point Cloud: A Simple Residual MLP Framework

Distillation with Contrast is All You Need for Self-Supervised Point Cloud Representation Learning

PointCLIP: Point Cloud Understanding by CLIP

DSPoint: Dual-scale Point Cloud Recognition with High-frequency Fusion

Tip-Adapter: Training-free CLIP-Adapter for Better Vision-Language Modeling

CLIP-Adapter: Better Vision-Language Models with Feature Adapters

An End-to-End Transformer Model for 3D Object Detection

Learning to Prompt for Vision-Language Models

PnP-3D: A Plug-and-Play for 3D Point Clouds

Revisiting Point Cloud Shape Classification with a Simple and Effective Baseline

Walk in the Cloud: Learning Curves for Point Clouds Shape Analysis

PAConv: Position Adaptive Convolution with Dynamic Kernel Assembling on Point Clouds

Learning Transferable Visual Models From Natural Language Supervision

Conditional Positional Encodings for Vision Transformers

Learning Geometry-Disentangled Representation for Complementary Understanding of 3D Object Point Cloud

PCT: Point cloud transformer

MVTN: Multi-View Transformation Network for 3D Shape Recognition

Point Transformer

An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale

Self-Supervised Few-Shot Learning on Point Clouds

A Closer Look at Local Aggregation Operators in Point Cloud Analysis

Transformer

Fourier Features Let Networks Learn High Frequency Functions in Low Dimensional Domains

Structure Aware Single-Stage 3D Object Detection From Point Cloud

Dense-Resolution Network for Point Cloud Classification and Segmentation

Generative PointNet: Deep Energy-Based Learning on Unordered Point Sets for 3D Generation, Reconstruction and Classification

PV-RCNN: Point-Voxel Feature Set Abstraction for 3D Object Detection

Geometric Back-Projection Network for Point Cloud Classification

DensePoint: Learning Densely Contextual Representation for Efficient Point Cloud Processing

Revisiting Point Cloud Classification: A New Benchmark Dataset and Classification Model on Real-World Data

DAR-Net: Dynamic Aggregation Network for Semantic Scene Segmentation

Point-Voxel CNN for Efficient 3D Deep Learning

Deep Learning Approach to Point Cloud Scene Understanding for Automated Scan to 3D Reconstruction

PointWeb: Enhancing Local Neighborhood Features for Point Cloud Processing

Structural Relational Reasoning of Point Clouds

6-DOF GraspNet: Variational Grasp Generation for Object Manipulation

KPConv: Flexible and Deformable Convolution for Point Clouds

VoteNet: A Deep Learning Label Fusion Method for Multi-Atlas Segmentation

Relation-Shape Convolutional Neural Network for Point Cloud Analysis

PointConv: Deep Convolutional Networks on 3D Point Clouds

VV-Net: Voxel VAE Net With Group Convolutions for Point Cloud Segmentation

3DmFV: Three-Dimensional Point Cloud Classification in Real-Time Using Convolutional Neural Networks

On the Spectral Bias of Neural Networks

RGCNN: Regularized Graph CNN for Point Cloud Segmentation

Monte Carlo convolution for learning on non-uniformly sampled point clouds

Tangent Convolutions for Dense Prediction in 3D

SpiderCNN: Deep Learning on Point Sets with Parameterized Convolutional Filters

3DMV: Joint 3D-Multi-View Prediction for 3D Semantic Scene Segmentation

Dynamic Graph CNN for Learning on Point Clouds

PointCNN: Convolution On X-Transformed Points

FoldingNet: Point Cloud Auto-Encoder via Deep Grid Deformation

Large-Scale Point Cloud Semantic Segmentation with Superpoint Graphs

Fast 3D point cloud segmentation using supervoxels with geometry and color for 3D scene understanding

Attention is All you Need

PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space

Convolutional Sequence to Sequence Learning

ScanNet: Richly-Annotated 3D Reconstructions of Indoor Scenes

PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation

Multi-view 3D Object Detection Network for Autonomous Driving

OctNet: Learning Deep 3D Representations at High Resolutions

A scalable active framework for region annotation in 3D shape collections