Lepard: Learning partial point cloud matching in rigid and deformable scenes (2021-11-24T00:00:00.000000Z)

TL;DR

Lepard, a Learning based approach for partial point cloud matching in rigid and deformable scenes with key characteristics are an architecture that disentangles point cloud representation into feature space and 3D position space and a position encoding method that explicitly reveals 3D relative distance information through the dot product of vectors.

Abstract

We present Lepard, a Learning based approach for partial point cloud matching in rigid and deformable scenes. The key characteristics are the following techniques that exploit 3D positional knowledge for point cloud matching: 1) An architecture that disentangles point cloud representation into feature space and 3D position space. 2) A position encoding method that explicitly reveals 3D relative distance information through the dot product of vectors. 3) A repositioning technique that modifies the cross-point-cloud relative positions. Ablation studies demonstrate the effectiveness of the above techniques. In rigid cases, Lepard combined with RANSAC and ICP demonstrates state-of-the-art registration recall of 93.9% / 71.3% on the 3DMatch / 3DLoMatch. In deformable cases, Lepard achieves +27.1% / +34.8% higher non-rigid feature matching recall than the prior art on our newly constructed 4DMatch / 4DLoMatch benchmark. Code and data are available at https://github.com/rabbityl/lepard.

Authors

T. Harada

4 papers

Yang Li

2 papers

TL;DR

Abstract

Authors

References86 items

Neural Scene Flow Prior

CoFiNet: Reliable Coarse-to-fine Correspondences for Robust Point Cloud Registration

DPFM: Deep Partial Functional Maps

Distinctiveness oriented Positional Equilibrium for Point Cloud Registration

You Only Hypothesize Once: Point Cloud Registration with Rotation-equivariant Descriptors

PoinTr: Diverse Point Cloud Completion with Geometry-Aware Transformers

Neighborhood Normalization for Robust Geometric Feature Learning

Bootstrap Your Own Correspondences

PointNetLK Revisited

4DComplete: Non-Rigid Motion Estimation Beyond the Observable Surface

RoFormer: Enhanced Transformer with Rotary Position Embedding

LoFTR: Detector-Free Local Feature Matching with Transformers

PointDSC: Robust Point Cloud Registration using Deep Spatial Consistency

Self-supervised Geometric Perception

OMNet: Learning Overlapping Mask for Partial-to-Partial Point Cloud Registration

UnsupervisedR&R: Unsupervised Point Cloud Registration via Differentiable Rendering

Weakly Supervised Learning of Rigid 3D Scene Flow

MultiBodySync: Multi-Body Segmentation and Motion Estimation via 3D Scan Synchronization

PREDATOR: Registration of 3D Point Clouds with Low Overlap

SpinNet: Learning a General Surface Descriptor for 3D Point Cloud Registration

A Laplacian for Nonmanifold Triangle Meshes

FLOT: Scene Flow on Point Clouds Guided by Optimal Transport

SplitFusion: Simultaneous Tracking and Mapping for Non-Rigid Scenes

Neural Non-Rigid Tracking

End-to-End Object Detection with Transformers

Deep Global Registration

Deep Geometric Functional Maps: Robust Feature Learning for Shape Correspondence

RPM-Net: Robust Point Matching Using Learned Features

Learning to Optimize Non-Rigid Tracking

End-to-End Learning Local Multi-View Descriptors for 3D Point Clouds

D3Feat: Joint Learning of Dense Detection and Description of 3D Local Features

A Bayesian Formulation of Coherent Point Drift

DeepDeform: Learning Non-Rigid RGB-D Reconstruction With Semi-Supervised Data

PointPWC-Net: A Coarse-to-Fine Network for Supervised and Self-Supervised Scene Flow Estimation on 3D Point Clouds

SuperGlue: Learning Feature Matching With Graph Neural Networks

PRNet: Self-Supervised Learning for Partial-to-Partial Registration

Fully Convolutional Geometric Features

Deep Closest Point: Learning Representations for Point Cloud Registration

4D Spatio-Temporal ConvNets: Minkowski Convolutional Neural Networks

KPConv: Flexible and Deformable Convolution for Point Clouds

3DRegNet: A Deep Neural Network for 3D Point Registration

PointNetLK: Robust & Efficient Point Cloud Registration Using PointNet

The Perfect Match: 3D Point Cloud Matching With Smoothed Densities

Neighbourhood Consensus Networks

PPF-FoldNet: Unsupervised Learning of Rotation Invariant 3D Local Descriptors

SurfelWarp: Efficient Non-Volumetric Single View Dynamic Reconstruction

3D-CODED: 3D Correspondences by Deep Deformation

FlowNet3D: Learning Scene Flow in 3D Point Clouds

PPFNet: Global Context Aware Local Features for Robust 3D Point Matching

Motion2fusion

Learning Compact Geometric Features

Focal Loss for Dense Object Detection

KillingFusion: Non-rigid 3D Reconstruction without Correspondences

Attention is All you Need

Self-Supervised Visual Descriptor Learning for Dense Correspondence

Fast Global Registration

Fusion4D

Non‐Rigid Puzzles

Go-ICP: A Globally Optimal Solution to 3D ICP Point-Set Registration

3DMatch: Learning Local Geometric Descriptors from RGB-D Reconstructions

VolumeDeform: Real-Time Volumetric Non-rigid Reconstruction

Registration with the Point Cloud Library: A Modular Framework for Aligning in 3-D

DynamicFusion: Reconstruction and tracking of non-rigid scenes in real-time

Robust reconstruction of indoor scenes

Fully convolutional networks for semantic segmentation

Real-time non-rigid reconstruction using an RGB-D camera

KinectFusion: Real-time dense surface mapping and tracking

KinectFusion: real-time 3D reconstruction and interaction using a moving depth camera

Unique shape context for 3d data description

Point Set Registration: Coherent Point Drift

Fast Point Feature Histograms (FPFH) for 3D registration

Aligning point cloud views using persistent feature histograms

Global Correspondence Optimization for Non‐Rigid Registration of Depth Scans

Embedded deformation for shape manipulation

As-rigid-as-possible surface modeling

Efficient RANSAC for Point‐Cloud Shape Detection