Self-Supervised Learning of Point Clouds via Orientation Estimation (2020-08-01T00:00:00.000000Z)

TL;DR

This paper leverages 3D self-supervision for learning downstream tasks on point clouds with fewer labels and demonstrates that its approach outperforms the state-of-the-art.

Abstract

Point clouds provide a compact and efficient representation of 3D shapes. While deep neural networks have achieved impressive results on point cloud learning tasks, they require massive amounts of manually labeled data, which can be costly and time-consuming to collect. In this paper, we leverage 3D self-supervision for learning downstream tasks on point clouds with fewer labels. A point cloud can be rotated in infinitely many ways, which provides a rich label-free source for self-supervision. We consider the auxiliary task of predicting rotations that in turn leads to useful features for other tasks such as shape classification and 3D keypoint prediction. Using experiments on ShapeNet and ModelNet, we demonstrate that our approach outperforms the state-of-the-art. Moreover, features learned by our model are complementary to other self-supervised methods and combining them leads to further performance improvement.

Authors

Vladimir G. Kim

4 papers

Omid Poursaeed

1 papers

Tianxing Jiang

1 papers

TL;DR

Abstract

Authors

References99 items

Coupling Explicit and Implicit Surface Representations for Generative 3D Modeling

Self-Supervised Learning of Local Features in 3D Point Clouds

A Comprehensive Survey on Geometric Deep Learning

Neural Puppet: Generative Layered Cartoon Characters

Unsupervised Multi-Task Feature Learning on Point Clouds

Unsupervised Feature Learning for Point Cloud Understanding by Contrasting and Clustering Using Graph Convolutional Neural Networks

DISN: Deep Implicit Surface Network for High-quality Single-view 3D Reconstruction

A critical analysis of self-supervision, or what we can learn from a single image

Relation-Shape Convolutional Neural Network for Point Cloud Analysis

Revisiting Self-Supervised Visual Representation Learning

Context Prediction for Unsupervised Deep Learning on Point Clouds

Self-Supervised Deep Learning on Point Clouds by Reconstructing Space

DeepSDF: Learning Continuous Signed Distance Functions for Shape Representation

3D Point Capsule Networks

On the Continuity of Rotation Representations in Neural Networks

Occupancy Networks: Learning 3D Reconstruction in Function Space

PointConv: Deep Convolutional Networks on 3D Point Clouds

View Inter-Prediction GAN: Unsupervised Representation Learning for 3D Shapes by Learning Global Shape Memories to Support Local View Predictions

PointGrow: Autoregressively Learned Point Cloud Generation with Self-Attention

Point Cloud GAN

Deep Fundamental Matrix Estimation without Correspondences

Generating 3D faces using Convolutional Mesh Autoencoders

Cross Pixel Optical Flow Similarity for Self-Supervised Learning

Deep Clustering for Unsupervised Learning of Visual Features

Multiresolution Tree Networks for 3D Point Cloud Processing

Representation Learning with Contrastive Predictive Coding

Self-Supervised Feature Learning by Learning to Spot Artifacts

Learning and Using the Arrow of Time

Geometry Guided Convolutional Neural Networks for Self-Supervised Video Representation Learning

Boosting Self-Supervised Learning via Knowledge Transfer

Pixel2Mesh: Generating 3D Mesh Models from Single RGB Images

Learning Category-Specific Mesh Reconstruction from Image Collections

Unsupervised Representation Learning by Predicting Image Rotations

Dynamic Graph CNN for Learning on Point Clouds

PointCNN: Convolution On X-Transformed Points

Mining Point Cloud Local Structures by Kernel Correlation and Graph Pooling

FoldingNet: Point Cloud Auto-Encoder via Deep Grid Deformation

Cross-Domain Self-Supervised Multi-task Feature Learning Using Synthetic Imagery

Improvements to Context Based Self-Supervised Learning

Representation Learning by Learning to Count

Transitive Invariance for Self-Supervised Visual Representation Learning

3D-PRNN: Generating Shape Primitives with Recurrent Neural Networks

Learning Representations and Generative Models for 3D Point Clouds

PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space

Look, Listen and Learn

Time-Contrastive Networks: Self-Supervised Learning from Video

Unsupervised Learning by Predicting Noise

Escape from Cells: Deep Kd-Networks for the Recognition of 3D Point Cloud Models

Octree Generating Networks: Efficient Convolutional Architectures for High-resolution 3D Outputs

ScanNet: Richly-Annotated 3D Reconstructions of Indoor Scenes

Learning Features by Watching Objects Move

Stacked Generative Adversarial Networks

SyncSpecCNN: Synchronized Spectral CNN for 3D Shape Segmentation

A Point Set Generation Network for 3D Object Reconstruction from a Single Image

PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation

Split-Brain Autoencoders: Unsupervised Learning by Cross-Channel Prediction

Object-Centric Representation Learning from Unlabeled Videos

OctNet: Learning Deep 3D Representations at High Resolutions

Learning a Probabilistic Latent Space of Object Shapes via 3D Generative-Adversarial Modeling

Generative and Discriminative Voxel Modeling with Convolutional Neural Networks

Unsupervised Learning of 3D Structure from Images

Conditional Image Generation with PixelCNN Decoders

Context Encoders: Feature Learning by Inpainting

VConv-DAE: Deep Volumetric Shape Learning Without Object Labels

Joint Unsupervised Learning of Deep Representations and Image Clusters

Volumetric and Multi-view CNNs for Object Classification on 3D Data

3D-R2N2: A Unified Approach for Single and Multi-view 3D Object Reconstruction

Unsupervised Learning of Visual Representations by Solving Jigsaw Puzzles

Colorful Image Colorization

Shuffle and Learn: Unsupervised Learning Using Temporal Order Verification

Learning Representations for Automatic Colorization

Pixel Recurrent Neural Networks

Visually Indicated Sounds

ShapeNet: An Information-Rich 3D Model Repository

Learning visual groups from co-occurrences in space and time

Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks