computer-vision-7

Zero-Shot Transfer 3D Point Cloud Classification

3260 papers • 126 benchmarks • 313 datasets

This task has no description! Would you like to contribute one?

(Image credit: Papersgraph)

Benchmarks

These leaderboards are used to track progress in zero-shot-transfer-3d-point-cloud-classification-7

Trend

Dataset

Best Model

Actions

ModelNet40

ScanObjectNN

ModelNet10

Libraries

i

Use these libraries to find zero-shot-transfer-3d-point-cloud-classification-7 models and implementations

qizekun/ReCon

2 papers 108

Datasets

ModelNet

ScanObjectNN

Subtasks

No subtasks available.

Most implemented papers

Contrast with Reconstruct: Contrastive 3D Representation Learning Guided by Generative Pretraining

Zekun Qi, Runpei Dong, Guo Fan, Zheng Ge, Xiangyu Zhang, Kaisheng Ma, Li Yi•Sat Feb 04 2023

ReCon is trained to learn from both generative modeling teachers and single/cross-modal contrastive teachers through ensemble distillation, where the generative student guides the contrastive student.

192

Content

runpeidong/act

2 papers 89

0

Paper Graph

ShapeLLM: Universal 3D Object Understanding for Embodied Interaction

Zekun Qi, Runpei Dong, Shaochen Zhang, Haoran Geng, Chunrui Han, Zheng Ge, Li Yi, Kaisheng Ma•Mon Feb 26 2024

ShapeLLM is the first 3D Multimodal Large Language Model designed for embodied interaction, exploring a universal 3D object understanding with 3D point clouds and languages, built upon an improved 3D encoder by extending ReCon to ReCon++ that benefits from multi-view image distillation for enhanced geometry understanding.

114 0

Paper Graph

PointCLIP V2: Prompting CLIP and GPT for Powerful 3D Open-world Learning

Shanghang Zhang, Renrui Zhang, Ziyu Guo, Peng Gao, Zipeng Qin, Xiangyang Zhu, Bowei He, Ziyao Zeng•Sun Nov 20 2022

This paper first collaborate CLIP and GPT to be a unified 3D open-world learner, named as Point-CLIP V2, which fully unleashes their potential for zero-shot 3D classification, segmentation, and detection, demonstrating the generalization ability for unified 3D open-world learning.

224 0

Paper Graph

PointCLIP: Point Cloud Understanding by CLIP

Hongsheng Li, Wei Zhang, Renrui Zhang, Ziyu Guo, Peng Gao, Kunchang Li, Y. Qiao, Xupeng Miao, Bin Cui•Fri Dec 03 2021

Recently, zero-shot and few-shot learning via Contrastive Vision-Language Pre-training (CLIP) have shown inspirational performance on 2D visual recognition, which learns to match images with their corresponding texts in open-vocabulary settings. However, it remains under explored that whether CLIP, pre-trained by large-scale image-text pairs in 2D, can be generalized to 3D recognition. In this paper, we identify such a setting is feasible by proposing PointCLIP, which conducts alignment between CLIP-encoded point clouds and 3D category texts. Specifically, we encode a point cloud by projecting it onto multi-view depth maps and aggregate the view-wise zero-shot prediction in an end-to-end manner, which achieves efficient knowledge transfer from 2D to 3D. We further design an inter-view adapter to better extract the global feature and adaptively fuse the 3D few-shot knowledge into CLIP pre-trained in 2D. By just fine-tuning the adapter under few-shot settings, the performance of PointCLIP could be largely improved. In addition, we observe the knowledge complementary property between PointCLIP and classical 3D-supervised networks. Via simple ensemble during inference, PointCLIP contributes to favorable performance enhancement over state-of-the-art 3D networks. Therefore, PointCLIP is a promising alternative for effective 3D point cloud understanding under low data regime with marginal resource cost. We conduct thorough experiments on Model-NetlO, ModelNet40 and ScanObjectNN to demonstrate the effectiveness of PointCLIP. Code is available at https://github.com/ZrrSkywalker/PointCLIP.

583 0

Paper Graph

CLIP2Point: Transfer CLIP to Point Cloud Classification with Image-Depth Pre-Training

Wanli Ouyang, W. Zuo, Rynson W. H. Lau, Tianyu Huang, Bowen Dong, Yunhan Yang, Xiaoshui Huang•Sun Oct 02 2022

Experimental results show that CLIP2Point is effective in transferring CLIP knowledge to 3D vision, and a novel Gated Dual-Path Adapter (GDPA) allows the ensemble of CLIP and CLIP2Point, tuning pre-training knowledge to downstream tasks in an efficient adaptation.

201 0

Paper Graph

Uni3D: Exploring Unified 3D Representation at Scale

Junsheng Zhou, Jinsheng Wang, Baorui Ma, Yu-Shen Liu, Tiejun Huang, Xinlong Wang•Mon Oct 09 2023

Uni3D, a 3D foundation model to explore the unified 3D representation at scale, efficiently scale up Uni3D to one billion parameters, and set new records on a broad range of 3D tasks, such as zero-shot classification, few- shot classification, open-world understanding and part segmentation.

174 0

Paper Graph

ULIP: Learning a Unified Representation of Language, Images, and Point Clouds for 3D Understanding

S. Savarese, Jiajun Wu, Juan Carlos Niebles, Caiming Xiong, Ran Xu, Roberto Mart'in-Mart'in, Le Xue, Mingfei Gao, Chen Xing•Fri Dec 09 2022

ULIP is introduced to learn a unified representation of image, text, and 3D point cloud by pre-training with object triplets from the three modalities, using a pre-trained vision-language model that has already learned a common visual and textual space by training with massive image-text pairs.

301 0

Paper Graph

Sculpting Holistic 3D Representation in Contrastive Language-Image-3D Pre-Training

Cihang Xie, Yuyin Zhou, Yipeng Gao, Zeyu Wang, Wei-Shi Zheng•Thu Nov 02 2023

MixCon3D, a simple yet effective method aiming to sculpt holistic 3D representation in contrastive language-image-3D pre-training, surpassing the previous state-of-the-art performance on the challenging 1,156- category Objaverse-LVIS dataset by 5.7%.

16 0

Paper Graph

ViT-Lens: Initiating Omni-Modal Exploration through 3D Insights

Weixian Lei, Yixiao Ge, Jianfeng Zhang, Dylan Sun, Kun Yi, Ying Shan, Mike Zheng Shou•Sat Aug 19 2023

ViT-Lens provides a unified solution for representation learning of increasing modalities with two appealing benefits: (i) Exploiting the pretrained ViT across tasks and domains effectively with efficient data regime; (ii) Emergent downstream capabilities of novel modalities are demonstrated due to the modality alignment space.

1 0

Paper Graph

OpenShape: Scaling Up 3D Shape Representation Towards Open-World Understanding

Hao Su, F. Porikli, Minghua Liu, Yinhao Zhu, Xuanlin Li, H. Cai, Ruoxi Shi, Kaiming Kuang, Shizhong Han•Wed May 17 2023

This work adopts the commonly used multi-modal contrastive learning framework for representation alignment, but with a specific focus on scaling up 3D representations to enable open-world 3D shape understanding, and proposes several strategies to automatically filter and enrich noisy text descriptions.

182 0

Paper Graph

Adding a benchmark result helps the community track progress.