3D AffordanceNet: A Benchmark for Visual Object Affordance Understanding (2021-03-30T00:00:00.000000Z)

TL;DR

Comprehensive results on the contributed dataset show the promise of visual affordance understanding as a valuable yet challenging benchmark.

Abstract

The ability to understand the ways to interact with objects from visual cues, a.k.a. visual affordance, is essential to vision-guided robotic research. This involves categorizing, segmenting and reasoning of visual affordance. Relevant studies in 2D and 2.5D image domains have been made previously, however, a truly functional understanding of object affordance requires learning and prediction in the 3D physical domain, which is still absent in the community. In this work, we present a 3D AffordanceNet dataset, a bench-mark of 23k shapes from 23 semantic object categories, annotated with 18 visual affordance categories. Based on this dataset, we provide three benchmarking tasks for evaluating visual affordance understanding, including full-shape, partial-view and rotation-invariant affordance estimations. Three state-of-the-art point cloud deep learning networks are evaluated on all tasks. In addition we also investigate a semi-supervised learning setup to explore the possibility to benefit from unlabeled data. Comprehensive results on our contributed dataset show the promise of visual affordance understanding as a valuable yet challenging benchmark.

Authors

K. Jia

12 papers

Chaozheng Wu

2 papers

Sheng Deng

1 papers

TL;DR

Abstract

Authors

References34 items

PointContrast: Unsupervised Pre-training for 3D Point Cloud Understanding

PartNet: A Recursive Part Decomposition Network for Fine-Grained and Hierarchical Shape Segmentation

PartNet: A Large-Scale Benchmark for Fine-Grained and Hierarchical Part-Level 3D Object Understanding

Patch-Based Progressive 3D Point Set Upsampling

Visual Affordance and Function Understanding

Dynamic Graph CNN for Learning on Point Clouds

Learning to Act Properly: Predicting and Explaining Affordances from Images

Learning SO(3) Equivariant Representations with Spherical CNNs

AffordanceNet: An End-to-End Deep Learning Approach for Object Affordance Detection

Object-based affordances detection with Convolutional Neural Networks and dense Conditional Random Fields

Predicting Human Activities Using Stochastic Grammar

Weakly Supervised Affordance Detection

PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space

Virtual Adversarial Training: A Regularization Method for Supervised and Semi-Supervised Learning

Weight-averaged consistency targets improve semi-supervised deep learning results

A scalable active framework for region annotation in 3D shape collections

A Multi-scale CNN for Affordance Segmentation in RGB Images

Temporal Ensembling for Semi-Supervised Learning

Semantic Understanding of Scenes Through the ADE20K Dataset

Learning how objects function via co-analysis of interactions

V-Net: Fully Convolutional Neural Networks for Volumetric Medical Image Segmentation

Learning to Detect Visual Grasp Affordance

ShapeNet: An Information-Rich 3D Model Repository

Structural-RNN: Deep Learning on Spatio-Temporal Graphs

Understanding tools: Task-oriented object modeling, learning and recognition

Affordance detection of tool parts from geometric features

Predicting Actions from Static Scenes

Anticipating Human Activities Using Object Affordances for Reactive Robotic Response

Learning human activities and object affordances from RGB-D videos

What makes a chair a chair?

Direct visibility of point sets

An efficient context-free parsing algorithm

An efficient context-free parsing algorithm

The Ecological Approach to Visual Perception

Field of Study

Journal Information

Name

Page

Venue Information

Name

Type

URL

Alternate Names