Detecting Human-Object Interaction via Fabricated Compositional Learning (2021-03-15T00:00:00.000000Z)

TL;DR

A novel HOI compositional learning framework is devised, termed as Fabricated Compositional Learning (FCL), to address the problem of open long-tailed HOI detection, which introduces an object fabricator to generate effective object representations, and then combines verbs and fabricated objects to compose new HOI samples.

Abstract

Human-Object Interaction (HOI) detection, inferring the relationships between human and objects from images/videos, is a fundamental task for high-level scene understanding. However, HOI detection usually suffers from the open long-tailed nature of interactions with objects, while human has extremely powerful compositional perception ability to cognize rare or unseen HOI samples. Inspired by this, we devise a novel HOI compositional learning framework, termed as Fabricated Compositional Learning (FCL), to address the problem of open long-tailed HOI detection. Specifically, we introduce an object fabricator to generate effective object representations, and then combine verbs and fabricated objects to compose new HOI samples. With the proposed object fabricator, we are able to generate large-scale HOI samples for rare and unseen categories to alleviate the open long-tailed issues in HOI detection. Extensive experiments on the most popular HOI detection dataset, HICO-DET, demonstrate the effectiveness of the proposed method for imbalanced HOI detection and significantly improve the state-of-the-art performance on rare and unseen HOI categories. Code is available at https://github.com/zhihou7/HOI-CL.

Authors

Y. Qiao

19 papers

D. Tao

37 papers

B. Yu

2 papers

TL;DR

Abstract

Authors

References70 items

Generative Adversarial Networks

DRG: Dual Relation Graph for Human-Object Interaction Detection

Perceiving 3D Human-Object Spatial Arrangements from a Single Image in the Wild

Visual Compositional Learning for Human-Object Interaction Detection

Discovering Human Interactions With Novel Objects via Zero-Shot Learning

Detailed 2D-3D Joint Representation for Human-Object Interaction

Generalized Zero-Shot Learning via Over-Complete Distribution

Learning Human-Object Interaction Detection Using Interaction Points

VSGNet: Spatial Attention Network for Detecting Human Object Interactions Using Graph Convolutions

PPDM: Parallel Point Detection and Matching for Real-Time Human-Object Interaction Detection

Decoupling Representation and Classifier for Long-Tailed Recognition

Deep Contextual Attention for Human-Object Interaction Detection

Relation Parsing Neural Network for Human-Object Interaction Detection

Gaussian Affinity for Max-Margin Class Imbalanced Learning

Pose-Aware Multi-Level Feature Network for Human Object Interaction Detection

Learning Imbalanced Datasets with Label-Distribution-Aware Margin Loss

Learning to Detect Human-Object Interactions With Knowledge

LVIS: A Dataset for Large Vocabulary Instance Segmentation

On Exploring Undetermined Relationships for Visual Relationship Detection

Large-Scale Long-Tailed Recognition in an Open World

Detecting Human-Object Interactions via Functional Generalization

LaSO: Label-Set Operations Networks for Multi-Label Few-Shot Learning

Class-Balanced Loss Based on Effective Number of Samples

Learning Compositional Representations for Few-Shot Recognition

Detecting Unseen Visual Relations Using Analogies

Generalized Zero- and Few-Shot Learning via Aligned Variational Autoencoders

Transferable Interactiveness Prior for Human-Object Interaction Detection

No-Frills Human-Object Interaction Detection: Factorization, Layout Encodings, and Training Techniques

Compositional Learning for Human Object Interaction

Learning Human-Object Interactions by Graph Parsing Neural Networks

iCAN: Instance-Centric Attention Network for Human-Object Interaction Detection

Scaling Human-Object Interaction Recognition Through Zero-Shot Learning

Low-Shot Learning from Imaginary Data

Generalized Zero-Shot Learning via Synthesized Examples

Feature Generating Networks for Zero-Shot Learning

Focal Loss for Dense Object Detection

Zero-Shot Learning—A Comprehensive Evaluation of the Good, the Bad and the Ugly

Detecting and Recognizing Human-Object Interactions

Prototypical Networks for Few-shot Learning

Learning to Detect Human-Object Interactions

Range Loss for Deep Face Recognition with Long-Tailed Training Data

Understanding deep learning requires rethinking generalization

Visual Relationship Detection with Language Priors

Learning Deep Representation for Imbalanced Classification

Matching Networks for One Shot Learning

Improved Techniques for Training GANs

Low-Shot Visual Recognition by Shrinking and Hallucinating Features

An Empirical Study and Analysis of Generalized Zero-Shot Learning for Object Recognition in the Wild

What can the brain teach us about building artificial intelligence?

Stacked Hourglass Networks for Human Pose Estimation

Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks

Visual Semantic Role Labeling

GloVe: Global Vectors for Word Representation

Microsoft COCO: Common Objects in Context

Auto-Encoding Variational Bayes

DeViSE: A Deep Visual-Semantic Embedding Model

Learning to detect unseen object classes by between-class attribute transfer

One-shot learning of object categories

Borderline-SMOTE: A New Over-Sampling Method in Imbalanced Data Sets Learning

The class imbalance problem: A systematic study

Recognition-by-components: a theory of human image understanding.

Parts of recognition

Learning From Imbalanced Data

Learning to Model the Tail

C4.5, Class Imbalance, and Cost Sensitivity: Why Under-Sampling beats Over-Sampling

This Paper Is Included in the Proceedings of the 12th Usenix Symposium on Operating Systems Design and Implementation (osdi '16). Tensorflow: a System for Large-scale Machine Learning Tensorflow: a System for Large-scale Machine Learning

For λ 3 , we provide the ablated experi

Additional Ablation Study Step-wise optimization. We also provide the comparison between step-wise optimization and one-step optimization in unseen object HOI detection in Table 15. Hyper-Parameters

Figure 7. Visual Comparison between FCL and our baseline. The two models use same detector. Figure 8. Illustration of failure cases

small objects also cause that the network detect object wrongly

Field of Study

Journal Information

Name

Page

Venue Information

Name