Relation-Aware Compositional Zero-Shot Learning for Attribute-Object Pair Recognition (2021-08-10T00:00:00.000000Z)

TL;DR

A novel model for recognizing images with composite attribute-object concepts, notably for composite concepts that are unseen during model training is proposed and a blocking mechanism is proposed that equalizes the information available to the model for both seen and unseen concepts.

Abstract

This paper proposes a novel model for recognizing images with composite attribute-object concepts, notably for composite concepts that are unseen during model training. We aim to explore the three key properties required by the task — relation-aware, consistent, and decoupled—to learn rich and robust features for primitive concepts that compose attribute-object pairs. To this end, we propose the Blocked Message Passing Network (BMP-Net). The model consists of two modules. The concept module generates semantically meaningful features for primitive concepts, whereas the visual module extracts visual features for attributes and objects from input images. A message passing mechanism is used in the concept module to capture the relations between primitive concepts. Furthermore, to prevent the model from being biased towards seen composite concepts and reduce the entanglement between attributes and objects, we propose a blocking mechanism that equalizes the information available to the model for both seen and unseen concepts. Extensive experiments and ablation studies on two benchmarks show the efficacy of the proposed model.

Authors

Mohan Kankanhalli

13 papers

Yongkang Wong

7 papers

Guangzhi Wang

2 papers

TL;DR

Abstract

Authors

References87 items

Cascaded Parsing of Human-Object Interaction Recognition

Generalized Zero-Shot Learning Via Multi-Modal Aggregated Posterior Aligning Neural Network

Joint Attribute Manipulation and Modality Alignment Learning for Composing Text and Image to Image Retrieval

Region Graph Embedding Network for Zero-Shot Learning

Invertible Zero-Shot Recognition Flows

A causal view of compositional zero-shot recognition

A Novel Perspective to Zero-Shot Learning: Towards an Alignment of Manifold Structures via Semantic Feature Expansion

Symmetry and Group in Attribute-Object Compositions

GradMix: Multi-source Transfer across Domains and Tasks

CI-GNN: Building a Category-Instance Graph for Zero-Shot Video Classification

12-in-1: Multi-Task Vision and Language Representation Learning

Erasing-based Attention Learning for Visual Question Answering

Zero-Shot Video Object Segmentation via Attentive Graph Neural Networks

Adversarial Fine-Grained Composition Learning for Unseen Attribute-Object Recognition

Attribute Attention for Semantic Disambiguation in Zero-Shot Learning

Spatiotemporal Co-Attention Recurrent Neural Networks for Human-Skeleton Motion Prediction

Embedding Symbolic Knowledge into Deep Networks

Explainable Video Action Reasoning via Prior Knowledge and State Transitions

Domain-Specific Embedding Network for Zero-Shot Recognition

ViLBERT: Pretraining Task-Agnostic Visiolinguistic Representations for Vision-and-Language Tasks

Recognizing Unseen Attribute-Object Pair with Generative Model

I Know the Relationships: Zero-Shot Action Recognition via Two-Stream Graph Convolutional Networks and Knowledge Graphs

Learning to Detect Human-Object Interactions With Knowledge

Task-Driven Modular Networks for Zero-Shot Compositional Learning

Reasoning Visual Dialogs With Structural and Partial Observations

F-VAEGAN-D2: A Feature Generating Framework for Any-Shot Learning

Interact as You Intend: Intention-Driven Human-Object Interaction Detection

Learning Human-Object Interactions by Graph Parsing Neural Networks

Attributes as Operators

Zero-Shot Recognition via Semantic Embeddings and Knowledge Graphs

Neural Algebra of Classifiers

Zero-Shot Visual Recognition Using Semantics-Preserving Adversarial Embedding Networks

Feature Generating Networks for Zero-Shot Learning

Jointly Recognizing Object Fluents and Tasks in Egocentric Videos

Zero-Shot Learning—A Comprehensive Evaluation of the Good, the Bad and the Ugly

From Red Wine to Red Tomato: Composition with Context

Removing Rain from Single Images via a Deep Detail Network

A Unified Approach for Conventional Zero-Shot, Generalized Zero-Shot, and Few-Shot Learning

Attention is All you Need

Complexity and compositionality in fluid intelligence

Semantic Autoencoder for Zero-Shot Learning

Joint Intermodal and Intramodal Label Transfers for Extremely Rare or Unseen Classes

Joint Discovery of Object States and Manipulation Actions

Beyond a Gaussian Denoiser: Residual Learning of Deep CNN for Image Denoising

Key-Value Memory Networks for Directly Reading Documents

A Decomposable Attention Model for Natural Language Inference

Recovering the Missing Link: Predicting Class-Attribute Associations for Unsupervised Zero-Shot Learning

FractalNet: Ultra-Deep Neural Networks without Residuals

An Empirical Study and Analysis of Generalized Zero-Shot Learning for Object Recognition in the Wild

Learning Attributes Equals Multi-Source Domain Generalization

Synthesized Classifiers for Zero-Shot Learning

Recognizing an Action Using Its Name: A Knowledge-Based Approach

Deep Residual Learning for Image Recognition

Towards Open Set Deep Networks

Inverting Visual Representations with Convolutional Networks

Discovering states and transformations in image collections

Human Action Recognition Bases on Local Action Attributes

End-To-End Memory Networks

Exploring Semantic Inter-Class Relationships (SIR) for Zero-Shot Action Recognition

Factorized Similarity Learning in Networks

GloVe: Global Vectors for Word Representation

Large-Scale Object Classification Using Label Relation Graphs

Neural Machine Translation by Jointly Learning to Align and Translate

Fine-Grained Visual Comparisons with Local Learning

Inferring Analogous Attributes

Auto-Encoding Variational Bayes

BabyTalk: Understanding and Generating Simple Image Descriptions

Label-Embedding for Attribute-Based Classification

Correlated attribute transfer with multi-task graph-guided fusion

Improving neural networks by preventing co-adaptation of feature detectors

Recommending Flickr groups with social topic model

Relative attributes

Recognition using visual phrases

Recognizing human actions by attributes

Sharing features between objects and their attributes

Grouplet: A structured image representation for recognizing human and object interactions