Person Search with Natural Language Description (2017-02-19T00:00:00.000000Z)

TL;DR

An Recurrent Neural Network with Gated Neural Attention mechanism (GNA-RNN) is proposed to establish the state-of-the art performance on person search and a large-scale person description dataset with detailed natural language annotations and person samples from various sources is collected.

Abstract

Searching persons in large-scale image databases with the query of natural language description has important applications in video surveillance. Existing methods mainly focused on searching persons with image-based or attribute-based queries, which have major limitations for a practical usage. In this paper, we study the problem of person search with natural language description. Given the textual description of a person, the algorithm of the person search is required to rank all the samples in the person database then retrieve the most relevant sample corresponding to the queried description. Since there is no person dataset or benchmark with textual description available, we collect a large-scale person description dataset with detailed natural language annotations and person samples from various sources, termed as CUHK Person Description Dataset (CUHK-PEDES). A wide range of possible models and baselines have been evaluated and compared on the person search benchmark. An Recurrent Neural Network with Gated Neural Attention mechanism (GNA-RNN) is proposed to establish the state-of-the art performance on person search.

Authors

Xiaogang Wang

21 papers

Bolei Zhou

20 papers

Tong Xiao

4 papers

TL;DR

Abstract

Authors

References45 items

Object Detection in Videos with Tubelet Proposal Networks

DualNet: Domain-invariant network for visual question answering

Multimodal Compact Bilinear Pooling for Visual Question Answering and Visual Grounding

Learning Deep Representations of Fine-Grained Visual Descriptions

Deep Attributes Driven Multi-Camera Person Re-identification

Object Detection from Video Tubelets with Convolutional Neural Networks

T-CNN: Tubelets With Convolutional Neural Networks for Object Detection From Videos

End-to-End Deep Learning for Person Search

Segmentation from Natural Language Expressions

Visual Genome: Connecting Language and Vision Using Crowdsourced Dense Image Annotations

Deep Residual Learning for Image Recognition

Simple Baseline for Visual Question Answering

DenseCap: Fully Convolutional Localization Networks for Dense Captioning

Image Question Answering Using Convolutional Neural Network with Dynamic Parameter Prediction

Natural Language Object Retrieval

Stacked Attention Networks for Image Question Answering

Multi-task deep visual-semantic embedding for video thumbnail selection

Are You Talking to a Machine? Dataset and Methods for Multilingual Image Question

Exploring Models and Data for Image Question Answering

Image Question Answering: A Visual Semantic Embedding Model and a New Dataset

Ask Your Neurons: A Neural-Based Approach to Answering Questions about Images

VQA: Visual Question Answering

Microsoft COCO Captions: Data Collection and Evaluation Server

Show, Attend and Tell: Neural Image Caption Generation with Visual Attention

Person Re-identification Meets Image Search

Deep Captioning with Multimodal Recurrent Neural Networks (m-RNN)

Deep visual-semantic alignments for generating image descriptions

Learning a Recurrent Visual Representation for Image Caption Generation

From captions to visual concepts and back

Show and tell: A neural image caption generator

Pedestrian Attribute Recognition At Far Distance

DeepReID: Deep Filter Pairing Neural Network for Person Re-identification

Person re-identification by Local Maximal Occurrence representation and metric learning

Microsoft COCO: Common Objects in Context

From image descriptions to visual denotations: New similarity metrics for semantic inference over event descriptions

DeViSE: A Deep Visual-Semantic Embedding Model

Framing Image Description as a Ranking Task: Data, Models and Evaluation Metrics

ImageNet classification with deep convolutional neural networks

Human Reidentification with Transferred Metric Learning

Person re-identification by probabilistic relative distance comparison

Caltech-UCSD Birds 200

Attribute-based people search in surveillance environments

Automated Flower Classification over a Large Number of Classes

Long Short-Term Memory

Evaluating Appearance Models for Recognition, Reacquisition, and Tracking

Field of Study

Journal Information

Name

Page

Venue Information

Name

Type

URL

Alternate Names