Re-Thinking Co-Salient Object Detection (2020-07-07T00:00:00.000000Z)

TL;DR

A novel CoEG-Net is proposed that augments the authors' prior model EGNet with a co-attention projection strategy to enable fast common information learning and fully leverages previous large-scale SOD datasets and significantly improves the model scalability and stability.

Abstract

In this article, we conduct a comprehensive study on the co-salient object detection (CoSOD) problem for images. CoSOD is an emerging and rapidly growing extension of salient object detection (SOD), which aims to detect the co-occurring salient objects in a group of images. However, existing CoSOD datasets often have a serious data bias, assuming that each group of images contains salient objects of similar visual appearances. This bias can lead to the ideal settings and effectiveness of models trained on existing datasets, being impaired in real-life situations, where similarities are usually semantic or conceptual. To tackle this issue, we first introduce a new benchmark, called CoSOD3k in the wild, which requires a large amount of semantic context, making it more challenging than existing CoSOD datasets. Our CoSOD3k consists of 3,316 high-quality, elaborately selected images divided into 160 groups with hierarchical annotations. The images span a wide range of categories, shapes, object sizes, and backgrounds. Second, we integrate the existing SOD techniques to build a unified, trainable CoSOD framework, which is long overdue in this field. Specifically, we propose a novel CoEG-Net that augments our prior model EGNet with a co-attention projection strategy to enable fast common information learning. CoEG-Net fully leverages previous large-scale SOD datasets and significantly improves the model scalability and stability. Third, we comprehensively summarize 40 cutting-edge algorithms, benchmarking 18 of them over three challenging CoSOD datasets (iCoSeg, CoSal2015, and our CoSOD3k), and reporting more detailed (i.e., group-level) performance analysis. Finally, we discuss the challenges and future works of CoSOD. We hope that our study will give a strong boost to growth in the CoSOD community. The benchmark toolbox and results are available on our project page at https://dpfan.net/CoSOD3K.

References134 items

Boundary-Aware Segmentation Network for Mobile and Web Applications

Revisiting Video Saliency Prediction in the Deep Learning Era

Uncertainty Inspired RGB-D Saliency Detection

Siamese Network for RGB-D Salient Object Detection and Beyond

Co-Saliency Detection With Co-Attention Fully Convolutional Network

RGB-D salient object detection: A survey

Co-Saliency Detection Using Collaborative Feature Extraction And High-To-Low Feature Integration

SRFlow: Learning the Super-Resolution Space with Normalizing Flow

Taking a Deeper Look at Co-Salient Object Detection

Camouflaged Object Detection

Gradient-Induced Co-Saliency Detection

JL-DCF: Joint Learning and Densely-Cooperative Fusion Framework for RGB-D Salient Object Detection

UC-Net: Uncertainty Inspired RGB-D Saliency Detection via Conditional Variational Autoencoders

Adaptive Graph Convolutional Network With Attention Graph Clustering for Co-Saliency Detection

ICNet: Information Conversion Network for RGB-D Based Salient Object Detection

Robust Deep Co-Saliency Detection With Group Semantic and Pyramid Attention

Co-saliency detection via integration of multi-layer convolutional features and inter-image propagation

Explorable Super Resolution

A Unified Multiple Graph Learning and Convolutional Network Model for Co-saliency Estimation

Co-saliency Detection Based on Hierarchical Consistency

Group-Wise Deep Object Co-Segmentation With Co-Attention Recurrent Neural Network

An easy-to-hard learning strategy for within-image co-saliency detection

EGNet: Edge Guidance Network for Salient Object Detection

Towards High-Resolution Salient Object Detection

Detecting Robust Co-Saliency with Recurrent Co-Attention Neural Network

Robust Deep Co-Saliency Detection with Group Semantic

Rethinking RGB-D Salient Object Detection: Models, Data Sets, and Large-Scale Benchmarks

Multiple Graph Convolutional Networks for Co-Saliency Detection

Nested Network With Two-Stream Pyramid for Salient Object Detection in Optical Remote Sensing Images

DeepCO3: Deep Instance Co-Segmentation by Co-Peak Search and Co-Saliency Detection

Shifting More Attention to Video Salient Object Detection

Co-Saliency Detection via Mask-Guided Fully Convolutional Networks With Multi-Scale Label Smoothing

CapSal: Leveraging Captioning to Boost Semantics for Salient Object Detection

Contrast Prior and Fluid Pyramid Integration for RGBD Salient Object Detection

Joint salient object detection and existence prediction

Video Saliency Detection via Sparsity-Based Reconstruction and Propagation

Salient Object Detection in the Deep Learning Era: An In-Depth Survey

Cascaded Partial Decoder for Fast and Accurate Salient Object Detection

Deep Group-Wise Fully Convolutional Network for Co-Saliency Detection With Graph Propagation

Res2Net: A New Multi-Scale Backbone Architecture

Unsupervised object discovery and co-localization by deep descriptor transformation

Image Co-Saliency Detection and Co-Segmentation via Progressive Joint Optimization

Selectivity or Invariance: Boundary-Aware Salient Object Detection

PartNet: A Large-Scale Benchmark for Fine-Grained and Hierarchical Part-Level 3D Object Understanding

A Feature-Adaptive Semi-Supervised Framework for Co-saliency Detection

A Unified Metric Learning-Based Framework for Co-Saliency Detection

Unsupervised CNN-Based Co-saliency Detection with Graphical Optimization

Co-attention CNNs for Unsupervised Object Co-segmentation

Enhanced-alignment Measure for Binary Foreground Map Evaluation

Co-Saliency Detection Within a Single Image

Salient Objects in Clutter: Bringing Salient Object Detection to the Foreground

Review of Visual Saliency Detection With Comprehensive Information

A Review of Co-Saliency Detection Algorithms

Quality-Guided Fusion-Based Co-Saliency Estimation for Image Co-Segmentation and Colocalization

Hierarchical Co-salient Object Detection via Color Names

Supervision by Fusion: Towards Unsupervised Learning of Deep Salient Object Detector

PiCANet: Learning Pixel-Wise Contextual Attention for Saliency Detection

Amulet: Aggregating Multi-level Convolutional Features for Salient Object Detection

Structure-Measure: A New Way to Evaluate Foreground Maps

Group-wise Deep Co-saliency Detection

Learning to Detect Salient Objects with Image-Level Supervision

What is and What is Not a Salient Object? Learning Salient Object Detector by Ensembling Linear Exemplar Regressors

Revisiting Co-Saliency Detection: A Novel Approach Based on Two-Stage Multi-View Spectral Rotation Co-clustering

Co-Salient Object Detection Based on Deep Saliency Networks and Seed Propagation Over an Integrated Graph

Co-Saliency Detection via a Self-Paced Multiple-Instance Learning Framework

Instance-Level Salient Object Segmentation

FLIC: Fast linear iterative clustering with active search

Saliency Detection with Recurrent Fully Convolutional Networks

Semi-Supervised Classification with Graph Convolutional Networks

DHSNet: Deep Hierarchical Saliency Network for Salient Object Detection

Cosaliency Detection Based on Intrasaliency Prior Transfer and Deep Intersaliency Mining

Detection of Co-salient Objects by Looking Deep and Wide

Higher-Order Image Co-segmentation

Deep Contrast Learning for Salient Object Detection

Learning Deep Features for Discriminative Localization

Deep Residual Learning for Image Recognition

A Self-Paced Multiple-Instance Learning Framework for Co-Saliency Detection

Co-Saliency Detection via Co-Salient Object Discovery and Recovery

Object-Based Multiple Foreground Video Co-Segmentation via Multi-State Selection Graph

Co-saliency detection via looking deep and wide

Efficient Saliency-Model-Guided Visual Co-Saliency Detection

Visual saliency based on multiscale deep features

Salient Object Detection: A Benchmark

Salient object detection: A survey

Co-Saliency Detection via Base Reconstruction

Going deeper with convolutions

Edge Boxes: Locating Object Proposals from Edges

Very Deep Convolutional Networks for Large-Scale Image Recognition

ImageNet Large Scale Visual Recognition Challenge

Co-Saliency Detection Based on Hierarchical Segmentation

Co-saliency detection based on region-level fusion and pixel-level refinement

Self-Adaptively Weighted Co-Saliency Detection via Rank Constraint

How to Evaluate Foreground Maps

Return of the Devil in the Details: Delving Deep into Convolutional Nets

Microsoft COCO: Common Objects in Context

OverFeat: Integrated Recognition, Localization and Detection using Convolutional Networks

Auto-Encoding Variational Bayes

A Model of Visual Attention for Natural Image Retrieval

Co-Salient Object Detection From Multiple Images

100

Cosegmentation and Cosketch by Unsupervised Learning

101

Efficient Salient Region Detection with Soft Image Abstraction

102

Cluster-Based Co-Saliency Detection

103

SalientShape: group saliency in image collections

104

Hierarchical Saliency Detection

105

Saliency Detection via Graph-Based Manifold Ranking

106

Looking Beyond the Image: Unsupervised Learning for Object Saliency and Detection

107

Efficient Inference in Fully Connected CRFs with Gaussian Edge Potentials

108

A Co-Saliency Model of Image Pairs

109

Efficient manifold ranking for image retrieval

110

Unbiased look at dataset bias

111

Global contrast based salient region detection

112

Contour Detection and Hierarchical Image Segmentation

113

Preattentive co-saliency detection

114

Cosaliency: where people look when comparing images

115

iCoseg: Interactive co-segmentation with intelligent scribble guidance

116

Frequency-tuned salient region detection

117

ImageNet: A large-scale hierarchical image database

118

Dynamic visual attention: searching for coding length increments

119

Learning to Detect A Salient Object

120

Saliency Detection: A Spectral Residual Approach

121

Image Segmentation by Probabilistic Bottom-Up Aggregation and Cue Integration

122

Pattern

123

Object categorization by learned universal visual dictionary

124

Distinctive Image Features from Scale-Invariant Keypoints

125

Learning with Local and Global Consistency

126

A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics

127

The discrimination of visual number.

128

LIII. On lines and planes of closest fit to systems of points in space

129

Co-Saliency Detection via a General Optimization Model and Adaptive Graph Learning

130

He joined Inception Institute of Artificial Intelligence (IIAI)

131

Deep Learning intra-image and inter-images features for Co-saliency detection

132

Learning Deep Architectures for AI

133

A Threshold Selection Method from Gray-Level Histograms

134