Rethinking Object Saliency Ranking: A Novel Whole-Flow Processing Paradigm (2023-12-06T00:00:00.000000Z)

TL;DR

An in-depth analysis of the causes and a whole-flow processing paradigm of saliency ranking task from the perspective of “GT data generation”, “network structure design” and “training protocol”, which outperforms existing state-of-the-art methods on the widely-used SALICON set.

Abstract

Existing salient object detection methods are capable of predicting binary maps that highlight visually salient regions. However, these methods are limited in their ability to differentiate the relative importance of multiple objects and the relationships among them, which can lead to errors and reduced accuracy in downstream tasks that depend on the relative importance of multiple objects. To conquer, this paper proposes a new paradigm for saliency ranking, which aims to completely focus on ranking salient objects by their “importance order”. While previous works have shown promising performance, they still face ill-posed problems. First, the saliency ranking ground truth (GT) orders generation methods are unreasonable since determining the correct ranking order is not well-defined, resulting in false alarms. Second, training a ranking model remains challenging because most saliency ranking methods follow the multi-task paradigm, leading to conflicts and trade-offs among different tasks. Third, existing regression-based saliency ranking methods are complex for saliency ranking models due to their reliance on instance mask-based saliency ranking orders. These methods require a significant amount of data to perform accurately and can be challenging to implement effectively. To solve these problems, this paper conducts an in-depth analysis of the causes and proposes a whole-flow processing paradigm of saliency ranking task from the perspective of “GT data generation”, “network structure design” and “training protocol”. The proposed approach outperforms existing state-of-the-art methods on the widely-used SALICON set, as demonstrated by extensive experiments with fair and reasonable comparisons. The saliency ranking task is still in its infancy, and our proposed unified framework can serve as a fundamental strategy to guide future work. The code and data will be available at https://github.com/MengkeSong/Saliency-Ranking-Paradigm.

References58 items

SegGPT: Towards Segmenting Everything In Context

Advances in deep concealed scene understanding

Segment Anything

Learning Complementary Spatial–Temporal Transformer for Video Salient Object Detection

Disentangled Capsule Routing for Fast Part-Object Relational Saliency

Improving RGB-D Salient Object Detection via Modality-Aware Decoder

Learning Implicit Class Knowledge for RGB-D Co-Salient Object Detection With Transformers

Contextual Similarity Distillation for Asymmetric Image Retrieval

Learning based Multi-modality Image and Video Compression

Rethinking Video Salient Object Ranking

Bi-directional Object-Context Prioritization Learning for Saliency Ranking

RGB-T salient object detection via CNN feature and result saliency map fusion

SalFBNet: Learning Pseudo-Saliency Distribution via Feedback Convolutional Networks

TranSalNet: Towards perceptually relevant visual saliency prediction

Looking at Boundary: Siamese Densely Cooperative Fusion for Salient Object Detection

Scene Context-Aware Salient Object Detection

RGB-D Saliency Detection via Cascaded Mutual Information Minimization

Instance-Level Relative Saliency Ranking With Graph Reasoning

Salient Object Ranking with Position-Preserved Attention

Simultaneously Localize, Segment and Rank the Camouflaged Objects

Concealed Object Detection

Salient Object Detection via Integrity Learning

Personal Fixations-Based Object Segmentation With Object Localization and Boundary Preservation

Sparse R-CNN: End-to-End Object Detection with Learnable Proposals

An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale

Uncertainty Inspired RGB-D Saliency Detection

RGBD Salient Object Detection via Disentangled Cross-Modal Fusion

Inferring Salient Objects from Human Fixations

Re-Thinking Co-Salient Object Detection

Inferring Attention Shift Ranks of Objects for Image Saliency

End-to-End Object Detection with Transformers

Unified Image and Video Saliency Modeling

Improved Saliency Detection in RGB-D Images Using Two-Phase Depth Estimation and Selective Deep Fusion

Visual Scanpath Prediction Using IOR-ROI Recurrent Mixture Density Network

EfficientDet: Scalable and Efficient Object Detection

Shifting More Attention to Video Salient Object Detection

An Iterative and Cooperative Top-Down and Bottom-Up Inference Network for Salient Object Detection

Salient Object Detection With Pyramid Attention and Salient Edges

Salient Object Detection in the Deep Learning Era: An In-Depth Survey

M3S-NIR: Multi-modal Multi-scale Noise-Insensitive Ranking for RGB-T Saliency Detection

Relative Saliency and Ranking: Models, Metrics, Data and Benchmarks

EML-NET: An Expandable Multi-Layer NETwork for Saliency Prediction

Revisiting Salient Object Detection: Simultaneous Detection, Ranking, and Subitizing of Multiple Salient Objects

Video Saliency Detection via Spatial-Temporal Fusion and Low-Rank Coherency Diffusion

Deep Visual Attention Prediction

SalGAN: Visual Saliency Prediction with Generative Adversarial Networks

A deep multi-level network for saliency prediction

Two-Stage Learning to Predict Human Eye Fixations via SDAEs

SALICON: Saliency in Context

Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks

Geodesic Saliency Using Background Priors

Global contrast based salient region detection

Predicting human gaze using low-level saliency combined with face detection

A Model of Saliency-Based Visual Attention for Rapid Scene Analysis

Unsupervised Multi-Subclass Saliency Classification for Salient Object Detection

at 06:31:44 UTC from IEEE Xplore. Restrictions apply

RETHINKING OBJECT SALIENCY RANKING: A NOVEL WHOLE-FLOW PROCESSING PARADIGM

license agreement with IEEE. Restrictions apply