n-Reference Transfer Learning for Saliency Prediction (2020-07-09T00:00:00.000000Z)

TL;DR

A few-shot transfer learning paradigm for saliency prediction is proposed, which enables efficient transfer of knowledge learned from the existing large-scale saliency datasets to a target domain with limited labeled examples.

Abstract

Benefiting from deep learning research and large-scale datasets, saliency prediction has achieved significant success in the past decade. However, it still remains challenging to predict saliency maps on images in new domains that lack sufficient data for data-hungry models. To solve this problem, we propose a few-shot transfer learning paradigm for saliency prediction, which enables efficient transfer of knowledge learned from the existing large-scale saliency datasets to a target domain with limited labeled examples. Specifically, very few target domain examples are used as the reference to train a model with a source domain dataset such that the training process can converge to a local minimum in favor of the target domain. Then, the learned model is further fine-tuned with the reference. The proposed framework is gradient-based and model-agnostic. We conduct comprehensive experiments and ablation study on various source domain and target domain pairs. The results show that the proposed framework achieves a significant performance improvement. The code is publicly available at \url{this https URL}.

Authors

Mohan Kankanhalli

13 papers

Qi Zhao

6 papers

Yongkang Wong

7 papers

TL;DR

Abstract

Authors

References53 items

Foundations of Machine Learning

GradMix: Multi-source Transfer across Domains and Tasks

Direction Concentration Learning: Enhancing Congruency in Machine Learning

Deep n-Shot Transfer Learning for Tactile Material Classification with a Flexible Pressure-Sensitive Skin

A Dilated Inception Network for Visual Saliency Prediction

Meta-Learning With Differentiable Convex Optimization

Revisiting Local Descriptor Based Image-To-Class Measure for Few-Shot Learning

SpotTune: Transfer Learning Through Adaptive Fine-Tuning

Explicit Inductive Bias for Transfer Learning with Convolutional Networks

Learning to Compare: Relation Network for Few-Shot Learning

Automatic differentiation in PyTorch

Two-Stage Transfer Learning of End-to-End Convolutional Neural Networks for Webpage Saliency Prediction

Attention Transfer from Web Images for Video Recognition

Five factors that guide attention in visual search

Borrowing Treasures from the Wealthy: Deep Transfer Learning through Selective Joint Fine-Tuning

SalGAN: Visual Saliency Prediction with Generative Adversarial Networks

Predicting Human Eye Fixations via an LSTM-Based Saliency Attentive Model

Aggregated Residual Transformations for Deep Neural Networks

A deep multi-level network for saliency prediction

Densely Connected Convolutional Networks

Matching Networks for One Shot Learning

What Do Different Evaluation Metrics Tell Us About Saliency Models?

Human-level concept learning through probabilistic program induction

Deep Residual Learning for Image Recognition

DeepFix: A Fully Convolutional Neural Network for Predicting Human Eye Fixations

SALICON: Saliency in Context

CAT2000: A Large Scale Fixation Dataset for Boosting Saliency Research

Adam: A Method for Stochastic Optimization

Deep Gaze I: Boosting Saliency Prediction with Feature Maps Trained on ImageNet

Saliency Detection: A Boolean Map Approach

Feature Selection for Multimedia Analysis by Sharing Information Among Multiple Tasks

ImageNet classification with deep convolutional neural networks

Foundations of Machine Learning

A Benchmark of Computational Models of Saliency to Predict Human Fixations

Data-driven visual similarity for cross-domain image matching

Deep Learning of Representations for Unsupervised and Transfer Learning

Learning to predict where humans look

Domain Adaptation via Transfer Component Analysis

ImageNet: A large-scale hierarchical image database

Graph-Based Visual Saliency

Contextual guidance of eye movements and attention in real-world scenes: the role of global features in object search.

Domain Adaptation for Statistical Classifiers

One-shot learning of object categories

Scene context guides eye movements during visual search

Realistic avatar eye and head animation using a neurobiological model of visual attention

A Model of Saliency-Based Visual Attention for Rapid Scene Analysis

A Comprehensive Survey on Domain Adaptation for Visual Applications

Webpage Saliency

Quantitative Analysis of Human-Model Agreement in Visual Saliency Modeling: A Comparative Study

Image Signature: Highlighting Sparse Salient Regions

One shot learning of simple visual concepts

Attention links sensing to recognition

Electronic Letters on Computer Vision and Image Analysis 3(1):13-24, 2004 Empirical Validation of the Saliency-based Model of Visual Attention

Field of Study

Venue Information

Name

Type

URL

Alternate Names