miscellaneous
Cross-Modal Retrieval
3260 papers • 126 benchmarks • 313 datasets
Cross-Modal Retrieval is used for implementing a retrieval task across different modalities. such as image-text, video-text, and audio-text Cross-Modal Retrieval. The main challenge of Cross-Modal Retrieval is the modality gap and the key solution of Cross-Modal Retrieval is to generate new representations from different modalities in the shared subspace, such that new generated features can be applied in the computation of distance metrics, such as cosine distance and Euclidean distance. References: [1] Scene-centric vs. Object-centric Image-Text Cross-modal Retrieval: A Reproducibility Study [2] Deep Triplet Neural Networks with Cluster-CCA for Audio-Visual Cross-modal Retrieval
(Image credit: Papersgraph)
Benchmarks
These leaderboards are used to track progress in cross-modal-retrieval
Trend
Dataset
Best Model
Actions
COCO 2014
COCO 2014
Flickr30k
Flickr30k
Libraries
iUse these libraries to find cross-modal-retrieval models and implementations
2 papers 49
Datasets
Subtasks
Most implemented papers
VSE++: Improving Visual-Semantic Embeddings with Hard Negatives
David J. Fleet, S. Fidler, Fartash Faghri, J. Kiros•Fri Jun 30 2017
A simple change to common loss functions used for multi-modal embeddings, inspired by hard negative mining, the use of hard negatives in structured prediction, and ranking loss functions, is introduced, which yields significant gains in retrieval performance.
1321
