MDMMT: Multidomain Multimodal Transformer for Video Retrieval
TL
TL;DR
A new state-of-the-art on the text-to-video retrieval task on MSRVTT and LSMDC benchmarks where the model outperforms all previous solutions by a large margin and is achieved using a single model and without finetuning.
Abstract
We present a new state-of-the-art on the text-to-video re-trieval task on MSRVTT and LSMDC benchmarks where our model outperforms all previous solutions by a large margin. Moreover, state-of-the-art results are achieved using a single model and without finetuning. This multidomain generalisation is achieved by a proper combination of different video caption datasets. We show that our practical approach for training on different datasets can improve test results of each other. Additionally, we check intersection between many popular datasets and show that MSRVTT as well as ActivityNet contains a significant overlap between the test and the training parts. More details are available at https://github.com/papermsucode/mdmmt.
References50 items
1
TRECVID 2020: A comprehensive campaign for evaluating video retrieval tasks across multiple application domains
2
Learning Transferable Visual Models From Natural Language Supervision
3
A Straightforward Framework For Video Retrieval Using CLIP
4
A Short Note on the Kinetics-700-2020 Human Action Dataset
5