A dataset for Movie Description
TL
TL;DR
Comparing ADs to scripts, it is found that ADs are far more visual and describe precisely what is shown rather than what should happen according to the scripts created prior to movie production.
Abstract
Audio Description (AD) provides linguistic descriptions of movies and allows visually impaired people to follow a movie along with their peers. Such descriptions are by design mainly visual and thus naturally form an interesting data source for computer vision and computational linguistics. In this work we propose a novel dataset which contains transcribed ADs, which are temporally aligned to full length HD movies. In addition we also collected the aligned movie scripts which have been used in prior work and compare the two different sources of descriptions. In total the MPII Movie Description dataset (MPII-MD) contains a parallel corpus of over 68K sentences and video snippets from 94 HD movies. We characterize the dataset by benchmarking different approaches for generating video descriptions. Comparing ADs to scripts, we find that ADs are far more visual and describe precisely what is shown rather than what should happen according to the scripts created prior to movie production.
References77 items
1
Using Descriptive Video Services to Create a Large Data Source for Video Annotation Research
2
Video Description Generation Incorporating Spatio-Temporal Features and a Soft-Attention Mechanism
3
Deep Captioning with Multimodal Recurrent Neural Networks (m-RNN)
4
Translating Videos to Natural Language Using Deep Recurrent Neural Networks
5