What Makes A Good Story? Designing Composite Rewards for Visual Storytelling
TL
TL;DR
This paper proposes three assessment criteria: relevance, coherence and expressiveness, which are observed through empirical analysis could constitute a “high-quality” story to the human eye and proposes a reinforcement learning framework, ReCo-RL, with reward functions designed to capture the essence of these quality criteria.
Abstract
Previous storytelling approaches mostly focused on optimizing traditional metrics such as BLEU, ROUGE and CIDEr. In this paper, we re-examine this problem from a different angle, by looking deep into what defines a natural and topically-coherent story. To this end, we propose three assessment criteria: relevance, coherence and expressiveness, which we observe through empirical analysis could constitute a “high-quality” story to the human eye. We further propose a reinforcement learning framework, ReCo-RL, with reward functions designed to capture the essence of these quality criteria. Experiments on the Visual Storytelling Dataset (VIST) with both automatic and human evaluation demonstrate that our ReCo-RL model achieves better performance than state-of-the-art baselines on both traditional metrics and the proposed new criteria.
References42 items
1
UNITER: UNiversal Image-TExt Representation Learning
2
UNITER: Learning UNiversal Image-TExt Representations
3
Domain Adaptive Text Style Transfer
4
Plan-And-Write: Towards Better Automatic Storytelling
5