A Simple Language Model for Task-Oriented Dialogue
Published in
Neural Information Processing Systems(2020)External Links:
TL
TL;DR
SimpleTOD is a simple approach to task-oriented dialogue that uses a single causal language model trained on all sub-tasks recast as a single sequence prediction problem, which allows it to fully leverage transfer learning from pre-trained, open domain, causal language models such as GPT-2.
Abstract
Task-oriented dialogue is often decomposed into three tasks: understanding user input, deciding actions, and generating a response. While such decomposition might suggest a dedicated model for each sub-task, we find a simple, unified approach leads to state-of-the-art performance on the MultiWOZ dataset. SimpleTOD is a simple approach to task-oriented dialogue that uses a single causal language model trained on all sub-tasks recast as a single sequence prediction problem. This allows SimpleTOD to fully leverage transfer learning from pre-trained, open domain, causal language models such as GPT-2. SimpleTOD improves over the prior state-of-the-art by 0.49 points in joint goal accuracy for dialogue state tracking. More impressively, SimpleTOD also improves the main metrics used to evaluate action decisions and response generation in an end-to-end setting for task-oriented dialog systems: inform rate by 8.1 points, success rate by 9.7 points, and combined score by 7.2 points.
References69 items
1
End-to-End Neural Pipeline for Goal-Oriented Dialogue Systems using GPT-2
2
SOLOIST: Few-shot Task-Oriented Dialog with A Single Pre-trained Auto-regressive Model
3
TripPy: A Triple Copy Strategy for Value Independent Neural Dialog State Tracking
4
Recipes for Building an Open-Domain Chatbot
5