Grounding Dialogue Systems via Knowledge Graph Aware Decoding with Pre-trained Transformers (2021-03-30T00:00:00.000000Z)

TL;DR

A novel architecture for integrating KGs into the response generation process by training a BERT model that learns to answer using the elements of the KG (entities and relations) in a multi-task, end-to-end setting is proposed.

Abstract

Generating knowledge grounded responses in both goal and non-goal oriented dialogue systems is an important research challenge. Knowledge Graphs (KG) can be viewed as an abstraction of the real world, which can potentially facilitate a dialogue system to produce knowledge grounded responses. However, integrating KGs into the dialogue generation process in an end-to-end manner is a non-trivial task. This paper proposes a novel architecture for integrating KGs into the response generation process by training a BERT model that learns to answer using the elements of the KG (entities and relations) in a multi-task, end-to-end setting. The k-hop subgraph of the KG is incorporated into the model during training and inference using Graph Laplacian. Empirical evaluation suggests that the model achieves better knowledge groundedness (measured via Entity F1 score) compared to other state-of-the-art models for both goal and non-goal oriented dialogues.

Authors

Jens Lehmann

7 papers

Md. Rashad Al Hasan Rony

2 papers

Debanjan Chaudhuri

3 papers

TL;DR

Abstract

Authors

References41 items

Improving Variational Autoencoder for Text Modelling with Timestep-Wise Regularisation

Hierarchical Transformer for Task Oriented Dialog Systems

DIALOGPT : Large-Scale Generative Pre-training for Conversational Response Generation

Transformer-based Model for Single Documents Neural Summarization

Using a KG-Copy Network for Non-goal Oriented Dialogues

DLGNet: A Transformer-based Model for Dialogue Response Generation

Introduction to Neural Network based Approaches for Question Answering over Knowledge Graphs

Learning Representation Mapping for Relation Detection in Knowledge Base Question Answering

Learning Deep Transformer Models for Machine Translation

Incorporating Joint Embeddings into Goal-Oriented Dialogues with Multi-task Learning

A Differentiable Generative Adversarial Network for Open Domain Dialogue

Transformer-Based Neural Network for Answer Selection in Question Answering

Global-to-local Memory Pointer Networks for Task-Oriented Dialogue

Interpretable Graph Convolutional Neural Networks for Inference on Noisy Knowledge Graphs

MultiWOZ - A Large-Scale Multi-Domain Wizard-of-Oz Dataset for Task-Oriented Dialogue Modelling

Multi-turn Dialogue Response Generation in an Adversarial Learning Framework

SimpleQuestions Nearly Solved: A New Upperbound and Baseline Approach

Mem2Seq: Effectively Incorporating Knowledge Bases into End-to-End Task-Oriented Dialog Systems

Strong Baselines for Simple Question Answering over Knowledge Graphs with and without Neural Networks

Graph Attention Networks

Core techniques of question answering systems over knowledge bases: a survey

Relevance of Unsupervised Metrics in Task-Oriented Dialogue for Evaluating Natural Language Generation

Attention is All you Need

Key-Value Retrieval Networks for Task-Oriented Dialogue

Improved Neural Relation Detection for Knowledge Base Question Answering

Learning Discourse-level Diversity for Neural Dialog Models using Conditional Variational Autoencoders

Geometric Deep Learning: Going beyond Euclidean data

Semi-Supervised Classification with Graph Convolutional Networks

Towards End-to-End Reinforcement Learning of Dialogue Agents for Information Access

Learning End-to-End Goal-Oriented Dialog

A Network-based End-to-End Trainable Task-oriented Dialogue System

Effective Approaches to Attention-based Neural Machine Translation

Large-scale Simple Question Answering with Memory Networks

Graph Pattern Matching: Do We Have to Reinvent the Wheel?

METEOR: An Automatic Metric for MT Evaluation with Improved Correlation with Human Judgments

Word Reordering and a Dynamic Programming Beam Search Algorithm for Statistical Machine Translation

Bleu: a Method for Automatic Evaluation of Machine Translation

Long Short-Term Memory

BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding

What Should Entity Linking link?

Grounding Dialogue Systems via Knowledge Graph Aware Decoding 15

Field of Study

Journal Information

Name

Volume

Venue Information

Name

Type

URL

Alternate Names