DialBERT: A Hierarchical Pre-Trained Model for Conversation Disentanglement
Published in
arXiv.org(2020)External Links:
TL
TL;DR
A new model, named Dialogue BERT (DialBERT), is proposed, which integrates local and global semantics in a single stream of messages to disentangle the conversations that mixed together and achieves a state-of-the-art result on the a new dataset proposed by IBM.
Abstract
Disentanglement is a problem in which multiple conversations occur in the same channel simultaneously, and the listener should decide which utterance is part of the conversation he will respond to. We propose a new model, named Dialogue BERT (DialBERT), which integrates local and global semantics in a single stream of messages to disentangle the conversations that mixed together. We employ BERT to capture the matching information in each utterance pair at the utterance-level, and use a BiLSTM to aggregate and incorporate the context-level information. With only a 3% increase in parameters, a 12% improvement has been attained in comparison to BERT, based on the F1-Score. The model achieves a state-of-the-art result on the a new dataset proposed by IBM and surpasses previous work by a substantial margin.
References34 items
1
Who did They Respond to? Conversation Structure Modeling using Masked Hierarchical Transformer
2
The Eighth Dialog System Technology Challenge
3
DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter
4
ALBERT: A Lite BERT for Self-supervised Learning of Language Representations
5