A Hybrid Neural Network Model for Commonsense Reasoning (2019-07-27T00:00:00.000000Z)

TL;DR

An ablation study shows that language models and semantic similarity models are complementary approaches to commonsense reasoning, and HNN effectively combines the strengths of both.

Abstract

This paper proposes a hybrid neural network(HNN) model for commonsense reasoning. An HNN consists of two component models, a masked language model and a semantic similarity model, which share a BERTbased contextual encoder but use different model-specific input and output layers. HNN obtains new state-of-the-art results on three classic commonsense reasoning tasks, pushing the WNLI benchmark to 89%, the Winograd Schema Challenge (WSC) benchmark to 75.1%, and the PDP60 benchmark to 90.0%. An ablation study shows that language models and semantic similarity models are complementary approaches to commonsense reasoning, and HNN effectively combines the strengths of both. The code and pre-trained models will be publicly available at https: //github.com/namisan/mt-dnn.

Authors

Jianfeng Gao

39 papers

Weizhu Chen

13 papers

Xiaodong Liu

8 papers

TL;DR

Abstract

Authors

References35 items

Probing Neural Network Comprehension of Natural Language Arguments

A Surprisingly Robust Trick for the Winograd Schema Challenge

Improving Multi-Task Deep Neural Networks via Knowledge Distillation for Natural Language Understanding

Unsupervised Deep Structured Semantic Models for Commonsense Reasoning

Multi-Task Deep Neural Networks for Natural Language Understanding

ReCoRD: Bridging the Gap between Human and Machine Commonsense Reading Comprehension

SWAG: A Large-Scale Adversarial Dataset for Grounded Commonsense Inference

Neural Approaches to Conversational AI

A Simple Method for Commonsense Reasoning

Event2Mind: Commonsense Inference on Events, Intents, and Reactions

GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding

Averaging Weights Leads to Wider Optima and Better Generalization

Combing Context and Commonsense Knowledge Through Neural Networks for Solving Winograd Schema Problems

Event Detection and Co-reference with Minimal Supervision

A Corpus and Cloze Evaluation for Deeper Understanding of Commonsense Stories

Commonsense reasoning and commonsense knowledge in artificial intelligence

Towards Addressing the Winograd Schema Challenge - Building and Using a Semantic Parser and a Knowledge Hunting Module

Representation Learning Using Multi-Task Deep Neural Networks for Semantic Classification and Information Retrieval

Adam: A Method for Stochastic Optimization

Tackling Winograd Schemas by Formalizing Relevance Theory in Knowledge Graphs

Learning deep structured semantic models for web search using clickthrough data

Resolving Complex Cases of Definite Pronouns: The Winograd Schema Challenge

The Winograd Schema Challenge

Choice of Plausible Alternatives: An Evaluation of Commonsense Causal Reasoning

Improving Machine Learning Approaches to Coreference Resolution

A Machine Learning Approach to Coreference Resolution of Noun Phrases

Multitask Learning

Language Models are Unsupervised Multitask Learners

BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding

During prediction, We ﬁrst replace the pronoun in a given sentence with its candidates one by one

These two models use the same approach as BERT LARGE-LM , but are trained with different additional training data

It is the unsupervised semantic matching model trained on the dataset generated with heuristic rules

Addressing the Winograd Schema Challenge as a Sequence Ranking Task

The Winograd Schema Challenge and Reasoning about Correlation

The trophy doesn’t ﬁt in the brown suitcase because it is too big. What is too big? A. The trophy B

Field of Study

Journal Information

Name

Volume

Venue Information

Name

Type

URL

Alternate Names