Neural Graph Embedding Methods for Natural Language Processing (2019-11-08T00:00:00.000000Z)

TL;DR

GCNs are utilized for Document Timestamping problem and for learning word embeddings using dependency context of a word instead of sequential context and two limitations of existing GCN models are addressed.

Abstract

Knowledge graphs are structured representations of facts in a graph, where nodes represent entities and edges represent relationships between them. Recent research has resulted in the development of several large KGs. However, all of them tend to be sparse with very few facts per entity. In the first part of the thesis, we propose two solutions to alleviate this problem: (1) KG Canonicalization, i.e., identifying and merging duplicate entities in a KG, (2) Relation Extraction which involves automating the process of extracting semantic relationships between entities from unstructured text. Traditional Neural Networks like CNNs and RNNs are constrained to handle Euclidean data. However, graphs in Natural Language Processing (NLP) are prominent. Recently, Graph Convolutional Networks (GCNs) have been proposed to address this shortcoming and have been successfully applied for several problems. In the second part of the thesis, we utilize GCNs for Document Timestamping problem and for learning word embeddings using dependency context of a word instead of sequential context. In this third part of the thesis, we address two limitations of existing GCN models, i.e., (1) The standard neighborhood aggregation scheme puts no constraints on the number of nodes that can influence the representation of a target node. This leads to a noisy representation of hub-nodes which coves almost the entire graph in a few hops. (2) Most of the existing GCN models are limited to handle undirected graphs. However, a more general and pervasive class of graphs are relational graphs where each edge has a label and direction associated with it. Existing approaches to handle such graphs suffer from over-parameterization and are restricted to learning representation of nodes only.

Authors

Shikhar Vashishth

4 papers

TL;DR

Abstract

Authors

References227 items

A Vectorized Relational Graph Convolutional Network for Multi-Relational Network Alignment

Adaptive Convolution for Multi-Relational Learning

Introduction to Data Mining

Confidence-based Graph Convolutional Networks for Semi-Supervised Learning

RESIDE: Improving Distantly-Supervised Neural Relation Extraction using Side Information

End-to-end Structure-Aware Convolutional Networks for Knowledge Base Completion

A Survey: Speech Recognition Approaches and Techniques

How Powerful are Graph Neural Networks?

RotatE: Knowledge Graph Embedding by Relational Rotation in Complex Space

Graph Convolution over Pruned Dependency Trees Improves Relation Extraction

Graph Convolutional Networks for Text Classification

Hypernetwork Knowledge Graph Embeddings

Object Detection With Deep Learning: A Review

Dating Documents using Graph Convolution Networks

Graph-to-Sequence Learning using Gated Graph Neural Networks

Representation Learning on Graphs with Jumping Knowledge Networks

Dual-Primal Graph Convolutional Networks

Lovasz Convolutional Networks

An End-to-End Deep Learning Architecture for Graph Classification

Graph Convolutional Networks With Argument-Aware Pooling for Event Detection

Training and Evaluating Improved Dependency-Based Word Embeddings

Linguistically-Informed Self-Attention for Semantic Role Labeling

Improving Distantly Supervised Relation Extraction using Word and Entity Based Attention

Higher-Order Coreference Resolution with Coarse-to-Fine Inference

CESI: Canonicalizing Open Knowledge Bases using Embeddings and Side Information

Jointly learning word embeddings using a corpus and a knowledge base

Omnidirectional CNN for Visual Place Recognition and Navigation

Deep Contextualized Word Representations

Learning Deep Generative Models of Graphs

Graph Partition Neural Networks for Semi-Supervised Classification

FastGCN: Fast Learning with Graph Convolutional Networks via Importance Sampling

SEE: Syntax-aware Entity Embedding for Neural Relation Extraction

A Novel Embedding Model for Knowledge Base Completion Based on Convolutional Neural Network

Knowledge Graph Embedding: A Survey of Approaches and Applications

Revisiting Simple Neural Networks for Learning Representations of Knowledge Graphs

Protein Interface Prediction using Graph Convolutional Networks

KBGAN: Adversarial Learning for Knowledge Graph Embeddings

Graph Attention Networks

Simple and Effective Multi-Paragraph Reading Comprehension

CANDiS: Coupled & Attention-Driven Neural Distant Supervision

KBlrn: End-to-End Learning of Knowledge Base Representations with Latent, Relational, and Numerical Features

A Soft-label Method for Noise-tolerant Distantly Supervised Relation Extraction

Gaussian Embeddings for Collaborative Filtering

A Hierarchical Algorithm for Extreme Clustering

Effective Deep Memory Networks for Distant Supervised Relation Extraction

Deep Gaussian Embedding of Graphs: Unsupervised Inductive Learning via Ranking

Convolutional 2D Knowledge Graph Embeddings

Bootstrapping for Numerical Open IE

Inductive Representation Learning on Large Graphs

Analogical Inference for Multi-relational Embeddings

Graph Convolutional Encoders for Syntax-aware Neural Machine Translation

Neural Message Passing for Quantum Chemistry

Modeling Relational Data with Graph Convolutional Networks

Encoding Sentences with Graph Convolutional Networks for Semantic Role Labeling

Distant Supervision for Relation Extraction with Sentence-Level Attention and Entity Descriptions

Noise Mitigation for Neural Entity Typing and Relation Extraction

Deep Learning on Lie Groups for Skeleton-Based Action Recognition

SyncSpecCNN: Synchronized Spectral CNN for 3D Shape Segmentation

CATENA: CAusal and TEmporal relation extraction from NAtural language texts

Geometric Deep Learning: Going beyond Euclidean data

Google’s Multilingual Neural Machine Translation System: Enabling Zero-Shot Translation

RDF2Vec: RDF Graph Embeddings for Data Mining

A Collection of Benchmark Datasets for Systematic Evaluations of Machine Learning on the Semantic Web

Xception: Deep Learning with Depthwise Separable Convolutions

Semi-Supervised Classification with Graph Convolutional Networks

Collaborative Knowledge Base Embedding for Recommender Systems

Neural Relation Extraction with Selective Attention over Instances

Open Information Extraction Systems and Downstream Applications

node2vec: Scalable Feature Learning for Networks

Convolutional Neural Networks on Graphs with Fast Localized Spectral Filtering

Complex Embeddings for Simple Link Prediction

SQuAD: 100,000+ Questions for Machine Comprehension of Text

Dependency Based Embeddings for Sentence Classification Tasks

Learning Convolutional Neural Networks for Graphs

Relation Schema Induction using Tensor Factorization with Side Information

As Time Goes By: Comprehensive Tagging of Textual Phrases with Temporal Scopes