Enhancing Interpretable Clauses Semantically using Pretrained Word Representation (2021-04-14T00:00:00.000000Z)

TL;DR

A novel way of using pre-trained word representations for TM is proposed, which enhances the performance and interpretability of TM by extracting semantically related words from pre- trained word representations as input features to the TM.

Abstract

Tsetlin Machine (TM) is an interpretable pattern recognition algorithm based on propositional logic, which has demonstrated competitive performance in many Natural Language Processing (NLP) tasks, including sentiment analysis, text classification, and Word Sense Disambiguation. To obtain human-level interpretability, legacy TM employs Boolean input features such as bag-of-words (BOW). However, the BOW representation makes it difficult to use any pre-trained information, for instance, word2vec and GloVe word representations. This restriction has constrained the performance of TM compared to deep neural networks (DNNs) in NLP. To reduce the performance gap, in this paper, we propose a novel way of using pre-trained word representations for TM. The approach significantly enhances the performance and interpretability of TM. We achieve this by extracting semantically related words from pre-trained word representations as input features to the TM. Our experiments show that the accuracy of the proposed approach is significantly higher than the previous BOW-based TM, reaching the level of DNN-based models.

Authors

Ole-Christoffer Granmo

9 papers

Lei Jiao

4 papers

Rohan Kumar Yadav

1 papers

TL;DR

Abstract

Authors

References38 items

Human-Level Interpretable Learning for Aspect-Based Sentiment Analysis

Measuring the Novelty of Natural Language Text Using the Conjunctive Clauses of a Tsetlin Machine Text Classifier

Why is Attention Not So Interpretable

Enriching feature engineering for short text samples by language time series analysis

Feature Projection for Improved Text Classification

BAE: BERT-based Adversarial Examples for Text Classification

Label-guided Learning for Text Classification

XLNet: Generalized Autoregressive Pretraining for Language Understanding

The Convolutional Tsetlin Machine

A Scheme for Continuous Input to the Tsetlin Machine with Applications to Forecasting Disease Outbreaks

Using the Tsetlin Machine to Learn Human-Interpretable Rules for High-Accuracy Text Categorization With Medical Applications

Baseline Needs More Love: On Simple Word-Embedding-Based Models and Associated Pooling Mechanisms

The Tsetlin Machine - A Game Theoretic Bandit Driven Approach to Optimal Pattern Recognition with Propositional Logic

Enriching Word Vectors with Subword Information

Bag of Tricks for Efficient Text Classification

Convolutional Neural Networks on Graphs with Fast Localized Spectral Filtering

Recurrent Neural Network for Text Classification with Multi-Task Learning

PTE: Predictive Text Embedding through Large-scale Heterogeneous Text Networks

Text Categorization as a Graph Classification Problem

GloVe: Global Vectors for Word Representation

Convolutional Neural Networks for Sentence Classification

Distributed Representations of Sentences and Documents

Distributed Representations of Words and Phrases and their Compositionality

Parsing with Compositional Vector Grammars

Baselines and Bigrams: Simple, Good Sentiment and Topic Classification

Word Representations: A Simple and General Method for Semi-Supervised Learning

Seeing Stars: Exploiting Class Relationships for Sentiment Categorization with Respect to Rating Scales

Learning Question Classifiers

Interpretability in Word Sense Disambiguation using Tsetlin Machine

Simple Spectral Graph Convolution

Question Classification using Interpretable Tsetlin Machine

Mining Interpretable Rules for Sentiment and Semantic Relation Analysis Using Tsetlin Machines

BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding

Is attention interpretable? In ACL, pages 2931–2951, Florence, Italy

Bridging semantics and syntax with graph algorithms - state-of-the-art of extracting biomedical relations

An Introduction to Information Retrieval

Distributional Structure

Proceedings of the Twenty-Second International Joint Conference on Artificial Intelligence Concept Labeling: Building Text Classifiers with Minimal Supervision

Field of Study

Venue Information

Name

Type

URL

Alternate Names