SCDV : Sparse Composite Document Vectors using soft clustering over distributional representations (2016-12-20T00:00:00.000000Z)

TL;DR

Through extensive experiments on multi-class and multi-label classification tasks, this work outperforms the previous state-of-the-art method, NTSG and achieves a significant reduction in training and prediction times compared to other representation methods.

Abstract

We present a feature vector formation technique for documents - Sparse Composite Document Vector (SCDV) - which overcomes several shortcomings of the current distributional paragraph vector representations that are widely used for text representation. In SCDV, word embeddings are clustered to capture multiple semantic contexts in which words occur. They are then chained together to form document topic-vectors that can express complex, multi-topic documents. Through extensive experiments on multi-class and multi-label classification tasks, we outperform the previous state-of-the-art method, NTSG. We also show that SCDV embeddings perform well on heterogeneous tasks like Topic Coherence, context-sensitive Learning and Information Retrieval. Moreover, we achieve a significant reduction in training and prediction times compared to other representation methods. SCDV achieves best of both worlds - better performance with lower time and space complexity.

Authors

Vivek Gupta

3 papers

Dheeraj Mekala

1 papers

Bhargavi Paranjape

2 papers

TL;DR

Abstract

Authors

References37 items

LTSG: Latent Topical Skip-Gram for Mutually Improving Topic Model and Vector Representations

Gaussian Mixture Models

Bag-of-concepts: Comprehending document representation through clustering words in distributed representation

LTSG: Latent Topical Skip-Gram for Mutually Learning Topic Model and Vector Representations

Integrating Topic Modeling with Word Embeddings by Mixtures of vMFs

Improving Distributed Word Representation and Topic Model by Word-Topic Mixture Model

Analysis of the Paragraph Vector Model for Information Retrieval

Improving Language Estimation with the Paragraph Vector Model for Ad-hoc Retrieval

Representing Documents and Queries as Sets of Word Embedded Vectors for Information Retrieval

Generative Topic Embedding: a Continuous Representation of Documents

Product Classification in E-Commerce using Distributional Semantics

Mixing Dirichlet Topic Models and Word Embeddings to Make lda2vec

Topic Modeling Using Distributed Word Embeddings

Linear Algebraic Structure of Word Senses, with Applications to Polysemy

Sparse Local Embeddings for Extreme Multi-label Classification

Words are not Equal: Graded Weighting Model for Building Composite Document Vectors

Learning Context-Sensitive Word Embeddings with Neural Tensor Skip-Gram Model

Gaussian LDA for Topic Models with Word Embeddings

Topic2Vec: Learning distributed representations of topics

Improving Topic Models with Latent Feature Word Representations

Unsupervised Topic Modeling for Short Texts Using Distributed Representations of Words

A Latent Variable Model Approach to PMI-based Word Embeddings

Topical Word Embeddings

Topic Modeling using Topics from Many Domains, Lifelong Learning and Big Data

Distributed Representations of Sentences and Documents

Distributed Representations of Words and Phrases and their Compositionality

Recursive Deep Models for Semantic Compositionality Over a Sentiment Treebank

Linguistic Regularities in Continuous Space Word Representations

A Practical Algorithm for Topic Modeling with Provable Guarantees

Optimizing Semantic Coherence in Topic Models

Reading Tea Leaves: How Humans Interpret Topic Models

A study of smoothing methods for language models applied to information retrieval

Model-based feedback in the language modeling approach to information retrieval

Two/Too Simple Adaptations of Word2Vec for Syntax Problems

Structured Regularizers for High-Dimensional Problems : Statistical and Computational Issues

Latent Dirichlet Allocation

Distributional Structure

Field of Study

Venue Information

Name

Type

URL

Alternate Names