Text2Mol: Cross-Modal Molecule Retrieval with Natural Language Queries (2021-01-01T00:00:00.000000Z)

TL;DR

This work constructs a paired dataset of molecules and their corresponding text descriptions, which it uses to learn an aligned common semantic embedding space for retrieval and extends this to create a cross-modal attention-based model for explainability and reranking by interpreting the attentions as association rules.

Abstract

We propose a new task, Text2Mol, to retrieve molecules using natural language descriptions as queries. Natural language and molecules encode information in very different ways, which leads to the exciting but challenging problem of integrating these two very different modalities. Although some work has been done on text-based retrieval and structure-based retrieval, this new task requires integrating molecules and natural language more directly. Moreover, this can be viewed as an especially challenging cross-lingual retrieval problem by considering the molecules as a language with a very unique grammar. We construct a paired dataset of molecules and their corresponding text descriptions, which we use to learn an aligned common semantic embedding space for retrieval. We extend this to create a cross-modal attention-based model for explainability and reranking by interpreting the attentions as association rules. We also employ an ensemble approach to integrate our different architectures, which significantly improves results from 0.372 to 0.499 MRR. This new multimodal approach opens a new perspective on solving problems in chemistry literature understanding and molecular machine learning.

Authors

Chengxiang Zhai

3 papers

Heng Ji

12 papers

Carl N. Edwards

4 papers

TL;DR

Abstract

Authors

References70 items

SAFRAN: An interpretable, rule-based link prediction method outperforming embedding models

Joint Biomedical Entity and Relation Extraction with Knowledge-Enhanced Collective Inference

Learning Transferable Visual Models From Natural Language Supervision

Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsity

Molecular representation learning with language models and domain-relevant auxiliary tasks

ChemBERTa: Large-Scale Self-Supervised Pretraining for Molecular Property Prediction

Deep Neural Matching Models for Graph Retrieval

An Overview of Cross-Language Information Retrieval

Cross-media Structured Common Space for Multimedia Event Extraction

Reinforced Anytime Bottom Up Rule Learning for Knowledge Graph Completion

Dataset’s chemical diversity limits the generalizability of machine learning predictions

Syntax-aware Multi-task Graph Convolutional Networks for Biomedical Relation Extraction

A Structure-Based Platform for Predicting Chemical Reactivity

Ensemble Learning

A Retrieval System of Medicine Molecules Based on Graph Similarity

UNITER: Learning UNiversal Image-TExt Representations

VL-BERT: Pre-training of Generic Visual-Linguistic Representations

ViLBERT: Pretraining Task-Agnostic Visiolinguistic Representations for Vision-and-Language Tasks

Anytime Bottom-Up Rule Learning for Knowledge Graph Completion

Trends in Integration of Vision and Language Research: A Survey of Tasks, Datasets, and Methods

SciBERT: A Pretrained Language Model for Scientific Text

Overview of the BioCreative VI Precision Medicine Track: mining protein interactions and mutations for precision medicine

PubChem 2019 update: improved access to chemical data

Deep learning for chemical reaction prediction

Sachem: a chemical cartridge for high-performance substructure search

Predicting reaction performance in C–N cross-coupling using machine learning

Mol2vec: Unsupervised Machine Learning Approach with Chemical Intuition

Chemical reaction vector embeddings: towards predicting drug metabolism in the human gut microbiome

Attention is All you Need

Information Retrieval and Text Mining Technologies for Chemistry.

Prediction of Organic Reaction Outcomes Using Machine Learning

Flexible Analog Search with Kernel PCA Embedded Molecule Vectors

MoleculeNet: a benchmark for molecular machine learning

Billion-Scale Similarity Search with GPUs

Improved Deep Metric Learning with Multi-class N-pair Loss Objective

Event Specific Multimodal Pattern Mining for Knowledge Base Construction

Using Word Embeddings for Query Translation for Hindi to English Cross Language Information Retrieval

Layer Normalization

ChEBI in 2016: Improved services and an expanding collection of metabolites

PubChem Substance and Compound databases

Monolingual and Cross-Lingual Information Retrieval Models Based on (Bilingual) Word Embeddings

Aiding face recognition with social context association rule based re-ranking

Adam: A Method for Stochastic Optimization

Entity linking for biomedical literature

Distributed Representations of Words and Phrases and their Compositionality

WBI-NER: The impact of domain-specific features on the performance of identifying and classifying mentions of drugs

Efficient Estimation of Word Representations in Vector Space

Mixture of experts: a literature survey

Chemical Name to Structure: OPSIN, an Open Source Solution

Chemical-Text Hybrid Search Engines

On linear mixture of expert approaches to information retrieval

Substructure similarity search in graph databases

Mining frequent patterns by pattern-growth: methodology and implications

Boosting and Other Ensemble Methods

Substructure searching methods: Old and new

Molecular Substructure Similarity Searching: Efficient Retrieval in Two-Dimensional Structure Databases.

Molecular substructure similarity searching: efficient retrieval in two-dimensional structure databases

SMILES. 2. Algorithm for generation of unique SMILES notation

SMILES, a chemical language and information system. 1. Introduction to methodology and encoding rules

Smiles

Fine-grained Information Extraction from Biomedical Literature based on Knowledge-enriched Abstract Meaning Representation

Rdkit: Open-source cheminformatics software

BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding

Biomedical Event Extraction based on Knowledge-driven Tree-LSTM

Accelerating structure search in small-molecule databases

Overview of the BioCreative VI Precision Medicine Track Mining protein interactions and mutations for precision medicine

Semi-supervised classiﬁcation with graph convolutional networks

Molecular fingerprint similarity search in virtual screening.

Extendedconnectivity fingerprints

Journal of Chemical Information and Modeling

Field of Study

Venue Information

Name

Type

URL

Alternate Names