A Molecular Multimodal Foundation Model Associating Molecule Graphs with Natural Language

Published in

arXiv.org(2022)

External Links:

Generate Graph DownloadPDF

TL;DR

A molecular multimodal foundation model which is pretrained from molecular graphs and their semantically related textual data via contrastive learning which enhances molecular property prediction and possesses capability to generate meaningful molecular graphs from natural language descriptions is proposed.

Abstract

Although artificial intelligence (AI) has made significant progress in understanding molecules in a wide range of fields, existing models generally acquire the single cognitive ability from the single molecular modality. Since the hierarchy of molecular knowledge is profound, even humans learn from different modalities including both intuitive diagrams and professional texts to assist their understanding. Inspired by this, we propose a molecular multimodal foundation model which is pretrained from molecular graphs and their semantically related textual data (crawled from published Scientific Citation Index papers) via contrastive learning. This AI model represents a critical attempt that directly bridges molecular graphs and natural language. Importantly, through capturing the specific and complementary information of the two modalities, our proposed model can better grasp molecular expertise. Experimental results show that our model not only exhibits promising performance in cross-modal tasks such as cross-modal retrieval and molecule caption, but also enhances molecular property prediction and possesses capability to generate meaningful molecular graphs from natural language descriptions. We believe that our model would have a broad impact on AI-empowered fields across disciplines such as biology, chemistry, materials, environment, and medicine, among others.

Authors

Anyi Rao

5 papers

Ji-rong Wen

11 papers

Yujie Zhou

2 papers

References80 items

Pre-training via Denoising for Molecular Property Prediction

Translation between Molecules and Natural Language

A deep-learning system bridging molecule structure and biomedical text with comprehension comparable to human professionals

MGCVAE: Multi-objective Inverse Design via Molecular Graph Conditional Variational Autoencoder

Bootstrapping Informative Graph Augmentation via A Meta Learning Approach

A Molecular Multimodal Foundation Model Associating Molecule Graphs with Natural Language

Published in

arXiv.org(2022)

External Links:

Generate Graph DownloadPDF

TL;DR

Abstract

Authors

Anyi Rao

5 papers

Ji-rong Wen

11 papers

Yujie Zhou

2 papers

References80 items

Pre-training via Denoising for Molecular Property Prediction

Translation between Molecules and Natural Language

A deep-learning system bridging molecule structure and biomedical text with comprehension comparable to human professionals

MGCVAE: Multi-objective Inverse Design via Molecular Graph Conditional Variational Autoencoder

Bootstrapping Informative Graph Augmentation via A Meta Learning Approach

Zhao-Qing Yang

1 papers

Jiangmeng Li

2 papers

An adaptive graph learning method for automated molecular interactions and properties predictions

Language models can learn complex molecular distributions

Towards artificial general intelligence via a multimodal foundation model

GF-VAE: A Flow-based Variational Autoencoder for Molecule Generation

Improving de novo molecular design with curriculum learning

Supervision Exists Everywhere: A Data Efficient Contrastive Language-Image Pre-training Paradigm

Multi-constraint molecular generation based on conditional transformer, knowledge distillation and reinforcement learning

Chemical-Reaction-Aware Molecule Representation Learning

Adversarial Graph Augmentation to Improve Graph Contrastive Learning

Algebraic graph-assisted bidirectional transformers for molecular property prediction

Learning Transferable Visual Models From Natural Language Supervision

Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervision

GraphDF: A Discrete Flow Model for Molecular Graph Generation

Applications of Deep Learning in Molecule Generation and Molecular Property Prediction.

A deep generative model for molecule optimization via one fragment modification

Optimizing molecules using efficient queries from property evaluations

Masked graph modeling for molecule generation

Graph Contrastive Learning with Augmentations

ChemBERTa: Large-Scale Self-Supervised Pretraining for Molecular Property Prediction

MoFlow: An Invertible Flow Model for Generating Molecular Graphs

Open Graph Benchmark: Datasets for Machine Learning on Graphs

Direct steering of de novo molecular generation with descriptor conditional recurrent neural networks

Oscar: Object-Semantics Aligned Pre-training for Vision-Language Tasks

5分で分かる!? 有名論文ナナメ読み：Jacob Devlin et al. : BERT : Pre-training of Deep Bidirectional Transformers for Language Understanding

A Simple Framework for Contrastive Learning of Visual Representations

Multi-Objective Molecule Generation using Interpretable Substructures

GraphAF: a Flow-based Autoregressive Model for Molecular Graph Generation

PyTorch: An Imperative Style, High-Performance Deep Learning Library

Momentum Contrast for Unsupervised Visual Representation Learning

SMILES Transformer: Pre-trained Molecular Fingerprint for Low Data Drug Discovery

SMILES-BERT: Large Scale Unsupervised Pre-Training for Molecular Property Prediction

MolecularRNN: Generating realistic molecular graphs with optimized properties

Strategies for Pre-training Graph Neural Networks

SciBERT: A Pretrained Language Model for Scientific Text

BioBERT: a pre-trained biomedical language representation model for biomedical text mining

A graph-convolutional neural network model for the prediction of chemical reactivity

How Powerful are Graph Neural Networks?

Deep Graph Infomax

Accelerated discovery of stable lead-free hybrid organic-inorganic perovskites via machine learning

Learning deep representations by mutual information estimation and maximization

Representation Learning with Contrastive Predictive Coding

Predicting molecular properties with covariant compositional networks.

Unsupervised Feature Learning via Non-parametric Instance Discrimination

Syntax-Directed Variational Autoencoder for Structured Data

Junction Tree Variational Autoencoder for Molecular Graph Generation

Generating Focused Molecule Libraries for Drug Discovery with Recurrent Neural Networks

Inductive Representation Learning on Large Graphs

Neural Message Passing for Quantum Chemistry

Grammar Variational Autoencoder

MoleculeNet: a benchmark for molecular machine learning

Computational Modeling of β-Secretase 1 (BACE-1) Inhibitors Using Ligand Based Approaches

Automatic Chemical Design Using a Data-Driven Continuous Representation of Molecules

ToxCast Chemical Landscape: Paving the Road to 21st Century Toxicology.

Counting on natural products for drug design.

The SIDER database of drugs and side effects

SWEETLEAD: an In Silico Database of Approved Drugs, Regulated Chemicals, and Herbal Isolates for Computer-Aided Drug Discovery

A Bayesian Approach to in Silico Blood-Brain Barrier Penetration Modeling

ZINC: A Free Tool to Discover Chemistry for Biology

Effectiveness of 2D fingerprints for scaffold hopping.

Design, synthesis and selection of DNA-encoded small-molecule libraries.

PubChem: a public information system for analyzing bioactivities of small molecules

A decade of fragment-based drug design: strategic advances and lessons learned

Text2Mol: Cross-Modal Molecule Retrieval with Natural Language Queries

S2ORC: The Semantic Scholar Open Research Corpus

BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding

Biomedical Event Extraction based on Knowledge-driven Tree-LSTM

Tox21 data challenge 2014

Visualizing Data using t-SNE

Bing Su, Dazhao Du, and Ji-Rong Wen designed the methodology of the study

and Zhao Yang wrote the original draft. All co-authors discussed the results, performed analyses, revised and reviewed the manuscript

Aids antiviral screen data

Competing interests statement

Molecular contrastive learning of representations via graph neural networks

Field of Study

Computer Science

Journal Information

Name

ArXiv

Volume

abs/2005.00687

Venue Information

Name

arXiv.org

Type

URL

https://arxiv.org

Alternate Names

ArXiv