Empowering Molecule Discovery for Molecule-Caption Translation With Large Language Models: A ChatGPT Perspective

Published in

IEEE Transactions on Knowledge and Data Enginee...(2023)

External Links:

Generate Graph DownloadPDF

TL;DR

This work proposes a novel LLM-based framework (MolReGPT) for molecule-caption translation, where an In-Context Few-Shot Molecule Learning paradigm is introduced to empower molecule discovery with LLMs like ChatGPT to perform their in-context learning capability without domain-specific pre-training and fine-tuning.

Abstract

Molecule discovery plays a crucial role in various scientific fields, advancing the design of tailored materials and drugs, which contributes to the development of society and human well-being. Specifically, molecule-caption translation is an important task for molecule discovery, aligning human understanding with molecular space. However, most of the existing methods heavily rely on domain experts, require excessive computational cost, or suffer from sub-optimal performance. On the other hand, Large Language Models (LLMs), like ChatGPT, have shown remarkable performance in various cross-modal tasks due to their powerful capabilities in natural language understanding, generalization, and in-context learning (ICL), which provides unprecedented opportunities to advance molecule discovery. Despite several previous works trying to apply LLMs in this task, the lack of domain-specific corpus and difficulties in training specialized LLMs still remain challenges. In this work, we propose a novel LLM-based framework (MolReGPT) for molecule-caption translation, where an In-Context Few-Shot Molecule Learning paradigm is introduced to empower molecule discovery with LLMs like ChatGPT to perform their in-context learning capability without domain-specific pre-training and fine-tuning. MolReGPT leverages the principle of molecular similarity to retrieve similar molecules and their text descriptions from a local database to enable LLMs to learn the task knowledge from context examples. We evaluate the effectiveness of MolReGPT on molecule-caption translation, including molecule understanding and text-based molecule generation. Experimental results show that compared to fine-tuned models, MolReGPT outperforms MolT5-base and is comparable to MolT5-large without additional training. To the best of our knowledge, MolReGPT is the first work to leverage LLMs via in-context learning in molecule-caption translation for advancing molecule discovery. Our work expands the scope of LLM applications, as well as providing a new paradigm for molecule discovery and design.

Empowering Molecule Discovery for Molecule-Caption Translation With Large Language Models: A ChatGPT Perspective

Published in

IEEE Transactions on Knowledge and Data Enginee...(2023)

External Links:

Generate Graph DownloadPDF

TL;DR

Abstract

References72 items

Neural scaling of deep chemical models

The future of chemistry is language

TALLRec: An Effective and Efficient Tuning Framework to Align Large Language Model with Recommendation

MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Models

Deep Learning Methods for Small Molecule Drug Discovery: A Survey

LLaMA: Open and Efficient Foundation Language Models

ChatGPT: A Meta-Analysis after 2.5 Months

Generative Diffusion Models on Graphs: Methods and Applications

Advances in Molecular Design and Photophysical Engineering of Perylene Bisimide-Containing Polyads and Multichromophores for Film-Based Fluorescent Sensors.

Predictive Molecular Design and Structure–Property Validation of Novel Terpene-Based, Sustainably Sourced Bacterial Biofilm-Resistant Materials

Plasma etching effect on the molecular structure of chitosan-based hydrogels and its biological properties.

COVID-19 Vaccines: Computational tools and Development

Multi-modal molecule structure–text model for text-based retrieval and editing

Automatic Chain of Thought Prompting in Large Language Models

Protein structure-based in-silico approaches to drug discovery: Guide to COVID-19 therapeutics

A Molecular Multimodal Foundation Model Associating Molecule Graphs with Natural Language

Identifying the kind behind SMILES - anatomical therapeutic chemical classification using structure-only representations

Material Design for Next-Generation mRNA Vaccines Using Lipid Nanoparticles

Autonomous Chemistry Enabling Environment-Adaptive Electrochemical Energy Storage Devices

Translation between Molecules and Natural Language

PaLM: Scaling Language Modeling with Pathways

MolGenSurvey: A Systematic Survey in Machine Learning Models for Molecule Design

Application of deep metric learning to molecular graph similarity

Training language models to follow instructions with human feedback

A deep-learning system bridging molecule structure and biomedical text with comprehension comparable to human professionals

LaMDA: Language Models for Dialog Applications

The Commoditization of AI for Molecule Design.

Learning To Retrieve Prompts for In-Context Learning

MetaICL: Learning to Learn In Context

MolGPT: Molecular Generation Using a Transformer-Decoder Model

Multi-constraint molecular generation based on conditional transformer, knowledge distillation and reinforcement learning

Late-Stage Photoredox C-H Amidation of N-Unprotected Indole Derivatives: Access to N-(Indol-2-yl)amides.

What Makes Good In-Context Examples for GPT-3?

ChemBERTa: Large-Scale Self-Supervised Pretraining for Molecular Property Prediction

Language Models are Few-Shot Learners

Bidirectional Molecule Generation with Recurrent Neural Networks

Selective Photoredox Trifluoromethylation of Tryptophan-Containing Peptides

SMILES Transformer: Pre-trained Molecular Fingerprint for Low Data Drug Discovery

Convolutional Neural Networks for the Design and Analysis of Non-Fullerene Acceptors

Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer

Randomized SMILES strings improve the quality of molecular generative models

iMotor-CNN: Identifying molecular functions of cytoskeleton motor proteins using 2D convolutional neural network via Chou's 5-step rule.

Syntax-Directed Variational Autoencoder for Structured Data

Attention is All you Need

Grammar Variational Autoencoder

Automatic Chemical Design Using a Data-Driven Continuous Representation of Molecules

Improving chemical similarity ensemble approach in target prediction

The high-throughput highway to computational materials design.

The Probabilistic Relevance Framework: BM25 and Beyond

A decade of fragment-based drug design: strategic advances and lessons learned

The process of structure-based drug design.

Unsupervised Data Base Clustering Based on Daylight's Fingerprint and Tanimoto Similarity: A Fast and Automated Way To Cluster Small and Large Data Sets

Bioisosterism: A Rational Approach in Drug Design.

SMILES, a chemical language and information system. 1. Introduction to methodology and encoding rules

Programming Techniques: Regular expression search algorithm

Measures of the Amount of Ecologic Association Between Species

ChemCrow: Augmenting large-language models with chemistry tools

Text2Mol: Cross-Modal Molecule Retrieval with Natural Language Queries

Guidelines for Recurrent Neural Network Transfer Learning-Based Molecular Generation of Focused Libraries

Language Models are Unsupervised Multitask Learners

Improving Language Understanding by Generative Pre-Training

An information-theoretic perspective of tf-idf measures

Submission and Formatting Instructions for Icml 2016

A new cyano (- cn) free molecular design perspective for constructing carbazole-thiophene based environmental friendly organic solar cells

underthesupervisionofDr currentlyworkingtowardthePhDdegreeoftheDe-partmentofComputing

Difficulty in learning chirality for transformer fed with smiles

PhDdegreeincomputersciencefromtheCityUniver-sityofHongKongandhasworkedasapostdoctoralfellowintheUniversityofCalifornia

research scholar with Michigan State University (MSU)

“Introducing chatgpt,”

isavisitingprofessoroftheDepartmentofCom-puting,TheHongKongPolytechnicUniversity

[ HongKongPoly-technicUniversity(PolyU

“Vicuna: An open-source chatbot impressing GPT-4 with 90% chatgpt quality,”