The language of proteins: NLP, machine learning & protein sequences

Published in

Computational and Structural Biotechnology Journal(2021)

External Links:

TL;DR

The success, promise and pitfalls of applying NLP algorithms to the study of proteins, and methods for encoding the information of proteins as text and analyzing it with NLP methods, reviewing classic concepts such as bag-of-words, k-mers/n-grams and text search.

Authors

Dan Ofer

3 papers

N. Brandes

2 papers

M. Linial

2 papers

References124 items

Continuous

MSA Transformer

Rethinking Attention with Performers

DNABERT: pre-trained Bidirectional Encoder Representations from Transformers model for DNA-language in genome

Deep Learning in Proteomics

The language of proteins: NLP, machine learning & protein sequences

Published in

Computational and Structural Biotechnology Journal(2021)

External Links:

Generate Graph DownloadPDF

TL;DR

Authors

Dan Ofer

3 papers

N. Brandes

2 papers

M. Linial

2 papers

References124 items

Continuous

MSA Transformer

Rethinking Attention with Performers

DNABERT: pre-trained Bidirectional Encoder Representations from Transformers model for DNA-language in genome

Deep Learning in Proteomics

Embeddings from deep learning transfer GO annotations beyond homology

Big Bird: Transformers for Longer Sequences

ProtTrans: Towards Cracking the Language of Life’s Code Through Self-Supervised Deep Learning and High Performance Computing

Signal Peptides Generated by Attention-Based Neural Networks.

Learning the language of viral evolution and escape

BERTology Meets Biology: Interpreting Attention in Protein Language Models

Deep learning models in genomics; are we there yet?

Big Self-Supervised Models are Strong Semi-Supervised Learners

Language Models are Few-Shot Learners

On Biases of Attention in Scientific Discovery

Language modelling for biological sequences – curated datasets and baselines

ProGen: Language Modeling for Protein Generation

Structural basis for the recognition of SARS-CoV-2 by full-length human ACE2

CodeBERT: A Pre-Trained Model for Programming and Natural Languages

Aligning the Pretraining and Finetuning Objectives of Language Models

Improved protein structure prediction using potentials from deep learning

PyTorch: An Imperative Style, High-Performance Deep Learning Library

Deep Learning for Symbolic Mathematics

Modeling aspects of the language of life through transfer-learning protein sequences

Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer

Critical assessment of methods of protein structure prediction (CASP)—Round XIII

Unified rational protein engineering with sequence-based deep representation learning

ALBERT: A Lite BERT for Self-supervised Learning of Language Representations

CTRL: A Conditional Transformer Language Model for Controllable Generation

KG-BERT: BERT for Knowledge Graph Completion

Neural Attentive Bag-of-Entities Model for Text Classification

RoBERTa: A Robustly Optimized BERT Pretraining Approach

UDSMProt: universal deep sequence models for protein classification

Evaluating Protein Transfer Learning with TAPE

XLNet: Generalized Autoregressive Pretraining for Language Understanding

Using deep learning to annotate the protein universe

Biological structure and function emerge from scaling unsupervised learning to 250 million protein sequences

One neuron versus deep learning in aftershock prediction

Learning protein sequence embeddings using information from structure

Contextual Word Representations: A Contextual Introduction

Grammar of protein domain architectures

A High Efficient Biological Language Model for Predicting Protein–Protein Interactions

Neural networks versus Logistic regression for 30 days all-cause readmission prediction

Learned protein embeddings for machine learning

Probabilistic variable-length segmentation of protein sequences for discriminative motif discovery (DiMotif) and sequence embedding (ProtVecX)

Deep Nets: What have They Ever Done for Vision?

Subword Regularization: Improving Neural Network Translation Models with Multiple Subword Candidates

GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding

Deep Contextualized Word Representations

DeepType: Multilingual Entity Linking by Neural Type System Evolution

Universal Language Model Fine-tuning for Text Classification

DeepSig: deep learning improves signal peptide detection in proteins

DeepLoc: prediction of protein subcellular localization using deep learning

MMseqs2 enables sensitive protein sequence searching for the analysis of massive data sets

Learned in Translation: Contextualized Word Vectors

Global analysis of protein folding using massively parallel design, synthesis, and testing

Attention is All you Need

Dilated Residual Networks

A Simple but Tough-to-Beat Baseline for Sentence Embeddings

OpenNMT: Open-Source Toolkit for Neural Machine Translation

Understanding deep learning requires rethinking generalization

FFPred 3: feature-based function prediction for all Gene Ontology domains

Enriching Word Vectors with Subword Information

Bag of Tricks for Efficient Text Classification

Deep learning for computational biology

Deep learning in bioinformatics

Machine Learning for Protein Function

ASAP: a machine learning framework for local protein properties

ProFET: Feature engineering captures high-level protein functions

Neural Machine Translation of Rare Words with Subword Units

Protein-protein interaction predictions using text mining methods.

GloVe: Global Vectors for Word Representation

Sequence to Sequence Learning with Neural Networks

Detecting "protein words" through unsupervised word segmentation

NeuroPID: a predictor for identifying neuropeptide precursors from metazoan proteomes

CNN Features Off-the-Shelf: An Astounding Baseline for Recognition

word2vec Explained: deriving Mikolov et al.'s negative-sampling word-embedding method

Distributed Representations of Words and Phrases and their Compositionality

Scalable Protein Sequence Similarity Search using Locality-Sensitive Hashing and MapReduce

Characterizing the state of the art in the computational assignment of gene function: lessons from the first critical assessment of functional annotation (CAFA)

Biases in the Experimental Annotations of Protein Function and Their Effect on Our Understanding of Protein Function Space

ImageNet classification with deep convolutional neural networks

Evaluation of Database Search Programs for Accurate Detection of Neuropeptides in Tandem Mass Spectrometry Experiments

HHblits: lightning-fast iterative protein sequence searching by HMM-HMM alignment

The complete peptide dictionary – A meta‐proteomics resource

FragBag, an accurate representation of protein structure, retrieves structural neighbors from the entire PDB quickly and accurately

Cooperativity within proximal phosphorylation sites is revealed from large-scale proteomics data

The origin of the genetic code and of the earliest oligopeptides.

Unsupervised learning of natural languages

Reduced amino acid alphabets exhibit an improved sensitivity and selectivity in fold assignment

ClanTox: a classifier of short animal toxins

The Unreasonable Effectiveness of Data

Machine learning methods for predictive proteomics

When Less Is More: Improving Classification of Protein Families with a Minimal Set of Global Features

Speeding up tandem mass spectrometry database search: metric embeddings and fast near neighbor search

MultiLoc: prediction of protein subcellular localization using N-terminal targeting sequences, sequence motifs and amino acid composition

Reduced amino acid alphabet is sufficient to accurately recognize intrinsically disordered protein

Mismatch string kernels for discriminative protein classification

100

Proteomic signatures: Amino acid and oligopeptide compositions differentiate among phyla

101

CAPRI: A Critical Assessment of PRedicted Interactions

102

The Spectrum Kernel: A String Kernel for SVM Protein Classification

103

Simplified amino acid alphabets for protein fold recognition and implications for folding.

104

Long Short-Term Memory

105

The Shannon information entropy of protein sequences.

106

How does protein synthesis give rise to the 3D‐structure?

107

Introduction to Modern Information Retrieval

108

ProtTrans: Towards Cracking the Language of Life’s Code Through Self-Supervised Learning

109

Deep Learning Embedder Method and Tool for Mass Spectra Similarity Search

110

ELECTRA: Pre-training Text Encoders as Discriminators Rather Than Generators.

111

BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding

112

Language Models are Unsupervised Multitask Learners

113

NLP’s imagenet moment has arrived

114

Evaluating Vector-Space Models of Word Representation, or, The Unreasonable Effectiveness of Counting Words Near Other Words

115

Demonstrations System, editor. OpenNMT: Open-Source Toolkit for Neural Machine Translation. Vancouver, Canada: Association for Computational Linguistics

116

UniProtKB/Swiss-Prot, the Manually Annotated Section of the UniProt KnowledgeBase: How to Use the Entry View.

117

Critical assessment of methods of protein structure prediction (CASP)—round IX

118

UniProtKB/Swiss-Prot.

119

Sequence Motifs: Highly Predictive Features of Protein Function

120

Families of membranous proteins can be characterized by the amino acid composition of their transmembrane domains

121

Ultraconservative Online Algorithms for Multiclass Problems

122

The Protein Data Bank

123

Prediction and Entropy of Printed English

124

The Aquila Digital Community The Aquila Digital Community

Field of Study

Computer ScienceMedicine

Journal Information

Name

Computational and Structural Biotechnology Journal

Page

1750 - 1758

Volume

Venue Information

Name

Computational and Structural Biotechnology Journal

Type

journal

URL

https://www.journals.elsevier.com/computational-and-structural-biotechnology-journal/

Alternate Names

Comput struct biotechnol j
Comput Struct Biotechnol J
Computational and structural biotechnology journal