Sequential Labelling and DNABERT For Splice Site Prediction in Homo Sapiens DNA (2022-12-15T00:00:00.000000Z)

TL;DR

This paper proposes sequential labelling to predict splice sites regardless their position in sequence and concludes that model experience overfitting is deemed not suitable for splice site prediction.

Abstract

Genome sequencing technology has improved significantly in few last years and resulted in abundance genetic data. Artificial intelligence has been employed to analyze genetic data in response to its sheer size and variability. Gene prediction on single DNA has been conducted using various deep learning architectures to discover splice sites and therefore discover intron and exon region. Recent predictions are carried out with models trained on sequence with fixed splice site location which eliminates possibility of multiple splice sites existence in single sequence. This paper proposes sequential labelling to predict splice sites regardless their position in sequence. Sequential labelling is carried out on DNA to determine intron and exon region and thus discover splice sites. Sequential labelling models used are based on pretrained DNABERT-3 which has been trained on human genome. Both fine-tuning and feature-based approach are tested. Proposed model is benchmarked against latest sequential labelling model designed for mutation type and location prediction. While achieving high F1 scores on validation data, both baseline and proposed model perform poorly on test data. Error and test results analysis reveal that model experience overfitting and therefore, model is deemed not suitable for splice site prediction.

Authors

A. Purwarianti

4 papers

M. A. Leksono

1 papers

TL;DR

Abstract

Authors

References50 items

EnsembleSplice: ensemble deep learning model for splice site prediction

Splice-site identification for exon prediction using bidirectional LSTM-RNN approach

Deep Splicer: A CNN Model for Splice Site Prediction in Genetic Sequences

Spliceator: multi-species splice site prediction using convolutional neural networks

Classification of DNA Sequences with k-mers Based Vector Representations

Representation learning applications in biological sequence analysis

A Survey on Recent Advances in Sequence Labeling from Deep Learning Models

Effects of Hidden Layers on the Efficiency of Neural networks

DNABERT: pre-trained Bidirectional Encoder Representations from Transformers model for DNA-language in genome

Splice2Deep: An ensemble of deep convolutional neural networks for improved splice site prediction in genomic DNA

PyTorch: An Imperative Style, High-Performance Deep Learning Library

Modeling aspects of the language of life through transfer-learning protein sequences

Recurrent Neural Network for Predicting Transcription Factor Binding Sites

A Survey on Recent Advances in Named Entity Recognition from Deep Learning models

DeepSS: Exploring Splice Site Motif Through Convolutional Neural Network Directly From DNA Sequence

Deep Contextualized Word Representations

Markov Chains: From Theory to Implementation and Experimentation

Attention is All you Need

dna2vec: Consistent vector representations of variable-length k-mers

A novel method for splice sites prediction using sequence component and hidden Markov model

Principal component analysis: a review and recent developments

Prediction of donor splice sites using random forest with a new sequence encoding approach

Reference sequence (RefSeq) database at NCBI: current status, taxonomic expansion, and functional annotation

GloVe: Global Vectors for Word Representation

Efficient Estimation of Word Representations in Vector Space

GENCODE: The reference human genome annotation for The ENCODE Project

Detection of Splice Sites Using Support Vector Machine

Accurate splice site prediction using support vector machines

Matplotlib: A 2D Graphics Environment

Splice site identification using probabilistic parameters and SVM classification

Automatic annotation of eukaryotic genes, pseudogenes and promoters

SpliceMachine: predicting splice sites from high-dimensional local context representations

Prediction of splice sites with dependency graphs and their expanded bayesian networks

MicroRNAs Genomics, Biogenesis, Mechanism, and Function

Feature subset selection for splice site prediction

Effective hidden Markov models for detecting splicing junction sites in DNA sequences

GeneSplicer: a new computational method for splice site prediction.

Long Short-Term Memory

Support-Vector Networks

Evolution of long-range fractal correlations and 1/f noise in DNA base sequences.

Identification of common molecular subsequences.

Statistical Inference for Probabilistic Functions of Finite State Markov Chains

Join Classifier of Type and Index Mutation on Lung Cancer DNA Using Sequential Labeling Model

Experiment Tracking with Weights and Biases.

BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding

Jupyter Notebooks - a publishing format for reproducible computational workflows

Gradient-based learning applied to document recognition

Handwritten Digit Recognition with a Back-Propagation Network

LeCun et al . , “ Handwritten Digit Recognition with a BackPropagation Network , ” in Advances in Neural Information Processing Systems ,

This Paper Is Included in the Proceedings of the 12th Usenix Symposium on Operating Systems Design and Implementation (osdi '16). Tensorflow: a System for Large-scale Machine Learning Tensorflow: a System for Large-scale Machine Learning

Field of Study