CU-UD: text-mining drug and chemical-protein interactions with ensembles of BERT-based models
Published in
arXiv.org(2021)External Links:
TL
TL;DR
This submission is an ensemble system, including multiple BERT-based language models, which combines the outputs of individual models using majority voting and multilayer perceptron, demonstrating the effectiveness of using ensembles of BERT - based language models for automatically detecting relations between chemicals and proteins.
Abstract
Identifying the relations between chemicals and proteins is an important text mining task. BioCreative VII track 1 DrugProt task aims to promote the development and evaluation of systems that can automatically detect relations between chemical compounds/drugs and genes/proteins in PubMed abstracts. In this paper, we describe our submission, which is an ensemble system, including multiple BERT-based language models. We combine the outputs of individual models using majority voting and multilayer perceptron. Our system obtained 0.7708 in precision and 0.7770 in recall, for an F1 score of 0.7739, demonstrating the effectiveness of using ensembles of BERT-based language models for automatically detecting relations between chemicals and proteins. Our code is available at https://github.com/bionlplab/drugprot_bcvii.
References15 items
1
Improving BERT Model Using Contrastive Learning for Biomedical Relation Extraction
2
Investigation of BERT Model on Biomedical Relation Extraction Based on Revised Fine-tuning Mechanism
3
Domain-Specific Language Model Pretraining for Biomedical Natural Language Processing
4
Stanza: A Python Natural Language Processing Toolkit for Many Human Languages
5