Code for the paper BERT might be Overkill: A Tiny but Effective Biomedical Entity Linker based on Residual Convolutional Neural Networks

Tuan Manh Lai

Last update: Oct 24, 2022

Related tags

Deep Learning rescnn_bioel

Overview

Biomedical Entity Linking

This repo provides the code for the paper BERT might be Overkill: A Tiny but Effective Biomedical Entity Linker based on Residual Convolutional Neural Networks (EMNLP 2021 Findings).

Download the pretrained embedding layer from this link. And set this line to the path of the downloaded file.

Basic running instructions

pip install -r requirements.txt
python cg_trainer.py --dataset bc5cdr-chemical

Please refer to the file constants.py for the list of all supported datasets. Note that for COMETA, you need to download the dataset from https://github.com/cambridgeltl/cometa.

Note that for ncbi-disease, bc5cdr-disease, and bc5cdr-chemical, we follow the protocol of BioSyn. We use development (dev) set to search the hyperparameters, and train on traindev (train+dev) set to report the final performance.

We are cleaning the codebase and we will add more running instructions soon.

Comments

Cannot reproduce results on NCBI-Diseases
I have downloaded embedding.pt and use

python cg_trainer.py --dataset ncbi-disease

to train NCBI-D.

I obtain results of {'top1_accuracy': 0.90833, 'top5_accuracy': 0.93958, 'top10_accuracy': 0.95625, 'top20_accuracy': 0.96042} which is lower than your reported 92.4. Do I miss something to reproduce your results.
opened by GanjinZero 6
Some files are still missing

Thanks for your reply, #1

I can't run this code for BC5CDR datasets, Because there is some difference between your code and the downloaded data(from BioSyn).

In your code, I need the .json file to initialize the Ontology class, but the downloaded data dosen't have .json file

Can you upload these files, or release the script to process the existing .txt and .concept files to get those .json.

Sorry to bother~

opened by SouthWindShiB 2
Can‘t run this code

I have read the paper, it's a interesting work. But I can't run this code withought a guildline.

When the detailed readme will be release? and those processed .json file?

opened by SouthWindShiB 2
What is Hard Negatives Mode?
Hey, thanks for sharing your work.

Can you describe what your hard negatives mode is doing? I can't find any reference to it in the publication.

Did you train for 100 epochs? I found this in your config, but not in your publication.

Thanks!
opened by waynchi 1
umls ontology preprocess

Dear author,

Thanks for making the code publicly available!

However, I still have some problems in running code on the MedMentions dataset because of the lack of UMLS 2017aa ontology file. I have downloaded the UMLS-2017aa-full file and installed it on my device. And the number of unique concepts in it is 3,415,665, the number of unique synonyms of all concepts in it is 7,135,041.But my experimental results of acc@1 on the MedMentions test set is 80%+, which is much higher than the results reported in your paper.

Is there something wrong with me when dealing with umls-2017aa-full?

opened by Annztt 1

Owner

Tuan Manh Lai

UIUC CS PhD student

GitHub

U-Net Implementation: Convolutional Networks for Biomedical Image Segmentation" using the Carvana Image Masking Dataset in PyTorch

U-Net Implementation By Christopher Ley This is my interpretation and implementation of the famous paper "U-Net: Convolutional Networks for Biomedical

1 Jan 6, 2022

A Pytorch implementation of CVPR 2021 paper "RSG: A Simple but Effective Module for Learning Imbalanced Datasets"

RSG: A Simple but Effective Module for Learning Imbalanced Datasets (CVPR 2021) A Pytorch implementation of our CVPR 2021 paper "RSG: A Simple but Eff

120 Dec 12, 2022

This repository implements and evaluates convolutional networks on the Möbius strip as toy model instantiations of Coordinate Independent Convolutional Networks.

Orientation independent Möbius CNNs This repository implements and evaluates convolutional networks on the Möbius strip as toy model instantiations of

59 Dec 9, 2022

Simple and Effective Few-Shot Named Entity Recognition with Structured Nearest Neighbor Learning

structshot Code and data for paper "Simple and Effective Few-Shot Named Entity Recognition with Structured Nearest Neighbor Learning", Yi Yang and Arz

47 Dec 27, 2022

PyTorch code for our ECCV 2018 paper "Image Super-Resolution Using Very Deep Residual Channel Attention Networks"

1.2k Dec 26, 2022

The source codes for ACL 2021 paper 'BoB: BERT Over BERT for Training Persona-based Dialogue Models from Limited Personalized Data'

BoB: BERT Over BERT for Training Persona-based Dialogue Models from Limited Personalized Data This repository provides the implementation details for

124 Dec 27, 2022

Source code for NAACL 2021 paper "TR-BERT: Dynamic Token Reduction for Accelerating BERT Inference"

TR-BERT Source code and dataset for "TR-BERT: Dynamic Token Reduction for Accelerating BERT Inference". The code is based on huggaface's transformers.

37 Oct 30, 2022

Pre-trained BERT Models for Ancient and Medieval Greek, and associated code for LaTeCH 2021 paper titled - "A Pilot Study for BERT Language Modelling and Morphological Analysis for Ancient and Medieval Greek"

Ancient Greek BERT The first and only available Ancient Greek sub-word BERT model! State-of-the-art post fine-tuning on Part-of-Speech Tagging and Mor

22 Dec 8, 2022

The official implementation of You Only Compress Once: Towards Effective and Elastic BERT Compression via Exploit-Explore Stochastic Nature Gradient.

You Only Compress Once: Towards Effective and Elastic BERT Compression via Exploit-Explore Stochastic Nature Gradient (paper) @misc{zhang2021compress,

46 Dec 7, 2022

[EMNLP 2021] MuVER: Improving First-Stage Entity Retrieval with Multi-View Entity Representations

MuVER This repo contains the code and pre-trained model for our EMNLP 2021 paper: MuVER: Improving First-Stage Entity Retrieval with Multi-View Entity

24 May 30, 2022

PyTorch version of the paper 'Enhanced Deep Residual Networks for Single Image Super-Resolution' (CVPRW 2017)

About PyTorch 1.2.0 Now the master branch supports PyTorch 1.2.0 by default. Due to the serious version problem (especially torch.utils.data.dataloade

2.1k Jan 1, 2023

Pre-trained model, code, and materials from the paper "Impact of Adversarial Examples on Deep Learning Models for Biomedical Image Segmentation" (MICCAI 2019).

Adaptive Segmentation Mask Attack This repository contains the implementation of the Adaptive Segmentation Mask Attack (ASMA), a targeted adversarial

53 Jul 4, 2022

Code for our ICASSP 2021 paper: SA-Net: Shuffle Attention for Deep Convolutional Neural Networks

SA-Net: Shuffle Attention for Deep Convolutional Neural Networks (paper) By Qing-Long Zhang and Yu-Bin Yang [State Key Laboratory for Novel Software T

199 Jan 8, 2023

Chinese clinical named entity recognition using pre-trained BERT model

Chinese clinical named entity recognition (CNER) using pre-trained BERT model Introduction Code for paper Chinese clinical named entity recognition wi

109 Dec 14, 2022

Example Of Fine-Tuning BERT For Named-Entity Recognition Task And Preparing For Cloud Deployment Using Flask, React, And Docker

Example Of Fine-Tuning BERT For Named-Entity Recognition Task And Preparing For Cloud Deployment Using Flask, React, And Docker This repository contai

12 Dec 14, 2022

Code for the paper BERT might be Overkill: A Tiny but Effective Biomedical Entity Linker based on Residual Convolutional Neural Networks

Related tags

Overview

Biomedical Entity Linking

Comments

Cannot reproduce results on NCBI-Diseases

Some files are still missing

Can‘t run this code

What is Hard Negatives Mode?

umls ontology preprocess

Owner

Tuan Manh Lai

U-Net Implementation: Convolutional Networks for Biomedical Image Segmentation" using the Carvana Image Masking Dataset in PyTorch

A Pytorch implementation of CVPR 2021 paper "RSG: A Simple but Effective Module for Learning Imbalanced Datasets"

This repository implements and evaluates convolutional networks on the Möbius strip as toy model instantiations of Coordinate Independent Convolutional Networks.

Simple and Effective Few-Shot Named Entity Recognition with Structured Nearest Neighbor Learning

PyTorch code for our ECCV 2018 paper "Image Super-Resolution Using Very Deep Residual Channel Attention Networks"

The source codes for ACL 2021 paper 'BoB: BERT Over BERT for Training Persona-based Dialogue Models from Limited Personalized Data'

Source code for NAACL 2021 paper "TR-BERT: Dynamic Token Reduction for Accelerating BERT Inference"

Pre-trained BERT Models for Ancient and Medieval Greek, and associated code for LaTeCH 2021 paper titled - "A Pilot Study for BERT Language Modelling and Morphological Analysis for Ancient and Medieval Greek"

The official implementation of You Only Compress Once: Towards Effective and Elastic BERT Compression via Exploit-Explore Stochastic Nature Gradient.

[EMNLP 2021] MuVER: Improving First-Stage Entity Retrieval with Multi-View Entity Representations

PyTorch version of the paper 'Enhanced Deep Residual Networks for Single Image Super-Resolution' (CVPRW 2017)

Pre-trained model, code, and materials from the paper "Impact of Adversarial Examples on Deep Learning Models for Biomedical Image Segmentation" (MICCAI 2019).

Code for our ICASSP 2021 paper: SA-Net: Shuffle Attention for Deep Convolutional Neural Networks

Chinese clinical named entity recognition using pre-trained BERT model

Example Of Fine-Tuning BERT For Named-Entity Recognition Task And Preparing For Cloud Deployment Using Flask, React, And Docker

PyTorch implementation of Wide Residual Networks with 1-bit weights by McDonnell (ICLR 2018)

A PyTorch implementation for PyramidNets (Deep Pyramidal Residual Networks)

Wide Residual Networks (WideResNets) in PyTorch

RMNet: Equivalently Removing Residual Connection from Networks