Code for the paper BERT might be Overkill: A Tiny but Effective Biomedical Entity Linker based on Residual Convolutional Neural Networks

Overview

Biomedical Entity Linking

This repo provides the code for the paper BERT might be Overkill: A Tiny but Effective Biomedical Entity Linker based on Residual Convolutional Neural Networks (EMNLP 2021 Findings).

Download the pretrained embedding layer from this link. And set this line to the path of the downloaded file.

Basic running instructions

pip install -r requirements.txt
python cg_trainer.py --dataset bc5cdr-chemical

Please refer to the file constants.py for the list of all supported datasets. Note that for COMETA, you need to download the dataset from https://github.com/cambridgeltl/cometa.

Note that for ncbi-disease, bc5cdr-disease, and bc5cdr-chemical, we follow the protocol of BioSyn. We use development (dev) set to search the hyperparameters, and train on traindev (train+dev) set to report the final performance.

We are cleaning the codebase and we will add more running instructions soon.

Comments
  • Cannot reproduce results on NCBI-Diseases

    Cannot reproduce results on NCBI-Diseases

    I have downloaded embedding.pt and use

    python cg_trainer.py --dataset ncbi-disease
    

    to train NCBI-D.

    I obtain results of {'top1_accuracy': 0.90833, 'top5_accuracy': 0.93958, 'top10_accuracy': 0.95625, 'top20_accuracy': 0.96042} which is lower than your reported 92.4. Do I miss something to reproduce your results.

    opened by GanjinZero 6
  • Some files are still missing

    Some files are still missing

    Thanks for your reply, #1

    I can't run this code for BC5CDR datasets, Because there is some difference between your code and the downloaded data(from BioSyn).

    In your code, I need the .json file to initialize the Ontology class, but the downloaded data dosen't have .json file

    Can you upload these files, or release the script to process the existing .txt and .concept files to get those .json.

    Sorry to bother~

    image

    image

    image

    opened by SouthWindShiB 2
  • Can‘t run this code

    Can‘t run this code

    I have read the paper, it's a interesting work. But I can't run this code withought a guildline.

    When the detailed readme will be release? and those processed .json file?

    opened by SouthWindShiB 2
  • What is Hard Negatives Mode?

    What is Hard Negatives Mode?

    Hey, thanks for sharing your work.

    1. Can you describe what your hard negatives mode is doing? I can't find any reference to it in the publication.
    2. Did you train for 100 epochs? I found this in your config, but not in your publication.

    Thanks!

    opened by waynchi 1
  • umls ontology preprocess

    umls ontology preprocess

    Dear author,

    Thanks for making the code publicly available!

    However, I still have some problems in running code on the MedMentions dataset because of the lack of UMLS 2017aa ontology file. I have downloaded the UMLS-2017aa-full file and installed it on my device. And the number of unique concepts in it is 3,415,665, the number of unique synonyms of all concepts in it is 7,135,041.But my experimental results of acc@1 on the MedMentions test set is 80%+, which is much higher than the results reported in your paper.

    Is there something wrong with me when dealing with umls-2017aa-full?

    opened by Annztt 1
Owner
Tuan Manh Lai
UIUC CS PhD student
Tuan Manh Lai
U-Net Implementation: Convolutional Networks for Biomedical Image Segmentation" using the Carvana Image Masking Dataset in PyTorch

U-Net Implementation By Christopher Ley This is my interpretation and implementation of the famous paper "U-Net: Convolutional Networks for Biomedical

Christopher Ley 1 Jan 6, 2022
A Pytorch implementation of CVPR 2021 paper "RSG: A Simple but Effective Module for Learning Imbalanced Datasets"

RSG: A Simple but Effective Module for Learning Imbalanced Datasets (CVPR 2021) A Pytorch implementation of our CVPR 2021 paper "RSG: A Simple but Eff

null 120 Dec 12, 2022
This repository implements and evaluates convolutional networks on the Möbius strip as toy model instantiations of Coordinate Independent Convolutional Networks.

Orientation independent Möbius CNNs This repository implements and evaluates convolutional networks on the Möbius strip as toy model instantiations of

Maurice Weiler 59 Dec 9, 2022
Simple and Effective Few-Shot Named Entity Recognition with Structured Nearest Neighbor Learning

structshot Code and data for paper "Simple and Effective Few-Shot Named Entity Recognition with Structured Nearest Neighbor Learning", Yi Yang and Arz

ASAPP Research 47 Dec 27, 2022
PyTorch code for our ECCV 2018 paper "Image Super-Resolution Using Very Deep Residual Channel Attention Networks"

PyTorch code for our ECCV 2018 paper "Image Super-Resolution Using Very Deep Residual Channel Attention Networks"

Yulun Zhang 1.2k Dec 26, 2022
The source codes for ACL 2021 paper 'BoB: BERT Over BERT for Training Persona-based Dialogue Models from Limited Personalized Data'

BoB: BERT Over BERT for Training Persona-based Dialogue Models from Limited Personalized Data This repository provides the implementation details for

null 124 Dec 27, 2022
Source code for NAACL 2021 paper "TR-BERT: Dynamic Token Reduction for Accelerating BERT Inference"

TR-BERT Source code and dataset for "TR-BERT: Dynamic Token Reduction for Accelerating BERT Inference". The code is based on huggaface's transformers.

THUNLP 37 Oct 30, 2022
Pre-trained BERT Models for Ancient and Medieval Greek, and associated code for LaTeCH 2021 paper titled - "A Pilot Study for BERT Language Modelling and Morphological Analysis for Ancient and Medieval Greek"

Ancient Greek BERT The first and only available Ancient Greek sub-word BERT model! State-of-the-art post fine-tuning on Part-of-Speech Tagging and Mor

Pranaydeep Singh 22 Dec 8, 2022
The official implementation of You Only Compress Once: Towards Effective and Elastic BERT Compression via Exploit-Explore Stochastic Nature Gradient.

You Only Compress Once: Towards Effective and Elastic BERT Compression via Exploit-Explore Stochastic Nature Gradient (paper) @misc{zhang2021compress,

null 46 Dec 7, 2022
[EMNLP 2021] MuVER: Improving First-Stage Entity Retrieval with Multi-View Entity Representations

MuVER This repo contains the code and pre-trained model for our EMNLP 2021 paper: MuVER: Improving First-Stage Entity Retrieval with Multi-View Entity

null 24 May 30, 2022
PyTorch version of the paper 'Enhanced Deep Residual Networks for Single Image Super-Resolution' (CVPRW 2017)

About PyTorch 1.2.0 Now the master branch supports PyTorch 1.2.0 by default. Due to the serious version problem (especially torch.utils.data.dataloade

Sanghyun Son 2.1k Jan 1, 2023
Pre-trained model, code, and materials from the paper "Impact of Adversarial Examples on Deep Learning Models for Biomedical Image Segmentation" (MICCAI 2019).

Adaptive Segmentation Mask Attack This repository contains the implementation of the Adaptive Segmentation Mask Attack (ASMA), a targeted adversarial

Utku Ozbulak 53 Jul 4, 2022
Code for our ICASSP 2021 paper: SA-Net: Shuffle Attention for Deep Convolutional Neural Networks

SA-Net: Shuffle Attention for Deep Convolutional Neural Networks (paper) By Qing-Long Zhang and Yu-Bin Yang [State Key Laboratory for Novel Software T

Qing-Long Zhang 199 Jan 8, 2023
Chinese clinical named entity recognition using pre-trained BERT model

Chinese clinical named entity recognition (CNER) using pre-trained BERT model Introduction Code for paper Chinese clinical named entity recognition wi

Xiangyang Li 109 Dec 14, 2022
Example Of Fine-Tuning BERT For Named-Entity Recognition Task And Preparing For Cloud Deployment Using Flask, React, And Docker

Example Of Fine-Tuning BERT For Named-Entity Recognition Task And Preparing For Cloud Deployment Using Flask, React, And Docker This repository contai

Nikita 12 Dec 14, 2022
PyTorch implementation of Wide Residual Networks with 1-bit weights by McDonnell (ICLR 2018)

1-bit Wide ResNet PyTorch implementation of training 1-bit Wide ResNets from this paper: Training wide residual networks for deployment using a single

Sergey Zagoruyko 122 Dec 7, 2022
A PyTorch implementation for PyramidNets (Deep Pyramidal Residual Networks)

A PyTorch implementation for PyramidNets (Deep Pyramidal Residual Networks) This repository contains a PyTorch implementation for the paper: Deep Pyra

Greg Dongyoon Han 262 Jan 3, 2023
Wide Residual Networks (WideResNets) in PyTorch

Wide Residual Networks (WideResNets) in PyTorch WideResNets for CIFAR10/100 implemented in PyTorch. This implementation requires less GPU memory than

Jason Kuen 296 Dec 27, 2022
RMNet: Equivalently Removing Residual Connection from Networks

RM Operation can equivalently convert ResNet to VGG, which is better for pruning; and can help RepVGG perform better when the depth is large.

null 8 Nov 4, 2021