nnFormer: Interleaved Transformer for Volumetric Segmentation

Overview

nnFormer: Interleaved Transformer for Volumetric Segmentation

Code for paper "nnFormer: Interleaved Transformer for Volumetric Segmentation ". Please read our preprint at the following link: paper_address.

Parts of codes are borrowed from nn-UNet.


Installation

1、System requirements

This software was originally designed and run on a system running Ubuntu 18.01, with Python 3.6, PyTorch 1.8.1, and CUDA 10.1. For a full list of software packages and version numbers, see the Conda environment file environment.yml.

This software leverages graphical processing units (GPUs) to accelerate neural network training and evaluation; systems lacking a suitable GPU will likely take an extremely long time to train or evaluate models. The software was tested with the NVIDIA RTX 2080 TI GPU, though we anticipate that other GPUs will also work, provided that the unit offers sufficient memory.

2、Installation guide

We recommend installation of the required packages using the Conda package manager, available through the Anaconda Python distribution. Anaconda is available free of charge for non-commercial use through Anaconda Inc. After installing Anaconda and cloning this repository, For use as integrative framework:

git clone https://github.com/282857341/nnFormer.git
cd nnFormer
conda env create -f environment.yml
source activate nnFormer
pip install -e .

3、The main downloaded file directory description

  • ACDC_dice: Calculate dice of ACDC dataset

  • Synapse_dice_and_hd: Calulate dice of the Synapse dataset

  • dataset_json: About how to divide the training and test set

  • inference: The entry program of the infernece.

  • network_architecture: The models are stored here.

  • run: The entry program of the training.

  • training: The trainers are stored here, the training of the network is conducted by the trainer.


Training

1、Datasets

Datasets can be downloaded at the following links:

And the division of the dataset can be seen in the files in the ./dataset_json/

Dataset I ACDC

Dataset II The Synapse multi-organ CT dataset

2、Setting up the datasets

While we provide code to load data for training a deep-learning model, you will first need to download images from the above repositories. Regarding the format setting and related preprocessing of the dataset, we operate based on nnFormer, so I won’t go into details here. You can see nnUNet for specific operations.

Regarding the downloaded data, I will not introduce too much here, you can go to the corresponding website to view it. Organize the downloaded DataProcessed as follows:

./Pretrained_weight/
./nnFormer/
./DATASET/
  ├── nnFormer_raw/
      ├── nnFormer_raw_data/
          ├── Task01_ACDC/
              ├── imagesTr/
              ├── imagesTs/
              ├── labelsTr/
              ├── labelsTs/
              ├── dataset.json
          ├── Task02_Synapse/
              ├── imagesTr/
              ├── imagesTs/
              ├── labelsTr/
              ├── labelsTs/
              ├── dataset.json
      ├── nnFormer_cropped_data/
  ├── nnFormer_trained_models/
  ├── nnFormer_preprocessed/

After that, you can preprocess the data using:

nnFormer_convert_decathlon_task -i ../DATASET/nnFormer_raw/nnFormer_raw_data/Task01_ACDC
nnFormer_convert_decathlon_task -i ../DATASET/nnFormer_raw/nnFormer_raw_data/Task02_Synapse
nnFormer_plan_and_preprocess -t 1
nnFormer_plan_and_preprocess -t 2

3 Training and Testing the models

A. Use the best model we have trained to infer the test set
(1).Put the downloaded the best training weights in the specified directory.

the download link is

Link:https://pan.baidu.com/s/1h1h8_DKvve8enyTiIyzfHw 
Extraction code:yimv

The Google Drive link is as follows:

Link:https://drive.google.com/drive/folders/16y1QYOQD4vjrR2hh8TpPB-tq5EYX--Az?usp=sharing

the specified directory is

../DATASET/nnFormer_trained_models/nnFormer/3d_fullres/Task001_ACDC/nnFormerTrainerV2_ACDC__nnFormerPlansv2.1/fold_0/model_best.model
../DATASET/nnFormer_trained_models/nnFormer/3d_fullres/Task001_ACDC/nnFormerTrainerV2_ACDC__nnFormerPlansv2.1/fold_0/model_best.model.pkl

../DATASET/nnFormer_trained_models/nnFormer/3d_fullres/Task002_Synapse/nnFormerTrainerV2_Synapse__nnFormerPlansv2.1/fold_0/model_best.model
../DATASET/nnFormer_trained_models/nnFormer/3d_fullres/Task002_Synapse/nnFormerTrainerV2_Synapse__nnFormerPlansv2.1/fold_0/model_best.model.pkl
(2).Evaluating the models
  • ACDC

Inference

nnFormer_predict -i ../DATASET/nnFormer_raw/nnFormer_raw_data/Task001_ACDC/imagesTs -o ../DATASET/nnFormer_raw/nnFormer_raw_data/Task001_ACDC/inferTs/output -m 3d_fullres -f 0 -t 1 -chk model_best -tr nnFormerTrainerV2_ACDC

Calculate DICE

python ./nnformer/ACDC_dice/inference.py
  • The Synapse multi-organ CT dataset

Inference

nnFormer_predict -i ../DATASET/nnFormer_raw/nnFormer_raw_data/Task002_Synapse/imagesTs -o ../DATASET/nnFormer_raw/nnFormer_raw_data/Task002_Synapse/inferTs/output -m 3d_fullres -f 0 -t 2 -chk model_best -tr nnFormerTrainerV2_Synapse

Calculate DICE

python ./nnformer/Synapse_dice_and_hd/inference.py

The dice result will be saved in ../DATASET/nnFormer_raw/nnFormer_raw_data/Task002_Synapse/inferTs/output

B. The complete process of retraining the model and inference
(1).Put the downloaded pre-training weights in the specified directory.

the download link is

Link:https://pan.baidu.com/s/1h1h8_DKvve8enyTiIyzfHw 
Extraction code:yimv

the specified directory is

../Pretrained_weight/pretrain_ACDC.model
../Pretrained_weight/pretrain_Synapse.model
(2).Training
  • ACDC
nnFormer_train 3d_fullres nnFormerTrainerV2_ACDC 1 0 
  • The Synapse multi-organ CT dataset
nnFormer_train 3d_fullres nnFormerTrainerV2_Synapse 2 0 
(3).Evaluating the models
  • ACDC

Inference

nnFormer_predict -i ../DATASET/nnFormer_raw/nnFormer_raw_data/Task001_ACDC/imagesTs -o ../DATASET/nnFormer_raw/nnFormer_raw_data/Task001_ACDC/inferTs/output -m 3d_fullres -f 0 -t 1 -chk model_best -tr nnFormerTrainerV2_ACDC

Calculate DICE

python ./nnformer/ACDC_dice/inference.py
  • The Synapse multi-organ CT dataset

Inference

nnFormer_predict -i ../DATASET/nnFormer_raw/nnFormer_raw_data/Task002_Synapse/imagesTs -o ../DATASET/nnFormer_raw/nnFormer_raw_data/Task002_Synapse/inferTs/output -m 3d_fullres -f 0 -t 2 -chk model_best -tr nnFormerTrainerV2_Synapse

Calculate DICE

python ./nnformer/Synapse_dice_and_hd/inference.py

The dice results will be saved in ../DATASET/nnFormer_raw/nnFormer_raw_data/Task002_Synapse/inferTs/output

Comments
  • postprocessing.json

    postprocessing.json

    (nnFormer) toolman@modeling506:~/Heart_model/zhang/nnFormer/nnformer$ nnFormer_predict -i ../DATASET/nnFormer_raw/nnFormer_raw_data/Task001_ACDC/imagesTs -o ../DATASET/nnFormer_raw/nnFormer_raw_data/Task001_ACDC/inferTs/output -m 3d_fullres -f 0 -t 1 -chk model_best -tr nnFormerTrainerV2_ACDC using model stored in /home/toolman/Heart_model/zhang/nnFormer/DATASET/nnFormer_trained_models/nnFormer/3d_fullres/Task001_ACDC/nnFormerTrainerV2_ACDC__nnFormerPlansv2.1 This model expects 1 input modalities for each image Found 40 unique case ids, here are some examples: ['patient009_frame13_0000' 'patient095_frame12_0000' 'patient079_frame01_0000' 'patient053_frame12_0000' 'patient081_frame07_0000' 'patient081_frame07_0000' 'patient009_frame13_0000' 'patient064_frame12_0000' 'patient088_frame01_0000' 'patient003_frame15_0000'] If they don't look right, make sure to double check your filenames. They must end with _0000.nii.gz etc number of cases: 40 number of cases that still need to be predicted: 0 emptying cuda cache loading parameters for folds, [0] 2021-09-28 14:03:56.302944: Using dummy2d data augmentation I am using the pre_train weight!! using the following model files: ['/home/toolman/Heart_model/zhang/nnFormer/DATASET/nnFormer_trained_models/nnFormer/3d_fullres/Task001_ACDC/nnFormerTrainerV2_ACDC__nnFormerPlansv2.1/fold_0/model_best.model'] starting preprocessing generator starting prediction... inference done. Now waiting for the segmentation export to finish... WARNING! Cannot run postprocessing because the postprocessing file is missing. Make sure to run consolidate_folds in the output folder of the model first! The folder you need to run this in is /home/toolman/Heart_model/zhang/nnFormer/DATASET/nnFormer_trained_models/nnFormer/3d_fullres/Task001_ACDC/nnFormerTrainerV2_ACDC__nnFormerPlansv2.1

    请问一下,为什么会出现上面问题?是缺少postprocessing.json文件还是我少了步骤?

    opened by zhangjin506 16
  • How to get

    How to get "splits_final.pkl"

    Hello, Mr.jsguo.

    I'm trying your nnFormer, but I am in trouble. I can't do "change_plan_swin.py", because there is not "splits_final.pkl" in Task001_ACDC and Task002_Synapse folder. Please tell me how to get this file, or send me this file.

    Best regard

    opened by kosuke-r 12
  • Quetion about the pre-trained model

    Quetion about the pre-trained model

    Hi,

    Thank you for your outstanding works. I tried to use the pre-trained model to run the ACDC task and I didn't get a high accuracy by myself. I just followed your instruction step by step, but I got the results as below. WeChat Image_20211026090756

    Can you teach me how to get a high accuracy like your paper, such as 91.78 for ACDC? I can train it by myself.

    Thank you

    opened by bxie9 8
  • About the plans.pkl file

    About the plans.pkl file

    When I use pickle.load() to read the 'plans.pkl' file, it returns a empty list. However, the file is not empty(21KB). I do not know what the problem is. Thanks very much if you can help me.

    opened by hytim 7
  • About the fold [0,1,2,3,4,5]

    About the fold [0,1,2,3,4,5]

    In the README.md, the command is used fold 0, Can I use other parameters, such as fold 1, fold 2, fold 3...? Does the modification affect the GPU memory and running time?

    opened by Jx-Tan 6
  • Inference the best model    no such file....

    Inference the best model no such file....

    when I Evaluating the models in the stage of Tainging/3.Training and Testing the models/A. Use the best model we have trained to infer the test set/(2).Evaluating the models/The Synapse multi-organ CT dataset Inference I meet this issue:

    (nnFormer) zheng@zheng-PC:~/PycharmProjects/nnFormer-main$ nnFormer_predict -i ../DATASET/nnFormer_raw/nnFormer_raw_data/Task002_Synapse/imagesTs -o ../DATASET/nnFormerr_raw/nnFormer_raw_data/Task002_Synapse/inferTs/output -m 3d_fullres -f 0 -t 2 -chk model_best -tr nnFormerTrainerV2_Synapse

    using model stored in /home/zheng/PycharmProjects/nnFormer-main/DATASET/nnFormer_trained_models/nnFormer/3d_fullres/Task002_Synapse/nnFormerTrainerV2_Synapse_nnFormerPlansv2.1 Traceback (most recent call last): File "/home/zheng/anaconda2/envs/nnFormer/bin/nnFormer_predict", line 33, in sys.exit(load_entry_point('nnformer', 'console_scripts', 'nnFormer_predict')()) File "/home/zheng/PycharmProjects/nnFormer-main/nnformer/inference/predict_simple.py", line 224, in main shutil.copy(plans_file, join(model_folder_name, "plans.pkl")) File "/home/zheng/anaconda2/envs/nnFormer/lib/python3.6/shutil.py", line 245, in copy copyfile(src, dst, follow_symlinks=follow_symlinks) File "/home/zheng/anaconda2/envs/nnFormer/lib/python3.6/shutil.py", line 120, in copyfile with open(src, 'rb') as fsrc: FileNotFoundError: [Errno 2] No such file or directory: '/home/zheng/PycharmProjects/nnFormer-main/DATASET/nnFormer_preprocessed/Task002_Synapse/nnFormerPlansv2.1_plans_3D.pkl'

    There seems to be no such document here.

    I have used the latest code version.

    opened by Jx-Tan 6
  • Problems relevant to using model_best weight for synapse task.

    Problems relevant to using model_best weight for synapse task.

    Hi there,when using the pretrained model_best weight relevant to synapse task,some error occur.

    No such file or directory: '/home/xychen/new_transformer/nnUNetFrame/DATASET/nnUNet_preprocessed/Task002_Abdomen/nnUNetPlansv2.1_plans_3D.pkl'

    I thought there maybe some setting errors but I coundn't find it.

    Could you please tell me which part setting lead to such error and how to fix it?

    Looking forward to your reply.

    opened by GYDDHPY 5
  • 关于网络配置文件nnFormerPlansv2.1_plans_3D.pkl

    关于网络配置文件nnFormerPlansv2.1_plans_3D.pkl

    您好,我使用您的代码预处理后得到的nnFormerPlansv2.1_plans_3D.pkl文件和nnunet预处理后相同的,直接使用导致错误如下: image

    论文中在提及在池化、深监督等方面是和nnunet有些许不同的,是您手动修改了nnFormerPlansv2.1_plans_3D文件吗?还是我没有进行正确的预处理?

    opened by lzy-whut 5
  • train own data using nnFormer

    train own data using nnFormer

    大佬您好,感谢开源,我想问问如果使用nnFormer来训练自己的数据,swin transformer的权重文件是必须的吗?我的数据集也是CT片所以我使用了和Synapse类似的change_plan_swim设置和您提供的Synapse预训练权重文件。前面步骤顺利到训练时报错:

    File "/home/lab/nnFormer/nnFormer/nnformer/run/run_training.py", line 140, in main raise RuntimeError("Could not find trainer class in nnformer.training.network_training") RuntimeError: Could not find trainer class in nnformer.training.network_training

    载入的pkl中很多参数返回都是None,不知道是哪出了问题,期待您的回复,祝好!

    opened by Cherryygq 5
  • Task056  Task083_Verse参数设置

    Task056 Task083_Verse参数设置

    大神你好, 感谢开源!你们的算法表现很惊艳,赞!我们想试试nnFormer在Verse数据上的表现。但是没太看明白你们change_plan_swin.py脚本里面的参数具体是怎么设置?Task001_ACDC和Task002_Synapse的配置有些区别, 如果换成VerSe数据集, 建议怎么修改配置参数?还需要多改动哪些地方? 另外一点,论文里面提到”V-MSA模块,让计算复杂度减少了超过98%“,那么平均推理速度相对nnUnet大概达到多少呢?

    opened by yangshunDragon 5
  • raise RuntimeError(

    raise RuntimeError("MultiThreadedAugmenter.abort_event was set, something went wrong.

    很抱歉打扰您!作者大大知道这个是什么原因吗? raise RuntimeError("MultiThreadedAugmenter.abort_event was set, something went wrong. Maybe one of " RuntimeError: MultiThreadedAugmenter.abort_event was set, something went wrong. Maybe one of your workers crashed. This is not the actual error message! Look further up your stdout to see what caused the error. Please also check whether your RAM was full`

    opened by nothingtosay-l 4
  • About 3090

    About 3090

    Can we use RTX3090 to train nnFormer? It always comes up with the following information: NVIDIA GeForce RTX 3090 with CUDA capability sm_86 is not compatible with the current PyTorch installation. The current PyTorch install supports CUDA capabilities sm_37 sm_50 sm_60 sm_70. If you want to use the NVIDIA GeForce RTX 3090 GPU with PyTorch, please check the instructions at https://pytorch.org/get-started/locally/

    And point out EOFError What should I do?

    opened by zzzyzh 3
  • training issue!

    training issue!

    Hi. everytime I use nnFormer to train, the whole learning process starts behaving very weird after 3-4 epochs. Please see below: (this happens regardless of dataset but this one was tested on synapse dataset)

    2022-11-22 23:23:13.618784:

    2022-11-22 23:23:13.619194: epoch: 0 2022-11-22 23:24:55.847692: train loss : 0.4745 2022-11-22 23:25:04.270859: validation loss: 0.3916 2022-11-22 23:25:04.271266: Average global foreground Dice: [0.0, 0.0001320296010365524, 2.274101208874301e-05, 0.0, 0.5969908676653919, 0.0, 0.0004676893599727684, 0.0] 2022-11-22 23:25:04.271303: (interpret this as an estimate for the Dice of the different classes. This is not exact.) 2022-11-22 23:25:04.607909: lr: 0.009991 2022-11-22 23:25:04.608021: current best_val_eval_criterion_MA is 0.07470 2022-11-22 23:25:04.608056: current val_eval_criterion_MA is 0.0747 2022-11-22 23:25:04.608111: This epoch took 110.988639 s

    2022-11-22 23:25:04.608142: epoch: 1 2022-11-22 23:26:39.660466: train loss : 0.4107 2022-11-22 23:26:48.049546: validation loss: 0.3228 2022-11-22 23:26:48.049927: Average global foreground Dice: [5.155274283790632e-06, 0.10146689407960259, 0.07402830708351602, 0.0, 0.5865517051479923, 0.0, 0.190082921281602, 0.0] 2022-11-22 23:26:48.049962: (interpret this as an estimate for the Dice of the different classes. This is not exact.) 2022-11-22 23:26:48.361166: lr: 0.009982 2022-11-22 23:26:48.361290: current best_val_eval_criterion_MA is 0.07470 2022-11-22 23:26:48.361320: current val_eval_criterion_MA is 0.0791 2022-11-22 23:26:48.541745: saving checkpoint... 2022-11-22 23:26:49.902999: done, saving took 1.54 seconds 2022-11-22 23:26:49.913620: This epoch took 105.305442 s

    2022-11-22 23:26:49.913674: epoch: 2 2022-11-22 23:28:23.650122: train loss : nan 2022-11-22 23:28:32.050157: validation loss: 0.2625 2022-11-22 23:28:32.050549: Average global foreground Dice: [8.664553959592852e-07, 0.035751748251748254, 5.627967535285733e-05, 0.0, 0.7476544526585733, 9.193226920710256e-06, 0.20471679386411343, 0.0] 2022-11-22 23:28:32.050584: (interpret this as an estimate for the Dice of the different classes. This is not exact.) 2022-11-22 23:28:32.350800: lr: 0.009973 2022-11-22 23:28:32.350904: current best_val_eval_criterion_MA is 0.07910 2022-11-22 23:28:32.350929: current val_eval_criterion_MA is 0.0836 2022-11-22 23:28:32.484009: saving checkpoint... 2022-11-22 23:28:33.946634: done, saving took 1.60 seconds 2022-11-22 23:28:33.959197: This epoch took 104.045484 s

    2022-11-22 23:28:33.959269: epoch: 3 2022-11-22 23:30:09.360566: train loss : nan 2022-11-22 23:30:17.759997: validation loss: 0.3103 2022-11-22 23:30:17.760396: Average global foreground Dice: [0.18516268021874827, 0.05451979106578396, 0.00011326764572626299, 0.0, 0.6808126343402319, 0.00023126412611603676, 0.34789594875600277, 0.0013141655534632897] 2022-11-22 23:30:17.760430: (interpret this as an estimate for the Dice of the different classes. This is not exact.) 2022-11-22 23:30:18.065005: lr: 0.009964 2022-11-22 23:30:18.065115: current best_val_eval_criterion_MA is 0.08360 2022-11-22 23:30:18.065141: current val_eval_criterion_MA is 0.0911 2022-11-22 23:30:18.211870: saving checkpoint... 2022-11-22 23:30:19.666188: done, saving took 1.60 seconds 2022-11-22 23:30:19.686091: This epoch took 105.726789 s

    2022-11-22 23:30:19.686258: epoch: 4 2022-11-22 23:31:57.169777: train loss : nan 2022-11-22 23:32:05.593204: validation loss: 1.4975 2022-11-22 23:32:05.593626: Average global foreground Dice: [0.19781398936287933, 0.0, 0.03799706011142289, 0.0, 0.036336266863399164, 0.023039737116651585, 5.222491180518019e-06, 0.0] 2022-11-22 23:32:05.593674: (interpret this as an estimate for the Dice of the different classes. This is not exact.) 2022-11-22 23:32:05.909654: lr: 0.009955 2022-11-22 23:32:05.909767: current best_val_eval_criterion_MA is 0.09110 2022-11-22 23:32:05.909795: current val_eval_criterion_MA is 0.0857 2022-11-22 23:32:05.909823: This epoch took 106.223495 s

    2022-11-22 23:32:05.909852: epoch: 5 2022-11-22 23:33:41.345607: train loss : nan 2022-11-22 23:33:49.781050: validation loss: 1.8927 2022-11-22 23:33:49.781520: Average global foreground Dice: [0.1855083287413658, 0.0, 0.01310647396050735, 0.0, 0.04130637519675208, 0.032110824761916984, 0.0, 0.0] 2022-11-22 23:33:49.781560: (interpret this as an estimate for the Dice of the different classes. This is not exact.) 2022-11-22 23:33:50.090640: lr: 0.009946 2022-11-22 23:33:50.090751: current best_val_eval_criterion_MA is 0.09110 2022-11-22 23:33:50.090780: current val_eval_criterion_MA is 0.0805 2022-11-22 23:33:50.090809: This epoch took 104.180931 s

    opened by Chadkowski 1
  • Is a pretrained model weights required for synapse dataset training?

    Is a pretrained model weights required for synapse dataset training?

    https://github.com/282857341/nnFormer/blob/d48aaeb05612a09538f70c38a2357149ea3a7ac0/nnformer/training/network_training/nnFormerTrainerV2_nnformer_synapse.py#L54

    https://github.com/282857341/nnFormer/blob/d48aaeb05612a09538f70c38a2357149ea3a7ac0/nnformer/training/network_training/nnFormerTrainerV2_nnformer_synapse.py#L177

    Since the defaut is True, I'm assuming it is necessary to load (ImageNet?) pretrained weights. However, I can't find the pretrained weights here, or do you mean this link?

    opened by SilenceMonk 0
  • problems encountered on my own dataset

    problems encountered on my own dataset

    hello,I want to ask where should I modify if I want to train nnFormer on my own dataset? image In the nnFormerTrainerV2_nnformer_acdc.py,the do_split() function,what are these code intended for? image

    opened by puppy2000 1
  • Difference between run_online_evaluation() and validate()

    Difference between run_online_evaluation() and validate()

    Hi @282857341,

    I was wondering what is the difference between the functionalities of run_online_evaluation() and validate() functions as I noticed the DICE scores are different for the validation during training and the actual validation.

    Thanks

    opened by mmaaz60 0
Owner
jsguo
jsguo
The undersampled DWI image using Slice-Interleaved Diffusion Encoding (SIDE) method can be reconstructed by the UNet network.

UNet-SIDE The undersampled DWI image using Slice-Interleaved Diffusion Encoding (SIDE) method can be reconstructed by the UNet network. For Super Reso

TIANTIAN XU 1 Jan 13, 2022
A PyTorch implementation for V-Net: Fully Convolutional Neural Networks for Volumetric Medical Image Segmentation

A PyTorch implementation of V-Net Vnet is a PyTorch implementation of the paper V-Net: Fully Convolutional Neural Networks for Volumetric Medical Imag

Matthew Macy 606 Dec 21, 2022
Unofficial implementation of Point-Unet: A Context-Aware Point-Based Neural Network for Volumetric Segmentation

Point-Unet This is an unofficial implementation of the MICCAI 2021 paper Point-Unet: A Context-Aware Point-Based Neural Network for Volumetric Segment

Namt0d 9 Dec 7, 2022
This repository contains the code for the CVPR 2020 paper "Differentiable Volumetric Rendering: Learning Implicit 3D Representations without 3D Supervision"

Differentiable Volumetric Rendering Paper | Supplementary | Spotlight Video | Blog Entry | Presentation | Interactive Slides | Project Page This repos

null 697 Jan 6, 2023
VSR-Transformer - This paper proposes a new Transformer for video super-resolution (called VSR-Transformer).

VSR-Transformer By Jiezhang Cao, Yawei Li, Kai Zhang, Luc Van Gool This paper proposes a new Transformer for video super-resolution (called VSR-Transf

Jiezhang Cao 225 Nov 13, 2022
This is an official implementation for "Swin Transformer: Hierarchical Vision Transformer using Shifted Windows" on Object Detection and Instance Segmentation.

Swin Transformer for Object Detection This repo contains the supported code and configuration files to reproduce object detection results of Swin Tran

Swin Transformer 1.4k Dec 30, 2022
Implementation of Transformer in Transformer, pixel level attention paired with patch level attention for image classification, in Pytorch

Transformer in Transformer Implementation of Transformer in Transformer, pixel level attention paired with patch level attention for image c

Phil Wang 272 Dec 23, 2022
Third party Pytorch implement of Image Processing Transformer (Pre-Trained Image Processing Transformer arXiv:2012.00364v2)

ImageProcessingTransformer Third party Pytorch implement of Image Processing Transformer (Pre-Trained Image Processing Transformer arXiv:2012.00364v2)

null 61 Jan 1, 2023
Implementation of Vision Transformer, a simple way to achieve SOTA in vision classification with only a single transformer encoder, in Pytorch

Implementation of Vision Transformer, a simple way to achieve SOTA in vision classification with only a single transformer encoder, in Pytorch

Phil Wang 12.6k Jan 9, 2023
Alex Pashevich 62 Dec 24, 2022
The implementation of "Shuffle Transformer: Rethinking Spatial Shuffle for Vision Transformer"

Shuffle Transformer The implementation of "Shuffle Transformer: Rethinking Spatial Shuffle for Vision Transformer" Introduction Very recently, window-

null 87 Nov 29, 2022
Unofficial implementation of "Swin Transformer: Hierarchical Vision Transformer using Shifted Windows" (https://arxiv.org/abs/2103.14030)

Swin-Transformer-Tensorflow A direct translation of the official PyTorch implementation of "Swin Transformer: Hierarchical Vision Transformer using Sh

null 52 Dec 29, 2022
CSWin Transformer: A General Vision Transformer Backbone with Cross-Shaped

CSWin-Transformer This repo is the official implementation of "CSWin Transformer: A General Vision Transformer Backbone with Cross-Shaped Windows". Th

Microsoft 409 Jan 6, 2023
3D-Transformer: Molecular Representation with Transformer in 3D Space

3D-Transformer: Molecular Representation with Transformer in 3D Space

null 55 Dec 19, 2022
This repository builds a basic vision transformer from scratch so that one beginner can understand the theory of vision transformer.

vision-transformer-from-scratch This repository includes several kinds of vision transformers from scratch so that one beginner can understand the the

null 1 Dec 24, 2021
Transformer - Transformer in PyTorch

Transformer 完成进度 Embeddings and PositionalEncoding with example. MultiHeadAttent

Tianyang Li 1 Jan 6, 2022
Transformer Huffman coding - Complete Huffman coding through transformer

Transformer_Huffman_coding Complete Huffman coding through transformer 2022/2/19

null 3 May 19, 2022
Implementation of the Transformer variant proposed in "Transformer Quality in Linear Time"

FLASH - Pytorch Implementation of the Transformer variant proposed in the paper Transformer Quality in Linear Time Install $ pip install FLASH-pytorch

Phil Wang 209 Dec 28, 2022