Breaking the Dilemma of Medical Image-to-image Translation

Related tags

Deep Learning Reg-GAN
Overview

Breaking the Dilemma of Medical Image-to-image Translation

Supervised Pix2Pix and unsupervised Cycle-consistency are two modes that dominate the field of medical image-to-image translation. However, neither modes are ideal. The Pix2Pix mode has excellent performance. But it requires paired and well pixel-wise aligned images, which may not always be achievable due to respiratory motion or anatomy change between times that paired images are acquired. The Cycle-consistency mode is less stringent with training data and works well on unpaired or misaligned images. But its performance may not be optimal. In order to break the dilemma of the existing modes, we propose a new unsupervised mode called RegGAN for medical image-to-image translation. It is based on the theory of "loss-correction". In RegGAN, the misaligned target images are considered as noisy labels and the generator is trained with an additional registration network to fit the misaligned noise distribution adaptively. The goal is to search for the common optimal solution to both image-to-image translation and registration tasks. We incorporated RegGAN into a few state-of-the-art image-to-image translation methods and demonstrated that RegGAN could be easily combined with these methods to improve their performances. Such as a simple CycleGAN in our mode surpasses latest NICEGAN even though using less network parameters. Based on our results, RegGAN outperformed both Pix2Pix on aligned data and Cycle-consistency on misaligned or unpaired data. RegGAN is insensitive to noises which makes it a better choice for a wide range of scenarios, especially for medical image-to-image translation tasks in which well pixel-wise aligned data are not available

This paper has been accepted by NeurIPS 2021. Get the full paper on Arxiv.

Comments
  • About the training details.

    About the training details.

    Thank for your work, Kid-Liet. Could you please complete the training process and important details? I'm looking forward to doing something about the Reg-GAN.

    opened by ElegantLee 15
  • Save the test results

    Save the test results 

                           Dear Kid-Liet,

    I have noticed that there is a "save_deformation" subfunction within CycTrainer.py. Please, what is it for? Besides, how can CycTrainer.py be modified in such a way that the results can be saved during testing. I am sorry, I am not very familiar with Pytorch.

    Thank you for your time and help.

    Kind regards

    opened by patricegaofei 8
  • CUDA version

    CUDA version 

                        Dear author,

    I hope you are having a productive day. Congratulations on your great work, and thank you very much for making the source codes publicly accessible! As you stated in the environment requirements, one has to install Cuda 11.1 to be able to run the codes. Unfortunately, I am having an older Cuda version (10.0) on my server, and the codes do not work with this version. Besides, I am running the codes on a server that is being used by many people and I am not the Admin, which makes it quite impossible for me to upgrade the Cuda version. Please, is there any way to adjust the codes in such a way that they can be run on an environment with older Cuda version?

    Thank you very much for your time and help!

    With kind regards

    opened by patricegaofei 2
  • are you really using the decay in learning rate?

    are you really using the decay in learning rate?

    Hello Author (@Kid-Liet), Thanks for your clean implementation.

    I am curious to know if you are really using the weight decay here in your implementation (NC+R)? if so in which line (can you please point me to that line)? Though I can see you have set decay_epoch=20 but I can not see its usage in CycTrainer.py file.

    Also in your paper you have mentioned that the batch size was set too 1 with weight decay 0.0001. what do you mean by this line?

    opened by zeeshannisar 1
  • about how to process my own dataset for this project

    about how to process my own dataset for this project

    Can you please describe the features in the dataset processing process, I am training with my own dataset, but the dataset format is not correct. I've converted the png image to npy format, but I can't understand why the original dataset is two-dimensional

    opened by 445193519 1
  • Should network R be used in the test phase?

    Should network R be used in the test phase?

    Great Work! I read your paper and realize your goal is to optimize three networks G, D, and R through L1loss(you named correction loss), SMloss, and adversarial loss.

    But there is one thing in the test phase that confuses me. Which of G(x) and R(G(x)) should I use as the final generated image? I noticed a test phase in your code that only use G(x) as the final translated image. Is it correct for Reg-GAN that use G(x) rather than R(G(x)) as the translated image?

    If there is some mistake in my understanding of RegGAN, hope you can point it out.

    opened by Casera-Meow 0
  • 关于unpaired datasets 的训练

    关于unpaired datasets 的训练

    尊敬的作者，您好！我想请教一下文中所说的刚性配准具体用的哪个算法。“For unpaired datasets, we can conduct rigid registration first in 3D space and then use RegGAN for training.” 这句话的意思是利用刚性配准尽可能的为源域图像找到配对图像后再用于reggan的训练吗？.

    opened by kkgoer 1
Owner
Kid Liet
Kid Liet
GitHub
Official implementation for "Low-light Image Enhancement via Breaking Down the Darkness"

Low-light Image Enhancement via Breaking Down the Darkness by Qiming Hu, Xiaojie Guo. 1. Dependencies Python3 PyTorch>=1.0 OpenCV-Python, TensorboardX

Qiming Hu 30 Jan 1, 2023
Pytorch Code for "Medical Transformer: Gated Axial-Attention for Medical Image Segmentation"

Medical-Transformer Pytorch Code for the paper "Medical Transformer: Gated Axial-Attention for Medical Image Segmentation" About this repo: This repo

Jeya Maria Jose 615 Dec 25, 2022
The Medical Detection Toolkit contains 2D + 3D implementations of prevalent object detectors such as Mask R-CNN, Retina Net, Retina U-Net, as well as a training and inference framework focused on dealing with medical images.

The Medical Detection Toolkit contains 2D + 3D implementations of prevalent object detectors such as Mask R-CNN, Retina Net, Retina U-Net, as well as a training and inference framework focused on dealing with medical images.

MIC-DKFZ 1.2k Jan 4, 2023
Build a medical knowledge graph based on Unified Language Medical System (UMLS)

UMLS-Graph Build a medical knowledge graph based on Unified Language Medical System (UMLS) Requisite Install MySQL Server 5.6 and import UMLS data int

Donghua Chen 6 Dec 25, 2022
Copy Paste positive polyp using poisson image blending for medical image segmentation

Copy Paste positive polyp using poisson image blending for medical image segmentation According poisson image blending I've completely used it for bio

Phạm Vũ Hùng 2 Oct 19, 2021
This repository contains several image-to-image translation models, whcih were tested for RGB to NIR image generation. The models are Pix2Pix, Pix2PixHD, CycleGAN and PointWise.

RGB2NIR_Experimental This repository contains several image-to-image translation models, whcih were tested for RGB to NIR image generation. The models

null 5 Jan 4, 2023
This repo holds code for TransUNet: Transformers Make Strong Encoders for Medical Image Segmentation

TransUNet This repo holds code for TransUNet: Transformers Make Strong Encoders for Medical Image Segmentation Usage

null 1.4k Jan 4, 2023
CoTr: Efficiently Bridging CNN and Transformer for 3D Medical Image Segmentation

CoTr: Efficient 3D Medical Image Segmentation by bridging CNN and Transformer This is the official pytorch implementation of the CoTr: Paper: CoTr: Ef

null 218 Dec 25, 2022
[CVPR'21] FedDG: Federated Domain Generalization on Medical Image Segmentation via Episodic Learning in Continuous Frequency Space

FedDG: Federated Domain Generalization on Medical Image Segmentation via Episodic Learning in Continuous Frequency Space by Quande Liu, Cheng Chen, Ji

Quande Liu 178 Jan 6, 2023
A collection of loss functions for medical image segmentation

A collection of loss functions for medical image segmentation

Jun 3.1k Jan 3, 2023
ivadomed is an integrated framework for medical image analysis with deep learning.

Repository on the collaborative IVADO medical imaging project between the Mila and NeuroPoly labs.

null 144 Dec 19, 2022
Vision Transformer for 3D medical image registration (Pytorch).

ViT-V-Net: Vision Transformer for Volumetric Medical Image Registration keywords: vision transformer, convolutional neural networks, image registratio

Junyu Chen 192 Dec 20, 2022
Medical Image Segmentation using Squeeze-and-Expansion Transformers

Medical Image Segmentation using Squeeze-and-Expansion Transformers Introduction This repository contains the code of the IJCAI'2021 paper 'Medical Im

askerlee 172 Dec 20, 2022
The codes for the work "Swin-Unet: Unet-like Pure Transformer for Medical Image Segmentation"

Swin-Unet The codes for the work "Swin-Unet: Unet-like Pure Transformer for Medical Image Segmentation"(https://arxiv.org/abs/2105.05537). A validatio

null 869 Jan 7, 2023
TorchIO is a Medical image preprocessing and augmentation toolkit for deep learning. Part of the PyTorch Ecosystem.

Medical image preprocessing and augmentation toolkit for deep learning. Part of the PyTorch Ecosystem.

Fernando Pérez-García 1.6k Jan 6, 2023
Segmentation for medical image.

EfficientSegmentation Introduction EfficientSegmentation is an open source, PyTorch-based segmentation framework for 3D medical image. Features A whol

null 68 Nov 28, 2022
A PyTorch implementation for V-Net: Fully Convolutional Neural Networks for Volumetric Medical Image Segmentation

A PyTorch implementation of V-Net Vnet is a PyTorch implementation of the paper V-Net: Fully Convolutional Neural Networks for Volumetric Medical Imag

Matthew Macy 606 Dec 21, 2022
Semi Supervised Learning for Medical Image Segmentation, a collection of literature reviews and code implementations.

Semi-supervised-learning-for-medical-image-segmentation. Recently, semi-supervised image segmentation has become a hot topic in medical image computin

Healthcare Intelligence Laboratory 1.3k Jan 3, 2023
A pytorch-based deep learning framework for multi-modal 2D/3D medical image segmentation

A 3D multi-modal medical image segmentation library in PyTorch We strongly believe in open and reproducible deep learning research. Our goal is to imp

Adaloglou Nikolas 1.2k Dec 27, 2022