Accelerated Multi-Modal MR Imaging with Transformers

Last update: Dec 16, 2022

Related tags

Deep Learning MTrans

Overview

Accelerated Multi-Modal MR Imaging with Transformers

Dependencies

numpy==1.18.5
scikit_image==0.16.2
torchvision==0.8.1
torch==1.7.0
runstats==1.8.0
pytorch_lightning==1.0.6
h5py==2.10.0
PyYAML==5.4

🔥 NEWS 🔥

We have uploaded the csv.files of the paired data.

multi gpu train

python -m torch.distributed.launch --nproc_per_node=8   train.py --experiment sr_multi_cross

single gpu train

python train.py --experiment sr_multi_cross

multi gpu test

python -m torch.distributed.launch --nproc_per_node=8   test.py --experiment sr_multi_cross

single gpu test

python test.py --experiment sr_multi_cross

--experiment is the experiment you running. And you can change the config file to set the parameter when training or testing.

Citation

@article{feng2021accelerated,
  title={Accelerated Multi-Modal MR Imaging with Transformers},
  author={Feng, Chun-Mei and Yan, Yunlu and Chen, Geng and Fu, Huazhu and Xu, Yong and Shao, Ling},
  journal={arXiv e-prints},
  pages={arXiv--2106},
  year={2021}
}

Multi-modal co-attention for drug-target interaction annotation and Its Application to SARS-CoV-2

CoaDTI Multi-modal co-attention for drug-target interaction annotation and Its Application to SARS-CoV-2 Abstract Environment The test was conducted i

7 Nov 14, 2022

Code of paper Interact, Embed, and EnlargE (IEEE): Boosting Modality-specific Representations for Multi-Modal Person Re-identification.

Interact, Embed, and EnlargE (IEEE): Boosting Modality-specific Representations for Multi-Modal Person Re-identification We provide the codes for repr

12 Dec 12, 2022

Multi-Modal Machine Learning toolkit based on PyTorch.

简体中文 | English TorchMM 简介多模态学习工具包 TorchMM 旨在于提供模态联合学习和跨模态学习算法模型库，为处理图片文本等多模态数据提供高效的解决方案，助力多模态学习应用落地。近期更新 2022.1.5 发布 TorchMM 初始版本 v1.0 特性丰富的任务场景：工具

1 Jan 5, 2022

Multi-Modal Machine Learning toolkit based on PaddlePaddle.

简体中文 | English PaddleMM 简介飞桨多模态学习工具包 PaddleMM 旨在于提供模态联合学习和跨模态学习算法模型库，为处理图片文本等多模态数据提供高效的解决方案，助力多模态学习应用落地。近期更新 2022.1.5 发布 PaddleMM 初始版本 v1.0 特性丰富的任务

520 Dec 28, 2022

Multi-modal Text Recognition Networks: Interactive Enhancements between Visual and Semantic Features

Multi-modal Text Recognition Networks: Interactive Enhancements between Visual and Semantic Features | paper | Official PyTorch implementation for Mul

48 Dec 28, 2022

4st place solution for the PBVS 2022 Multi-modal Aerial View Object Classification Challenge - Track 1 (SAR) at PBVS2022

A Two-Stage Shake-Shake Network for Long-tailed Recognition of SAR Aerial View Objects 4st place solution for the PBVS 2022 Multi-modal Aerial View Ob

5 Nov 9, 2022

[LREC] MMChat: Multi-Modal Chat Dataset on Social Media

MMChat This repo contains the code and data for the LREC2022 paper MMChat: Multi-Modal Chat Dataset on Social Media. Dataset MMChat is a large-scale d

47 Jan 3, 2023

[CVPR 2022 Oral] Versatile Multi-Modal Pre-Training for Human-Centric Perception

Versatile Multi-Modal Pre-Training for Human-Centric Perception Fangzhou Hong1 Liang Pan1 Zhongang Cai1,2,3 Ziwei Liu1* 1S-Lab, Nanyang Technologic

96 Jan 3, 2023

GPU-Accelerated Deep Learning Library in Python

Hebel GPU-Accelerated Deep Learning Library in Python Hebel is a library for deep learning with neural networks in Python using GPU acceleration with

1.2k Dec 21, 2022

Comments

ModuleNotFoundError

from .contrast_model.unet import build_model as UNET from .contrast_model.unet_multi import build_model as UNETMULTI

from .contrast_model.mcsr import build_model as MCSR from .contrast_model.edsr import build_model as EDSR

ModuleNotFoundError: No module named 'models.contrast_model'

I can not find these modules in project,can you help me?

opened by massica17 5
Problem in code

When I ran the code 'train.py' encountered an unsolvable problem, so I came to ask for help. This error looks like this:

/home/img/anaconda3/envs/bin/python3.6 /home/img/Desktop/lff/MTrans-main/train.py doing sr_single DATASET: CHALLENGE: singlecoil MODE: train ROOT: /home/img/Desktop/lff/Dataset/single INPUT_SIZE: 160 MODEL: HEAD_HIDDEN_DIM: 16 INPUT_DIM: 1 OUTPUT_DIM: 1 PATCH_SIZE: 16 TRANSFORMER_DEPTH: 4 TRANSFORMER_MLP_RATIO: 3 TRANSFORMER_NUM_HEADS: 4 MULTI:

OUTPUTDIR: /home/img/Desktop/lff/MTrans-main/output/weights_SR_single_50x RESUME: SCALE: 2 SEED: 42 SOLVER: BATCH_SIZE: 8 DEVICE: cuda DEVICE_IDS: [0, 1] LR: 0.001 LR_DROP: [40] NUM_WORKERS: 16 PRINT_FREQ: 10 WEIGHT_DECAY: 0.0001 TEST_OUTPUTDIR: outputs/sr_single TRAIN: EPOCHS: 50 TRANSFORMS: ACCELERATIONS: [4] CENTER_FRACTIONS: [0.08] MASKTYPE: random USE_CL1_LOSS: False USE_MULTI_MODEL: False WORK_TYPE: sr dist_url: env:// world_size: 1 Not using distributed mode number of params: 641.73 M Traceback (most recent call last): File "/home/img/Desktop/lff/MTrans-main/train.py", line 170, in main(cfg, args.experiment) File "/home/img/Desktop/lff/MTrans-main/train.py", line 57, in main dataset_train = build_dataset(args, mode='train') File "/home/img/Desktop/lff/MTrans-main/data/fastmri.py", line 224, in build_dataset sample_rate=sample_rate, mode=mode) File "/home/img/Desktop/lff/MTrans-main/data/fastmri.py", line 119, in init with open(self.csv_file, 'r') as f: FileNotFoundError: [Errno 2] No such file or directory: '/home/img/Desktop/lff/Dataset/single/singlecoil_train/singlecoil_train_split_less.csv'

Process finished with exit code 1

I don’t know how to deal with it, so, please. Thanks for the help.

opened by Aristot1e 26

Accelerated Multi-Modal MR Imaging with Transformers

Related tags

Overview

Accelerated Multi-Modal MR Imaging with Transformers

Dependencies

🔥 NEWS 🔥

You might also like...

Multi-modal co-attention for drug-target interaction annotation and Its Application to SARS-CoV-2

Code of paper Interact, Embed, and EnlargE (IEEE): Boosting Modality-specific Representations for Multi-Modal Person Re-identification.

Multi-Modal Machine Learning toolkit based on PyTorch.

Multi-Modal Machine Learning toolkit based on PaddlePaddle.

Multi-modal Text Recognition Networks: Interactive Enhancements between Visual and Semantic Features

4st place solution for the PBVS 2022 Multi-modal Aerial View Object Classification Challenge - Track 1 (SAR) at PBVS2022

[LREC] MMChat: Multi-Modal Chat Dataset on Social Media

[CVPR 2022 Oral] Versatile Multi-Modal Pre-Training for Human-Centric Perception

GPU-Accelerated Deep Learning Library in Python

Comments

ModuleNotFoundError

Problem in code

Owner

Code of U2Fusion: a unified unsupervised image fusion network for multiple image fusion tasks, including multi-modal, multi-exposure and multi-focus image fusion.

Code and pre-trained models for MultiMAE: Multi-modal Multi-task Masked Autoencoders

Official PyTorch implementation for Generic Attention-model Explainability for Interpreting Bi-Modal and Encoder-Decoder Transformers, a novel method to visualize any Transformer-based network. Including examples for DETR, VQA.

[CVPR'21] Multi-Modal Fusion Transformer for End-to-End Autonomous Driving

Deep RGB-D Saliency Detection with Depth-Sensitive Attention and Automatic Multi-Modal Fusion (CVPR'2021, Oral)

A Multi-modal Model Chinese Spell Checker Released on ACL2021.

We present a framework for training multi-modal deep learning models on unlabelled video data by forcing the network to learn invariances to transformations applied to both the audio and video streams.

A pytorch-based deep learning framework for multi-modal 2D/3D medical image segmentation

AdaMML: Adaptive Multi-Modal Learning for Efficient Video Recognition

Self-supervised Multi-modal Hybrid Fusion Network for Brain Tumor Segmentation