Code of U2Fusion: a unified unsupervised image fusion network for multiple image fusion tasks, including multi-modal, multi-exposure and multi-focus image fusion.

Han Xu

Last update: Dec 11, 2022

Related tags

Deep Learning U2Fusion

Overview

U2Fusion

Code of U2Fusion: a unified unsupervised image fusion network for multiple image fusion tasks, including multi-modal (VIS-IR, medical), multi-exposure and multi-focus image fusion.

Tips:

Large files should be downloaded separately, including the following files:

For training:

Training dataset
The code to create your own training dataset can be found here.
vgg16 (put it in 'VGGnet/')

If this work is helpful to you, please cite it as:

@article{xu2020u2fusion,
  title={U2Fusion: A Unified Unsupervised Image Fusion Network},
  author={Xu, Han and Ma, Jiayi and Jiang, Junjun and Guo, Xiaojie and Ling, Haibin},
  journal={IEEE Transactions on Pattern Analysis and Machine Intelligence},
  year={2020},
  publisher={IEEE}
}

If you have any question, please email to me ([email protected]).

Comments

Open-source code about RF-Net

Hi, Thank your great contributio for this community. Recently, I read this paper "RFNet: Unsupervised Network for Mutually Reinforcing Multi-modal Image Registration and Fusion" Is there any plan for the open-source code of RF-Net? Or could you share this code with me. Appreciate it!

opened by lajipeng 0
HELP!!

Hi, Han Xu

You can provide the papers' RFNet: Unsupervised Network for Mutually Reinforcing Multi - modal Image Registration and Fusion 'code?

Thank you!

opened by songwenhao123 0
Are the medical datasets used when training on task1?

hi, thank you for sharing your code. I have some questions for you.

are the medical dataset used when training on task1? It seems that the datasets you provide did not include the medical dataset, and the dataset did not seems to be used in your code. if used, could you please provide a link to download it and update the code to use it? looking forward to an early reply, thank you.

opened by L-D-Luffy 0
about output image bit width

Hi, Han Xu

Thank you for your excellent work!

In this work, is the image after the fusion of the two LDR images still 8bit? 12bit or higher.

Thank you!

opened by Qirui-Y 1
Fusing Multiple Images

Thanks for the nice work! I have a question regarding fusing multiple images. The paper mentions fusing multiple images sequentially (two at a time.)

But it appears to me that the loss used for training can be easiliy generalized to more than two images and hence achieving multiple-image (more than two images) fusion as one single pass to the DenseNet. Am I right or do I miss something?

Thanks!

opened by macrohuang1993 0

Code of U2Fusion: a unified unsupervised image fusion network for multiple image fusion tasks, including multi-modal, multi-exposure and multi-focus image fusion.

Related tags

Overview

U2Fusion

Tips:

For training:

Comments

Open-source code about RF-Net

HELP!!

Are the medical datasets used when training on task1?

about output image bit width

Fusing Multiple Images

Owner

Han Xu

implementation of paper - You Only Learn One Representation: Unified Network for Multiple Tasks

We propose a new method for effective shadow removal by regarding it as an exposure fusion problem.

Official PyTorch implementation for Generic Attention-model Explainability for Interpreting Bi-Modal and Encoder-Decoder Transformers, a novel method to visualize any Transformer-based network. Including examples for DETR, VQA.

Self-supervised Multi-modal Hybrid Fusion Network for Brain Tumor Segmentation

Unified unsupervised and semi-supervised domain adaptation network for cross-scenario face anti-spoofing, Pattern Recognition

Deep RGB-D Saliency Detection with Depth-Sensitive Attention and Automatic Multi-Modal Fusion (CVPR'2021, Oral)

[CVPR'21] Multi-Modal Fusion Transformer for End-to-End Autonomous Driving

Code for Piggyback: Adapting a Single Network to Multiple Tasks by Learning to Mask Weights

Code for PackNet: Adding Multiple Tasks to a Single Network by Iterative Pruning

Python package for multiple object tracking research with focus on laboratory animals tracking.

Exposure Time Calculator (ETC) and radial velocity precision estimator for the Near InfraRed Planet Searcher (NIRPS) spectrograph

TorchDistiller - a collection of the open source pytorch code for knowledge distillation, especially for the perception tasks, including semantic segmentation, depth estimation, object detection and instance segmentation.

Outlier Exposure with Confidence Control for Out-of-Distribution Detection

Deep Anomaly Detection with Outlier Exposure (ICLR 2019)

We evaluate our method on different datasets (including ShapeNet, CUB-200-2011, and Pascal3D+) and achieve state-of-the-art results, outperforming all the other supervised and unsupervised methods and 3D representations, all in terms of performance, accuracy, and training time.

This is the official implementation of TrivialAugment and a mini-library for the application of multiple image augmentation strategies including RandAugment and TrivialAugment.

Code and pre-trained models for MultiMAE: Multi-modal Multi-task Masked Autoencoders

A lightweight Python-based 3D network multi-agent simulator. Uses a cell-based congestion model. Calculates risk, loudness and battery capacities of the agents. Suitable for 3D network optimization tasks.

We present a framework for training multi-modal deep learning models on unlabelled video data by forcing the network to learn invariances to transformations applied to both the audio and video streams.