Code of U2Fusion: a unified unsupervised image fusion network for multiple image fusion tasks, including multi-modal, multi-exposure and multi-focus image fusion.

Overview

U2Fusion

Code of U2Fusion: a unified unsupervised image fusion network for multiple image fusion tasks, including multi-modal (VIS-IR, medical), multi-exposure and multi-focus image fusion.

Tips:

Large files should be downloaded separately, including the following files:

For training:

If this work is helpful to you, please cite it as:

@article{xu2020u2fusion,
  title={U2Fusion: A Unified Unsupervised Image Fusion Network},
  author={Xu, Han and Ma, Jiayi and Jiang, Junjun and Guo, Xiaojie and Ling, Haibin},
  journal={IEEE Transactions on Pattern Analysis and Machine Intelligence},
  year={2020},
  publisher={IEEE}
}

If you have any question, please email to me ([email protected]).

Issues
  • Are the medical datasets used when training on task1?

    Are the medical datasets used when training on task1?

    hi, thank you for sharing your code. I have some questions for you.

    are the medical dataset used when training on task1? It seems that the datasets you provide did not include the medical dataset, and the dataset did not seems to be used in your code. if used, could you please provide a link to download it and update the code to use it? looking forward to an early reply, thank you.

    opened by L-D-Luffy 0
  • about output image bit width

    about output image bit width

    Hi, Han Xu

    Thank you for your excellent work!

    In this work, is the image after the fusion of the two LDR images still 8bit? 12bit or higher.

    Thank you!

    opened by Qirui-Y 1
  • 请问可以提供训练代码吗?

    请问可以提供训练代码吗?

    作者您好,我对U2Fusion的思路很感兴趣,想先试试自己的数据集训练效果,不过看到train_task.py中没有定义main函数,请问方便的话可以提供一下对应的训练代码吗?

    opened by SSyangguang 1
  • Fusing Multiple Images

    Fusing Multiple Images

    Thanks for the nice work! I have a question regarding fusing multiple images. The paper mentions fusing multiple images sequentially (two at a time.)

    But it appears to me that the loss used for training can be easiliy generalized to more than two images and hence achieving multiple-image (more than two images) fusion as one single pass to the DenseNet. Am I right or do I miss something?

    Thanks!

    opened by macrohuang1993 0
  • 请问怎么实现RGB图像的融合

    请问怎么实现RGB图像的融合

    我正在学习1通道的红外+3通道的RGB图像的融合,您的论文中提到对RGB使用YCbCr转换,只用Y分量去融合,请问是如何实现的?

    opened by playerben 5
  • 关于图像通道数

    关于图像通道数

    你好,请问训练的图片通道数必须为1嘛? 还有test的时候,可以输入通道数为4的图像嘛?

    opened by dreamckk 1
Owner
Han Xu
Han Xu
We propose a new method for effective shadow removal by regarding it as an exposure fusion problem.

Auto-exposure fusion for single-image shadow removal We propose a new method for effective shadow removal by regarding it as an exposure fusion proble

Qing Guo 97 Jan 7, 2022
Official PyTorch implementation for Generic Attention-model Explainability for Interpreting Bi-Modal and Encoder-Decoder Transformers, a novel method to visualize any Transformer-based network. Including examples for DETR, VQA.

PyTorch Implementation of Generic Attention-model Explainability for Interpreting Bi-Modal and Encoder-Decoder Transformers 1 Using Colab Please notic

Hila Chefer 279 Jan 13, 2022
Self-supervised Multi-modal Hybrid Fusion Network for Brain Tumor Segmentation

JBHI-Pytorch This repository contains a reference implementation of the algorithms described in our paper "Self-supervised Multi-modal Hybrid Fusion N

FeiyiFANG 5 Dec 13, 2021
Deep RGB-D Saliency Detection with Depth-Sensitive Attention and Automatic Multi-Modal Fusion (CVPR'2021, Oral)

DSA^2 F: Deep RGB-D Saliency Detection with Depth-Sensitive Attention and Automatic Multi-Modal Fusion (CVPR'2021, Oral) This repo is the official imp

如今我已剑指天涯 40 Jan 8, 2022
[CVPR'21] Multi-Modal Fusion Transformer for End-to-End Autonomous Driving

TransFuser This repository contains the code for the CVPR 2021 paper Multi-Modal Fusion Transformer for End-to-End Autonomous Driving. If you find our

null 384 Jan 14, 2022
Unified unsupervised and semi-supervised domain adaptation network for cross-scenario face anti-spoofing, Pattern Recognition

USDAN The implementation of Unified unsupervised and semi-supervised domain adaptation network for cross-scenario face anti-spoofing, which is accepte

null 6 Jan 5, 2022
Outlier Exposure with Confidence Control for Out-of-Distribution Detection

OOD-detection-using-OECC This repository contains the essential code for the paper Outlier Exposure with Confidence Control for Out-of-Distribution De

Nazim Shaikh 54 Sep 11, 2021
Deep Anomaly Detection with Outlier Exposure (ICLR 2019)

Outlier Exposure This repository contains the essential code for the paper Deep Anomaly Detection with Outlier Exposure (ICLR 2019). Requires Python 3

Dan Hendrycks 406 Jan 16, 2022
Code for Piggyback: Adapting a Single Network to Multiple Tasks by Learning to Mask Weights

Piggyback: https://arxiv.org/abs/1801.06519 Pretrained masks and backbones are available here: https://uofi.box.com/s/c5kixsvtrghu9yj51yb1oe853ltdfz4q

Arun Mallya 153 Dec 24, 2021
Code for PackNet: Adding Multiple Tasks to a Single Network by Iterative Pruning

PackNet: https://arxiv.org/abs/1711.05769 Pretrained models are available here: https://uofi.box.com/s/zap2p03tnst9dfisad4u0sfupc0y1fxt Datasets in Py

Arun Mallya 194 Dec 20, 2021
Python package for multiple object tracking research with focus on laboratory animals tracking.

motutils is a Python package for multiple object tracking research with focus on laboratory animals tracking. Features loads: MOTChallenge CSV, sleap

Matěj Šmíd 1 Jan 6, 2022
TorchDistiller - a collection of the open source pytorch code for knowledge distillation, especially for the perception tasks, including semantic segmentation, depth estimation, object detection and instance segmentation.

This project is a collection of the open source pytorch code for knowledge distillation, especially for the perception tasks, including semantic segmentation, depth estimation, object detection and instance segmentation.

yifan liu 102 Jan 17, 2022
We present a framework for training multi-modal deep learning models on unlabelled video data by forcing the network to learn invariances to transformations applied to both the audio and video streams.

Multi-Modal Self-Supervision using GDT and StiCa This is an official pytorch implementation of papers: Multi-modal Self-Supervision from Generalized D

Facebook Research 30 Dec 12, 2021
This is the official implementation of TrivialAugment and a mini-library for the application of multiple image augmentation strategies including RandAugment and TrivialAugment.

Trivial Augment This is the official implementation of TrivialAugment (https://arxiv.org/abs/2103.10158), as was used for the paper. TrivialAugment is

AutoML-Freiburg-Hannover 59 Jan 17, 2022
A lightweight Python-based 3D network multi-agent simulator. Uses a cell-based congestion model. Calculates risk, loudness and battery capacities of the agents. Suitable for 3D network optimization tasks.

AMAZ3DSim AMAZ3DSim is a lightweight python-based 3D network multi-agent simulator. It uses a cell-based congestion model. It calculates risk, battery

Daniel Hirsch 9 Jan 19, 2022
Object detection on multiple datasets with an automatically learned unified label space.

Simple multi-dataset detection An object detector trained on multiple large-scale datasets with a unified label space; Winning solution of E

Xingyi Zhou 323 Jan 12, 2022
Code of paper Interact, Embed, and EnlargE (IEEE): Boosting Modality-specific Representations for Multi-Modal Person Re-identification.

Interact, Embed, and EnlargE (IEEE): Boosting Modality-specific Representations for Multi-Modal Person Re-identification We provide the codes for repr

null 4 Dec 22, 2021
Propose a principled and practically effective framework for unsupervised accuracy estimation and error detection tasks with theoretical analysis and state-of-the-art performance.

Detecting Errors and Estimating Accuracy on Unlabeled Data with Self-training Ensembles This project is for the paper: Detecting Errors and Estimating

Jiefeng Chen 4 Dec 11, 2021