[ACM MM 2021] Joint Implicit Image Function for Guided Depth Super-Resolution

Related tags

Deep Learning jiif
Overview

Joint Implicit Image Function for Guided Depth Super-Resolution

This repository contains the code for:

Joint Implicit Image Function for Guided Depth Super-Resolution
Jiaxiang Tang, Xiaokang Chen, Gang Zeng
ACM MM 2021

model

Installation

Environments:

  • Python >= 3.6
  • PyTorch >= 1.6.0
  • tensorboardX
  • tqdm, opencv-python, Pillow
  • NVIDIA apex (python-only build is ok.)

Data preparation

Please see data/prepare_data.md for the details.

Training

You can use the provided scripts (scripts/train*) to train models.

For example:

# train JIIF with scale = 8 on the NYU dataset.
OMP_NUM_THREADS=8 CUDA_VISIBLE_DEVICES=2 python main.py \
    --name jiif_8 --model JIIF --scale 8 \
    --sample_q 30720 --input_size 256 --train_batch 1 \
    --epoch 200 --eval_interval 10 \
    --lr 0.0001 --lr_step 60 --lr_gamma 0.2

Testing

To test the performance of the models on difference datasets, you can use the provided scripts (scripts/test*).

For example:

# test the best checkpoint on MiddleBury dataest with scale = 8
OMP_NUM_THREADS=8 CUDA_VISIBLE_DEVICES=1 python main.py \
    --test --checkpoint best \
    --name jiif_8 --model JIIF \
    --dataset Middlebury --scale 8 --data_root ./data/depth_enhance/01_Middlebury_Dataset

Pretrained models and Reproducing

We provide the pretrained models here.

To test the performance of the pretrained models, please download the corresponding models and put them under pretrained folder. Then you can use scripts/test_jiif_pretrained.sh and scripts/test_denoise_jiif_pretrained.sh to reproduce the results reported in our paper.

Citation

If you find the code useful for your research, please use the following BibTeX entry:

@article{tang2021joint,
    title        = {Joint Implicit Image Function for Guided Depth Super-Resolution},
    author       = {Jiaxiang Tang, Xiaokang Chen, Gang Zeng},
    year         = 2021,
    journal      = {arXiv preprint arXiv:2107.08717}
}

Acknowledgment

The model implementation is based on liif.

You might also like...
Official PyTorch code for Hierarchical Conditional Flow: A Unified Framework for Image Super-Resolution and Image Rescaling (HCFlow, ICCV2021)
Official PyTorch code for Hierarchical Conditional Flow: A Unified Framework for Image Super-Resolution and Image Rescaling (HCFlow, ICCV2021)

Hierarchical Conditional Flow: A Unified Framework for Image Super-Resolution and Image Rescaling (HCFlow, ICCV2021) This repository is the official P

Jingyun Liang 159 Dec 30, 2022
Repository for
Repository for "Exploring Sparsity in Image Super-Resolution for Efficient Inference", CVPR 2021

SMSR Reposity for "Exploring Sparsity in Image Super-Resolution for Efficient Inference" [arXiv] Highlights Locate and skip redundant computation in S

Longguang Wang 225 Dec 26, 2022
PyTorch implementation of Graph Convolutional Networks in Feature Space for Image Deblurring and Super-resolution, IJCNN 2021.

GCResNet PyTorch implementation of Graph Convolutional Networks in Feature Space for Image Deblurring and Super-resolution, IJCNN 2021. The code will

null 11 May 19, 2022
Code for 'Self-Guided and Cross-Guided Learning for Few-shot segmentation. (CVPR' 2021)'

SCL Introduction Code for 'Self-Guided and Cross-Guided Learning for Few-shot segmentation. (CVPR' 2021)' We evaluated our approach using two baseline

null 34 Oct 8, 2022
Official PyTorch implementation of the paper
Official PyTorch implementation of the paper "Recycling Discriminator: Towards Opinion-Unaware Image Quality Assessment Using Wasserstein GAN", accepted to ACM MM 2021 BNI Track.

RecycleD Official PyTorch implementation of the paper "Recycling Discriminator: Towards Opinion-Unaware Image Quality Assessment Using Wasserstein GAN

Yunan Zhu 23 Nov 5, 2022
[ACM MM 2021] Diverse Image Inpainting with Bidirectional and Autoregressive Transformers

Diverse Image Inpainting with Bidirectional and Autoregressive Transformers Installation pip install -r requirements.txt Dataset Preparation Given the

Yingchen Yu 25 Nov 9, 2022
Monocular Depth Estimation - Weighted-average prediction from multiple pre-trained depth estimation models
Monocular Depth Estimation - Weighted-average prediction from multiple pre-trained depth estimation models

merged_depth runs (1) AdaBins, (2) DiverseDepth, (3) MiDaS, (4) SGDepth, and (5) Monodepth2, and calculates a weighted-average per-pixel absolute dept

Pranav 39 Nov 21, 2022
The implemention of Video Depth Estimation by Fusing Flow-to-Depth Proposals

Flow-to-depth (FDNet) video-depth-estimation This is the implementation of paper Video Depth Estimation by Fusing Flow-to-Depth Proposals Jiaxin Xie,

null 32 Jun 14, 2022
Monocular Depth Estimation Using Laplacian Pyramid-Based Depth Residuals
Monocular Depth Estimation Using Laplacian Pyramid-Based Depth Residuals

LapDepth-release This repository is a Pytorch implementation of the paper "Monocular Depth Estimation Using Laplacian Pyramid-Based Depth Residuals" M

Minsoo Song 205 Dec 30, 2022
Comments
  • the input of mlp

    the input of mlp

    Great work!

    But in model/jiif.py: 79~81

    q_guide = torch.cat([q_guide_hr, q_guide_hr - q_guide_lr], dim=-1)

inp = torch.cat([q_feat, q_guide, rel_coord], dim=-1)

    It seems inconsistent with the equation (8) of your paper. In eq.(8), the second input feature is g_i, which is a feature sampled from the low-resolution feature map. But in the code, q_guide_hr is used, which is sampled from the high-resolution features, corresponding to g_q but not g_i.

    opened by Philipflyg 2
Owner
hawkey
nameless kiui.
hawkey
GitHub
A framework for joint super-resolution and image synthesis, without requiring real training data

SynthSR This repository contains code to train a Convolutional Neural Network (CNN) for Super-resolution (SR), or joint SR and data synthesis. The met

null 83 Jan 1, 2023
RGB-D Local Implicit Function for Depth Completion of Transparent Objects

RGB-D Local Implicit Function for Depth Completion of Transparent Objects [Project Page] [Paper] Overview This repository maintains the official imple

NVIDIA Research Projects 43 Dec 12, 2022
This repository contains the code for the paper "PIFu: Pixel-Aligned Implicit Function for High-Resolution Clothed Human Digitization"

PIFu: Pixel-Aligned Implicit Function for High-Resolution Clothed Human Digitization News: [2020/05/04] Added EGL rendering option for training data g

Shunsuke Saito 1.5k Jan 3, 2023
Beyond Image to Depth: Improving Depth Prediction using Echoes (CVPR 2021)

Beyond Image to Depth: Improving Depth Prediction using Echoes (CVPR 2021) Kranti Kumar Parida, Siddharth Srivastava, Gaurav Sharma. We address the pr

Kranti Kumar Parida 33 Jun 27, 2022
Learning Continuous Image Representation with Local Implicit Image Function

LIIF This repository contains the official implementation for LIIF introduced in the following paper: Learning Continuous Image Representation with Lo

Yinbo Chen 1k Dec 25, 2022
Boosting Monocular Depth Estimation Models to High-Resolution via Content-Adaptive Multi-Resolution Merging

Boosting Monocular Depth Estimation Models to High-Resolution via Content-Adaptive Multi-Resolution Merging This repository contains an implementation

Computational Photography Lab @ SFU 1.1k Jan 2, 2023
ICRA 2021 "Towards Precise and Efficient Image Guided Depth Completion"

PENet: Precise and Efficient Depth Completion This repo is the PyTorch implementation of our paper to appear in ICRA2021 on "Towards Precise and Effic

null 232 Dec 25, 2022
Aerial Single-View Depth Completion with Image-Guided Uncertainty Estimation (RA-L/ICRA 2020)

Aerial Depth Completion This work is described in the letter "Aerial Single-View Depth Completion with Image-Guided Uncertainty Estimation", by Lucas

ETHZ V4RL 70 Dec 22, 2022
An unofficial personal implementation of UM-Adapt, specifically to tackle joint estimation of panoptic segmentation and depth prediction for autonomous driving datasets.

Semisupervised Multitask Learning This repository is an unofficial and slightly modified implementation of UM-Adapt[1] using PyTorch. This code primar

Abhinav Atrishi 11 Nov 25, 2022
Official PyTorch code for Hierarchical Conditional Flow: A Unified Framework for Image Super-Resolution and Image Rescaling (HCFlow, ICCV2021)

Hierarchical Conditional Flow: A Unified Framework for Image Super-Resolution and Image Rescaling (HCFlow, ICCV2021) This repository is the official P

Jingyun Liang 159 Dec 30, 2022