Bi-level feature alignment for versatile image translation and manipulation (Under submission of TPAMI)

Related tags

Deep Learning RABIT
Overview

Bi-level feature alignment for versatile image translation and manipulation (Under submission of TPAMI)

Teaser

Preparation

Clone the Synchronized-BatchNorm-PyTorch repository.

cd models/networks/
git clone https://github.com/vacancy/Synchronized-BatchNorm-PyTorch
cp -rf Synchronized-BatchNorm-PyTorch/sync_batchnorm .
cd ../../

VGG model for computing loss. Download from here, move it to models/.

For the preparation of datasets, please refer to CoCosNet.

Training

Then run the command

bash train_ade.sh

Citation

If you use this code for your research, please cite our papers.

@article{zhan2021rabit,
  title={Bi-level feature alignment for versatile image translation and manipulation},
  author={Zhan, Fangneng and Yu, Yingchen and Wu, Rongliang and Cui, Kaiwen and Xiao, Aoran and Lu, Shijian and Shao, Ling},
  journal={arXiv preprint arXiv:2107.03021},
  year={2021}
}

Acknowledgments

This code borrows heavily from CoCosNet. We also thank SPADE, Synchronized Normalization.

You might also like...
Implementation of Transformer in Transformer, pixel level attention paired with patch level attention for image classification, in Pytorch
Implementation of Transformer in Transformer, pixel level attention paired with patch level attention for image classification, in Pytorch

Transformer in Transformer Implementation of Transformer in Transformer, pixel level attention paired with patch level attention for image c

Learning Pixel-level Semantic Affinity with Image-level Supervision for Weakly Supervised Semantic Segmentation, CVPR 2018
Learning Pixel-level Semantic Affinity with Image-level Supervision for Weakly Supervised Semantic Segmentation, CVPR 2018

Learning Pixel-level Semantic Affinity with Image-level Supervision This code is deprecated. Please see https://github.com/jiwoon-ahn/irn instead. Int

Unbalanced Feature Transport for Exemplar-based Image Translation (CVPR 2021)

UNITE and UNITE+ Unbalanced Feature Transport for Exemplar-based Image Translation (CVPR 2021) Unbalanced Intrinsic Feature Transport for Exemplar-bas

An official implementation of
An official implementation of "SFNet: Learning Object-aware Semantic Correspondence" (CVPR 2019, TPAMI 2020) in PyTorch.

PyTorch implementation of SFNet This is the implementation of the paper "SFNet: Learning Object-aware Semantic Correspondence". For more information,

(ImageNet pretrained models) The official pytorch implemention of the TPAMI paper
(ImageNet pretrained models) The official pytorch implemention of the TPAMI paper "Res2Net: A New Multi-scale Backbone Architecture"

Res2Net The official pytorch implemention of the paper "Res2Net: A New Multi-scale Backbone Architecture" Our paper is accepted by IEEE Transactions o

DVG-Face: Dual Variational Generation for Heterogeneous Face Recognition, TPAMI 2021

DVG-Face: Dual Variational Generation for HFR This repo is a PyTorch implementation of DVG-Face: Dual Variational Generation for Heterogeneous Face Re

 Deep Learning for 3D Point Clouds: A Survey (IEEE TPAMI, 2020)
Deep Learning for 3D Point Clouds: A Survey (IEEE TPAMI, 2020)

🔥Deep Learning for 3D Point Clouds (IEEE TPAMI, 2020)

[TPAMI 2021] iOD: Incremental Object Detection via Meta-Learning
[TPAMI 2021] iOD: Incremental Object Detection via Meta-Learning

Incremental Object Detection via Meta-Learning To appear in an upcoming issue of the IEEE Transactions on Pattern Analysis and Machine Intelligence (T

🔥RandLA-Net in Tensorflow (CVPR 2020, Oral & IEEE TPAMI 2021)
🔥RandLA-Net in Tensorflow (CVPR 2020, Oral & IEEE TPAMI 2021)

RandLA-Net: Efficient Semantic Segmentation of Large-Scale Point Clouds (CVPR 2020) This is the official implementation of RandLA-Net (CVPR2020, Oral

Comments
  • about bi-level warp function

    about bi-level warp function "align_feature" and "align_feature_v"

    dear author, I cannot understand the bi-level warp function "align_feature" and "align_feature_v". The align_feature_v seems like the inverse function of the align_feature. Can you explain more about them, especially about details of bi-level warp and its inverse function. Thank you very much!

    opened by JAYatBUAA 1
  • about non-differentiable top-k

    about non-differentiable top-k

    Nice Works!

    Being very interested in your work,But I found that your top-k ranking removes the differentiable top-k method and uses softmax instead, what is the reason for this ?

    opened by xiewende 3
  • about some bugs in you code

    about some bugs in you code

    Hi zhan, I appreciate your work so much! However , I met some bugs when I tried to use your code.

    1、models/networks/correspondence.py (line 385)

           elif self.opt.warp_mask_losstype == 'cycle':
                # f_div_C_v = F.softmax(f_WTA.transpose(1, 2), dim=-1)
                f_WTA_v = f.transpose(1, 2)
                f_div_C_v = f_WTA_v / f_WTA_v.sum(-1).view(-1, N, 1)
    
                seg = F.interpolate(seg_map, scale_factor=1 / self.down, mode='nearest')
                channel = seg.shape[1]
                seg = seg.view(batch_size, channel, -1)
                seg = seg.permute(0, 2, 1)
                warp_mask_to_ref = torch.matmul(f_div_C_v, seg)  # 2*1936*channel
                warp_mask = torch.matmul(f_div_C, warp_mask_to_ref)  # 2*1936*channel
                warp_mask = warp_mask.permute(0, 2, 1).contiguous()
                coor_out['warp_mask'] = warp_mask.view(batch_size, channel, feat_height, feat_width)  # 2*3*44*44
    

    'f' and 'f_div_C' is not defined

    2、models/networks/generator.py (line 47)

         self.conv_img1 = nn.Conv2d(1, seg, 3, padding=1)
         self.up = nn.Upsample(scale_factor=2)
    

    'seg' is not defined

    would you please fix these bugs?

    opened by rorschach-xiao 2
Owner
Fangneng Zhan
Computer Vision, Deep Learning.
Fangneng Zhan
🏆 The 1st Place Submission to AICity Challenge 2021 Natural Language-Based Vehicle Retrieval Track (Alibaba-UTS submission)

AI City 2021: Connecting Language and Vision for Natural Language-Based Vehicle Retrieval ?? The 1st Place Submission to AICity Challenge 2021 Natural

null 82 Dec 29, 2022
(under submission) Bayesian Integration of a Generative Prior for Image Restoration

BIGPrior: Towards Decoupling Learned Prior Hallucination and Data Fidelity in Image Restoration Authors: Majed El Helou, and Sabine Süsstrunk {Note: p

Majed El Helou 22 Dec 17, 2022
PFENet: Prior Guided Feature Enrichment Network for Few-shot Segmentation (TPAMI).

PFENet This is the implementation of our paper PFENet: Prior Guided Feature Enrichment Network for Few-shot Segmentation that has been accepted to IEE

DV Lab 230 Dec 31, 2022
ManiSkill-Learn is a framework for training agents on SAPIEN Open-Source Manipulation Skill Challenge (ManiSkill Challenge), a large-scale learning-from-demonstrations benchmark for object manipulation.

ManiSkill-Learn ManiSkill-Learn is a framework for training agents on SAPIEN Open-Source Manipulation Skill Challenge, a large-scale learning-from-dem

Hao Su's Lab, UCSD 48 Dec 30, 2022
A Transformer-Based Feature Segmentation and Region Alignment Method For UAV-View Geo-Localization

University1652-Baseline [Paper] [Slide] [Explore Drone-view Data] [Explore Satellite-view Data] [Explore Street-view Data] [Video Sample] [中文介绍] This

Zhedong Zheng 335 Jan 6, 2023
Pytorch implementation for "Implicit Feature Alignment: Learn to Convert Text Recognizer to Text Spotter".

Implicit Feature Alignment: Learn to Convert Text Recognizer to Text Spotter This is a pytorch-based implementation for paper Implicit Feature Alignme

wangtianwei 61 Nov 12, 2022
An official implementation of the paper Exploring Sequence Feature Alignment for Domain Adaptive Detection Transformers

Sequence Feature Alignment (SFA) By Wen Wang, Yang Cao, Jing Zhang, Fengxiang He, Zheng-jun Zha, Yonggang Wen, and Dacheng Tao This repository is an o

WangWen 79 Dec 24, 2022
Code for Quantifying Ignorance in Individual-Level Causal-Effect Estimates under Hidden Confounding

?? quince Code for Quantifying Ignorance in Individual-Level Causal-Effect Estimates under Hidden Confounding ?? Installation $ git clone [email protected]

Andrew Jesson 19 Jun 23, 2022
Official implementation of Generalized Data Weighting via Class-level Gradient Manipulation (NeurIPS 2021).

Generalized Data Weighting via Class-level Gradient Manipulation This repository is the official implementation of Generalized Data Weighting via Clas

null 9 Nov 3, 2021