Photographic Image Synthesis with Cascaded Refinement Networks - Pytorch Implementation

Related tags

Deep Learning deep-learning neural-network high-resolution vgg19 semantic-segmentation cityscapes photorealistic-based-rendering
Overview

Photographic Image Synthesis with Cascaded Refinement Networks-Pytorch (https://arxiv.org/abs/1707.09405)

This is a Pytorch implementation of cascaded refinement networks to synthesize photographic images from semantic layouts. Now the pretrained model and codes for training the network from scratch are available for 256x512 resolution. Thanks to Qifeng Chen for his tensorflow implementation which helped a lot in developing this pytorch version. Output

Testing

  1. Download this package and keep all the subsequent mentioned files in the same folder.
  2. Download the pretrained VGG19 Net from VGG19
  3. Download the pretrained weights for the CRN network for 256x512 CRN
  4. Keep the mode=test and mention the semantic image name to be tested in the Cascadaed_Network_LM_256.py
  5. The synthesized images will be saved in current folder.

Training

  1. Follow steps 1 to 3 from the testing steps.
  2. Resize all the training images to 256x512. Keep the semantic segmentated training images in Label256Full folder and
    the RGB training images in RGB256Full (without any subfolders).
  3. Set mode=train in Cascadaed_Network_LM_256.py and run it for desired epochs (default is 200).

Future Work

  1. Soon the pretrained weights for resolution 512x1024 and 1024x20148 will be available along with training scripts.

Note

  1. All the codes are written to run on GPU. Suitable changes should be done if you want to run on CPU. Also feel free to
    customize it according to your need.
You might also like...
Official implementation for (Refine Myself by Teaching Myself : Feature Refinement via Self-Knowledge Distillation, CVPR-2021)

FRSKD Official implementation for Refine Myself by Teaching Myself : Feature Refinement via Self-Knowledge Distillation (CVPR-2021) Requirements Pytho

null 75 Dec 28, 2022
Official Implementation for
Official Implementation for "ReStyle: A Residual-Based StyleGAN Encoder via Iterative Refinement" https://arxiv.org/abs/2104.02699

ReStyle: A Residual-Based StyleGAN Encoder via Iterative Refinement Recently, the power of unconditional image synthesis has significantly advanced th

null 967 Jan 4, 2023
Paddle implementation for "Cross-Lingual Word Embedding Refinement by ℓ1 Norm Optimisation" (NAACL 2021)

L1-Refinement Paddle implementation for "Cross-Lingual Word Embedding Refinement by ℓ1 Norm Optimisation" (NAACL 2021) 🙈 A more detailed readme is co

Lincedo Lab 4 Jun 9, 2021
Implementation of the paper
Implementation of the paper "Self-Promoted Prototype Refinement for Few-Shot Class-Incremental Learning"

Self-Promoted Prototype Refinement for Few-Shot Class-Incremental Learning This is the implementation of the paper "Self-Promoted Prototype Refinement

Kai Zhu 78 Dec 2, 2022
π-GAN: Periodic Implicit Generative Adversarial Networks for 3D-Aware Image Synthesis

π-GAN: Periodic Implicit Generative Adversarial Networks for 3D-Aware Image Synthesis Project Page | Paper | Data Eric Ryan Chan*, Marco Monteiro*, Pe

null 375 Dec 31, 2022
Multi-View Consistent Generative Adversarial Networks for 3D-aware Image Synthesis (CVPR2022)
Multi-View Consistent Generative Adversarial Networks for 3D-aware Image Synthesis (CVPR2022)

Multi-View Consistent Generative Adversarial Networks for 3D-aware Image Synthesis Multi-View Consistent Generative Adversarial Networks for 3D-aware

Xuanmeng Zhang 78 Dec 10, 2022
Pytorch implementation of few-shot semantic image synthesis
Pytorch implementation of few-shot semantic image synthesis

Few-shot Semantic Image Synthesis Using StyleGAN Prior Our method can synthesize photorealistic images from dense or sparse semantic annotations using

null 40 Sep 26, 2022
PyTorch implementation for SDEdit: Image Synthesis and Editing with Stochastic Differential Equations
PyTorch implementation for SDEdit: Image Synthesis and Editing with Stochastic Differential Equations

SDEdit: Image Synthesis and Editing with Stochastic Differential Equations Project | Paper | Colab PyTorch implementation of SDEdit: Image Synthesis a

null 536 Jan 5, 2023
A PyTorch implementation of the paper
A PyTorch implementation of the paper "Semantic Image Synthesis via Adversarial Learning" in ICCV 2017

Semantic Image Synthesis via Adversarial Learning This is a PyTorch implementation of the paper Semantic Image Synthesis via Adversarial Learning. Req

Seonghyeon Nam 146 Nov 25, 2022
Comments
  • Information on training process

    Information on training process

    Hi there,

    I'm just wondering if it is possible to get some information on what hardware you used to train the model, as well as how long you took? I am currently writing my own version of the CRN, however I am encountering extremely long training times and massive memory requirements, and so I thought is worthwhile to find out more about other implementations and their requirments.

    Kind regards

    opened by Swidilator 2
  • pretrained VGG19 and CRN weights are no longer able to download

    pretrained VGG19 and CRN weights are no longer able to download

    Hello, I am trying to reproduce your work with the readme instruction. I found I currently do not have access to you pretrained models in google drive, could you release it?

    opened by ConvMech 1
  • The way that you get D and D_m seems computation heavy.

    The way that you get D and D_m seems computation heavy. 

    def recursive_img(label,res): #Resulution may refers to the final image output i.e. 256x512 or 512x1024
     dim=512 if res>=128 else 1024
#    #M_low will start from 4x8 to resx2*res
     if res == 4:
         downsampled = label #torch.unsqueeze(torch.from_numpy(label).float().permute(2,0,1), dim=0)
     else:
         max1=nn.AvgPool2d(kernel_size=2, padding=0, stride=2)
         downsampled=max1(label)
         img = recursive_img(downsampled, res//2)
         
     global D
     global count
     global D_m

     D.insert(count, downsampled)
     D_m.insert(count, dim)
     count+=1
     return downsampled

    Why not directly assign each D_i and D_m_i with specific values.

    opened by Naruto-Sasuke 1
  • Several questions

    Several questions

    1. Can we use recursive_generator similar with the official code?
    2. LayerNorm is in here http://pytorch.org/docs/master/nn.html#layernorm.(Seems in this week v0.4 will be released)
    opened by Naruto-Sasuke 1
Owner
Soumya Tripathy
Doctoral student
Soumya Tripathy
GitHub
Face Detection and Alignment using Multi-task Cascaded Convolutional Networks (MTCNN)

Face-Detection-with-MTCNN Face detection is a computer vision problem that involves finding faces in photos. It is a trivial problem for humans to sol

Chetan Hirapara 3 Oct 7, 2022
Unoffical implementation about Image Super-Resolution via Iterative Refinement by Pytorch

Image Super-Resolution via Iterative Refinement Paper | Project Brief This is a unoffical implementation about Image Super-Resolution via Iterative Re

LiangWei Jiang 2.5k Jan 2, 2023
QueryDet: Cascaded Sparse Query for Accelerating High-Resolution SmallObject Detection

QueryDet-PyTorch This repository is the official implementation of our paper: QueryDet: Cascaded Sparse Query for Accelerating High-Resolution Small O

Chenhongyi Yang 276 Dec 31, 2022
Cascaded Pyramid Network (CPN) based on Keras (Tensorflow backend)

ML2 Takehome Project Reimplementing the paper: Cascaded Pyramid Network for Multi-Person Pose Estimation Dataset The model uses the COCO dataset which

Vo Van Tu 1 Nov 22, 2021
Cascaded Deep Video Deblurring Using Temporal Sharpness Prior and Non-local Spatial-Temporal Similarity

This repository is the official PyTorch implementation of Cascaded Deep Video Deblurring Using Temporal Sharpness Prior and Non-local Spatial-Temporal Similarity

hippopmonkey 4 Dec 11, 2022
Pytorch implementation for reproducing StackGAN_v2 results in the paper StackGAN++: Realistic Image Synthesis with Stacked Generative Adversarial Networks

StackGAN-v2 StackGAN-v1: Tensorflow implementation StackGAN-v1: Pytorch implementation Inception score evaluation Pytorch implementation for reproduci

Han Zhang 809 Dec 16, 2022
PyTorch 1.5 implementation for paper DECOR-GAN: 3D Shape Detailization by Conditional Refinement.

DECOR-GAN PyTorch 1.5 implementation for paper DECOR-GAN: 3D Shape Detailization by Conditional Refinement, Zhiqin Chen, Vladimir G. Kim, Matthew Fish

Zhiqin Chen 72 Dec 31, 2022
Pytorch implementation of “Recursive Non-Autoregressive Graph-to-Graph Transformer for Dependency Parsing with Iterative Refinement”

Graph-to-Graph Transformers Self-attention models, such as Transformer, have been hugely successful in a wide range of natural language processing (NL

Idiap Research Institute 40 Aug 14, 2022
Implementation of Invariant Point Attention, used for coordinate refinement in the structure module of Alphafold2, as a standalone Pytorch module

Invariant Point Attention - Pytorch Implementation of Invariant Point Attention as a standalone module, which was used in the structure module of Alph

Phil Wang 113 Jan 5, 2023
PyTorch implementation of convolutional neural networks-based text-to-speech synthesis models

Deepvoice3_pytorch PyTorch implementation of convolutional networks-based text-to-speech synthesis models: arXiv:1710.07654: Deep Voice 3: Scaling Tex

Ryuichi Yamamoto 1.8k Jan 8, 2023