Official Implementation of Domain-Aware Universal Style Transfer

Related tags

Deep Learning style-transfer
Overview

Domain Aware Universal Style Transfer

Official Pytorch Implementation of 'Domain Aware Universal Style Transfer' (ICCV 2021)

teaser

Domain Aware Universal Style Transfer

Kibeom Hong (Yonsei Univ.), Seogkyu Jeon (Yonsei Univ.), Jianlong Fu (Microsoft Research), Huan Yang (Microsoft Research), Hyeran Byun (Yonsei Univ.)

Paper : https://arxiv.org/abs/2108.04441

Abstract: Style transfer aims to reproduce content images with the styles from reference images. Existing universal style transfer methods successfully deliver arbitrary styles to original images either in an artistic or a photo-realistic way. However, the range of “arbitrary style” defined by existing works is bounded in the particular domain due to their structural limitation. Specifically, the degrees of content preservation and stylization are established according to a predefined target domain. As a result, both photo-realistic and artistic models have difficulty in performing the desired style transfer for the other domain. To overcome this limitation, we propose a unified architecture, Domain-aware Style Transfer Networks (DSTN) that transfer not only the style but also the property of domain (i.e., domainness) from a given reference image. To this end, we design a novel domainness indicator that captures the domainness value from the texture and structural features of reference images. Moreover, we introduce a unified framework with domain-aware skip connection to adaptively transfer the stroke and palette to the input contents guided by the domainness indicator. Our extensive experiments validate that our model produces better qualitative results and outperforms previous methods in terms of proxy metrics on both artistic and photo-realistic stylizations.

Prerequisites

Dependency

  • Python 3.6
  • CUDA 11.0
  • Pytorch 1.7
  • Check the requirements.txt
pip install -r requirements.txt

Usage

Set pretrained weights

  • Pretrained models for encoder(VGG-19) can be found in the ./baseline_checkpoints

  • Prepare pretrained models for Domainnes Indicator

  • Prepare pretrained models for Decoder

  • Move these pretrained weights to each folders:

    • style_indicator.pth -> ./train_results/StyleIndicator/log/
    • decoder.pth -> ./train_results/Decoder/log/
    • decoder_adversarial.pth -> ./train_results/Decoder_adversarial/log/

    (Please rename decoder_adversarial.pth -> decoder.pth)

Inference (Automatic)

  • Vanilla decoder
bash scripts/transfer.sh
  • Decoder with adversarial loss
bash scripts/transfer_adversarial.sh

Training

Available soon

Evaluation

Available soon

Ciation

If you find this work useful for your research, please cite:

@article{Hong2021DomainAwareUS,
  title={Domain-Aware Universal Style Transfer},
  author={Kibeom Hong and Seogkyu Jeon and Huan Yang and Jianlong Fu and H. Byun},
  journal={ArXiv},
  year={2021},
  volume={abs/2108.04441}
}

Contact

If you have any question or comment, please contact the first author of this paper - Kibeom Hong

[email protected]

Comments
  • Inference command

    Inference command

    Hello,

    Your paper is amazing, nice works ! Could you release on Readme the Test inference command ?

    I read your main.py but I don't know how to make an artistic style transfer ? Could you provide a command example for a simple style transfert ?

    Thanks a lot.

    opened by crenaudineau1 2
  • Training process.

    Training process.

    Hi.

    I have a question.

    Did you use the total loss or only reconstruction loss when you train your decoder for reconstruction?

    Reconstruction and stylization are both trained?

    thanks.

    opened by sunwoo76 1
  • pretrained weights are unreached

    pretrained weights are unreached

    Seems the links for the 3 pretrained weights (style_indicator.pth, pretrained weights, pretrained weights )on google drive are broken. can you please share correct links? Thank you!

    opened by dnevo 0
  • Error in the code

    Error in the code

    There is an error in the main.py file

    Traceback (most recent call last): File "C:\NEURAL\Domain-Aware-Style-Transfer\main.py", line 50, in from baseline import Baseline as Baseline File "C:\NEURAL\Domain-Aware-Style-Transfer\baseline.py", line 520 dir_path = os.path.join(self.result_img_dir, 'interpolate', , self.DA_comment+'_'+self.ST_comment) -------------------------------------------------------------^ SyntaxError: invalid syntaxST_comment)

    opened by surfingnirvana 0
  • AttributeError: 'Baseline' object has no attribute 'DA_Net_trained_epoch'

    AttributeError: 'Baseline' object has no attribute 'DA_Net_trained_epoch'

    When I running the test code : bash scripts/interpolate.sh The error occured： AttributeError: 'Baseline' object has no attribute 'DA_Net_trained_epoch' How to fix it....

    opened by AeYeung 1
  • Add Domain Aware Style Transfer API project

    Add Domain Aware Style Transfer API project

    Principal features:

    1. Add a Domain Aware Style Transfer API scaffolding project using the FastAPI library.
    2. Add API endpoints to support the developed models (vanilla, adversarial loss) and the interpolation case.
    3. Create configurations files (.json) for each model inference and baseline process.
    4. Refactor and re-structure the code. Remove unnecessary dependencies, clean the code and modify the code to avoid GPU out of memory during inference process.
    5. Add a Docker configuration (Dockerfile) and environment.yml to execute locally (cmd or Docker) or execute the project in the cloud using Container Instances with GPU.
    6. Add README.md as a user guide to install dependencies locally or deploy the API in an Azure Container Instance using Azure Portal.
    opened by alexandergg 0
  • About GPU use

    About GPU use

    I trained StyleIndicator and Decoder respectively. However, I found the Volatile GPU-Util is 0%. It seems that the whole network computed in CPU. Have you ever had this situation？

    opened by KevinMarkVine 3
Owner
KibeomHong
* Ph.D. student in Yonsei Univ. (2018.03.~present)
KibeomHong
GitHub
Official pytorch implementation of "Feature Stylization and Domain-aware Contrastive Loss for Domain Generalization" ACMMM 2021 (Oral)

Feature Stylization and Domain-aware Contrastive Loss for Domain Generalization This is an official implementation of "Feature Stylization and Domain-

null 22 Sep 22, 2022
Simple Tensorflow implementation of "Adaptive Convolutions for Structure-Aware Style Transfer" (CVPR 2021)

AdaConv — Simple TensorFlow Implementation [Paper] : Adaptive Convolutions for Structure-Aware Style Transfer (CVPR 2021) Note This repository does no

Junho Kim 26 Nov 18, 2022
[CVPR2021] Domain Consensus Clustering for Universal Domain Adaptation

[CVPR2021] Domain Consensus Clustering for Universal Domain Adaptation [Paper] Prerequisites To install requirements: pip install -r requirements.txt

Guangrui Li 84 Dec 26, 2022
Reference code for the paper CAMS: Color-Aware Multi-Style Transfer.

CAMS: Color-Aware Multi-Style Transfer Mahmoud Afifi1, Abdullah Abuolaim*1, Mostafa Hussien*2, Marcus A. Brubaker1, Michael S. Brown1 1York University

Mahmoud Afifi 36 Dec 4, 2022
code for our paper "Source Data-absent Unsupervised Domain Adaptation through Hypothesis Transfer and Labeling Transfer"

SHOT++ Code for our TPAMI submission "Source Data-absent Unsupervised Domain Adaptation through Hypothesis Transfer and Labeling Transfer" that is ext

null 75 Dec 16, 2022
Official PyTorch implementation of "ArtFlow: Unbiased Image Style Transfer via Reversible Neural Flows"

ArtFlow Official PyTorch implementation of the paper: ArtFlow: Unbiased Image Style Transfer via Reversible Neural Flows Jie An*, Siyu Huang*, Yibing

null 123 Dec 27, 2022
Official PyTorch implementation of Retrieve in Style: Unsupervised Facial Feature Transfer and Retrieval.

Retrieve in Style: Unsupervised Facial Feature Transfer and Retrieval PyTorch This is the PyTorch implementation of Retrieve in Style: Unsupervised Fa

null 60 Oct 12, 2022
Code for CVPR2021 "Visualizing Adapted Knowledge in Domain Transfer". Visualization for domain adaptation. #explainable-ai

Visualizing Adapted Knowledge in Domain Transfer @inproceedings{hou2021visualizing, title={Visualizing Adapted Knowledge in Domain Transfer}, auth

Yunzhong Hou 80 Dec 25, 2022
The official code of Anisotropic Stroke Control for Multiple Artists Style Transfer

ASMA-GAN Anisotropic Stroke Control for Multiple Artists Style Transfer Proceedings of the 28th ACM International Conference on Multimedia The officia

Six_God 146 Nov 21, 2022
UFT - Universal File Transfer With Python

UFT 2.0.0 UFT (Universal File Transfer) is a CLI tool , which can be used to upl

Merwin 1 Feb 18, 2022
Official Implementation of HRDA: Context-Aware High-Resolution Domain-Adaptive Semantic Segmentation

HRDA: Context-Aware High-Resolution Domain-Adaptive Semantic Segmentation by Lukas Hoyer, Dengxin Dai, and Luc Van Gool [Arxiv] [Paper] Overview Unsup

Lukas Hoyer 149 Dec 28, 2022
Official pytorch code for SSAT: A Symmetric Semantic-Aware Transformer Network for Makeup Transfer and Removal

SSAT: A Symmetric Semantic-Aware Transformer Network for Makeup Transfer and Removal This is the official pytorch code for SSAT: A Symmetric Semantic-

ForeverPupil 57 Dec 13, 2022
PyTorch implementation of neural style transfer algorithm

neural-style-pt This is a PyTorch implementation of the paper A Neural Algorithm of Artistic Style by Leon A. Gatys, Alexander S. Ecker, and Matthias

null 770 Jan 2, 2023
An implementation of "Optimal Textures: Fast and Robust Texture Synthesis and Style Transfer through Optimal Transport"

Optex An implementation of Optimal Textures: Fast and Robust Texture Synthesis and Style Transfer through Optimal Transport for TU Delft CS4240. You c

Hans Brouwer 33 Jan 5, 2023
Pytorch implementation of Nueral Style transfer

Nueral Style Transfer Pytorch implementation of Nueral style transfer algorithm , it is used to apply artistic styles to content images . Content is t

Abhinav 9 Oct 15, 2022
Implementation of CVPR 2021 paper "Spatially-invariant Style-codes Controlled Makeup Transfer"

SCGAN Implementation of CVPR 2021 paper "Spatially-invariant Style-codes Controlled Makeup Transfer" Prepare The pre-trained model is avaiable at http

null 118 Dec 12, 2022
PyTorch implementation of MuseMorphose, a Transformer-based model for music style transfer.

MuseMorphose This repository contains the official implementation of the following paper: Shih-Lun Wu, Yi-Hsuan Yang MuseMorphose: Full-Song and Fine-

Yating Music, Taiwan AI Labs 142 Jan 8, 2023
PyTorch implementation of paper: AdaAttN: Revisit Attention Mechanism in Arbitrary Neural Style Transfer, ICCV 2021.

AdaAttN: Revisit Attention Mechanism in Arbitrary Neural Style Transfer [Paper] [PyTorch Implementation] [Paddle Implementation] Overview This reposit

null 148 Dec 30, 2022
Implementation of Neural Style Transfer in Pytorch

PytorchNeuralStyleTransfer Code to run Neural Style Transfer from our paper Image Style Transfer Using Convolutional Neural Networks. Also includes co

Leon Gatys 396 Dec 1, 2022