Official Pytorch implementation of "CLIPstyler:Image Style Transfer with a Single Text Condition"

Related tags

Deep Learning style-transfer clip
Overview

CLIPstyler

Official Pytorch implementation of "CLIPstyler:Image Style Transfer with a Single Text Condition"

MAIN3_e2-min

Environment

Pytorch 1.7.1, Python 3.6

$ conda create -n CLIPstyler python=3.6
$ conda install --yes -c pytorch pytorch=1.7.1 torchvision cudatoolkit=11.0
$ pip install ftfy regex tqdm
$ conda install -c anaconda git
$ pip install git+https://github.com/openai/CLIP.git

Style Transfer with Single-image

To train the model and obtain the image, run

python train_CLIPstyler.py --content_path ./test_set/face.jpg \
--content_name face --exp_name exp1 \
--text "Sketch with black pencil"

To change the style of custom image, please change the --content_path argument

edit the text condition with --text argument

For easy demo, we provide Google Colab Open In Colab.

*Warning : Due to slow computation speed of colab, it may take several minutes in colab environment

Fast Style Transfer

Before training, plase download DIV2K dataset LINK.

We recomment to use Training data of High-Resolution(HR) images.

To train the model, please download the pre-trained vgg encoder & decoder models in LINK.

Please save the downloaded models in ./models directory

Then, run the command

python train_fast.py --content_path $DIV2K_DIR$ \
--name exp1 \
--text "Sketch with black pencil" --test_dir ./test_set

Please set the $DIV2K_DIR$ as the directory in which DIV2K images are saved.

To test the fast style transfer model,

python test_fast.py --test_dir ./test_set --decoder ./model_fast/clip_decoder_iter_200.pth.tar

Change the argument --decoder to other trained models for testing on different text conditions.

We provide several fine-tuned decoders for several text conditions. LINK

To use high-resolution image, please add --hr_dir ./hr_set to test command.

We provide colab notebook for testing fast transfer model Open In Colab

Comments
  • NameError: name 'net' is not defined

    NameError: name 'net' is not defined

    Hi, I got the following error while testing the fast style transfer model,

    Traceback (most recent call last):
  File "test_fast.py", line 76, in <module>
    network = net.Net(vgg, decoder)
NameError: name 'net' is not define

    Seems that net.Net should be fast_stylenet.Net in https://github.com/cyclomon/CLIPstyler/blob/c8ba7a088b566d40dea3771bc298c08189123eca/test_fast.py#L76

    There seems to be another typo that --content_path should be changed to --content_dir in README for train_fast.py

    https://github.com/cyclomon/CLIPstyler/blob/main/README.md#fast-style-transfer

    opened by xunings 1
  • Comparison with StyleGAN-NADA

    Comparison with StyleGAN-NADA

    Hello authors,

    What hyperparameters/settings did you used for producing the StyleGAN-NADA? Im having very different results.

    Thank you for your atttention.

    opened by josejhlee 1
  • Issue regarding non 1:1 images and higer resolutions

    Issue regarding non 1:1 images and higer resolutions

    First I must say, this tool is pure magic! Thanks for developing it.

    I see that in the research paper, there is a demonstration image which gets style-transferred to its original format, which was a non-square, it was a image of a field and skies. I also never got any image output over the resolution of 512, which my 24GB card would surely handle.

    I tried settings the --crop_size, --num_crop and --image_size options, but it never did anything. I'm no programmer (and still have a github lol, I want to learn rust eventually) and I looked through the various .py files, but couldn't find anything obvious to change it.

    Is there not any way to change this in the released model, or did I just miss something?

    opened by SomeOrdinaryDude 1
  • Add Docker environment & web demo

    Add Docker environment & web demo

    Hey @paper11667 ! šŸ‘‹

    This pull request makes it possible to run your model inside a Docker environment, which makes it easier for other people to run it. We're using an open source tool called Cog to make this process easier.

    This also means we can make a web page where other people can try out your model! View it here: https://replicate.com/paper11667/clipstyler

    Do claim your page here so you can own the page, customise the Example gallery as you like, push any future update to the web demo, and we'll feature it on our website and tweet about it too.

    In case you're wondering who I am, I'm fromĀ Replicate, where we're trying to make machine learning reproducible. We got frustrated that we couldn't run all the really interesting ML work being done. So, we're going round implementing models we like.Ā šŸ˜Š

    opened by chenxwh 0
  • Upgrade to Cog version 0.1

    Upgrade to Cog version 0.1

    The new version of Cog improves the Python API, along with several other changes.

    This PR upgrades CLIPstyler to Cog version >= 0.1. I have already pushed this to Replicate for you, so you don't need to do anything for the demo to keep working: https://replicate.com/paper11667/clipstyler

    opened by andreasjansson 0
Owner
null
GitHub
StyleGAN2-ADA - Official PyTorch implementation

Abstract: Training generative adversarial networks (GAN) using too little data typically leads to discriminator overfitting, causing training to diverge. We propose an adaptive discriminator augmentation mechanism that significantly stabilizes training in limited data regimes.

NVIDIA Research Projects 3.2k Dec 30, 2022
Official PyTorch implementation of Joint Object Detection and Multi-Object Tracking with Graph Neural Networks

This is the official PyTorch implementation of our paper: "Joint Object Detection and Multi-Object Tracking with Graph Neural Networks". Our project website and video demos are here.

RichardĀ Wang 443 Dec 6, 2022
Official pytorch implementation of paper "Image-to-image Translation via Hierarchical Style Disentanglement".

HiSD: Image-to-image Translation via Hierarchical Style Disentanglement Official pytorch implementation of paper "Image-to-image Translation

null 364 Dec 14, 2022
Official pytorch implementation of paper "Inception Convolution with Efficient Dilation Search" (CVPR 2021 Oral).

IC-Conv This repository is an official implementation of the paper Inception Convolution with Efficient Dilation Search. Getting Started Download Imag

Jie Liu 111 Dec 31, 2022
Official PyTorch Implementation of Unsupervised Learning of Scene Flow Estimation Fusing with Local Rigidity

UnRigidFlow This is the official PyTorch implementation of UnRigidFlow (IJCAI2019). Here are two sample results (~10MB gif for each) of our unsupervis

Liang Liu 28 Nov 16, 2022
Official implementation of our paper "LLA: Loss-aware Label Assignment for Dense Pedestrian Detection" in Pytorch.

LLA: Loss-aware Label Assignment for Dense Pedestrian Detection This project provides an implementation for "LLA: Loss-aware Label Assignment for Dens

null 35 Dec 6, 2022
An official implementation of "SFNet: Learning Object-aware Semantic Correspondence" (CVPR 2019, TPAMI 2020) in PyTorch.

PyTorch implementation of SFNet This is the implementation of the paper "SFNet: Learning Object-aware Semantic Correspondence". For more information,

CV Lab @ Yonsei University 87 Dec 30, 2022
Old Photo Restoration (Official PyTorch Implementation)

Bringing Old Photo Back to Life (CVPR 2020 oral)

Microsoft 11.3k Dec 30, 2022
Official PyTorch implementation of Spatial Dependency Networks.

Spatial Dependency Networks: Neural Layers for Improved Generative Image Modeling Đorđe Miladinović ā€ƒ Aleksandar Stanić ā€ƒ Stefan Bauer ā€ƒ JĆ¼rgen Schmid

Djordje Miladinovic 34 Jan 19, 2022
Official implementation of our CVPR2021 paper "OTA: Optimal Transport Assignment for Object Detection" in Pytorch.

OTA: Optimal Transport Assignment for Object Detection This project provides an implementation for our CVPR2021 paper "OTA: Optimal Transport Assignme

null 217 Jan 3, 2023
This is the official PyTorch implementation of the paper "TransFG: A Transformer Architecture for Fine-grained Recognition" (Ju He, Jie-Neng Chen, Shuai Liu, Adam Kortylewski, Cheng Yang, Yutong Bai, Changhu Wang, Alan Yuille).

TransFG: A Transformer Architecture for Fine-grained Recognition Official PyTorch code for the paper: TransFG: A Transformer Architecture for Fine-gra

Ju He 307 Jan 3, 2023
StyleGAN2-ADA - Official PyTorch implementation

Need Help? If youā€™re new to StyleGAN2-ADA and looking to get started, please check out this video series from a course Lia Coleman and I taught in Oct

Derrick Schultz 217 Jan 4, 2023
Official PyTorch implementation of "ArtFlow: Unbiased Image Style Transfer via Reversible Neural Flows"

ArtFlow Official PyTorch implementation of the paper: ArtFlow: Unbiased Image Style Transfer via Reversible Neural Flows Jie An*, Siyu Huang*, Yibing

null 123 Dec 27, 2022
Official PyTorch implementation of RobustNet (CVPR 2021 Oral)

RobustNet (CVPR 2021 Oral): Official Project Webpage Codes and pretrained models will be released soon. This repository provides the official PyTorch

Sungha Choi 173 Dec 21, 2022
Official PyTorch implementation for Generic Attention-model Explainability for Interpreting Bi-Modal and Encoder-Decoder Transformers, a novel method to visualize any Transformer-based network. Including examples for DETR, VQA.

PyTorch Implementation of Generic Attention-model Explainability for Interpreting Bi-Modal and Encoder-Decoder Transformers 1 Using Colab Please notic

Hila Chefer 489 Jan 7, 2023
[PyTorch] Official implementation of CVPR2021 paper "PointDSC: Robust Point Cloud Registration using Deep Spatial Consistency". https://arxiv.org/abs/2103.05465

PointDSC repository PyTorch implementation of PointDSC for CVPR'2021 paper "PointDSC: Robust Point Cloud Registration using Deep Spatial Consistency",

null 153 Dec 14, 2022
Official PyTorch implementation of MX-Font (Multiple Heads are Better than One: Few-shot Font Generation with Multiple Localized Experts)

Introduction Pytorch implementation of Multiple Heads are Better than One: Few-shot Font Generation with Multiple Localized Expert. | paper Song Park1

Clova AI Research 97 Dec 23, 2022
Official Pytorch implementation of 'GOCor: Bringing Globally Optimized Correspondence Volumes into Your Neural Network' (NeurIPS 2020)

Official implementation of GOCor This is the official implementation of our paper : GOCor: Bringing Globally Optimized Correspondence Volumes into You

Prune Truong 71 Nov 18, 2022
Official PyTorch Implementation of Hypercorrelation Squeeze for Few-Shot Segmentation, arXiv 2021

Hypercorrelation Squeeze for Few-Shot Segmentation This is the implementation of the paper "Hypercorrelation Squeeze for Few-Shot Segmentation" by Juh

Juhong Min 165 Dec 28, 2022