PyTorch implementation of paper: AdaAttN: Revisit Attention Mechanism in Arbitrary Neural Style Transfer, ICCV 2021.

Overview

AdaAttN: Revisit Attention Mechanism in Arbitrary Neural Style Transfer

[Paper] [PyTorch Implementation] [Paddle Implementation]

Overview

This repository contains the official PyTorch implementation of paper:

AdaAttN: Revisit Attention Mechanism in Arbitrary Neural Style Transfer,

Songhua Liu, Tianwei Lin, Dongliang He, Fu Li, Meiling Wang, Xin Li, Zhengxing Sun, Qian Li, Errui Ding

ICCV 2021

Prerequisites

  • Linux or macOS
  • Python 3
  • PyTorch 1.7+ and other dependencies (torchvision, visdom, dominate, and other common python libs)

Getting Started

  • Clone this repository:

    git clone https://github.com/Huage001/AdaAttN
    cd AdaAttN
  • Inference:

    • Make a directory for checkpoints if there is not:

      mkdir checkpoints
    • Download pretrained model from Google Drive, move it to checkpoints directory, and unzip:

      mv [Download Directory]/AdaAttN_model.zip checkpoints/
      unzip checkpoints/AdaAttN_model.zip
      rm checkpoints/AdaAttN_model.zip
    • Configure content_path and style_path in test_adaattn.sh firstly, indicating paths to folders of testing content images and testing style images respectively.

    • Then, simply run:

      bash test_adaattn.sh
    • Check the results under results/AdaAttN folder.

  • Train:

    • Download COCO dataset and WikiArt dataset and then extract them.

    • Configure content_path and style_path in train_adaattn.sh, indicating paths to folders of training content images and training style images respectively.

    • Before training, start visdom server:

      python -m visdom.server
    • Then, simply run:

      bash train_adaattn.sh
    • You can monitor training status at http://localhost:8097/ and models would be saved at checkpoints/AdaAttN folder.

    • You may feel free to try other training options written in train_adaattn.sh.

Citation

  • If you find ideas or codes useful for your research, please cite:

    @inproceedings{liu2021adaattn,
      title={AdaAttN: Revisit Attention Mechanism in Arbitrary Neural Style Transfer},
      author={Liu, Songhua and Lin, Tianwei and He, Dongliang and Li, Fu and Wang, Meiling and Li, Xin and Sun, Zhengxing and Li, Qian and Ding, Errui},
      booktitle={Proceedings of the IEEE International Conference on Computer Vision},
      year={2021}
    }
    

Acknowledgments

Comments
  • Mean-variance-norm and Instance Norm

    Mean-variance-norm and Instance Norm

    Hi @Huage001 I read the paper and found that mean variance norm mean 'mean-variance channel-wise norm' works quite like instance norm. Can you explain to me why use mean-variance-norm function instead of instance norm? Thank you so much.

    opened by sonnguyen129 10
  • About the inference_frame.py in video infererce

    About the inference_frame.py in video infererce

    Hi , thanks for your code. As for the code in inference_frame.py, the line 110 in class AttnAdaINCos G_norm = torch.sqrt((G ** 2).sum(1).view(b, 1, -1)) b, _, h, w = F.size() F = F.view(b, -1, w * h) F_norm = torch.sqrt((F ** 2).sum(1).view(b, -1, 1)) F = F.permute(0, 2, 1) S = torch.relu(torch.bmm(F, G) / (F_norm + 1e-5) / (G_norm + 1e-5) + 1) # S: b, n_c, n_s S = S / (S.sum(dim=-1, keepdim=True) + 1e-5)

    1. i think the F_norm and the G_norm should be calculated by sum(2) not sum(1)
    2. why you add the 1e-5 in three places in code?
    opened by diaodeyi 9
  • About the adaattn_model.py

    About the adaattn_model.py

    Hi , thanks for your code. I have two questions about it.

    1. I don't understand what self.device means here, it doesn't seem to be defined. 1647223113(1)

    2. In addition, I would like to ask whether local feature loss is taking up too much memory.Because I ran out of memory when I tried to import this module into other code.

    Thanks!

    opened by weiyang001 3
  • what should i set the numbers of lambda_content??

    what should i set the numbers of lambda_content??

    In my screen, the lambda_content is always zero, i find the lambda_content is zero by default, i dont know how to set it , could you tell me how to set? Thanks!

    opened by zhongyuning 2
  • pre-trained model for video style transfer

    pre-trained model for video style transfer

    Thanks for the great work! It looks like this repo currently only hosts code and model for image style transfer. Could you provide a pre-trained model (and perhaps test scripts) for video style transfer as discussed in the paper? Thank you.

    opened by fmu2 2
  • Q. Traning time

    Q. Traning time

    The results of your paper is interesting.

    So, I want to train your model from scratch.

    Before training, I want to know approximate training time when you trained your model.

    Thanks.

    opened by sunwoo76 1
  • Add Docker environment & web demo

    Add Docker environment & web demo

    Hey @Huage001! 👋

    This pull request makes it possible to run your model inside a Docker environment, which makes it easier for other people to run it. We're using an open source tool called Cog to make this process easier.

    This also means we can make a web page where other people can try out your model! View it here: https://replicate.ai/huage001/adaattn

    Do claim your page here so you can own the page, customize the Example gallery as you like, and we'll feature it on our website and tweet about it too.

    In case you're wondering who I am, I'm from Replicate, where we're trying to make machine learning reproducible. We got frustrated that we couldn't run all the really interesting ML work being done. So, we're going round implementing models we like. 😊

    opened by chenxwh 0
  • video training code

    video training code

    Hi, thanks for the repo and for the video test code posted in the issues! It would be extremely helpful for the video training script to be posted as well, so we can experiment training on specific datasets.

    Thanks anyways

    opened by or-toledano 0
  • how to train video style transfer

    how to train video style transfer

    Hello, I have tried using the pre-training model you provided, and the effect is very good. Is the pre-training model you provided trained on Wikiart? If I want to train in more styles. Can you provide more details on how to train video style transfer? Just like readme said, is it OK to train directly on the coco dataset?

    opened by fengxiaoming520 0
Owner
null
Unofficial pytorch implementation of 'Arbitrary Style Transfer in Real-time with Adaptive Instance Normalization'

pytorch-AdaIN This is an unofficial pytorch implementation of a paper, Arbitrary Style Transfer in Real-time with Adaptive Instance Normalization [Hua

Naoto Inoue 873 Jan 6, 2023
Pytorch implementation of the paper "Enhancing Content Preservation in Text Style Transfer Using Reverse Attention and Conditional Layer Normalization"

Pytorch implementation of the paper "Enhancing Content Preservation in Text Style Transfer Using Reverse Attention and Conditional Layer Normalization"

Dongkyu Lee 4 Sep 18, 2022
Two-Stage Peer-Regularized Feature Recombination for Arbitrary Image Style Transfer

Two-Stage Peer-Regularized Feature Recombination for Arbitrary Image Style Transfer Paper on arXiv Public PyTorch implementation of two-stage peer-reg

NNAISENSE 38 Oct 14, 2022
Investigating Attention Mechanism in 3D Point Cloud Object Detection (arXiv 2021)

Investigating Attention Mechanism in 3D Point Cloud Object Detection (arXiv 2021) This repository is for the following paper: "Investigating Attention

null 52 Nov 19, 2022
code for paper"A High-precision Semantic Segmentation Method Combining Adversarial Learning and Attention Mechanism"

PyTorch implementation of UAGAN(U-net Attention Generative Adversarial Networks) This repository contains the source code for the paper "A High-precis

Tong 8 Apr 25, 2022
PyTorch implementation of neural style transfer algorithm

neural-style-pt This is a PyTorch implementation of the paper A Neural Algorithm of Artistic Style by Leon A. Gatys, Alexander S. Ecker, and Matthias

null 770 Jan 2, 2023
Official PyTorch implementation of "ArtFlow: Unbiased Image Style Transfer via Reversible Neural Flows"

ArtFlow Official PyTorch implementation of the paper: ArtFlow: Unbiased Image Style Transfer via Reversible Neural Flows Jie An*, Siyu Huang*, Yibing

null 123 Dec 27, 2022
Implementation of Neural Style Transfer in Pytorch

PytorchNeuralStyleTransfer Code to run Neural Style Transfer from our paper Image Style Transfer Using Convolutional Neural Networks. Also includes co

Leon Gatys 396 Dec 1, 2022
Implementation of CVPR 2021 paper "Spatially-invariant Style-codes Controlled Makeup Transfer"

SCGAN Implementation of CVPR 2021 paper "Spatially-invariant Style-codes Controlled Makeup Transfer" Prepare The pre-trained model is avaiable at http

null 118 Dec 12, 2022
Fast Neural Style for Image Style Transform by Pytorch

FastNeuralStyle by Pytorch Fast Neural Style for Image Style Transform by Pytorch This is famous Fast Neural Style of Paper Perceptual Losses for Real

Bengxy 81 Sep 3, 2022
Pytorch implementation for our ICCV 2021 paper "TRAR: Routing the Attention Spans in Transformers for Visual Question Answering".

TRAnsformer Routing Networks (TRAR) This is an official implementation for ICCV 2021 paper "TRAR: Routing the Attention Spans in Transformers for Visu

Ren Tianhe 49 Nov 10, 2022
This is the official pytorch implementation for our ICCV 2021 paper "TRAR: Routing the Attention Spans in Transformers for Visual Question Answering" on VQA Task

?? ERASOR (RA-L'21 with ICRA Option) Official page of "ERASOR: Egocentric Ratio of Pseudo Occupancy-based Dynamic Object Removal for Static 3D Point C

Hyungtae Lim 225 Dec 29, 2022
An implementation demo of the ICLR 2021 paper Neural Attention Distillation: Erasing Backdoor Triggers from Deep Neural Networks in PyTorch.

Neural Attention Distillation This is an implementation demo of the ICLR 2021 paper Neural Attention Distillation: Erasing Backdoor Triggers from Deep

Yige-Li 84 Jan 4, 2023
Attention mechanism with MNIST dataset

[TensorFlow] Attention mechanism with MNIST dataset Usage $ python run.py Result Training Loss graph. Test Each figure shows input digit, attention ma

YeongHyeon Park 12 Jun 10, 2022
Stacked Hourglass Network with a Multi-level Attention Mechanism: Where to Look for Intervertebral Disc Labeling

⚠️ ‎‎‎ A more recent and actively-maintained version of this code is available in ivadomed Stacked Hourglass Network with a Multi-level Attention Mech

Reza Azad 14 Oct 24, 2022
Learnable Multi-level Frequency Decomposition and Hierarchical Attention Mechanism for Generalized Face Presentation Attack Detection

LMFD-PAD Note This is the official repository of the paper: LMFD-PAD: Learnable Multi-level Frequency Decomposition and Hierarchical Attention Mechani

null 28 Dec 2, 2022
Neural style transfer in PyTorch.

style-transfer-pytorch An implementation of neural style transfer (A Neural Algorithm of Artistic Style) in PyTorch, supporting CPUs and Nvidia GPUs.

Katherine Crowson 395 Jan 6, 2023
Neural style transfer as a class in PyTorch

pt-styletransfer Neural style transfer as a class in PyTorch Based on: https://github.com/alexis-jacq/Pytorch-Tutorials Adds: StyleTransferNet as a cl

Tyler Kvochick 31 Jun 27, 2022
Code for our CVPR 2021 Paper "Rethinking Style Transfer: From Pixels to Parameterized Brushstrokes".

Rethinking Style Transfer: From Pixels to Parameterized Brushstrokes (CVPR 2021) Project page | Paper | Colab | Colab for Drawing App Rethinking Style

CompVis Heidelberg 153 Jan 4, 2023