Implementation of Neural Style Transfer in Pytorch

Overview
Comments
  • Link for downloading vgg unavailable

    Link for downloading vgg unavailable

    Hi,

    The link https://bethgelab.org/media/uploads/pytorch_models/vgg_conv.pth from the shell script is currently unavailable. Is there any other way to download it?

    Thank you in advance!

    opened by LucasAlegre 3
  • How the

    How the "vgg_conv.pth" is generated?

    I found that "vgg_conv.pth" only has the conv layers of the original vgg net, and the fc layers are removed. May I ask that how this model is generated?

    opened by panovr 3
  • Confusion in gram matrix normalization division step's denominator values

    Confusion in gram matrix normalization division step's denominator values

    In GramMatrix class, at 3rd cell and 7th line, shouldn't we divide the gram matrix by the product of channels, height, and width (i.e. G.div_(c*h*w) instead of G.div_(h*w))? Or am I missing something?

    Edit: I do notice better results with your gram matrix implementation but not sure how this is a correct normalization. 🤔

    opened by RahulBhalley 2
  • Loss value not decreasing with LBFGS

    Loss value not decreasing with LBFGS

    I run the same code, with the output initialized to the content image. When running the optimization with LBFGS, the loss values does not decrease.

    Iteration: 50, loss: 479620896.000000 Iteration: 100, loss: 479620896.000000 ....

    There are no updates to opt_img at all. Is there any reason that this could be happening? EDIT: There is an exploding gradient as well. I am wondering if there is any clamping that is required.

    opened by nithya4 2
  • Imagenet normalization

    Imagenet normalization

    The pytorch docs (link) say to normalize images via

    normalize = transforms.Normalize(mean=[0.485, 0.456, 0.406],
                                     std=[0.229, 0.224, 0.225])
    

    However, the notebook in this repo normalizes via

     transforms.Normalize(mean=[0.40760392, 0.45795686, 0.48501961],
                                                    std=[1,1,1]
    

    I am trying to re-create the results from the original paper, so I am just curious about this. Is this method of normalization specific to this task, did the imagenet normalizations for pytorch change over time, or is there some other reason I may be missing?

    opened by Nick-Morgan 1
  • division by zero error on L-BGFS / python3.6 no cuda

    division by zero error on L-BGFS / python3.6 no cuda

    Thanks for the code Leon, cloning and running on linux, python 3.6 with cuda, works just fine for source images or any other img i threw at this. Running on py3.6 no cuda on a mac I got a division by zero on the optimizer Not that it makes any sense to run this without cuda but thought you ought to know. Running it with RMSProp this worked - although results were less pronounced.

    ZeroDivisionError Traceback (most recent call last) in () 20 return loss 21 ---> 22 optimizer.step(closure) 23 24 #display result

    ~/anaconda3/envs/abc/lib/python3.6/site-packages/torch/optim/lbfgs.py in step(self, closure) 151 152 # update scale of initial Hessian approximation --> 153 H_diag = ys / y.dot(y) # (y*y) 154 155 # compute the approximate (L-BFGS) inverse Hessian

    ZeroDivisionError: float division by zero

    opened by javiervdp 1
  • Runtime error when moving vgg to FP16

    Runtime error when moving vgg to FP16

    This is indeed a pytorch issue, not the issue of the original code by Leon.

    I am trying to speed up the neural style transfer on Nvidia Tesla V100 by using FP16. I modified the code to move the vgg to cuda().half(). In addition, all three images, style image, content image, and opt_img, are in FP16. I tried to keep the loss layers in FP32 because it easily can generate NaN and infinity in FP16. The code is at https://gist.github.com/michaelhuang74/009e149a2002b84696731fb599408c90

    When I ran the code, I encountered the following error. +++++++++++++++++++++++++++++++++++++++++++++++++++++++ Traceback (most recent call last): File "neural-style-Gatys-half.py", line 167, in style_targets = [GramMatrix()(A).detach().cuda() for A in vgg(style_image, style_layers)] File "/home/mqhuang2/anaconda2/lib/python2.7/site-packages/torch/nn/modules/module.py", line 319, in call result = self.forward(input, **kwargs) File "neural-style-Gatys-half.py", line 86, in forward G.div_(hw) RuntimeError: value cannot be converted to type Half without overflow: 960000 +++++++++++++++++++++++++++++++++++++++++++++++++++++++

    It seems that although I tried to keep the GramMatrix and loss functions in FP32, somehow, pytorch tried to convert FP32 to FP16 in the GramMatrix forward() method.

    Any idea how to resolve this error?

    opened by michaelhuang74 1
  • Why cov4 layer is repeated 2 times in VGG module ?

    Why cov4 layer is repeated 2 times in VGG module ?

    I am a beginner in ANN and I am working on your paper on style transfer.

    But I have a question regarding defining VGG module. Why there is layer4 is repeated 2 times? (Check below code) Thanks in advance.

        self.conv4_1 = nn.Conv2d(256, 512, kernel_size=3, padding=1)
          self.conv4_2 = nn.Conv2d(512, 512, kernel_size=3, padding=1)
          self.conv4_3 = nn.Conv2d(512, 512, kernel_size=3, padding=1)
          self.conv4_4 = nn.Conv2d(512, 512, kernel_size=3, padding=1)
          self.conv4_1 = nn.Conv2d(256, 512, kernel_size=3, padding=1)
          self.conv4_2 = nn.Conv2d(512, 512, kernel_size=3, padding=1)
          self.conv4_3 = nn.Conv2d(512, 512, kernel_size=3, padding=1)
          self.conv4_4 = nn.Conv2d(512, 512, kernel_size=3, padding=1)
    
    opened by fazlerabbitanjil 1
  • loss = sum(layer_losses)

    loss = sum(layer_losses)

    Hello, I am new to pytorch and this scene in general, but when I run the code I get an error;torch.sum() Use tensor.detach().numpy() instead.

    I am not familiar with how to detach or couldnt find an example of detaching each tensor on the list. Is the following equivalent code? : for elem in layer_losses[1:]: loss += elem

    opened by 0x54daydreamer 1
  • Link for downloading the model is unavailable

    Link for downloading the model is unavailable

    Hi, The link for download : https://bethgelab.org/media/uploads/pytorch_models/vgg_conv.pth is currently unavailable. In fact, I think the whole webpage http://bethgelab.org/ itself is unavailable.

    Could you kindly solve the issue, or is there another way to download it?

    Thanks!

    opened by mesahwi 2
  • Runtime Error: Mismatch of size of tensor

    Runtime Error: Mismatch of size of tensor

    Hi, I have been trying to debug the whole day, but have yet to solve the problem that only occurred to some images.

    Below is the error message:

    Traceback (most recent call last):
      File "NeuralStyleTransfer.py", line 279, in <module>
        optimizer.step(closure)
      File "/home/user/miniconda3/envs/gatys/lib/python3.6/site-packages/torch/optim/lbfgs.py", line 103, in step
        orig_loss = closure()
      File "NeuralStyleTransfer.py", line 268, in closure
        layer_losses = [weights[a] * loss_fns[a](A, targets[a]) for a, A in enumerate(out)]
      File "NeuralStyleTransfer.py", line 268, in <listcomp>
        layer_losses = [weights[a] * loss_fns[a](A, targets[a]) for a, A in enumerate(out)]
      File "/home/user/miniconda3/envs/gatys/lib/python3.6/site-packages/torch/nn/modules/module.py", line 493, in __call__
        result = self.forward(*input, **kwargs)
      File "/home/user/miniconda3/envs/gatys/lib/python3.6/site-packages/torch/nn/modules/loss.py", line 443, in forward
        return F.mse_loss(input, target, reduction=self.reduction)
      File "/home/user/miniconda3/envs/gatys/lib/python3.6/site-packages/torch/nn/functional.py", line 2256, in mse_loss
        expanded_input, expanded_target = torch.broadcast_tensors(input, target)
      File "/home/user/miniconda3/envs/gatys/lib/python3.6/site-packages/torch/functional.py", line 62, in broadcast_tensors
        return torch._C._VariableFunctions.broadcast_tensors(tensors)
    RuntimeError: The size of tensor a (159) must match the size of tensor b (160) at non-singleton dimension 3
    

    Really appreciate for any assistance.

    opened by lQingRu 1
  • Please add any kind of license

    Please add any kind of license

    MIT or Apache or even BSD. Or of course anything else, it's your code after all. :P Just anything to get a handle on this because as it stands, this code is "look don't touch", which is really a shame. :)

    opened by black-puppydog 3
Owner
Leon Gatys
Leon Gatys
Official PyTorch implementation of "ArtFlow: Unbiased Image Style Transfer via Reversible Neural Flows"

ArtFlow Official PyTorch implementation of the paper: ArtFlow: Unbiased Image Style Transfer via Reversible Neural Flows Jie An*, Siyu Huang*, Yibing

null 123 Dec 27, 2022
PyTorch implementation of paper: AdaAttN: Revisit Attention Mechanism in Arbitrary Neural Style Transfer, ICCV 2021.

AdaAttN: Revisit Attention Mechanism in Arbitrary Neural Style Transfer [Paper] [PyTorch Implementation] [Paddle Implementation] Overview This reposit

null 148 Dec 30, 2022
Implementation of Neural Style Transfer in Pytorch

PytorchNeuralStyleTransfer Code to run Neural Style Transfer from our paper Image Style Transfer Using Convolutional Neural Networks. Also includes co

Leon Gatys 396 Dec 1, 2022
Fast Neural Style for Image Style Transform by Pytorch

FastNeuralStyle by Pytorch Fast Neural Style for Image Style Transform by Pytorch This is famous Fast Neural Style of Paper Perceptual Losses for Real

Bengxy 81 Sep 3, 2022
Neural style transfer in PyTorch.

style-transfer-pytorch An implementation of neural style transfer (A Neural Algorithm of Artistic Style) in PyTorch, supporting CPUs and Nvidia GPUs.

Katherine Crowson 395 Jan 6, 2023
Neural style transfer as a class in PyTorch

pt-styletransfer Neural style transfer as a class in PyTorch Based on: https://github.com/alexis-jacq/Pytorch-Tutorials Adds: StyleTransferNet as a cl

Tyler Kvochick 31 Jun 27, 2022
Pytorch implementation of Nueral Style transfer

Nueral Style Transfer Pytorch implementation of Nueral style transfer algorithm , it is used to apply artistic styles to content images . Content is t

Abhinav 9 Oct 15, 2022
Official PyTorch implementation of Retrieve in Style: Unsupervised Facial Feature Transfer and Retrieval.

Retrieve in Style: Unsupervised Facial Feature Transfer and Retrieval PyTorch This is the PyTorch implementation of Retrieve in Style: Unsupervised Fa

null 60 Oct 12, 2022
PyTorch implementation of MuseMorphose, a Transformer-based model for music style transfer.

MuseMorphose This repository contains the official implementation of the following paper: Shih-Lun Wu, Yi-Hsuan Yang MuseMorphose: Full-Song and Fine-

Yating Music, Taiwan AI Labs 142 Jan 8, 2023
Unofficial pytorch implementation of 'Arbitrary Style Transfer in Real-time with Adaptive Instance Normalization'

pytorch-AdaIN This is an unofficial pytorch implementation of a paper, Arbitrary Style Transfer in Real-time with Adaptive Instance Normalization [Hua

Naoto Inoue 873 Jan 6, 2023
Pytorch implementation of the paper "Enhancing Content Preservation in Text Style Transfer Using Reverse Attention and Conditional Layer Normalization"

Pytorch implementation of the paper "Enhancing Content Preservation in Text Style Transfer Using Reverse Attention and Conditional Layer Normalization"

Dongkyu Lee 4 Sep 18, 2022
Towards Ultra-Resolution Neural Style Transfer via Thumbnail Instance Normalization

Towards Ultra-Resolution Neural Style Transfer via Thumbnail Instance Normalization Official PyTorch implementation for our URST (Ultra-Resolution Sty

czczup 148 Dec 27, 2022
CVPR '21: In the light of feature distributions: Moment matching for Neural Style Transfer

In the light of feature distributions: Moment matching for Neural Style Transfer (CVPR 2021) This repository provides code to recreate results present

Nikolai Kalischek 49 Oct 13, 2022
This script runs neural style transfer against the provided content image.

Neural Style Transfer Content Style Output Description: This script runs neural style transfer against the provided content image. The content image m

Martynas Subonis 0 Nov 25, 2021
An implementation of "Optimal Textures: Fast and Robust Texture Synthesis and Style Transfer through Optimal Transport"

Optex An implementation of Optimal Textures: Fast and Robust Texture Synthesis and Style Transfer through Optimal Transport for TU Delft CS4240. You c

Hans Brouwer 33 Jan 5, 2023
Implementation of CVPR 2021 paper "Spatially-invariant Style-codes Controlled Makeup Transfer"

SCGAN Implementation of CVPR 2021 paper "Spatially-invariant Style-codes Controlled Makeup Transfer" Prepare The pre-trained model is avaiable at http

null 118 Dec 12, 2022
Official Implementation of Domain-Aware Universal Style Transfer

Domain Aware Universal Style Transfer Official Pytorch Implementation of 'Domain Aware Universal Style Transfer' (ICCV 2021) Domain Aware Universal St

KibeomHong 80 Dec 30, 2022
Simple Tensorflow implementation of "Adaptive Convolutions for Structure-Aware Style Transfer" (CVPR 2021)

AdaConv — Simple TensorFlow Implementation [Paper] : Adaptive Convolutions for Structure-Aware Style Transfer (CVPR 2021) Note This repository does no

Junho Kim 26 Nov 18, 2022
Only a Matter of Style: Age Transformation Using a Style-Based Regression Model

Only a Matter of Style: Age Transformation Using a Style-Based Regression Model The task of age transformation illustrates the change of an individual

null 444 Dec 30, 2022