An implementation of "Optimal Textures: Fast and Robust Texture Synthesis and Style Transfer through Optimal Transport"

Hans Brouwer

Last update: Jan 5, 2023

Related tags

Deep Learning OptimalTextures

Overview

Optex

An implementation of Optimal Textures: Fast and Robust Texture Synthesis and Style Transfer through Optimal Transport for TU Delft CS4240.

You can find a more in-depth summary of the implementation in this blog post.

Installation

git clone https://github.com/JCBrouwer/OptimalTextures
cd OptimalTextures
pip install -r requirements.txt
python optex.py -h

Texture synthesis

Generate a texture based on an example:

python optex.py --style style/graffiti.jpg --size 512

Style transfer

Supply two images and synthesize one in the style of the other.

python optex.py --style style/lava-small.jpg --content content/rocket.jpg --content_strength 0.2

Texture mixing

Blend two textures together.

python optex.py --style style/zebra.jpg style/pattern-small.jpg --mixing_alpha 0.5

Color transfer

Perform style transfer but keep the original colors of the content.

python optex.py --style style/green-paint-large.jpg --content content/city.jpg --style_scale 0.5 --content_strength 0.2 --color_transfer opt --size 1024

Comments

numpy can't convert cuda tensor

Out of the box (using conda python 3.8.5 and pytorch 1.8.1) was getting this error:

TypeError: can't convert cuda:0 device type tensor to numpy. Use Tensor.cpu() to copy the tensor to host memory first.

With this trace:

  File "optex.py", line 304, in <module>
    output = optimal_texture(**vars(args))
  File "optex.py", line 55, in optimal_texture
    style_layers, style_eigvs, content_layers = encode_inputs(styles, content, use_pca=use_pca)
  File "optex.py", line 141, in encode_inputs
    style_layers[l], eigvecs = fit_pca(style_layers[l])  # PCA
  File "optex.py", line 160, in fit_pca
    k = np.argmax(np.cumsum([i / total_variance for i in eigvals.cpu()]) > 0.9)
  File "<__array_function__ internals>", line 5, in cumsum
  File ".../numpy/core/fromnumeric.py", line 2511, in cumsum
    return _wrapfunc(a, 'cumsum', axis=axis, dtype=dtype, out=out)
  File ".../numpy/core/fromnumeric.py", line 55, in _wrapfunc
    return _wrapit(obj, method, *args, **kwds)
  File ".../numpy/core/fromnumeric.py", line 44, in _wrapit
    result = getattr(asarray(obj), method)(*args, **kwds)
  File ".../numpy/core/_asarray.py", line 102, in asarray
    return array(a, dtype, copy=False, order=order)
  File ".../torch/tensor.py", line 621, in __array__
    return self.numpy()

Converting the total_variance and eigvals in fit_pca() to cpu seemed to fix it though.

Really enjoying playing with this!

opened by RKelln 2

🌳 A Python-inspired implementation of the Optimum-Path Forest classifier.

OPFython: A Python-Inspired Optimum-Path Forest Classifier Welcome to OPFython. Note that this implementation relies purely on the standard LibOPF. Th

30 Jan 4, 2023

Implementation of Geometric Vector Perceptron, a simple circuit for 3d rotation equivariance for learning over large biomolecules, in Pytorch. Idea proposed and accepted at ICLR 2021

Geometric Vector Perceptron Implementation of Geometric Vector Perceptron, a simple circuit with 3d rotation equivariance for learning over large biom

59 Nov 24, 2022

Official implementation of AAAI-21 paper "Label Confusion Learning to Enhance Text Classification Models"

Description: This is the official implementation of our AAAI-21 accepted paper Label Confusion Learning to Enhance Text Classification Models. The str

101 Nov 25, 2022

Official PyTorch implementation for paper Context Matters: Graph-based Self-supervised Representation Learning for Medical Images

Context Matters: Graph-based Self-supervised Representation Learning for Medical Images Official PyTorch implementation for paper Context Matters: Gra

49 Nov 23, 2022

PyTorch implementation of "Conformer: Convolution-augmented Transformer for Speech Recognition" (INTERSPEECH 2020)

PyTorch implementation of Conformer: Convolution-augmented Transformer for Speech Recognition. Transformer models are good at capturing content-based

565 Jan 4, 2023

An essential implementation of BYOL in PyTorch + PyTorch Lightning

Essential BYOL A simple and complete implementation of Bootstrap your own latent: A new approach to self-supervised Learning in PyTorch + PyTorch Ligh

48 Sep 27, 2022

The official implementation of NeMo: Neural Mesh Models of Contrastive Features for Robust 3D Pose Estimation [ICLR-2021]. https://arxiv.org/pdf/2101.12378.pdf

NeMo: Neural Mesh Models of Contrastive Features for Robust 3D Pose Estimation [ICLR-2021] Release Notes The offical PyTorch implementation of NeMo, p

76 Nov 23, 2022

A PyTorch re-implementation of the paper 'Exploring Simple Siamese Representation Learning'. Reproduced the 67.8% Top1 Acc on ImageNet.

Exploring simple siamese representation learning This is a PyTorch re-implementation of the SimSiam paper on ImageNet dataset. The results match that

72 Nov 9, 2022

PyTorch implementation of "A Full-Band and Sub-Band Fusion Model for Real-Time Single-Channel Speech Enhancement."

FullSubNet This Git repository for the official PyTorch implementation of "A Full-Band and Sub-Band Fusion Model for Real-Time Single-Channel Speech E

357 Jan 4, 2023

An implementation of "Optimal Textures: Fast and Robust Texture Synthesis and Style Transfer through Optimal Transport"

Related tags

Overview

Optex

Installation

Texture synthesis

Style transfer

Texture mixing

Color transfer

You might also like...

🌳 A Python-inspired implementation of the Optimum-Path Forest classifier.

Implementation of Geometric Vector Perceptron, a simple circuit for 3d rotation equivariance for learning over large biomolecules, in Pytorch. Idea proposed and accepted at ICLR 2021

Official implementation of AAAI-21 paper "Label Confusion Learning to Enhance Text Classification Models"

Official PyTorch implementation for paper Context Matters: Graph-based Self-supervised Representation Learning for Medical Images

PyTorch implementation of "Conformer: Convolution-augmented Transformer for Speech Recognition" (INTERSPEECH 2020)

An essential implementation of BYOL in PyTorch + PyTorch Lightning

The official implementation of NeMo: Neural Mesh Models of Contrastive Features for Robust 3D Pose Estimation [ICLR-2021]. https://arxiv.org/pdf/2101.12378.pdf

A PyTorch re-implementation of the paper 'Exploring Simple Siamese Representation Learning'. Reproduced the 67.8% Top1 Acc on ImageNet.

PyTorch implementation of "A Full-Band and Sub-Band Fusion Model for Real-Time Single-Channel Speech Enhancement."

Comments

numpy can't convert cuda tensor

Owner

Hans Brouwer

ALBERT-pytorch-implementation - ALBERT pytorch implementation

Numenta Platform for Intelligent Computing is an implementation of Hierarchical Temporal Memory (HTM), a theory of intelligence based strictly on the neuroscience of the neocortex.

PyTorch implementation of neural style transfer algorithm

PyTorch implementation of DeepDream algorithm

The project is an official implementation of our CVPR2019 paper "Deep High-Resolution Representation Learning for Human Pose Estimation"

Image-to-Image Translation with Conditional Adversarial Networks (Pix2pix) implementation in keras

Python implementation of cover trees, near-drop-in replacement for scipy.spatial.kdtree

Home repository for the Regularized Greedy Forest (RGF) library. It includes original implementation from the paper and multithreaded one written in C++, along with various language-specific wrappers.

Implementation of Restricted Boltzmann Machine (RBM) and its variants in Tensorflow

A fast Evolution Strategy implementation in Python