Official PaddlePaddle implementation of Paint Transformer

TianweiLin

Last update: Dec 31, 2022

Related tags

Deep Learning PaintTransformer

Overview

Paint Transformer: Feed Forward Neural Painting with Stroke Prediction

[Paper] [Paddle Implementation]

Update

We have optimized the serial inference procedure to achieve better rendering quality and faster speed.

Overview

This repository contains the official PaddlePaddle implementation of paper:

Paint Transformer: Feed Forward Neural Painting with Stroke Prediction,

Songhua Liu*, Tianwei Lin*, Dongliang He, Fu Li, Ruifeng Deng, Xin Li, Errui Ding, Hao Wang (* indicates equal contribution)

ICCV 2021 (Oral)

Prerequisites

Linux or macOS
Python 3.6+
PaddlePaddle 2.0+ and other dependencies (numpy, cv2, and other common python libs)
```
python -m pip install paddlepaddle-gpu
```

Getting Started

Clone this repository:

git clone https://github.com/wzmsltw/PaintTransformer
cd PaintTransformer

Download pretrained model from Google Drive and move it to inference directory:
```
mv [Download Directory]/paint_best.pdparams inference/
cd inference
```
Inference:
```
python inference.py
```
- Input image path, output path, and etc can be set in the main function.
- Notably, there is a flag serial as one parameter of the main function:
  - If serial is True, strokes would be rendered serially. The consumption of video memory will be low but it requires more time. Serial inference can achieve better rendering quality.
  - If serial is False, strokes would be rendered in parallel. The consumption of video memory will be high but it would be faster.
  - If animated results are required, serial must be True.
Train:
- You can send email to us for the training codes.

More Results

Input	Animated Output

App

Do not want to run the code? Try an App 一刻相册 downloaded from here!

Citation

If you find ideas or codes useful for your research, please cite:

@inproceedings{liu2021paint,
  title={Paint Transformer: Feed Forward Neural Painting with Stroke Prediction},
  author={Liu, Songhua and Lin, Tianwei and He, Dongliang and Li, Fu and Deng, Ruifeng and Li, Xin and Ding, Errui and Wang, Hao},
  booktitle={Proceedings of the IEEE International Conference on Computer Vision},
  year={2021}
}

Contact

For any question, please file an issue or contact

Songhua Liu: [email protected]
Tianwei Lin: [email protected]

Comments

The vision of paddlepaddle-gpu.

The vision of paddlepaddle-gpu that I used is 2.1.3, and meet the requirements of version greater than 2.0 . It doesn't work. In the definition of the Transformer function, the number of initial parameters of the TransformerEncoder class does not correspond. The number of initial parameters in the 2.0 and 2.1 versions is 3, but 4 actual parameters are passed in in network.py. I want to know which version of paddlepaddle-gpu you have chosen.

opened by ChengYan-Yan 1
IndexError
environment：

windows python3.6

CUDAv10.2 and Corresponding cdnn、paddle（2.4.0.port102），

where:

render_utils.py line 56：rush = paddle.to_tensor(meta_brushes_resize[index])

IndexError index 30238054144 is out of bounds for axis 0 with size 2
opened by lishufood 0
Brush stroke suggestion

I think the brush stroke should have some variable transparency, not just be 100% opaque. Something similar to what is being done here: http://3dstereophoto.blogspot.com/2018/07/non-photorealistic-rendering-software.html

This is a stroke based renderer that simulates oil painting/digital painting using traditional methods, which means it probably creates too many strokes.

By having the brushstroke as a grayscale image where white means fully opaque and black fully transparent, some of the color that's on the canvas would still be visible after the brush stroke is applied, which would give in my opinion a more natural/realistic look.

opened by ugocapeto 0
Can you provide Google CoLab support?

As a mac user, I do not have valid GPU to use. I have downloaded 一刻相机 to try your painting style, but it seems very slow. Could you please provide online running demo like CoLab? Or does PaddlePaddle has online running feature? Thanks in advance.

opened by tianye2856 1
How to set do not resize?

I want to do not resize and set resize_h and resize_w as None, but got the error: ValueError: (InvalidArgument) The 'shape' in Reshape0p is invalid.The input tensor X'size must be equal to the capacity of 'shape'......

opened by WellTung666 2

Owner

TianweiLin

Graduate student in SJTU

GitHub

Homepage of paper: Paint Transformer: Feed Forward Neural Painting with Stroke Prediction, ICCV 2021.

Paint Transformer: Feed Forward Neural Painting with Stroke Prediction [Paper] [PaddlePaddle Implementation] Homepage of paper: Paint Transformer: Fee

442 Dec 16, 2022

PaddleViT: State-of-the-art Visual Transformer and MLP Models for PaddlePaddle 2.0+

PaddlePaddle Vision Transformers State-of-the-art Visual Transformer and MLP Models for PaddlePaddle ?? PaddlePaddle Visual Transformers (PaddleViT or

1k Dec 28, 2022

An implementation of paper `Real-time Convolutional Neural Networks for Emotion and Gender Classification` with PaddlePaddle.

简介通过PaddlePaddle框架复现了论文 Real-time Convolutional Neural Networks for Emotion and Gender Classification 中提出的两个模型，分别是SimpleCNN和MiniXception。利用 imdb_crop

8 Mar 11, 2022

A PaddlePaddle implementation of Time Interval Aware Self-Attentive Sequential Recommendation.

TiSASRec.paddle A PaddlePaddle implementation of Time Interval Aware Self-Attentive Sequential Recommendation. Introduction 论文：Time Interval Aware Sel

2 Nov 28, 2021

A PaddlePaddle implementation of STGCN with a few modifications in the model architecture in order to forecast traffic jam.

About This repository contains the code of a PaddlePaddle implementation of STGCN based on the paper Spatio-Temporal Graph Convolutional Networks: A D

1 Jan 11, 2022

This is an official implementation for "Swin Transformer: Hierarchical Vision Transformer using Shifted Windows" on Object Detection and Instance Segmentation.

Swin Transformer for Object Detection This repo contains the supported code and configuration files to reproduce object detection results of Swin Tran

1.4k Dec 30, 2022

Official PaddlePaddle implementation of Paint Transformer

Related tags

Overview

Paint Transformer: Feed Forward Neural Painting with Stroke Prediction

Update

Overview

Prerequisites

Getting Started

More Results

App

Citation

Contact

Comments

The vision of paddlepaddle-gpu.

IndexError

Brush stroke suggestion

Can you provide Google CoLab support?

How to set do not resize?

Owner

TianweiLin

Homepage of paper: Paint Transformer: Feed Forward Neural Painting with Stroke Prediction, ICCV 2021.

PaddleViT: State-of-the-art Visual Transformer and MLP Models for PaddlePaddle 2.0+

An implementation of paper `Real-time Convolutional Neural Networks for Emotion and Gender Classification` with PaddlePaddle.

A PaddlePaddle implementation of Time Interval Aware Self-Attentive Sequential Recommendation.

A PaddlePaddle implementation of STGCN with a few modifications in the model architecture in order to forecast traffic jam.

This is an official implementation for "Swin Transformer: Hierarchical Vision Transformer using Shifted Windows" on Object Detection and Instance Segmentation.

VSR-Transformer - This paper proposes a new Transformer for video super-resolution (called VSR-Transformer).

Object detection and instance segmentation toolkit based on PaddlePaddle.

A PaddlePaddle version image model zoo.

Paddle-Adversarial-Toolbox (PAT) is a Python library for Deep Learning Security based on PaddlePaddle.

Plaything for Autistic Children (demo for PaddlePaddle/Wechaty/Mixlab project)

Remote sensing change detection tool based on PaddlePaddle

End-to-end image segmentation kit based on PaddlePaddle.

Large-scale open domain KNOwledge grounded conVERsation system based on PaddlePaddle

🔥🔥High-Performance Face Recognition Library on PaddlePaddle & PyTorch🔥🔥

🔥🔥High-Performance Face Recognition Library on PaddlePaddle & PyTorch🔥🔥

Classical OCR DCNN reproduction based on PaddlePaddle framework.

Generate images from texts. In Russian. In PaddlePaddle

buildseg is a building extraction plugin of QGIS based on PaddlePaddle.