Official pytorch code for SSAT: A Symmetric Semantic-Aware Transformer Network for Makeup Transfer and Removal

ForeverPupil

Last update: Dec 13, 2022

Related tags

Deep Learning SSAT

Overview

SSAT: A Symmetric Semantic-Aware Transformer Network for Makeup Transfer and Removal

This is the official pytorch code for SSAT: A Symmetric Semantic-Aware Transformer Network for Makeup Transfer and Removal, which has been accepted by AAAI-2022. Note that only trained models and test code are provided in pytorch code. We will provide complete training code in Mindspore code in the future.

Example

We have provided test samples and trained models, you only need to run the "test.py" file and the results will be in "./results" folder .

How to run

Prepare face parsing. Face parsing is used in this code. In our experiment, face parsing is generated by https://github.com/zllrunning/face-parsing.PyTorch.
Put the results of face parsing in the .\test\seg1\makeup and \test\seg1\non-makeup
python test.py.

Our results

This source code is implemented using keras library based on "Automatic ocular artifacts removal in EEG using deep learning"

CSP_Deep_EEG This source code is implemented using keras library based on "Automatic ocular artifacts removal in EEG using deep learning" {https://www

2 Nov 8, 2022

Reference code for the paper CAMS: Color-Aware Multi-Style Transfer.

CAMS: Color-Aware Multi-Style Transfer Mahmoud Afifi1, Abdullah Abuolaim*1, Mostafa Hussien*2, Marcus A. Brubaker1, Michael S. Brown1 1York University

36 Dec 4, 2022

Code for the ICME 2021 paper "Exploring Driving-Aware Salient Object Detection via Knowledge Transfer"

TSOD Code for the ICME 2021 paper "Exploring Driving-Aware Salient Object Detection via Knowledge Transfer" Usage For training, open train_test, run p

2 Dec 23, 2021

Implementation of ICCV2021(Oral) paper - VMNet: Voxel-Mesh Network for Geodesic-aware 3D Semantic Segmentation

VMNet: Voxel-Mesh Network for Geodesic-Aware 3D Semantic Segmentation Created by Zeyu HU Introduction This work is based on our paper VMNet: Voxel-Mes

82 Dec 27, 2022

Official PyTorch implementation of Segmenter: Transformer for Semantic Segmentation

Segmenter: Transformer for Semantic Segmentation Segmenter: Transformer for Semantic Segmentation by Robin Strudel*, Ricardo Garcia*, Ivan Laptev and

594 Jan 6, 2023

Official Implementation of HRDA: Context-Aware High-Resolution Domain-Adaptive Semantic Segmentation

HRDA: Context-Aware High-Resolution Domain-Adaptive Semantic Segmentation by Lukas Hoyer, Dengxin Dai, and Luc Van Gool [Arxiv] [Paper] Overview Unsup

149 Dec 28, 2022

Official code for CVPR2022 paper: Depth-Aware Generative Adversarial Network for Talking Head Video Generation

📖 Depth-Aware Generative Adversarial Network for Talking Head Video Generation (CVPR 2022) 🔥 If DaGAN is helpful in your photos/projects, please hel

503 Jan 4, 2023

Learning from Synthetic Shadows for Shadow Detection and Removal [Inoue+, IEEE TCSVT 2020].

Learning from Synthetic Shadows for Shadow Detection and Removal (IEEE TCSVT 2020) Overview This repo is for the paper "Learning from Synthetic Shadow

67 Dec 28, 2022

A Web API for automatic background removal using Deep Learning. App is made using Flask and deployed on Heroku.

Automatic_Background_Remover A Web API for automatic background removal using Deep Learning. App is made using Flask and deployed on Heroku. 👉 https:

16 Oct 29, 2022

Comments

About Partial Makeup Transfer

Hi @Snowfallingplum, I see your paper provides partial makeup transfer. Could you give me some hints how to do this in your repo code.

Thank you very much!

opened by haile-vnu 4
Generate artifacts that appear in the final image

Thank you very much for your work and the improved generalization robustness of this algorithm, but I would still like to ask you what causes the phenomenon shown in the figure.

opened by YifanWangCN 2
关于模型架构的一些想法，想探讨请教一下

就是对于模型设计，我很喜欢就是输入人脸解析这个语义分割结果这个idea，，但是感觉这边处理人脸解析信息这里有点粗暴，就是只是parse 经过 semantic_encoder 然后直接和content_encoder的人脸内容信息进行torch.cat会不会有点粗暴呢？如果是我的想法会至少进行人脸解析和人脸内容进行一个逐元素相乘的操作，我想通过类似这种逐元素相乘更好的避免一些背景的干扰会不会好一点呢？我之所以想到这个原因是因为我把pytorch你们团队给出的代码用paddle写了一份，并且参数迁移过去我实验了多次，发现就是人脸需要占据整张图片的很大一部分效果才好，就是生成出来的妆容迁移往往人物背景会进行改变和变色，其中输入图片的背景也会对于模型妆容迁移进行错误干扰。

不知道你们团队对于我说的这个人脸解析语义分割信息与人脸图片融合的想法是否有过类似的idea和实验呢？就是我不是故意找茬的，就是你们的这篇论文让我受益匪浅，想更加深入的探讨一下，不知道可不可以。

如果是需要光照信息的话，人脸应该也可以获得光照信息，并不依赖背景，我这边测试了证件照还有一个其他的照片，这里给出证件照片的不太好的效果。

opened by hl123-123 1
我觉得这里是不是代码有点小瑕疵，关于这个softmax
https://github.com/Snowfallingplum/SSAT/blob/master/networks.py class SymmetryAttention其中269到272行

energy_ab_T = torch.bmm(fb.transpose(-2, -1), fa) * alpha corr_ab_T = F.softmax(energy_ab_T, dim=2) # n*HW*C @ n*C*HW -> n*HW*HW # print(softmax_weights.shape, b_raw.shape) b_warp = torch.bmm(b_raw.view(n, raw_c, h * w), corr_ab_T) # n*HW*1

这里面相当于b_content[b,hw,c] a_content[b,c,hw]再softmax得到attention,然后b_makeupattention。也就是说b的妆容针对于a的脸部进行一个内容扭曲，但是这个softmax应该针对的是1dim啊，因为你是b_makeup*attention，应该是a每个空间点对应的b所有空间信息进行一个softmax，对吧，这边你们团队给出的代码dim等于2逻辑上与传统的qkv self attention机制思维逻辑不一样吧
opened by hl123-123 2

Owner

ForeverPupil

GitHub

Official pytorch code for SSAT: A Symmetric Semantic-Aware Transformer Network for Makeup Transfer and Removal

Related tags

Overview

SSAT: A Symmetric Semantic-Aware Transformer Network for Makeup Transfer and Removal

Example

How to run

Our results

You might also like...

This source code is implemented using keras library based on "Automatic ocular artifacts removal in EEG using deep learning"

Reference code for the paper CAMS: Color-Aware Multi-Style Transfer.

Code for the ICME 2021 paper "Exploring Driving-Aware Salient Object Detection via Knowledge Transfer"

Implementation of ICCV2021(Oral) paper - VMNet: Voxel-Mesh Network for Geodesic-aware 3D Semantic Segmentation

Official PyTorch implementation of Segmenter: Transformer for Semantic Segmentation

Official Implementation of HRDA: Context-Aware High-Resolution Domain-Adaptive Semantic Segmentation

Official code for CVPR2022 paper: Depth-Aware Generative Adversarial Network for Talking Head Video Generation

Learning from Synthetic Shadows for Shadow Detection and Removal [Inoue+, IEEE TCSVT 2020].

A Web API for automatic background removal using Deep Learning. App is made using Flask and deployed on Heroku.

Comments

About Partial Makeup Transfer

Generate artifacts that appear in the final image

关于模型架构的一些想法，想探讨请教一下

我觉得这里是不是代码有点小瑕疵，关于这个softmax

Owner

ForeverPupil

Implementation of CVPR 2021 paper "Spatially-invariant Style-codes Controlled Makeup Transfer"

MPRNet-Cloud-removal: Progressive cloud removal

Implementation of Cross Transformer for spatially-aware few-shot transfer, in Pytorch

Official code for "Stereo Waterdrop Removal with Row-wise Dilated Attention (IROS2021)"

RCDNet: A Model-driven Deep Neural Network for Single Image Rain Removal (CVPR2020)

An official implementation of "SFNet: Learning Object-aware Semantic Correspondence" (CVPR 2019, TPAMI 2020) in PyTorch.

Official Implementation of Domain-Aware Universal Style Transfer

code for our paper "Source Data-absent Unsupervised Domain Adaptation through Hypothesis Transfer and Labeling Transfer"

【CVPR 2021, Variational Inference Framework, PyTorch】 From Rain Generation to Rain Removal

Code for CVPR2021 paper "Robust Reflection Removal with Reflection-free Flash-only Cues"