Reimplement of SimSwap training code

seeprettyface.com

Last update: Dec 31, 2022

Related tags

Deep Learning SimSwap-train

Overview

SimSwap-train

Reimplement of SimSwap training code

Instructions

1.Environment Preparation

（1）Refer to the README document of SIMSWAP to configure the environment and download the pretrained model;
（2）In order to support custom resolution, you need to modify two places in /*your envs*/site-packages/insightface/utils/face_align.py:
line28: src_all = np.array([src1, src2, src3, src4, src5])
line53: src = src_all * image_size / 112

2.Making Training Data

python make_dataset.py --dataroot ./dataset/CelebA --extract_size 512 --output_img_dir ./dataset/CelebA/imgs --output_latent_dir ./dataset/CelebA/latents

The face images and latents will be recored in the output_img_dir and output_latent_dir directories.

3.Start Training

（1）New Training

CUDA_VISIBLE_DEVICES=0 python train.py --name CelebA_512 --dataroot ./dataset/CelebA --image_size 512 --display_winsize 512

Training visualization, loss log-files and model weights will be stored in chekpoints/name folder.

（2）Finetuning

CUDA_VISIBLE_DEVICES=0 python train.py --name CelebA_512_finetune --dataroot ./dataset/CelebA --image_size 512 --display_winsize 512 --continue_train

If chekpoints/name is an un-existed folder, it will first copy the official model from chekpoints/people to chekpoints/name; then finetuning.

4.Training Result

（1）CelebA with 224x224 res

（2）CelebA with 512x512 res

Comments

The training code is not working, face does not swap, it only tries to restore the same as in the video

The training is not working correctly. The face does not swap, it only tries to restore the same as in the video. Looks like this training implementation is not working, or this repo its a kinda bad joke for people, who want to train his own model for SimSwap... I train with different dataset size - 512, 224 and 104 but result is same - face is not swaping... Maybe did something wrong and did not correctly train model? I prepare dataset from celebA with make_dataset.py and train with command provided in readme.

In general, there is at least someone who managed to train the model and get the result and not just like mine?

Or can someone tell me why this happened? I don't think the author of this repo reads the issues and will answer my message.

Please write about your results and if this training code does not really work - write so that other users do not waste their time.

培训工作不正常。脸部不会交换，它只会尝试恢复与视频中的相同。看起来这个训练实现不起作用，或者这个 repo 对于想要为 SimSwap 训练他自己的模型的人来说是一个糟糕的笑话...... 我使用不同的数据集大小进行训练 - 512、224 和 104 但结果相同 - 面部没有交换......也许做错了什么并且没有正确训练模型？我使用 make_dataset.py 从 celebA 准备数据集，并使用自述文件中提供的命令进行训练。一般来说，至少有人设法训练模型并获得结果，而不仅仅是像我这样的？或者有人可以告诉我为什么会这样？我认为这个 repo 的作者不会阅读这些问题，也不会回答我的信息。请写下您的结果，如果此培训代码不起作用 - 写下来以免其他用户浪费时间。

opened by netrunner-exe 7

Issue after adding src_all = np.array([src1, src2, src3, src4, src5]) src = src_all * image_size / 112

Trying to train in Google Collab

If i change face_align.py, from readme, it show:

Traceback (most recent call last):
  File "make_dataset.py", line 10, in <module>
    from insightface_func.face_detect_crop_single import Face_detect_crop
  File "/content/SimSwap-train/insightface_func/face_detect_crop_single.py", line 8, in <module>
    from insightface.model_zoo import model_zoo
  File "/usr/local/lib/python3.7/dist-packages/insightface/__init__.py", line 16, in <module>
    from . import model_zoo
  File "/usr/local/lib/python3.7/dist-packages/insightface/model_zoo/__init__.py", line 1, in <module>
    from .model_zoo import get_model
  File "/usr/local/lib/python3.7/dist-packages/insightface/model_zoo/model_zoo.py", line 11, in <module>
    from .arcface_onnx import *
  File "/usr/local/lib/python3.7/dist-packages/insightface/model_zoo/arcface_onnx.py", line 12, in <module>
    from ..utils import face_align
  File "/usr/local/lib/python3.7/dist-packages/insightface/utils/face_align.py", line 29, in <module>
    src_map = {112: src, 224: src * 2}
NameError: name 'src' is not defined

If i dont change face_align.py, it just show:

find model: ./insightface_func/models/antelope/glintr100.onnx recognition
find model: ./insightface_func/models/antelope/scrfd_10g_bnkps.onnx detection
set det-size: (640, 640)
0it [00:00, ?it/s]

and not make dataset, just make empty folders - imgs and latents.

Images from CelebA dataset are in the root of the folderSimSwap-train/dataset/CelebA

If i run CUDA_VISIBLE_DEVICES=0 python train.py --name CelebA_512_finetune --dataroot /content/SimSwap-train/dataset/CelebA --image_size 512 --display_winsize 512 --continue_train it show

Traceback (most recent call last):
  File "train.py", line 14, in <module>
    from util.visualizer import Visualizer
  File "/content/SimSwap-train/util/visualizer.py", line 6, in <module>
    from . import html
  File "/content/SimSwap-train/util/html.py", line 1, in <module>
    import dominate
ModuleNotFoundError: No module named 'dominate'

I would be glad for any help or if someone shared his pretrained model.

opened by netrunner-exe 3

Can't download 256px-pretrained-weight

Hi @a312863063, please upload 256px-pretrained-weight to Google drive or DropBox in order to be able to download for those users who cannot download from Baidu drive. 嗨 @a312863063，请将 256px-pretrained-weight 上传到谷歌驱动器或 DropBox，以便那些无法从百度驱动器下载的用户能够下载。

opened by netrunner-exe 1

训练过程中计算Recloss是不是有问题?

首先感谢大佬的开源实现, 给大佬点赞👍 对比了下大佬的训练代码和当前offical的实现还是有挺多不一样的. 这里训练用的数据集为CelebA, 原始使用的是VGGFace, VGGFace中有按人划分文件夹, CelebA中的都混在一起了. 这里在读取数据的时候, 用文件路径区分不同的人, 认为不同图片路径即为不同id, 感觉不是很恰当 (当然针对后续训练流程貌似也没大问题) 在 dataloader环节, 将是否相同图片标记了 data_type:

        data_type = self.type
        if self.count == self.bs:
            #self.type = 1 - type
            self.type = (self.type + 1) % 5  # better
            self.count = 0
        
        return img_id, img_att, latent_id, latent_att, data_type

然后在训练流程中, 将data_type给到了 forward的 for_G参数, 但是!!! 在forward过程中并没有实际用到, 导致针对不同图片换脸也计算了 RecLoss

        #G_ID
        img_fake_down = F.interpolate(img_fake, scale_factor=112/img_fake.shape[-1])
        latent_fake = self.netArc(img_fake_down)
        latent_fake = F.normalize(latent_fake, p=2, dim=1)
        loss_G_ID = (1 - self.cosin_metric(latent_fake, latent_id)) * self.opt.lambda_id
        #print('=====================G_ID========================')
        #print(loss_G_ID)

        #G_Rec
        loss_G_Rec = self.criterionRec(img_fake, img_att) * self.opt.lambda_rec

        # Only return the fake_B image if necessary to save BW
        return [self.loss_filter(loss_G_GAN, loss_G_GAN_Feat, loss_G_VGG, loss_G_ID, loss_G_Rec, loss_D_GP, loss_D_real, loss_D_fake),
                img_fake]

这样出来应该是不合适的吧? 会导致RecLoss失效? 请问我的理解是对的吗? 再次感谢大佬的开源贡献

opened by iloveOREO 1

opt.nThreads leads to possibly wrong data.

I set the option --nThreads to 8, but the dataloader seems to not work as expected. I guess that could be duplicate datasets are working with independent attributes in these 8 threads. I've got all the data_type printed at the beginning of each batch. One type of data is fed successively for nThreads times until it gets increased. Is there any way to fix this without loss of much performance?

------------ Options -------------
Arc_path: arcface_model/arcface_checkpoint.tar
batchSize: 8
beta1: 0.5
checkpoints_dir: ./checkpoints
continue_train: False
data_type: 32
dataroot: ./dataset/CelebA
debug: False
display_freq: 1
display_winsize: 224
feat_num: 3
fineSize: 512
fp16: False
gan_mode: hinge
gpu_ids: [0]
image_size: 224
input_nc: 3
instance_feat: False
isTrain: True
label_feat: False
label_nc: 0
lambda_GP: 1e-05
lambda_feat: 10.0
lambda_id: 10.0
lambda_rec: 10.0
latent_size: 512
loadSize: 1024
load_features: False
load_pretrain: 
local_rank: 0
lr: 0.0002
max_dataset_size: inf
model: pix2pixHD
nThreads: 8
n_blocks_global: 6
n_blocks_local: 3
n_clusters: 10
n_downsample_E: 4
n_downsample_global: 3
n_layers_D: 4
n_local_enhancers: 1
name: CelebA_224
ndf: 64
nef: 16
netG: global
ngf: 64
niter: 10000
niter_decay: 10000
niter_fix_global: 0
no_flip: False
no_ganFeat_loss: False
no_html: False
no_instance: False
no_vgg_loss: False
norm: batch
norm_G: spectralspadesyncbatch3x3
num_D: 2
output_nc: 3
phase: train
pool_size: 0
print_freq: 1
resize_or_crop: scale_width
save_epoch_freq: 10000
save_latest_freq: 10000
semantic_nc: 3
serial_batches: False
tf_log: False
times_G: 1
use_dropout: False
verbose: False
which_epoch: latest
-------------- End ----------------
CustomDatasetDataLoader
dataset [FaceDataSet] was created
#training images = 202129
create web directory ./checkpoints\CelebA_224\web...
same ID: tensor([0, 0, 0, 0, 0, 0, 0, 0])
(epoch: 1, iters: 8, time: 0.955) G_GAN: -0.215 G_GAN_Feat: 23.784 G_ID: 9.646 G_Rec: 11.735 D_GP: 0.001 D_real: 0.808 D_fake: 1.215 
same ID: tensor([0, 0, 0, 0, 0, 0, 0, 0])
(epoch: 1, iters: 16, time: 0.092) G_GAN: 0.023 G_GAN_Feat: 16.439 G_ID: 9.752 G_Rec: 7.549 D_GP: 0.000 D_real: 1.191 D_fake: 1.277 
same ID: tensor([0, 0, 0, 0, 0, 0, 0, 0])
(epoch: 1, iters: 24, time: 0.081) G_GAN: 0.035 G_GAN_Feat: 14.662 G_ID: 9.737 G_Rec: 6.130 D_GP: 0.000 D_real: 1.055 D_fake: 1.185 
same ID: tensor([0, 0, 0, 0, 0, 0, 0, 0])
(epoch: 1, iters: 32, time: 0.082) G_GAN: 0.003 G_GAN_Feat: 12.767 G_ID: 9.161 G_Rec: 5.381 D_GP: 0.000 D_real: 1.105 D_fake: 1.121 
same ID: tensor([0, 0, 0, 0, 0, 0, 0, 0])
(epoch: 1, iters: 40, time: 0.081) G_GAN: 0.142 G_GAN_Feat: 12.409 G_ID: 9.250 G_Rec: 5.028 D_GP: 0.000 D_real: 1.126 D_fake: 1.014 
same ID: tensor([0, 0, 0, 0, 0, 0, 0, 0])
(epoch: 1, iters: 48, time: 0.084) G_GAN: 0.018 G_GAN_Feat: 11.651 G_ID: 8.848 G_Rec: 4.848 D_GP: 0.000 D_real: 0.995 D_fake: 1.036 
same ID: tensor([0, 0, 0, 0, 0, 0, 0, 0])
(epoch: 1, iters: 56, time: 0.086) G_GAN: -0.180 G_GAN_Feat: 10.410 G_ID: 7.619 G_Rec: 4.273 D_GP: 0.000 D_real: 0.858 D_fake: 1.186 
same ID: tensor([0, 0, 0, 0, 0, 0, 0, 0])
(epoch: 1, iters: 64, time: 0.081) G_GAN: 0.026 G_GAN_Feat: 11.858 G_ID: 8.109 G_Rec: 5.547 D_GP: 0.000 D_real: 1.032 D_fake: 1.011 
same ID: tensor([1, 1, 1, 1, 1, 1, 1, 1])
(epoch: 1, iters: 72, time: 0.082) G_GAN: 0.103 G_GAN_Feat: 10.665 G_ID: 9.722 G_Rec: 4.928 D_GP: 0.000 D_real: 0.956 D_fake: 0.912 
same ID: tensor([1, 1, 1, 1, 1, 1, 1, 1])
(epoch: 1, iters: 80, time: 0.086) G_GAN: 0.070 G_GAN_Feat: 10.263 G_ID: 9.511 G_Rec: 4.405 D_GP: 0.000 D_real: 0.933 D_fake: 0.958 
same ID: tensor([1, 1, 1, 1, 1, 1, 1, 1])
(epoch: 1, iters: 88, time: 0.085) G_GAN: 0.052 G_GAN_Feat: 9.233 G_ID: 9.924 G_Rec: 3.842 D_GP: 0.000 D_real: 0.872 D_fake: 0.979 
same ID: tensor([1, 1, 1, 1, 1, 1, 1, 1])
(epoch: 1, iters: 96, time: 0.082) G_GAN: -0.091 G_GAN_Feat: 10.016 G_ID: 9.510 G_Rec: 4.408 D_GP: 0.000 D_real: 0.833 D_fake: 1.098 
same ID: tensor([1, 1, 1, 1, 1, 1, 1, 1])
(epoch: 1, iters: 104, time: 0.085) G_GAN: 0.041 G_GAN_Feat: 9.755 G_ID: 9.953 G_Rec: 4.054 D_GP: 0.000 D_real: 0.950 D_fake: 0.988 
same ID: tensor([1, 1, 1, 1, 1, 1, 1, 1])
(epoch: 1, iters: 112, time: 0.086) G_GAN: 0.002 G_GAN_Feat: 9.763 G_ID: 10.223 G_Rec: 4.298 D_GP: 0.000 D_real: 0.786 D_fake: 1.010 
same ID: tensor([1, 1, 1, 1, 1, 1, 1, 1])
(epoch: 1, iters: 120, time: 0.080) G_GAN: 0.310 G_GAN_Feat: 9.213 G_ID: 9.937 G_Rec: 3.761 D_GP: 0.000 D_real: 1.077 D_fake: 0.826 
same ID: tensor([1, 1, 1, 1, 1, 1, 1, 1])
(epoch: 1, iters: 128, time: 0.080) G_GAN: -0.077 G_GAN_Feat: 11.179 G_ID: 10.065 G_Rec: 4.348 D_GP: 0.000 D_real: 0.824 D_fake: 1.104 
same ID: tensor([2, 2, 2, 2, 2, 2, 2, 2])
(epoch: 1, iters: 136, time: 0.082) G_GAN: -0.130 G_GAN_Feat: 10.189 G_ID: 9.776 G_Rec: 4.090 D_GP: 0.000 D_real: 0.703 D_fake: 1.133 
same ID: tensor([2, 2, 2, 2, 2, 2, 2, 2])
(epoch: 1, iters: 144, time: 0.079) G_GAN: 0.337 G_GAN_Feat: 13.464 G_ID: 9.856 G_Rec: 7.010 D_GP: 0.000 D_real: 1.178 D_fake: 0.791 
same ID: tensor([2, 2, 2, 2, 2, 2, 2, 2])
(epoch: 1, iters: 152, time: 0.082) G_GAN: -0.030 G_GAN_Feat: 10.651 G_ID: 9.827 G_Rec: 4.328 D_GP: 0.000 D_real: 0.848 D_fake: 1.090 
same ID: tensor([2, 2, 2, 2, 2, 2, 2, 2])
(epoch: 1, iters: 160, time: 0.080) G_GAN: 0.254 G_GAN_Feat: 7.981 G_ID: 10.843 G_Rec: 3.265 D_GP: 0.000 D_real: 1.095 D_fake: 0.847 
same ID: tensor([2, 2, 2, 2, 2, 2, 2, 2])
(epoch: 1, iters: 168, time: 0.083) G_GAN: 0.036 G_GAN_Feat: 9.070 G_ID: 10.492 G_Rec: 4.058 D_GP: 0.000 D_real: 0.949 D_fake: 0.989 
same ID: tensor([2, 2, 2, 2, 2, 2, 2, 2])
(epoch: 1, iters: 176, time: 0.084) G_GAN: 0.152 G_GAN_Feat: 8.269 G_ID: 9.705 G_Rec: 3.551 D_GP: 0.000 D_real: 0.938 D_fake: 0.874 
same ID: tensor([2, 2, 2, 2, 2, 2, 2, 2])
(epoch: 1, iters: 184, time: 0.087) G_GAN: 0.261 G_GAN_Feat: 7.916 G_ID: 9.935 G_Rec: 3.237 D_GP: 0.000 D_real: 0.908 D_fake: 0.766 
same ID: tensor([2, 2, 2, 2, 2, 2, 2, 2])
(epoch: 1, iters: 192, time: 0.081) G_GAN: 0.237 G_GAN_Feat: 9.409 G_ID: 10.117 G_Rec: 4.025 D_GP: 0.000 D_real: 0.885 D_fake: 0.810 
same ID: tensor([3, 3, 3, 3, 3, 3, 3, 3])
(epoch: 1, iters: 200, time: 0.081) G_GAN: 0.158 G_GAN_Feat: 8.814 G_ID: 10.127 G_Rec: 3.870 D_GP: 0.000 D_real: 0.777 D_fake: 0.883 
same ID: tensor([3, 3, 3, 3, 3, 3, 3, 3])
(epoch: 1, iters: 208, time: 0.087) G_GAN: 0.842 G_GAN_Feat: 8.845 G_ID: 10.243 G_Rec: 3.296 D_GP: 0.001 D_real: 1.372 D_fake: 0.526 
same ID: tensor([3, 3, 3, 3, 3, 3, 3, 3])
(epoch: 1, iters: 216, time: 0.080) G_GAN: -0.421 G_GAN_Feat: 9.650 G_ID: 9.546 G_Rec: 3.654 D_GP: 0.001 D_real: 0.506 D_fake: 1.439 
same ID: tensor([3, 3, 3, 3, 3, 3, 3, 3])
(epoch: 1, iters: 224, time: 0.080) G_GAN: -0.022 G_GAN_Feat: 8.847 G_ID: 10.050 G_Rec: 3.227 D_GP: 0.001 D_real: 0.724 D_fake: 1.052 
same ID: tensor([3, 3, 3, 3, 3, 3, 3, 3])
(epoch: 1, iters: 232, time: 0.082) G_GAN: 0.445 G_GAN_Feat: 9.311 G_ID: 10.803 G_Rec: 4.323 D_GP: 0.001 D_real: 1.091 D_fake: 0.649 
same ID: tensor([3, 3, 3, 3, 3, 3, 3, 3])
(epoch: 1, iters: 240, time: 0.080) G_GAN: 0.361 G_GAN_Feat: 8.344 G_ID: 9.757 G_Rec: 3.223 D_GP: 0.001 D_real: 0.910 D_fake: 0.693 
same ID: tensor([3, 3, 3, 3, 3, 3, 3, 3])
(epoch: 1, iters: 248, time: 0.083) G_GAN: 0.114 G_GAN_Feat: 8.736 G_ID: 9.966 G_Rec: 4.477 D_GP: 0.001 D_real: 0.693 D_fake: 0.935 
same ID: tensor([3, 3, 3, 3, 3, 3, 3, 3])
(epoch: 1, iters: 256, time: 0.082) G_GAN: 0.224 G_GAN_Feat: 8.297 G_ID: 9.602 G_Rec: 3.353 D_GP: 0.001 D_real: 0.794 D_fake: 0.909 
same ID: tensor([4, 4, 4, 4, 4, 4, 4, 4])
(epoch: 1, iters: 264, time: 0.080) G_GAN: 0.028 G_GAN_Feat: 8.113 G_ID: 9.754 G_Rec: 3.604 D_GP: 0.001 D_real: 0.768 D_fake: 1.041 
same ID: tensor([4, 4, 4, 4, 4, 4, 4, 4])
(epoch: 1, iters: 272, time: 0.082) G_GAN: 0.275 G_GAN_Feat: 7.745 G_ID: 10.281 G_Rec: 3.235 D_GP: 0.001 D_real: 0.924 D_fake: 0.809 
same ID: tensor([4, 4, 4, 4, 4, 4, 4, 4])
(epoch: 1, iters: 280, time: 0.082) G_GAN: 0.199 G_GAN_Feat: 9.014 G_ID: 9.414 G_Rec: 5.138 D_GP: 0.001 D_real: 1.058 D_fake: 0.861 
same ID: tensor([4, 4, 4, 4, 4, 4, 4, 4])
(epoch: 1, iters: 288, time: 0.081) G_GAN: 0.195 G_GAN_Feat: 8.024 G_ID: 10.019 G_Rec: 3.524 D_GP: 0.001 D_real: 0.952 D_fake: 0.870 
same ID: tensor([4, 4, 4, 4, 4, 4, 4, 4])
(epoch: 1, iters: 296, time: 0.082) G_GAN: 0.190 G_GAN_Feat: 7.509 G_ID: 10.013 G_Rec: 2.861 D_GP: 0.001 D_real: 0.862 D_fake: 0.838 
same ID: tensor([4, 4, 4, 4, 4, 4, 4, 4])
(epoch: 1, iters: 304, time: 0.082) G_GAN: 0.253 G_GAN_Feat: 8.452 G_ID: 10.450 G_Rec: 3.357 D_GP: 0.001 D_real: 0.854 D_fake: 0.776 
same ID: tensor([4, 4, 4, 4, 4, 4, 4, 4])
(epoch: 1, iters: 312, time: 0.080) G_GAN: 0.571 G_GAN_Feat: 8.652 G_ID: 10.282 G_Rec: 3.527 D_GP: 0.002 D_real: 1.114 D_fake: 0.527 
same ID: tensor([4, 4, 4, 4, 4, 4, 4, 4])
(epoch: 1, iters: 320, time: 0.080) G_GAN: -0.004 G_GAN_Feat: 8.572 G_ID: 9.927 G_Rec: 3.243 D_GP: 0.002 D_real: 0.462 D_fake: 1.041 
same ID: tensor([0, 0, 0, 0, 0, 0, 0, 0])
(epoch: 1, iters: 328, time: 0.080) G_GAN: 0.565 G_GAN_Feat: 9.724 G_ID: 7.166 G_Rec: 4.114 D_GP: 0.003 D_real: 1.065 D_fake: 0.671 
same ID: tensor([0, 0, 0, 0, 0, 0, 0, 0])
(epoch: 1, iters: 336, time: 0.078) G_GAN: 0.118 G_GAN_Feat: 8.737 G_ID: 6.280 G_Rec: 3.815 D_GP: 0.002 D_real: 0.868 D_fake: 1.031 
same ID: tensor([0, 0, 0, 0, 0, 0, 0, 0])
(epoch: 1, iters: 344, time: 0.081) G_GAN: 0.062 G_GAN_Feat: 7.814 G_ID: 5.888 G_Rec: 3.160 D_GP: 0.002 D_real: 0.717 D_fake: 1.016 
same ID: tensor([0, 0, 0, 0, 0, 0, 0, 0])
(epoch: 1, iters: 352, time: 0.079) G_GAN: 0.521 G_GAN_Feat: 8.553 G_ID: 5.634 G_Rec: 4.576 D_GP: 0.002 D_real: 1.117 D_fake: 0.566 
same ID: tensor([0, 0, 0, 0, 0, 0, 0, 0])
(epoch: 1, iters: 360, time: 0.081) G_GAN: 0.107 G_GAN_Feat: 9.291 G_ID: 6.601 G_Rec: 5.221 D_GP: 0.002 D_real: 0.888 D_fake: 0.912 
same ID: tensor([0, 0, 0, 0, 0, 0, 0, 0])
(epoch: 1, iters: 368, time: 0.082) G_GAN: 0.060 G_GAN_Feat: 9.828 G_ID: 5.915 G_Rec: 5.354 D_GP: 0.003 D_real: 0.724 D_fake: 0.975 
same ID: tensor([0, 0, 0, 0, 0, 0, 0, 0])
(epoch: 1, iters: 376, time: 0.082) G_GAN: 0.892 G_GAN_Feat: 9.091 G_ID: 5.354 G_Rec: 4.656 D_GP: 0.002 D_real: 1.482 D_fake: 0.421 
same ID: tensor([0, 0, 0, 0, 0, 0, 0, 0])
(epoch: 1, iters: 384, time: 0.081) G_GAN: -0.054 G_GAN_Feat: 6.884 G_ID: 5.002 G_Rec: 2.787 D_GP: 0.003 D_real: 0.627 D_fake: 1.072 
same ID: tensor([1, 1, 1, 1, 1, 1, 1, 1])
(epoch: 1, iters: 392, time: 0.079) G_GAN: 0.188 G_GAN_Feat: 8.042 G_ID: 10.007 G_Rec: 3.202 D_GP: 0.004 D_real: 0.744 D_fake: 0.826 
same ID: tensor([1, 1, 1, 1, 1, 1, 1, 1])
(epoch: 1, iters: 400, time: 0.081) G_GAN: 0.454 G_GAN_Feat: 7.759 G_ID: 10.022 G_Rec: 3.154 D_GP: 0.004 D_real: 1.167 D_fake: 0.638 
same ID: tensor([1, 1, 1, 1, 1, 1, 1, 1])
(epoch: 1, iters: 408, time: 0.081) G_GAN: 0.461 G_GAN_Feat: 7.816 G_ID: 10.175 G_Rec: 3.143 D_GP: 0.003 D_real: 0.967 D_fake: 0.618 
same ID: tensor([1, 1, 1, 1, 1, 1, 1, 1])
(epoch: 1, iters: 416, time: 0.081) G_GAN: 0.018 G_GAN_Feat: 8.015 G_ID: 9.605 G_Rec: 2.684 D_GP: 0.004 D_real: 0.704 D_fake: 1.041 
same ID: tensor([1, 1, 1, 1, 1, 1, 1, 1])
(epoch: 1, iters: 424, time: 0.098) G_GAN: -0.196 G_GAN_Feat: 7.417 G_ID: 9.965 G_Rec: 2.688 D_GP: 0.004 D_real: 0.761 D_fake: 1.274 
same ID: tensor([1, 1, 1, 1, 1, 1, 1, 1])
(epoch: 1, iters: 432, time: 0.088) G_GAN: 0.521 G_GAN_Feat: 9.091 G_ID: 9.760 G_Rec: 5.096 D_GP: 0.003 D_real: 1.240 D_fake: 0.668 
same ID: tensor([1, 1, 1, 1, 1, 1, 1, 1])
(epoch: 1, iters: 440, time: 0.087) G_GAN: -0.066 G_GAN_Feat: 7.830 G_ID: 9.920 G_Rec: 3.300 D_GP: 0.002 D_real: 0.889 D_fake: 1.080 
same ID: tensor([1, 1, 1, 1, 1, 1, 1, 1])
(epoch: 1, iters: 448, time: 0.124) G_GAN: 0.099 G_GAN_Feat: 6.673 G_ID: 10.154 G_Rec: 2.599 D_GP: 0.002 D_real: 0.788 D_fake: 0.948 
same ID: tensor([2, 2, 2, 2, 2, 2, 2, 2])
(epoch: 1, iters: 456, time: 0.091) G_GAN: 0.049 G_GAN_Feat: 7.493 G_ID: 9.551 G_Rec: 3.624 D_GP: 0.002 D_real: 0.827 D_fake: 0.965 
same ID: tensor([2, 2, 2, 2, 2, 2, 2, 2])
(epoch: 1, iters: 464, time: 0.088) G_GAN: 0.286 G_GAN_Feat: 8.065 G_ID: 9.909 G_Rec: 3.528 D_GP: 0.002 D_real: 0.839 D_fake: 0.755 
same ID: tensor([2, 2, 2, 2, 2, 2, 2, 2])
(epoch: 1, iters: 472, time: 0.094) G_GAN: 0.415 G_GAN_Feat: 7.930 G_ID: 9.979 G_Rec: 4.074 D_GP: 0.003 D_real: 1.199 D_fake: 0.658 
same ID: tensor([2, 2, 2, 2, 2, 2, 2, 2])
(epoch: 1, iters: 480, time: 0.084) G_GAN: 0.360 G_GAN_Feat: 7.778 G_ID: 9.684 G_Rec: 2.843 D_GP: 0.003 D_real: 0.868 D_fake: 0.673 
same ID: tensor([2, 2, 2, 2, 2, 2, 2, 2])
(epoch: 1, iters: 488, time: 0.082) G_GAN: 0.247 G_GAN_Feat: 7.325 G_ID: 9.680 G_Rec: 3.086 D_GP: 0.004 D_real: 0.891 D_fake: 0.775 
same ID: tensor([2, 2, 2, 2, 2, 2, 2, 2])
(epoch: 1, iters: 496, time: 0.081) G_GAN: 0.197 G_GAN_Feat: 8.729 G_ID: 9.770 G_Rec: 4.807 D_GP: 0.005 D_real: 0.694 D_fake: 0.859 
same ID: tensor([2, 2, 2, 2, 2, 2, 2, 2])
(epoch: 1, iters: 504, time: 0.080) G_GAN: 0.332 G_GAN_Feat: 9.149 G_ID: 9.967 G_Rec: 3.969 D_GP: 0.006 D_real: 0.948 D_fake: 0.722 
same ID: tensor([2, 2, 2, 2, 2, 2, 2, 2])
(epoch: 1, iters: 512, time: 0.080) G_GAN: 0.243 G_GAN_Feat: 7.408 G_ID: 9.460 G_Rec: 2.800 D_GP: 0.006 D_real: 0.974 D_fake: 0.839 
same ID: tensor([3, 3, 3, 3, 3, 3, 3, 3])
(epoch: 1, iters: 520, time: 0.079) G_GAN: 0.039 G_GAN_Feat: 8.930 G_ID: 9.992 G_Rec: 4.354 D_GP: 0.004 D_real: 0.958 D_fake: 0.989 
same ID: tensor([3, 3, 3, 3, 3, 3, 3, 3])
(epoch: 1, iters: 528, time: 0.081) G_GAN: -0.200 G_GAN_Feat: 7.089 G_ID: 9.590 G_Rec: 3.049 D_GP: 0.004 D_real: 0.714 D_fake: 1.220 
same ID: tensor([3, 3, 3, 3, 3, 3, 3, 3])
(epoch: 1, iters: 536, time: 0.083) G_GAN: 0.464 G_GAN_Feat: 7.312 G_ID: 10.589 G_Rec: 3.485 D_GP: 0.003 D_real: 1.075 D_fake: 0.588 
same ID: tensor([3, 3, 3, 3, 3, 3, 3, 3])
(epoch: 1, iters: 544, time: 0.081) G_GAN: 0.038 G_GAN_Feat: 8.542 G_ID: 9.659 G_Rec: 4.123 D_GP: 0.005 D_real: 0.745 D_fake: 0.990 
same ID: tensor([3, 3, 3, 3, 3, 3, 3, 3])

opened by Ailon-Island 1

RuntimeError: mat1 and mat2 shapes cannot be multiplied (1x131072 and 25088x512)

When i use python test_video_swapsingle.py --image_size 512 it show an error:

load pretrained model:./checkpoints/CelebA_512_finetune/latest_net_G.pth
input mean and std: 127.5 127.5
find model: ./insightface_func/models/antelope/glintr100.onnx recognition
find model: ./insightface_func/models/antelope/scrfd_10g_bnkps.onnx detection
set det-size: (640, 640)
Traceback (most recent call last):
  File "test_video_swapsingle.py", line 68, in <module>
    latend_id = model.netArc(img_id_downsample)
  File "/usr/local/lib/python3.7/dist-packages/torch/nn/modules/module.py", line 1102, in _call_impl
    return forward_call(*input, **kwargs)
  File "/content/SimSwap-train/models/models.py", line 145, in forward
    x = self.fc(x)
  File "/usr/local/lib/python3.7/dist-packages/torch/nn/modules/module.py", line 1102, in _call_impl
    return forward_call(*input, **kwargs)
  File "/usr/local/lib/python3.7/dist-packages/torch/nn/modules/linear.py", line 103, in forward
    return F.linear(input, self.weight, self.bias)
  File "/usr/local/lib/python3.7/dist-packages/torch/nn/functional.py", line 1848, in linear
    return torch._C._nn.linear(input, weight, bias)
RuntimeError: mat1 and mat2 shapes cannot be multiplied (1x131072 and 25088x512)

If change 512 to 244 all works

opened by netrunner-exe 1

NameError: name 'img_id' is not defined

Traceback (most recent call last):
  File "test_video_swapsingle.py", line 72, in <module>
    no_simswaplogo=opt.no_simswaplogo,use_mask=opt.use_mask, name=opt.name)
  File "/content/SimSwap-train/util/videoswap.py", line 82, in video_swap
    swap_result = swap_model(img_id, input_norm, id_vetor, None)[0]
NameError: name 'img_id' is not defined

opened by netrunner-exe 1

Can't create faceset from celebA dataset
When i run make_dataset.py, it just show 0it [00:00, ?it/s]

find model: ./insightface_func/models/antelope/glintr100.onnx recognition find model: ./insightface_func/models/antelope/scrfd_10g_bnkps.onnx detection set det-size: (640, 640) 0it [00:00, ?it/s]

and not make dataset, just make empty folders - imgs and latents.

Images from CelebA dataset (around 10.000) are in the root of the folder SimSwap-train/dataset/CelebA Lines in face_align.py was modified like in readme.
opened by netrunner-exe 1
Need help and advice from those who with sucsessful training SimSwap
I just need advice from people who know and are able to train SimSwap. Some of my questions will seem silly, but I ask you to treat them with understanding. I am currently working to launch a train on Google Colab. I managed to do this with -image_size 400 --display_winsize 400 (this is max), if 512 - cuda is out of memory. The training is in progress, img files are being created in the web/images folder.

How can I pause the training and continue from the same place in other session? If possible - please give a complete example of the command.

After the end of the training - if I understood correctly - are the files latest_net_D1.pth, latest_net_D2.pth, latest_net_G.pth created? If I understand correctly - is this my trained model?

Can I change the amount of End of epoch? It shows 20,000 for me. Example -> (End of epoch 4 / 20,000). May i set 400 or 500?

Unfortunately, I have not been involved in training models before, this is a new experience for me and I am trying to learn more and more in detail.

Perhaps someone who has already been trained in SimSwap will agree to write more detailed instructions on how to properly train a model, with all the stages and examples, this would greatly help the entire community.
opened by netrunner-exe 0
No such file

File "C:\ProgramData\Anaconda3\envs\simswap\lib\site-packages\torch\utils\data_utils\fetch.py", line 44, in fetch data = [self.dataset[idx] for idx in possibly_batched_index] File "C:\ProgramData\Anaconda3\envs\simswap\lib\site-packages\torch\utils\data_utils\fetch.py", line 44, in data = [self.dataset[idx] for idx in possibly_batched_index] File "C:\Users\Administrator\Desktop\simswap\data\CelebA_class.py", line 73, in getitem latent_id = np.load(latent_id_dir)[0] # 512 File "C:\ProgramData\Anaconda3\envs\simswap\lib\site-packages\numpy\lib\npyio.py", line 416, in load fid = stack.enter_context(open(os_fspath(file), "rb")) FileNotFoundError: [Errno 2] No such file or directory: './dataset/CelebA\latents/27270.npy'

opened by Rhenm091619 0
Asian-celeb dataset download link
[Asian-celeb dataset]

Training data(Asian-celeb)

The dataset consists of the crawled images of celebrities on the he web.The ima images are covered under a Creative Commons Attribution-NonCommercial 4.0 International license (Please read the license terms here. e. http://creativecommons.org/licenses/by-nc/4.0/).

[train_msra.tar.gz]

MD5:c5b668f2204c400099b14f367069aef5

Content: Train dataset called MS-Celeb-1M-v1c with 86,876 ids/3,923,399 aligned images cleaned from MS-Celeb-1M dataset.

This dataset has been excluded from both LFW and Asian-Celeb.

Format: *.jpg

Google: https://drive.google.com/file/d/1aaPdI0PkmQzRbWErazOgYtbLA1mwJIfK/view?usp=sharing

[msra_lmk.tar.gz]

MD5:7c053dd0462b4af243bb95b7b31da6e6

Content: A list of five-point landmarks for the 3,923,399 images in MS-Celeb-1M-v1c.

Format: .....

while is the path of images in tar file train_msceleb.tar.gz.

Label is an integer ranging from 0 to 86,875.

(x,y) is the coordinate of a key point on the aligned images.

left eye right eye nose tip mouth left mouth right

Google: https://drive.google.com/file/d/1FQ7P4ItyKCneNEvYfJhW2Kff7cOAFpgk/view?usp=sharing

[train_celebrity.tar.gz]

MD5:9f2e9858afb6c1032c4f9d7332a92064

Content: Train dataset called Asian-Celeb with 93,979 ids/2,830,146 aligned images.

This dataset has been excluded from both LFW and MS-Celeb-1M-v1c.

Format: *.jpg

Google: https://drive.google.com/file/d/1-p2UKlcX06MhRDJxJukSZKTz986Brk8N/view?usp=sharing

[celebrity_lmk.tar.gz]

MD5:9c0260c77c13fbb32692fc06a5dbfaf0

Content: A list of five-point landmarks for the 2,830,146 images in Asian-Celeb.

Format: .....

while is the path of images in tar file train_celebrity.tar.gz.

Label is an integer ranging from 86,876 to 196,319.

(x,y) is the coordinate of a key point on the aligned images.

left eye right eye nose tip mouth left mouth right

Google: https://drive.google.com/file/d/1sQVV9epoF_8jS3ge6DqbilpWk3UNE8U7/view?usp=sharing

[testdata.tar.gz]

MD5:f17c4712f7562ea6d45f0a158e59b792

Content: Test dataset with 1,862,120 aligned images.

Format: *.jpg

Google: https://drive.google.com/file/d/1ghzuEQqmUFN3nVujfrZfBx_CeGUpWzuw/view?usp=sharing

[testdata_lmk.tar]

MD5:7e4995eb9976a2cfd2b23db05d76572c

Content: A list of five-point landmarks for the 1,862,120 images in testdata.tar.gz.

Features should be extracted in the same sequence and with the same amount with this list.

Format: .....

while is the path of images in tar file testdata.tar.gz.

(x,y) is the coordinate of a key point on the aligned images.

left eye right eye nose tip mouth left mouth right

Google: https://drive.google.com/file/d/1lYzqnPyHXRVgXJYbEVh6zTXn3Wq4JO-I/view?usp=sharing

[feature_tools.tar.gz]

MD5:227b069d7a83aa43b0cb738c2252dbc4

Content: Feature format transform tool and a sample feature file.

Format: We use the same format as Megaface(http://megaface.cs.washington.edu/) except that we merge all files into a single binary file.

Google: https://drive.google.com/file/d/1bjZwOonyZ9KnxecuuTPVdY95mTIXMeuP/view?usp=sharing
opened by AmesianX 1
Dataset choices

Thanks for your contribution of SimSwap training code. I saw you choose to use CelebA as your training set, instead of VGGFace2 which is choosen by the author. Considering VGGFace2 has more images, I am just wondering if CelebA can achieve a better performance than VGGFace2. Do you have any opinion about this? Thanks!

opened by cenalzw 0

Dataset creation issue

Hi,

When using make_dataset.py on the CelebA dataset, most of my images are failing on the face detection part? I am passing the 512x512 directory. Any idea what might be wrong? The part failing is here:

    def get(self, img, crop_size, max_num=0):
        bboxes, kpss = self.det_model.detect(img,
                                             threshold=self.det_thresh,
                                             max_num=max_num,
                                             metric='default')
        if bboxes.shape[0] == 0:
            return None, None

bboxes is empty and returns None, None for most images.

Thanks!

opened by antonsanchez 2

Is this good results so far from training?
HI all,

Here is my setup:

46,000 ultra high quality female headshots I painstakingly collected from the web.

Here is a picture from Epoch #24 of my training:

Obviously some changes have occurred: eyesbrows are lighter, eyes are a bit more blue, and the nose is slightly modified.

My questions are:

How many epoch until I see good results?

What values of G_GAN, G_GAN_Feat, G_ID, G_Rec, D_GP, D_real, & D_fake should I be looked for?

Should I combine my custom set with CelebA? Would it produce better results?

Should I set --use_mask to False?

What other parameters should I change?

Thanks!
opened by antonsanchez 2

Reimplement of SimSwap training code

Related tags

Overview

SimSwap-train

Instructions

1.Environment Preparation

2.Making Training Data

3.Start Training

（1）New Training

（2）Finetuning

4.Training Result

（1）CelebA with 224x224 res

（2）CelebA with 512x512 res

Comments

Google: https://drive.google.com/file/d/1aaPdI0PkmQzRbWErazOgYtbLA1mwJIfK/view?usp=sharing

Google: https://drive.google.com/file/d/1FQ7P4ItyKCneNEvYfJhW2Kff7cOAFpgk/view?usp=sharing

Google: https://drive.google.com/file/d/1-p2UKlcX06MhRDJxJukSZKTz986Brk8N/view?usp=sharing

Google: https://drive.google.com/file/d/1sQVV9epoF_8jS3ge6DqbilpWk3UNE8U7/view?usp=sharing

Google: https://drive.google.com/file/d/1ghzuEQqmUFN3nVujfrZfBx_CeGUpWzuw/view?usp=sharing

Google: https://drive.google.com/file/d/1lYzqnPyHXRVgXJYbEVh6zTXn3Wq4JO-I/view?usp=sharing

Google: https://drive.google.com/file/d/1bjZwOonyZ9KnxecuuTPVdY95mTIXMeuP/view?usp=sharing

Owner

seeprettyface.com

The official project of SimSwap (ACM MM 2020)

This is the code for our KILT leaderboard submission to the T-REx and zsRE tasks. It includes code for training a DPR model then continuing training with RAG.

This is the official source code for SLATE. We provide the code for the model, the training code, and a dataset loader for the 3D Shapes dataset. This code is implemented in Pytorch.

Super-Fast-Adversarial-Training - A PyTorch Implementation code for developing super fast adversarial training

Ultra-Data-Efficient GAN Training: Drawing A Lottery Ticket First, Then Training It Toughly

Learning recognition/segmentation models without end-to-end training. 40%-60% less GPU memory footprint. Same training time. Better performance.

ActNN: Reducing Training Memory Footprint via 2-Bit Activation Compressed Training

BERT model training impelmentation using 1024 A100 GPUs for MLPerf Training v1.1

FuseDream: Training-Free Text-to-Image Generationwith Improved CLIP+GAN Space OptimizationFuseDream: Training-Free Text-to-Image Generationwith Improved CLIP+GAN Space Optimization

Reference implementation of code generation projects from Facebook AI Research. General toolkit to apply machine learning to code, from dataset creation to model training and evaluation. Comes with pretrained models.

Code for the prototype tool in our paper "CoProtector: Protect Open-Source Code against Unauthorized Training Usage with Data Poisoning".

Code to use Augmented Shapiro Wilks Stopping, as well as code for the paper "Statistically Signifigant Stopping of Neural Network Training"

Code for the paper "Training GANs with Stronger Augmentations via Contrastive Discriminator" (ICLR 2021)

Code for our paper at ECCV 2020: Post-Training Piecewise Linear Quantization for Deep Neural Networks

Code for the paper: Adversarial Training Against Location-Optimized Adversarial Patches. ECCV-W 2020.

Code for pre-training CharacterBERT models (as well as BERT models).

Provided is code that demonstrates the training and evaluation of the work presented in the paper: "On the Detection of Digital Face Manipulation" published in CVPR 2020.

Official code of our work, Unified Pre-training for Program Understanding and Generation [NAACL 2021].