StudioGAN is a Pytorch library providing implementations of representative Generative Adversarial Networks (GANs) for conditional/unconditional image generation.

Related tags

PyTorch-StudioGAN
Overview


StudioGAN is a Pytorch library providing implementations of representative Generative Adversarial Networks (GANs) for conditional/unconditional image generation. StudioGAN aims to offer an identical playground for modern GANs so that machine learning researchers can readily compare and analyze a new idea.

Features

  • Extensive GAN implementations for PyTorch
  • Comprehensive benchmark of GANs using CIFAR10, Tiny ImageNet, and ImageNet datasets
  • Better performance and lower memory consumption than original implementations
  • Providing pre-trained models that are fully compatible with up-to-date PyTorch environment
  • Support Multi-GPU (DP, DDP, and Multinode DistributedDataParallel), Mixed Precision, Synchronized Batch Normalization, LARS, Tensorboard Visualization, and other analysis methods

Implemented GANs

Name Venue Architecture G_type* D_type* Loss EMA**
DCGAN arXiv' 15 CNN/ResNet*** N/A N/A Vanilla False
LSGAN ICCV' 17 CNN/ResNet*** N/A N/A Least Sqaure False
GGAN arXiv' 17 CNN/ResNet*** N/A N/A Hinge False
WGAN-WC ICLR' 17 ResNet N/A N/A Wasserstein False
WGAN-GP NIPS' 17 ResNet N/A N/A Wasserstein False
WGAN-DRA arXiv' 17 ResNet N/A N/A Wasserstein False
ACGAN ICML' 17 ResNet cBN AC Hinge False
ProjGAN ICLR' 18 ResNet cBN PD Hinge False
SNGAN ICLR' 18 ResNet cBN PD Hinge False
SAGAN ICML' 19 ResNet cBN PD Hinge False
BigGAN ICLR' 18 Big ResNet cBN PD Hinge True
BigGAN-Deep ICLR' 18 Big ResNet Deep cBN PD Hinge True
CRGAN ICLR' 20 Big ResNet cBN PD/CL Hinge True
ICRGAN arXiv' 20 Big ResNet cBN PD/CL Hinge True
LOGAN arXiv' 19 Big ResNet cBN PD Hinge True
DiffAugGAN Neurips' 20 Big ResNet cBN PD/CL Hinge True
ADAGAN Neurips' 20 Big ResNet cBN PD/CL Hinge True
ContraGAN Neurips' 20 Big ResNet cBN CL Hinge True
FreezeD CVPRW' 20 - - - - -

*G/D_type indicates the way how we inject label information to the Generator or Discriminator. **EMA means applying an exponential moving average update to the generator. ***Experiments on Tiny ImageNet are conducted using the ResNet architecture instead of CNN.

cBN : conditional Batch Normalization. AC : Auxiliary Classifier. PD : Projection Discriminator. CL : Contrastive Learning.

To be Implemented

Name Venue Architecture G_type* D_type* Loss EMA**
StyleGAN2 CVPR' 20 StyleNet AdaIN - Vanilla True

AdaIN : Adaptive Instance Normalization.

Requirements

  • Anaconda
  • Python >= 3.6
  • 6.0.0 <= Pillow <= 7.0.0
  • scipy == 1.1.0 (Recommended for fast loading of Inception Network)
  • sklearn
  • seaborn
  • h5py
  • tqdm
  • torch >= 1.6.0 (Recommended for mixed precision training and knn analysis)
  • torchvision >= 0.7.0
  • tensorboard
  • 5.4.0 <= gcc <= 7.4.0 (Recommended for proper use of adaptive discriminator augmentation module)
  • torchlars (need to use LARS optimizer, can install by typing "pip install torchlars" in the command line)

You can install the recommended environment as follows:

conda env create -f environment.yml -n studiogan

With docker, you can use:

docker pull mgkang/studiogan:latest

This is my command to make a container named "studioGAN".

Also, you can use port number 6006 to connect the tensoreboard.

docker run -it --gpus all --shm-size 128g -p 6006:6006 --name studioGAN -v /home/USER:/root/code --workdir /root/code mgkang/studiogan:latest /bin/bash

Quick Start

  • Train (-t) and evaluate (-e) the model defined in CONFIG_PATH using GPU 0
CUDA_VISIBLE_DEVICES=0 python3 src/main.py -t -e -c CONFIG_PATH
  • Train (-t) and evaluate (-e) the model defined in CONFIG_PATH using GPUs (0, 1, 2, 3) and DataParallel
CUDA_VISIBLE_DEVICES=0,1,2,3 python3 src/main.py -t -e -c CONFIG_PATH

Try python3 src/main.py to see available options.

Via Tensorboard, you can monitor trends of IS, FID, F_beta, Authenticity Accuracies, and the largest singular values:

~ PyTorch-StudioGAN/logs/RUN_NAME>>> tensorboard --logdir=./ --port PORT

Dataset

  • CIFAR10: StudioGAN will automatically download the dataset once you execute main.py.

  • Tiny Imagenet, Imagenet, or a custom dataset:

    1. download Tiny Imagenet and Imagenet. Prepare your own dataset.
    2. make the folder structure of the dataset as follows:
┌── docs
├── src
└── data
    └── ILSVRC2012 or TINY_ILSVRC2012 or CUSTOM
        ├── train
        │   ├── cls0
        │   │   ├── train0.png
        │   │   ├── train1.png
        │   │   └── ...
        │   ├── cls1
        │   └── ...
        └── valid
            ├── cls0
            │   ├── valid0.png
            │   ├── valid1.png
            │   └── ...
            ├── cls1
            └── ...

Supported Training Techniques

  • DistributedDataParallel (Please refer to Here)
    ### NODE_0, 4_GPUs, All ports are open to NODE_1
    docker run -it --gpus all --shm-size 128g --name studioGAN --network=host -v /home/USER:/root/code --workdir /root/code mgkang/studiogan:latest /bin/bash
    
    ~/code>>> export NCCL_SOCKET_IFNAME=^docker0,lo
    ~/code>>> export MASTER_ADDR=PUBLIC_IP_OF_NODE_0
    ~/code>>> export MASTER_PORT=AVAILABLE_PORT_OF_NODE_0
    
    ~/code/PyTorch-StudioGAN>>> CUDA_VISIBLE_DEVICES=0,1,2,3 python3 src/main.py -t -e -DDP -n 2 -nr 0 -c CONFIG_PATH
    ### NODE_1, 4_GPUs, All ports are open to NODE_0
    docker run -it --gpus all --shm-size 128g --name studioGAN --network=host -v /home/USER:/root/code --workdir /root/code mgkang/studiogan:latest /bin/bash
    
    ~/code>>> export NCCL_SOCKET_IFNAME=^docker0,lo
    ~/code>>> export MASTER_ADDR=PUBLIC_IP_OF_NODE_0
    ~/code>>> export MASTER_PORT=AVAILABLE_PORT_OF_NODE_0
    
    ~/code/PyTorch-StudioGAN>>> CUDA_VISIBLE_DEVICES=0,1,2,3 python3 src/main.py -t -e -DDP -n 2 -nr 1 -c CONFIG_PATH

※ StudioGAN does not support DDP training for ContraGAN. This is because conducting contrastive learning requires a 'gather' operation to calculate the exact conditional contrastive loss.

  • Mixed Precision Training (Narang et al.)
    CUDA_VISIBLE_DEVICES=0,...,N python3 src/main.py -t -mpc -c CONFIG_PATH
  • Standing Statistics (Brock et al.)
    CUDA_VISIBLE_DEVICES=0,...,N python3 src/main.py -e -std_stat --standing_step STANDING_STEP -c CONFIG_PATH
  • Synchronized BatchNorm
    CUDA_VISIBLE_DEVICES=0,...,N python3 src/main.py -t -sync_bn -c CONFIG_PATH
  • Load All Data in Main Memory
    CUDA_VISIBLE_DEVICES=0,...,N python3 src/main.py -t -l -c CONFIG_PATH
  • LARS
    CUDA_VISIBLE_DEVICES=0,...,N python3 src/main.py -t -l -c CONFIG_PATH -LARS

To Visualize and Analyze Generated Images

The StudioGAN supports Image visualization, K-nearest neighbor analysis, Linear interpolation, and Frequency analysis. All results will be saved in ./figures/RUN_NAME/*.png.

  • Image Visualization
CUDA_VISIBLE_DEVICES=0,...,N python3 src/main.py -iv -std_stat --standing_step STANDING_STEP -c CONFIG_PATH --checkpoint_folder CHECKPOINT_FOLDER --log_output_path LOG_OUTPUT_PATH

  • K-Nearest Neighbor Analysis (we have fixed K=7, the images in the first column are generated images.)
CUDA_VISIBLE_DEVICES=0,...,N python3 src/main.py -knn -std_stat --standing_step STANDING_STEP -c CONFIG_PATH --checkpoint_folder CHECKPOINT_FOLDER --log_output_path LOG_OUTPUT_PATH

  • Linear Interpolation (applicable only to conditional Big ResNet models)
CUDA_VISIBLE_DEVICES=0,...,N python3 src/main.py -itp -std_stat --standing_step STANDING_STEP -c CONFIG_PATH --checkpoint_folder CHECKPOINT_FOLDER --log_output_path LOG_OUTPUT_PATH

  • Frequency Analysis
CUDA_VISIBLE_DEVICES=0,...,N python3 src/main.py -fa -std_stat --standing_step STANDING_STEP -c CONFIG_PATH --checkpoint_folder CHECKPOINT_FOLDER --log_output_path LOG_OUTPUT_PATH

  • TSNE Analysis
CUDA_VISIBLE_DEVICES=0,...,N python3 src/main.py -tsne -std_stat --standing_step STANDING_STEP -c CONFIG_PATH --checkpoint_folder CHECKPOINT_FOLDER --log_output_path LOG_OUTPUT_PATH

Metrics

Inception Score (IS)

Inception Score (IS) is a metric to measure how much GAN generates high-fidelity and diverse images. Calculating IS requires the pre-trained Inception-V3 network, and recent approaches utilize OpenAI's TensorFlow implementation.

To compute official IS, you have to make a "samples.npz" file using the command below:

CUDA_VISIBLE_DEVICES=0,...,N python3 src/main.py -s -c CONFIG_PATH --checkpoint_folder CHECKPOINT_FOLDER --log_output_path LOG_OUTPUT_PATH

It will automatically create the samples.npz file in the path ./samples/RUN_NAME/fake/npz/samples.npz. After that, execute TensorFlow official IS implementation. Note that we do not split a dataset into ten folds to calculate IS ten times. We use the entire dataset to compute IS only once, which is the evaluation strategy used in the CompareGAN repository.

CUDA_VISIBLE_DEVICES=0,...,N python3 src/inception_tf13.py --run_name RUN_NAME --type "fake"

Keep in mind that you need to have TensorFlow 1.3 or earlier version installed!

Note that StudioGAN logs Pytorch-based IS during the training.

Frechet Inception Distance (FID)

FID is a widely used metric to evaluate the performance of a GAN model. Calculating FID requires the pre-trained Inception-V3 network, and modern approaches use Tensorflow-based FID. StudioGAN utilizes the PyTorch-based FID to test GAN models in the same PyTorch environment. We show that the PyTorch based FID implementation provides almost the same results with the TensorFlow implementation (See Appendix F of our paper).

Precision and Recall (PR)

Precision measures how accurately the generator can learn the target distribution. Recall measures how completely the generator covers the target distribution. Like IS and FID, calculating Precision and Recall requires the pre-trained Inception-V3 model. StudioGAN uses the same hyperparameter settings with the original Precision and Recall implementation, and StudioGAN calculates the F-beta score suggested by Sajjadi et al.

Benchmark

※ We always welcome your contribution if you find any wrong implementation, bug, and misreported score.

We report the best IS, FID, and F_beta values of various GANs. B. S. means batch size for training.

CR, ICR, DiffAug, ADA, and LO refer to regularization or optimization techiniques: CR (Consistency Regularization), ICR (Improved Consistency Regularization), DiffAug (Differentiable Augmentation), ADA (Adaptive Discriminator Augmentation), and LO (Latent Optimization), respectively.

CIFAR10 (3x32x32)

When training, we used the command below.

CUDA_VISIBLE_DEVICES=0 python3 src/main.py -t -e -l -stat_otf -c CONFIG_PATH --eval_type "test"

With a single TITAN RTX GPU, training BigGAN takes about 13-15 hours.

Name B. S. IS(⭡) FID(⭣) F_1/8(⭡) F_8(⭡) Config Log Weights
DCGAN 64 6.638 49.030 0.833 0.795 Config Log Link
LSGAN 64 5.577 66.686 0.757 0.720 Config Log Link
GGAN 64 6.227 42.714 0.916 0.822 Config Log Link
WGAN-WC 64 2.579 159.090 0.190 0.199 Config Log Link
WGAN-GP 64 7.458 25.852 0.962 0.929 Config Log Link
WGAN-DRA 64 6.432 41.586 0.922 0.863 Config Log Link
ACGAN 64 6.629 45.571 0.857 0.847 Config Log Link
ProjGAN 64 7.539 33.830 0.952 0.855 Config Log Link
SNGAN 64 8.677 13.248 0.983 0.978 Config Log Link
SAGAN 64 8.680 14.009 0.982 0.970 Config Log Link
BigGAN 64 9.746 8.034 0.995 0.994 Config Log Link
BigGAN + CR 64 10.380 7.178 0.994 0.993 Config Log Link
BigGAN + ICR 64 10.153 7.430 0.994 0.993 Config Log Link
BigGAN + DiffAug 64 9.775 7.157 0.996 0.993 Config Log Link
BigGAN + ADA 64 10.136 7.881 0.993 0.994 Config Log Link
BigGAN + LO 64 9.701 8.369 0.992 0.989 Config Log Link
ContraGAN 64 9.729 8.065 0.993 0.992 Config Log Link
ContraGAN + CR 64 9.812 7.685 0.995 0.993 Config Log Link
ContraGAN + ICR 64 10.117 7.547 0.996 0.993 Config Log Link
ContraGAN + DiffAug 64 9.996 7.193 0.995 0.990 Config Log Link
ContraGAN + ADA 64 9.411 10.830 0.990 0.964 Config Log Link

※ IS, FID, and F_beta values are computed using 10K test and 10K generated Images.

※ When evaluating, the statistics of batch normalization layers are calculated on the fly (statistics of a batch).

CUDA_VISIBLE_DEVICES=0 python3 src/main.py -e -l -stat_otf -c CONFIG_PATH --checkpoint_folder CHECKPOINT_FOLDER --eval_type "test"

Tiny ImageNet (3x64x64)

When training, we used the command below.

CUDA_VISIBLE_DEVICES=0,...,N python3 src/main.py -t -e -l -stat_otf -c CONFIG_PATH --eval_type "valid"

With 4 TITAN RTX GPUs, training BigGAN takes about 2 days.

Name B. S. IS(⭡) FID(⭣) F_1/8(⭡) F_8(⭡) Config Log Weights
DCGAN 256 5.640 91.625 0.606 0.391 Config Log Link
LSGAN 256 5.381 90.008 0.638 0.390 Config Log Link
GGAN 256 5.146 102.094 0.503 0.307 Config Log Link
WGAN-WC 256 9.696 41.454 0.940 0.735 Config Log Link
WGAN-GP 256 1.322 311.805 0.016 0.000 Config Log Link
WGAN-DRA 256 9.564 40.655 0.938 0.724 Config Log Link
ACGAN 256 6.342 78.513 0.668 0.518 Config Log Link
ProjGAN 256 6.224 89.175 0.626 0.428 Config Log Link
SNGAN 256 8.412 53.590 0.900 0.703 Config Log Link
SAGAN 256 8.342 51.414 0.898 0.698 Config Log Link
BigGAN 1024 11.998 31.920 0.956 0.879 Config Log Link
BigGAN + CR 1024 14.887 21.488 0.969 0.936 Config Log Link
BigGAN + ICR 1024 5.605 91.326 0.525 0.399 Config Log Link
BigGAN + DiffAug 1024 17.075 16.338 0.979 0.971 Config Log Link
BigGAN + ADA 1024 15.158 24.121 0.953 0.942 Config Log Link
BigGAN + LO 256 6.964 70.660 0.857 0.621 Config Log Link
ContraGAN 1024 13.494 27.027 0.975 0.902 Config Log Link
ContraGAN + CR 1024 15.623 19.716 0.983 0.941 Config Log Link
ContraGAN + ICR 1024 15.830 21.940 0.980 0.944 Config Log Link
ContraGAN + DiffAug 1024 17.303 15.755 0.984 0.962 Config Log Link
ContraGAN + ADA 1024 8.398 55.025 0.878 0.677 Config Log Link

※ IS, FID, and F_beta values are computed using 50K validation and 50K generated Images.

※ When evaluating, the statistics of batch normalization layers are calculated on the fly (statistics of a batch).

CUDA_VISIBLE_DEVICES=0,...,N python3 src/main.py -e -l -stat_otf -c CONFIG_PATH --checkpoint_folder CHECKPOINT_FOLDER --eval_type "valid"

ImageNet (3x128x128)

When training, we used the command below.

CUDA_VISIBLE_DEVICES=0,...,N python3 src/main.py -t -e -l -sync_bn -stat_otf -c CONFIG_PATH --eval_type "valid"

With 8 TESLA V100 GPUs, training BigGAN2048 takes about a month.

Name B. S. IS(⭡) FID(⭣) F_1/8(⭡) F_8(⭡) Config Log Weights
SNGAN 256 32.247 26.792 0.938 0.913 Config Log Link
SAGAN 256 29.848 34.726 0.849 0.914 Config Log Link
BigGAN 256 28.633 24.684 0.941 0.921 Config Log Link
BigGAN 2048 99.705 7.893 0.985 0.989 Config Log Link
ContraGAN 256 25.249 25.161 0.947 0.855 Config Log Link

※ IS, FID, and F_beta values are computed using 50K validation and 50K generated Images.

※ When evaluating, the statistics of batch normalization layers are calculated in advance (moving average of the previous statistics).

CUDA_VISIBLE_DEVICES=0,...,N python3 src/main.py -e -l -sync_bn -c CONFIG_PATH --checkpoint_folder CHECKPOINT_FOLDER --eval_type "valid"

References

[1] Exponential Moving Average: https://github.com/ajbrock/BigGAN-PyTorch

[2] Synchronized BatchNorm: https://github.com/vacancy/Synchronized-BatchNorm-PyTorch

[3] Self-Attention module: https://github.com/voletiv/self-attention-GAN-pytorch

[4] Implementation Details: https://github.com/ajbrock/BigGAN-PyTorch

[5] Architecture Details: https://github.com/google/compare_gan

[6] DiffAugment: https://github.com/mit-han-lab/data-efficient-gans

[7] Adaptive Discriminator Augmentation: https://github.com/rosinality/stylegan2-pytorch

[8] Tensorflow IS: https://github.com/openai/improved-gan

[9] Tensorflow FID: https://github.com/bioinf-jku/TTUR

[10] Pytorch FID: https://github.com/mseitzer/pytorch-fid

[11] Tensorflow Precision and Recall: https://github.com/msmsajjadi/precision-recall-distributions

[12] torchlars: https://github.com/kakaobrain/torchlars

Citation

StudioGAN is established for the following research project. Please cite our work if you use StudioGAN.

@inproceedings{kang2020ContraGAN,
  title   = {{ContraGAN: Contrastive Learning for Conditional Image Generation}},
  author  = {Minguk Kang and Jaesik Park},
  journal = {Conference on Neural Information Processing Systems (NeurIPS)},
  year    = {2020}
}
Issues
  • confuse about result

    confuse about result

    Hi, thank you for your sharing the good repo for reimplement for GAN.

    I run the command : python main.py --eval -t -c "./configs/Table2/biggan32_cifar_hinge_no.json" the result is

    model=G-step=80000-Inception_mean=9.465-Inception_std=0.00551-FID=10.344.pth

    but when I use contraGAN with the same dataset

    python main.py --eval -t -c "./configs/Table2/contragan32_cifar_hinge_no.json"

    the result is:

    Current best model (FID) is ..... step=78000-Inception_mean=8.89-Inception_std=0.1667-FID=12.648

    when contraGAN is not better than biggan?

    opened by Johnson-yue 14
  • truncation trick

    truncation trick

    Dear author,

    I am reading your implementation on latent sampling from sample.py (function: sample_latents). For a gaussian sampling as implemented by latents = torch.randn(batch_size, dim, device=device)/truncated_factor.

    I notice that the above implementation is not a standard truncation trick, which is defined by

    The Truncation Trick is a latent sampling procedure for generative adversarial networks, where we sample from a truncated normal (where values which fall outside a range are resampled to fall inside that range)

    opened by lihuiknight 11
  • bug report: function latent_optimise in losses.py

    bug report: function latent_optimise in losses.py

    the transport_cost is not defined before it has been used in line

    opened by lihuiknight 11
  • ContraGAN no improvement at all

    ContraGAN no improvement at all

    I am not getting any improvement from contraGAN at all. For the main contribution, the FID improvement is not even 0.1. If i rerun FID score, I get worse score many times. Same for IS. other metrics not improving at all, as your table show. Including other method like diffaug is not fair as biggan can benefit from those method too.

    fid score can change a lot more than 0.1, sometimes more than 1 point if you run it many times. how many times do you train the model or get the fid score?

    for imagenet 128x128, ContraGAN score is even worse than bigGAN baseline. Since contragan is build on biggan, the result is no improvement or worse performance.

    I want to use your work but it is hard to believe the score now, can you help explain? thanks

    opened by curiousbyte19 11
  • Can I train GANs with this repository on CIFAR-100?

    Can I train GANs with this repository on CIFAR-100?

    Thanks for the great GAN repo! I want to generate some images on CIFAR-100, and how can I use the repo to achieve this?

    opened by LeavesLei 7
  • GPU util is not 100%

    GPU util is not 100%

    Hi, thank you for a great repository!

    I'm trying to train SNGAN on my custom dataset (128x128) in 4 GPU, but found that the GPU utils are ~90%. I also increased num_workers=16 and applied -rm_API but it is still not addressed. Do you have any suspicion on this issue?

    By the way, even BigGAN_256 raised out-of-memory (OOM) error for 4 GPUs of 12G memory. Do I need 8 GPU server or distributed training for BigGAN_256?

    Thank you for your help!

    p.s. typo: load_frameowrk -> load_framework

    opened by sangwoomo 7
  • Reproducing BigGAN256 results

    Reproducing BigGAN256 results

    Hi Mniguk,

    Recently, I want to reproduce results of ProjGAN on ImageNet with config file src/configs/ILSVRC2012/BigGAN256.json. I use the command

    python src/main.py -t -e -sync_bn -c src/configs/ILSVRC2012/BigGAN256.json --eval_type "valid"
    

    From logging file logs/IMAGENET/BigGAN256-train-2021_01_24_03_52_15.log the FID at iteration 44000 is 46.32, however, I get 43.87 when I run it. The difference is not negligible. I train BigGAN256 on 4 A100 gpus. I am wondering is this normal behavior? or is there anything I potentially did wrong?

    The only differences of my experiment setting from using the command in README file are:

    1. I use CenterCrop during training, but I think that would only make FID score worse.
    2. I did not use -l option to save loading time.
    3. -stat_otf flag is also off since it contradicts sync_bn.
    4. I wrapped valid folder in another one since ILSVRC2012's valid folder is not organized in subfolders and causes problem in ImageFolder.

    Thanks a lot in advance!

    opened by phymhan 6
  • docker issue

    docker issue

    Hey, when I try to run your docker command, I get this:

    Error response from daemon: manifest for mgkang/studiogan:latest not found: manifest unknown: manifest unknown

    opened by jlim13 6
  • Can not reproduce the result on ImageNet with ContraGAN256

    Can not reproduce the result on ImageNet with ContraGAN256

    Hi,

    Recently, I want to reproduce the results on the imagenet with the config of **src/configs/ILSVRC2012/ContraGAN256.json** . The command is: 
    

    python src/main.py -t -e -sync_bn -c src/configs/ILSVRC2012/ContraGAN256.json --eval_type 'valid' --save_every 10000

    Some running environment:

    Pytorch=1.4.0 scikit-learn=0.19.1 tensorflow-gpu=1.15.0

    The code is running on 4 V100 GPUS, and each GPU has 32G memories. However, the corresponding FID score is 60.38. The reported numebr is 25.161.

    Are there any problems with my experiments, or the command?

    logs>
    
    [INFO] 2021-02-20 21:08:35 > Run name : ContraGAN256-train-2021_02_20_21_08_34
    [INFO] 2021-02-20 21:08:35 > {'config_path': 'src/configs/ILSVRC2012/ContraGAN256.json', 'checkpoint_folder': None, 'load_current': False, 'log_output_path': None, 'seed': -1, 'distributed_data_parallel': False, 'num_workers': 8, 'synchronized_bn': True, 'mixed_precision': False, 'LARS_optimizer': False, 'disable_debugging_API': False, 'reduce_train_dataset': 1.0, 'standing_statistics': False, 'standing_step': -1, 'freeze_layers': -1, 'load_all_data_in_memory': False, 'train': True, 'eval': True, 'save_images': False, 'image_visualization': False, 'k_nearest_neighbor': False, 'interpolation': False, 'frequency_analysis': False, 'tsne_analysis': False, 'nrow': 10, 'ncol': 8, 'print_every': 100, 'save_every': 10000, 'eval_type': 'valid'}
    [INFO] 2021-02-20 21:08:35 > {'data_processing': {'dataset_name': 'imagenet', 'data_path': './data/ILSVRC2012', 'img_size': 128, 'num_classes': 1000, 'batch_size4prcsing': 256, 'chunk_size': 500, 'compression': False}, 'train': {'model': {'architecture': 'big_resnet', 'conditional_strategy': 'ContraGAN', 'pos_collected_numerator': False, 'hypersphere_dim': 1536, 'nonlinear_embed': False, 'normalize_embed': True, 'g_spectral_norm': True, 'd_spectral_norm': True, 'activation_fn': 'ReLU', 'attention': True, 'attention_after_nth_gen_block': 4, 'attention_after_nth_dis_block': 1, 'z_dim': 120, 'shared_dim': 128, 'g_conv_dim': 96, 'd_conv_dim': 96, 'G_depth': 'N/A', 'D_depth': 'N/A'}, 'optimization': {'optimizer': 'Adam', 'batch_size': 256, 'accumulation_steps': 1, 'd_lr': 0.0002, 'g_lr': 5e-05, 'momentum': 'N/A', 'nesterov': 'N/A', 'alpha': 'N/A', 'beta1': 0.0, 'beta2': 0.999, 'g_steps_per_iter': 1, 'd_steps_per_iter': 2, 'total_step': 200000}, 'loss_function': {'adv_loss': 'hinge', 'contrastive_lambda': 1.0, 'margin': 0.0, 'tempering_type': 'constant', 'tempering_step': 'N/A', 'start_temperature': 1.0, 'end_temperature': 1.0, 'weight_clipping_for_dis': False, 'weight_clipping_bound': 'N/A', 'gradient_penalty_for_dis': False, 'gradient_penalty_lambda': 'N/A', 'deep_regret_analysis_for_dis': False, 'regret_penalty_lambda': 'N/A', 'cr': False, 'cr_lambda': 'N/A', 'bcr': False, 'real_lambda': 'N/A', 'fake_lambda': 'N/A', 'zcr': False, 'gen_lambda': 'N/A', 'dis_lambda': 'N/A', 'sigma_noise': 'N/A'}, 'initialization': {'g_init': 'ortho', 'd_init': 'ortho'}, 'training_and_sampling_setting': {'random_flip_preprocessing': True, 'diff_aug': False, 'ada': False, 'ada_target': 'N/A', 'ada_length': 'N/A', 'prior': 'gaussian', 'truncated_factor': 1, 'latent_op': False, 'latent_op_rate': 'N/A', 'latent_op_step': 'N/A', 'latent_op_step4eval': 'N/A', 'latent_op_alpha': 'N/A', 'latent_op_beta': 'N/A', 'latent_norm_reg_weight': 'N/A', 'ema': True, 'ema_decay': 0.9999, 'ema_start': 20000}}}
    [INFO] 2021-02-20 21:08:35 > Load train datasets...
    [INFO] 2021-02-20 21:08:40 > Train dataset size : 1281167
    [INFO] 2021-02-20 21:08:40 > Load valid datasets...
    [INFO] 2021-02-20 21:08:41 > Eval dataset size : 50000
    [INFO] 2021-02-20 21:08:41 > Build model...
    [INFO] 2021-02-20 21:08:41 > Modules are located on models.big_resnet.
    [INFO] 2021-02-20 21:09:12 > Prepare EMA for G with decay of 0.9999.
    [INFO] 2021-02-20 21:09:13 > Number of parameters: 70433988
    [INFO] 2021-02-20 21:09:13 > Generator(
      (linear0): Linear(in_features=20, out_features=24576, bias=True)
      (shared): Embedding(1000, 128)
      (blocks): ModuleList(
        (0): ModuleList(
          (0): GenBlock(
            (bn1): ConditionalBatchNorm2d_for_skip_and_shared(
              (bn): BatchNorm2d(1536, eps=0.0001, momentum=0.1, affine=False, track_running_stats=True)
              (gain): Linear(in_features=148, out_features=1536, bias=False)
              (bias): Linear(in_features=148, out_features=1536, bias=False)
            )
            (bn2): ConditionalBatchNorm2d_for_skip_and_shared(
              (bn): BatchNorm2d(1536, eps=0.0001, momentum=0.1, affine=False, track_running_stats=True)
              (gain): Linear(in_features=148, out_features=1536, bias=False)
              (bias): Linear(in_features=148, out_features=1536, bias=False)
            )
            (activation): ReLU(inplace=True)
            (conv2d0): Conv2d(1536, 1536, kernel_size=(1, 1), stride=(1, 1))
            (conv2d1): Conv2d(1536, 1536, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
            (conv2d2): Conv2d(1536, 1536, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
          )
        )
        (1): ModuleList(
          (0): GenBlock(
            (bn1): ConditionalBatchNorm2d_for_skip_and_shared(
              (bn): BatchNorm2d(1536, eps=0.0001, momentum=0.1, affine=False, track_running_stats=True)
              (gain): Linear(in_features=148, out_features=1536, bias=False)
              (bias): Linear(in_features=148, out_features=1536, bias=False)
            )
            (bn2): ConditionalBatchNorm2d_for_skip_and_shared(
              (bn): BatchNorm2d(768, eps=0.0001, momentum=0.1, affine=False, track_running_stats=True)
              (gain): Linear(in_features=148, out_features=768, bias=False)
              (bias): Linear(in_features=148, out_features=768, bias=False)
            )
            (activation): ReLU(inplace=True)
            (conv2d0): Conv2d(1536, 768, kernel_size=(1, 1), stride=(1, 1))
            (conv2d1): Conv2d(1536, 768, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
            (conv2d2): Conv2d(768, 768, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
          )
        )
        (2): ModuleList(
          (0): GenBlock(
            (bn1): ConditionalBatchNorm2d_for_skip_and_shared(
              (bn): BatchNorm2d(768, eps=0.0001, momentum=0.1, affine=False, track_running_stats=True)
              (gain): Linear(in_features=148, out_features=768, bias=False)
              (bias): Linear(in_features=148, out_features=768, bias=False)
            )
            (bn2): ConditionalBatchNorm2d_for_skip_and_shared(
              (bn): BatchNorm2d(384, eps=0.0001, momentum=0.1, affine=False, track_running_stats=True)
              (gain): Linear(in_features=148, out_features=384, bias=False)
              (bias): Linear(in_features=148, out_features=384, bias=False)
            )
            (activation): ReLU(inplace=True)
            (conv2d0): Conv2d(768, 384, kernel_size=(1, 1), stride=(1, 1))
            (conv2d1): Conv2d(768, 384, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
            (conv2d2): Conv2d(384, 384, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
          )
        )
        (3): ModuleList(
          (0): GenBlock(
            (bn1): ConditionalBatchNorm2d_for_skip_and_shared(
              (bn): BatchNorm2d(384, eps=0.0001, momentum=0.1, affine=False, track_running_stats=True)
              (gain): Linear(in_features=148, out_features=384, bias=False)
              (bias): Linear(in_features=148, out_features=384, bias=False)
            )
            (bn2): ConditionalBatchNorm2d_for_skip_and_shared(
              (bn): BatchNorm2d(192, eps=0.0001, momentum=0.1, affine=False, track_running_stats=True)
              (gain): Linear(in_features=148, out_features=192, bias=False)
              (bias): Linear(in_features=148, out_features=192, bias=False)
            )
            (activation): ReLU(inplace=True)
            (conv2d0): Conv2d(384, 192, kernel_size=(1, 1), stride=(1, 1))
            (conv2d1): Conv2d(384, 192, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
            (conv2d2): Conv2d(192, 192, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
          )
        )
        (4): ModuleList(
          (0): Self_Attn(
            (conv1x1_theta): Conv2d(192, 24, kernel_size=(1, 1), stride=(1, 1), bias=False)
            (conv1x1_phi): Conv2d(192, 24, kernel_size=(1, 1), stride=(1, 1), bias=False)
            (conv1x1_g): Conv2d(192, 96, kernel_size=(1, 1), stride=(1, 1), bias=False)
            (conv1x1_attn): Conv2d(96, 192, kernel_size=(1, 1), stride=(1, 1), bias=False)
            (maxpool): MaxPool2d(kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)
            (softmax): Softmax(dim=-1)
          )
        )
        (5): ModuleList(
          (0): GenBlock(
            (bn1): ConditionalBatchNorm2d_for_skip_and_shared(
              (bn): BatchNorm2d(192, eps=0.0001, momentum=0.1, affine=False, track_running_stats=True)
              (gain): Linear(in_features=148, out_features=192, bias=False)
              (bias): Linear(in_features=148, out_features=192, bias=False)
            )
            (bn2): ConditionalBatchNorm2d_for_skip_and_shared(
              (bn): BatchNorm2d(96, eps=0.0001, momentum=0.1, affine=False, track_running_stats=True)
              (gain): Linear(in_features=148, out_features=96, bias=False)
              (bias): Linear(in_features=148, out_features=96, bias=False)
            )
            (activation): ReLU(inplace=True)
            (conv2d0): Conv2d(192, 96, kernel_size=(1, 1), stride=(1, 1))
            (conv2d1): Conv2d(192, 96, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
            (conv2d2): Conv2d(96, 96, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
          )
        )
      )
      (bn4): BatchNorm2d(96, eps=0.0001, momentum=0.1, affine=True, track_running_stats=True)
      (activation): ReLU(inplace=True)
      (conv2d5): Conv2d(96, 3, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
      (tanh): Tanh()
    )
    [INFO] 2021-02-20 21:09:13 > Number of parameters: 90343202
    [INFO] 2021-02-20 21:09:13 > Discriminator(
      (blocks): ModuleList(
        (0): ModuleList(
          (0): DiscOptBlock(
            (conv2d0): Conv2d(3, 96, kernel_size=(1, 1), stride=(1, 1))
            (conv2d1): Conv2d(3, 96, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
            (conv2d2): Conv2d(96, 96, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
            (activation): ReLU(inplace=True)
            (average_pooling): AvgPool2d(kernel_size=2, stride=2, padding=0)
          )
        )
        (1): ModuleList(
          (0): Self_Attn(
            (conv1x1_theta): Conv2d(96, 12, kernel_size=(1, 1), stride=(1, 1), bias=False)
            (conv1x1_phi): Conv2d(96, 12, kernel_size=(1, 1), stride=(1, 1), bias=False)
            (conv1x1_g): Conv2d(96, 48, kernel_size=(1, 1), stride=(1, 1), bias=False)
            (conv1x1_attn): Conv2d(48, 96, kernel_size=(1, 1), stride=(1, 1), bias=False)
            (maxpool): MaxPool2d(kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)
            (softmax): Softmax(dim=-1)
          )
        )
        (2): ModuleList(
          (0): DiscBlock(
            (activation): ReLU(inplace=True)
            (conv2d0): Conv2d(96, 192, kernel_size=(1, 1), stride=(1, 1))
            (conv2d1): Conv2d(96, 192, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
            (conv2d2): Conv2d(192, 192, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
            (average_pooling): AvgPool2d(kernel_size=2, stride=2, padding=0)
          )
        )
        (3): ModuleList(
          (0): DiscBlock(
            (activation): ReLU(inplace=True)
            (conv2d0): Conv2d(192, 384, kernel_size=(1, 1), stride=(1, 1))
            (conv2d1): Conv2d(192, 384, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
            (conv2d2): Conv2d(384, 384, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
            (average_pooling): AvgPool2d(kernel_size=2, stride=2, padding=0)
          )
        )
        (4): ModuleList(
          (0): DiscBlock(
            (activation): ReLU(inplace=True)
            (conv2d0): Conv2d(384, 768, kernel_size=(1, 1), stride=(1, 1))
            (conv2d1): Conv2d(384, 768, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
            (conv2d2): Conv2d(768, 768, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
            (average_pooling): AvgPool2d(kernel_size=2, stride=2, padding=0)
          )
        )
        (5): ModuleList(
          (0): DiscBlock(
            (activation): ReLU(inplace=True)
            (conv2d0): Conv2d(768, 1536, kernel_size=(1, 1), stride=(1, 1))
            (conv2d1): Conv2d(768, 1536, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
            (conv2d2): Conv2d(1536, 1536, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
            (average_pooling): AvgPool2d(kernel_size=2, stride=2, padding=0)
          )
        )
        (6): ModuleList(
          (0): DiscBlock(
            (activation): ReLU(inplace=True)
            (conv2d1): Conv2d(1536, 1536, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
            (conv2d2): Conv2d(1536, 1536, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
            (average_pooling): AvgPool2d(kernel_size=2, stride=2, padding=0)
          )
        )
      )
      (activation): ReLU(inplace=True)
      (linear1): Linear(in_features=1536, out_features=1, bias=True)
      (linear2): Linear(in_features=1536, out_features=1536, bias=True)
      (embedding): Embedding(1000, 1536)
    )
    [INFO] 2021-02-20 21:13:20 > Calculate moments of valid dataset
    [INFO] 2021-02-20 21:14:33 > Save calculated means and covariances to disk...
    [INFO] 2021-02-20 21:14:33 > calculate inception score of valid dataset.
    [INFO] 2021-02-20 21:15:30 > Inception score=173.24171447753906-Inception_std=nan
    [INFO] 2021-02-20 21:15:30 > Start training....
    [INFO] 2021-02-20 21:19:36 > Step:     100 Progress: 0.1% Elapsed: 0:04:06 temperature: 1.0 ada_p: No Discriminator_loss: 7.47376 Generator_loss: 5.68739 
    [INFO] 2021-02-20 21:23:23 > Step:     200 Progress: 0.1% Elapsed: 0:07:53 temperature: 1.0 ada_p: No Discriminator_loss: 7.06227 Generator_loss: 5.19838 
    [INFO] 2021-02-20 21:27:11 > Step:     300 Progress: 0.1% Elapsed: 0:11:41 temperature: 1.0 ada_p: No Discriminator_loss: 7.02717 Generator_loss: 6.363 
    [INFO] 2021-02-20 21:30:58 > Step:     400 Progress: 0.2% Elapsed: 0:15:28 temperature: 1.0 ada_p: No Discriminator_loss: 7.30094 Generator_loss: 5.90407 
    [INFO] 2021-02-20 21:34:46 > Step:     500 Progress: 0.2% Elapsed: 0:19:16 temperature: 1.0 ada_p: No Discriminator_loss: 7.08733 Generator_loss: 5.71347 
    [INFO] 2021-02-20 21:38:34 > Step:     600 Progress: 0.3% Elapsed: 0:23:04 temperature: 1.0 ada_p: No Discriminator_loss: 6.88163 Generator_loss: 6.63448 
    [INFO] 2021-02-20 21:42:22 > Step:     700 Progress: 0.4% Elapsed: 0:26:52 temperature: 1.0 ada_p: No Discriminator_loss: 7.0539 Generator_loss: 6.41777 
    [INFO] 2021-02-20 21:46:09 > Step:     800 Progress: 0.4% Elapsed: 0:30:39 temperature: 1.0 ada_p: No Discriminator_loss: 6.49477 Generator_loss: 4.61053 
    [INFO] 2021-02-20 21:49:57 > Step:     900 Progress: 0.4% Elapsed: 0:34:27 temperature: 1.0 ada_p: No Discriminator_loss: 6.55702 Generator_loss: 6.68302 
    [INFO] 2021-02-20 21:53:45 > Step:    1000 Progress: 0.5% Elapsed: 0:38:15 temperature: 1.0 ada_p: No Discriminator_loss: 6.66745 Generator_loss: 6.78327 
    [INFO] 2021-02-20 21:57:32 > Step:    1100 Progress: 0.5% Elapsed: 0:42:02 temperature: 1.0 ada_p: No Discriminator_loss: 6.68186 Generator_loss: 5.39558 
    [INFO] 2021-02-20 22:01:20 > Step:    1200 Progress: 0.6% Elapsed: 0:45:50 temperature: 1.0 ada_p: No Discriminator_loss: 6.39857 Generator_loss: 5.21004 
    [INFO] 2021-02-20 22:05:07 > Step:    1300 Progress: 0.7% Elapsed: 0:49:37 temperature: 1.0 ada_p: No Discriminator_loss: 6.12846 Generator_loss: 5.68461 
    [INFO] 2021-02-20 22:08:55 > Step:    1400 Progress: 0.7% Elapsed: 0:53:25 temperature: 1.0 ada_p: No Discriminator_loss: 6.20334 Generator_loss: 5.6225 
    [INFO] 2021-02-20 22:12:43 > Step:    1500 Progress: 0.8% Elapsed: 0:57:13 temperature: 1.0 ada_p: No Discriminator_loss: 6.16044 Generator_loss: 5.232 
    [INFO] 2021-02-20 22:16:31 > Step:    1600 Progress: 0.8% Elapsed: 1:01:01 temperature: 1.0 ada_p: No Discriminator_loss: 6.02397 Generator_loss: 4.82499 
    [INFO] 2021-02-20 22:20:19 > Step:    1700 Progress: 0.9% Elapsed: 1:04:49 temperature: 1.0 ada_p: No Discriminator_loss: 6.43629 Generator_loss: 5.86295 
    [INFO] 2021-02-20 22:24:07 > Step:    1800 Progress: 0.9% Elapsed: 1:08:37 temperature: 1.0 ada_p: No Discriminator_loss: 6.3647 Generator_loss: 5.57718 
    [INFO] 2021-02-20 22:27:54 > Step:    1900 Progress: 0.9% Elapsed: 1:12:24 temperature: 1.0 ada_p: No Discriminator_loss: 6.2285 Generator_loss: 5.21836 
    [INFO] 2021-02-20 22:31:42 > Step:    2000 Progress: 1.0% Elapsed: 1:16:12 temperature: 1.0 ada_p: No Discriminator_loss: 6.13226 Generator_loss: 5.1835 
    [INFO] 2021-02-20 22:35:30 > Step:    2100 Progress: 1.1% Elapsed: 1:20:00 temperature: 1.0 ada_p: No Discriminator_loss: 6.295 Generator_loss: 5.24376 
    [INFO] 2021-02-20 22:39:18 > Step:    2200 Progress: 1.1% Elapsed: 1:23:48 temperature: 1.0 ada_p: No Discriminator_loss: 6.17141 Generator_loss: 5.57421 
    [INFO] 2021-02-20 22:43:06 > Step:    2300 Progress: 1.1% Elapsed: 1:27:36 temperature: 1.0 ada_p: No Discriminator_loss: 6.06213 Generator_loss: 5.75431 
    [INFO] 2021-02-20 22:46:54 > Step:    2400 Progress: 1.2% Elapsed: 1:31:24 temperature: 1.0 ada_p: No Discriminator_loss: 6.20347 Generator_loss: 5.13625 
    [INFO] 2021-02-20 22:50:42 > Step:    2500 Progress: 1.2% Elapsed: 1:35:12 temperature: 1.0 ada_p: No Discriminator_loss: 5.99916 Generator_loss: 4.948 
    [INFO] 2021-02-20 22:54:34 > Step:    2600 Progress: 1.3% Elapsed: 1:39:04 temperature: 1.0 ada_p: No Discriminator_loss: 6.16895 Generator_loss: 5.17007 
    [INFO] 2021-02-20 22:58:22 > Step:    2700 Progress: 1.4% Elapsed: 1:42:52 temperature: 1.0 ada_p: No Discriminator_loss: 6.12119 Generator_loss: 5.08095 
    [INFO] 2021-02-20 23:02:10 > Step:    2800 Progress: 1.4% Elapsed: 1:46:40 temperature: 1.0 ada_p: No Discriminator_loss: 6.09806 Generator_loss: 5.81261 
    [INFO] 2021-02-20 23:05:58 > Step:    2900 Progress: 1.5% Elapsed: 1:50:28 temperature: 1.0 ada_p: No Discriminator_loss: 6.15333 Generator_loss: 5.27119 
    [INFO] 2021-02-20 23:09:46 > Step:    3000 Progress: 1.5% Elapsed: 1:54:16 temperature: 1.0 ada_p: No Discriminator_loss: 6.20096 Generator_loss: 5.62279 
    [INFO] 2021-02-20 23:13:35 > Step:    3100 Progress: 1.6% Elapsed: 1:58:05 temperature: 1.0 ada_p: No Discriminator_loss: 6.05894 Generator_loss: 5.40782 
    [INFO] 2021-02-20 23:17:23 > Step:    3200 Progress: 1.6% Elapsed: 2:01:53 temperature: 1.0 ada_p: No Discriminator_loss: 6.0715 Generator_loss: 5.5504 
    [INFO] 2021-02-20 23:21:11 > Step:    3300 Progress: 1.7% Elapsed: 2:05:41 temperature: 1.0 ada_p: No Discriminator_loss: 5.71572 Generator_loss: 5.81254 
    [INFO] 2021-02-20 23:25:00 > Step:    3400 Progress: 1.7% Elapsed: 2:09:29 temperature: 1.0 ada_p: No Discriminator_loss: 6.05644 Generator_loss: 5.45611 
    [INFO] 2021-02-20 23:28:48 > Step:    3500 Progress: 1.8% Elapsed: 2:13:18 temperature: 1.0 ada_p: No Discriminator_loss: 5.74054 Generator_loss: 5.4326 
    [INFO] 2021-02-20 23:32:36 > Step:    3600 Progress: 1.8% Elapsed: 2:17:06 temperature: 1.0 ada_p: No Discriminator_loss: 5.70841 Generator_loss: 5.55402 
    [INFO] 2021-02-20 23:36:24 > Step:    3700 Progress: 1.8% Elapsed: 2:20:54 temperature: 1.0 ada_p: No Discriminator_loss: 6.15396 Generator_loss: 5.44167 
    [INFO] 2021-02-20 23:40:12 > Step:    3800 Progress: 1.9% Elapsed: 2:24:42 temperature: 1.0 ada_p: No Discriminator_loss: 5.9868 Generator_loss: 5.8448 
    [INFO] 2021-02-20 23:44:00 > Step:    3900 Progress: 1.9% Elapsed: 2:28:30 temperature: 1.0 ada_p: No Discriminator_loss: 6.05152 Generator_loss: 5.64723 
    [INFO] 2021-02-20 23:47:48 > Step:    4000 Progress: 2.0% Elapsed: 2:32:18 temperature: 1.0 ada_p: No Discriminator_loss: 5.57659 Generator_loss: 5.62673 
    [INFO] 2021-02-20 23:51:37 > Step:    4100 Progress: 2.1% Elapsed: 2:36:06 temperature: 1.0 ada_p: No Discriminator_loss: 5.6117 Generator_loss: 5.64639 
    [INFO] 2021-02-20 23:55:25 > Step:    4200 Progress: 2.1% Elapsed: 2:39:55 temperature: 1.0 ada_p: No Discriminator_loss: 5.56909 Generator_loss: 5.71393 
    [INFO] 2021-02-20 23:59:13 > Step:    4300 Progress: 2.1% Elapsed: 2:43:43 temperature: 1.0 ada_p: No Discriminator_loss: 5.82648 Generator_loss: 5.44368 
    [INFO] 2021-02-21 00:03:02 > Step:    4400 Progress: 2.2% Elapsed: 2:47:31 temperature: 1.0 ada_p: No Discriminator_loss: 5.62373 Generator_loss: 5.5879 
    [INFO] 2021-02-21 00:06:51 > Step:    4500 Progress: 2.2% Elapsed: 2:51:21 temperature: 1.0 ada_p: No Discriminator_loss: 5.80013 Generator_loss: 5.53241 
    [INFO] 2021-02-21 00:10:39 > Step:    4600 Progress: 2.3% Elapsed: 2:55:09 temperature: 1.0 ada_p: No Discriminator_loss: 5.4291 Generator_loss: 6.23477 
    [INFO] 2021-02-21 00:14:27 > Step:    4700 Progress: 2.4% Elapsed: 2:58:57 temperature: 1.0 ada_p: No Discriminator_loss: 5.69974 Generator_loss: 5.74164 
    [INFO] 2021-02-21 00:18:16 > Step:    4800 Progress: 2.4% Elapsed: 3:02:46 temperature: 1.0 ada_p: No Discriminator_loss: 5.59367 Generator_loss: 5.84235 
    [INFO] 2021-02-21 00:22:05 > Step:    4900 Progress: 2.5% Elapsed: 3:06:35 temperature: 1.0 ada_p: No Discriminator_loss: 5.79493 Generator_loss: 5.76193 
    [INFO] 2021-02-21 00:25:53 > Step:    5000 Progress: 2.5% Elapsed: 3:10:23 temperature: 1.0 ada_p: No Discriminator_loss: 5.66132 Generator_loss: 5.79669 
    [INFO] 2021-02-21 00:29:45 > Step:    5100 Progress: 2.5% Elapsed: 3:14:15 temperature: 1.0 ada_p: No Discriminator_loss: 5.67316 Generator_loss: 5.68072 
    [INFO] 2021-02-21 00:33:33 > Step:    5200 Progress: 2.6% Elapsed: 3:18:03 temperature: 1.0 ada_p: No Discriminator_loss: 5.64375 Generator_loss: 5.80585 
    [INFO] 2021-02-21 00:37:21 > Step:    5300 Progress: 2.6% Elapsed: 3:21:51 temperature: 1.0 ada_p: No Discriminator_loss: 5.58143 Generator_loss: 5.6846 
    [INFO] 2021-02-21 00:41:10 > Step:    5400 Progress: 2.7% Elapsed: 3:25:40 temperature: 1.0 ada_p: No Discriminator_loss: 5.66504 Generator_loss: 5.82986 
    [INFO] 2021-02-21 00:44:58 > Step:    5500 Progress: 2.8% Elapsed: 3:29:28 temperature: 1.0 ada_p: No Discriminator_loss: 5.60681 Generator_loss: 5.68068 
    [INFO] 2021-02-21 00:48:48 > Step:    5600 Progress: 2.8% Elapsed: 3:33:17 temperature: 1.0 ada_p: No Discriminator_loss: 5.821 Generator_loss: 5.67333 
    [INFO] 2021-02-21 00:52:36 > Step:    5700 Progress: 2.9% Elapsed: 3:37:06 temperature: 1.0 ada_p: No Discriminator_loss: 5.71505 Generator_loss: 5.64875 
    [INFO] 2021-02-21 00:56:24 > Step:    5800 Progress: 2.9% Elapsed: 3:40:54 temperature: 1.0 ada_p: No Discriminator_loss: 5.70555 Generator_loss: 6.37336 
    [INFO] 2021-02-21 01:00:13 > Step:    5900 Progress: 2.9% Elapsed: 3:44:43 temperature: 1.0 ada_p: No Discriminator_loss: 5.70239 Generator_loss: 5.66964 
    [INFO] 2021-02-21 01:04:01 > Step:    6000 Progress: 3.0% Elapsed: 3:48:31 temperature: 1.0 ada_p: No Discriminator_loss: 5.33738 Generator_loss: 5.96527 
    [INFO] 2021-02-21 01:07:50 > Step:    6100 Progress: 3.0% Elapsed: 3:52:20 temperature: 1.0 ada_p: No Discriminator_loss: 5.81752 Generator_loss: 5.64988 
    [INFO] 2021-02-21 01:11:39 > Step:    6200 Progress: 3.1% Elapsed: 3:56:09 temperature: 1.0 ada_p: No Discriminator_loss: 5.51375 Generator_loss: 5.72314 
    [INFO] 2021-02-21 01:15:27 > Step:    6300 Progress: 3.1% Elapsed: 3:59:57 temperature: 1.0 ada_p: No Discriminator_loss: 5.74 Generator_loss: 5.57064 
    [INFO] 2021-02-21 01:19:16 > Step:    6400 Progress: 3.2% Elapsed: 4:03:46 temperature: 1.0 ada_p: No Discriminator_loss: 5.56962 Generator_loss: 5.88552 
    [INFO] 2021-02-21 01:23:05 > Step:    6500 Progress: 3.2% Elapsed: 4:07:35 temperature: 1.0 ada_p: No Discriminator_loss: 5.43333 Generator_loss: 6.443 
    [INFO] 2021-02-21 01:26:53 > Step:    6600 Progress: 3.3% Elapsed: 4:11:23 temperature: 1.0 ada_p: No Discriminator_loss: 5.38937 Generator_loss: 6.40246 
    [INFO] 2021-02-21 01:30:42 > Step:    6700 Progress: 3.4% Elapsed: 4:15:12 temperature: 1.0 ada_p: No Discriminator_loss: 5.5843 Generator_loss: 5.76042 
    [INFO] 2021-02-21 01:34:30 > Step:    6800 Progress: 3.4% Elapsed: 4:19:00 temperature: 1.0 ada_p: No Discriminator_loss: 5.63685 Generator_loss: 5.73143 
    [INFO] 2021-02-21 01:38:19 > Step:    6900 Progress: 3.5% Elapsed: 4:22:49 temperature: 1.0 ada_p: No Discriminator_loss: 5.55004 Generator_loss: 5.8436 
    [INFO] 2021-02-21 01:42:07 > Step:    7000 Progress: 3.5% Elapsed: 4:26:37 temperature: 1.0 ada_p: No Discriminator_loss: 5.62071 Generator_loss: 5.85883 
    [INFO] 2021-02-21 01:45:56 > Step:    7100 Progress: 3.5% Elapsed: 4:30:26 temperature: 1.0 ada_p: No Discriminator_loss: 5.64066 Generator_loss: 5.85629 
    [INFO] 2021-02-21 01:49:45 > Step:    7200 Progress: 3.6% Elapsed: 4:34:15 temperature: 1.0 ada_p: No Discriminator_loss: 5.51869 Generator_loss: 5.75115 
    [INFO] 2021-02-21 01:53:33 > Step:    7300 Progress: 3.6% Elapsed: 4:38:03 temperature: 1.0 ada_p: No Discriminator_loss: 5.5455 Generator_loss: 5.87317 
    [INFO] 2021-02-21 01:57:22 > Step:    7400 Progress: 3.7% Elapsed: 4:41:52 temperature: 1.0 ada_p: No Discriminator_loss: 5.81166 Generator_loss: 5.96341 
    [INFO] 2021-02-21 02:01:11 > Step:    7500 Progress: 3.8% Elapsed: 4:45:41 temperature: 1.0 ada_p: No Discriminator_loss: 5.7403 Generator_loss: 5.56252 
    [INFO] 2021-02-21 02:05:03 > Step:    7600 Progress: 3.8% Elapsed: 4:49:33 temperature: 1.0 ada_p: No Discriminator_loss: 5.49518 Generator_loss: 5.72996 
    [INFO] 2021-02-21 02:08:52 > Step:    7700 Progress: 3.9% Elapsed: 4:53:22 temperature: 1.0 ada_p: No Discriminator_loss: 5.77958 Generator_loss: 6.01992 
    [INFO] 2021-02-21 02:12:41 > Step:    7800 Progress: 3.9% Elapsed: 4:57:11 temperature: 1.0 ada_p: No Discriminator_loss: 5.77388 Generator_loss: 5.91599 
    [INFO] 2021-02-21 02:16:30 > Step:    7900 Progress: 4.0% Elapsed: 5:01:00 temperature: 1.0 ada_p: No Discriminator_loss: 5.67232 Generator_loss: 5.86226 
    [INFO] 2021-02-21 02:20:19 > Step:    8000 Progress: 4.0% Elapsed: 5:04:49 temperature: 1.0 ada_p: No Discriminator_loss: 5.55205 Generator_loss: 5.74382 
    [INFO] 2021-02-21 02:24:07 > Step:    8100 Progress: 4.0% Elapsed: 5:08:37 temperature: 1.0 ada_p: No Discriminator_loss: 5.71738 Generator_loss: 5.97565 
    [INFO] 2021-02-21 02:27:57 > Step:    8200 Progress: 4.1% Elapsed: 5:12:27 temperature: 1.0 ada_p: No Discriminator_loss: 5.69044 Generator_loss: 5.7908 
    [INFO] 2021-02-21 02:31:46 > Step:    8300 Progress: 4.2% Elapsed: 5:16:16 temperature: 1.0 ada_p: No Discriminator_loss: 5.47848 Generator_loss: 5.65925 
    [INFO] 2021-02-21 02:35:34 > Step:    8400 Progress: 4.2% Elapsed: 5:20:04 temperature: 1.0 ada_p: No Discriminator_loss: 5.46204 Generator_loss: 5.83104 
    [INFO] 2021-02-21 02:39:23 > Step:    8500 Progress: 4.2% Elapsed: 5:23:53 temperature: 1.0 ada_p: No Discriminator_loss: 5.76954 Generator_loss: 5.44432 
    [INFO] 2021-02-21 02:43:12 > Step:    8600 Progress: 4.3% Elapsed: 5:27:42 temperature: 1.0 ada_p: No Discriminator_loss: 5.66066 Generator_loss: 5.83158 
    [INFO] 2021-02-21 02:47:00 > Step:    8700 Progress: 4.3% Elapsed: 5:31:30 temperature: 1.0 ada_p: No Discriminator_loss: 5.53432 Generator_loss: 5.82898 
    [INFO] 2021-02-21 02:50:49 > Step:    8800 Progress: 4.4% Elapsed: 5:35:18 temperature: 1.0 ada_p: No Discriminator_loss: 5.68213 Generator_loss: 5.75823 
    [INFO] 2021-02-21 02:54:37 > Step:    8900 Progress: 4.5% Elapsed: 5:39:07 temperature: 1.0 ada_p: No Discriminator_loss: 5.60052 Generator_loss: 5.83647 
    [INFO] 2021-02-21 02:58:26 > Step:    9000 Progress: 4.5% Elapsed: 5:42:56 temperature: 1.0 ada_p: No Discriminator_loss: 5.8928 Generator_loss: 5.57909 
    [INFO] 2021-02-21 03:02:15 > Step:    9100 Progress: 4.5% Elapsed: 5:46:44 temperature: 1.0 ada_p: No Discriminator_loss: 5.76009 Generator_loss: 5.84614 
    [INFO] 2021-02-21 03:06:03 > Step:    9200 Progress: 4.6% Elapsed: 5:50:33 temperature: 1.0 ada_p: No Discriminator_loss: 5.53428 Generator_loss: 6.0749 
    [INFO] 2021-02-21 03:09:52 > Step:    9300 Progress: 4.7% Elapsed: 5:54:22 temperature: 1.0 ada_p: No Discriminator_loss: 5.59954 Generator_loss: 5.87722 
    [INFO] 2021-02-21 03:13:40 > Step:    9400 Progress: 4.7% Elapsed: 5:58:10 temperature: 1.0 ada_p: No Discriminator_loss: 5.59304 Generator_loss: 5.68887 
    [INFO] 2021-02-21 03:17:29 > Step:    9500 Progress: 4.8% Elapsed: 6:01:59 temperature: 1.0 ada_p: No Discriminator_loss: 5.4157 Generator_loss: 6.01136 
    [INFO] 2021-02-21 03:21:17 > Step:    9600 Progress: 4.8% Elapsed: 6:05:47 temperature: 1.0 ada_p: No Discriminator_loss: 5.52159 Generator_loss: 6.03692 
    [INFO] 2021-02-21 03:25:06 > Step:    9700 Progress: 4.9% Elapsed: 6:09:36 temperature: 1.0 ada_p: No Discriminator_loss: 5.63729 Generator_loss: 5.42364 
    [INFO] 2021-02-21 03:28:55 > Step:    9800 Progress: 4.9% Elapsed: 6:13:25 temperature: 1.0 ada_p: No Discriminator_loss: 5.62768 Generator_loss: 5.89667 
    [INFO] 2021-02-21 03:32:43 > Step:    9900 Progress: 5.0% Elapsed: 6:17:13 temperature: 1.0 ada_p: No Discriminator_loss: 5.44132 Generator_loss: 6.03905 
    [INFO] 2021-02-21 03:36:32 > Step:   10000 Progress: 5.0% Elapsed: 6:21:02 temperature: 1.0 ada_p: No Discriminator_loss: 5.89578 Generator_loss: 5.9393 
    [INFO] 2021-02-21 03:36:32 > Start Evaluation (10000 Step): ContraGAN256-train-2021_02_20_21_08_34
    [INFO] 2021-02-21 03:36:32 > Start Evaluation FID score ... 
    [INFO] 2021-02-21 03:36:32 > Calculating FID Score....
    [INFO] 2021-02-21 03:37:48 > Start Evaluation IS score ... 
    [INFO] 2021-02-21 03:37:48 > Calculating Inception Score....
    [INFO] 2021-02-21 03:38:48 > Start Evaluation F Beta score ... 
    [INFO] 2021-02-21 03:38:48 > Calculate F_beta Score....
    [INFO] 2021-02-21 03:52:26 > Calculate Accuracies....
    [INFO] 2021-02-21 03:53:45 > Calculate Accuracies....
    [INFO] 2021-02-21 04:00:05 > F_8 score (Step: 10000, Using valid images): 0.3978422218675434
    [INFO] 2021-02-21 04:00:05 > F_1/8 score (Step: 10000, Using valid images): 0.47360188071189163
    [INFO] 2021-02-21 04:00:05 > FID score (Step: 10000, Using valid moments): 102.34712951952213
    [INFO] 2021-02-21 04:00:05 > Inception score (Step: 10000, 50000 generated images): 9.047901153564453
    [INFO] 2021-02-21 04:00:05 > Best FID score (Step: 10000, Using valid moments): 102.34712951952213
    [INFO] 2021-02-21 04:00:13 > Save model to checkpoints/ContraGAN256-train-2021_02_20_21_08_34
    [INFO] 2021-02-21 04:04:03 > Step:   10100 Progress: 5.1% Elapsed: 6:48:33 temperature: 1.0 ada_p: No Discriminator_loss: 5.61549 Generator_loss: 5.59227 
    [INFO] 2021-02-21 04:07:50 > Step:   10200 Progress: 5.1% Elapsed: 6:52:20 temperature: 1.0 ada_p: No Discriminator_loss: 5.47054 Generator_loss: 6.01183 
    [INFO] 2021-02-21 04:11:38 > Step:   10300 Progress: 5.1% Elapsed: 6:56:08 temperature: 1.0 ada_p: No Discriminator_loss: 5.4844 Generator_loss: 5.90196 
    [INFO] 2021-02-21 04:15:26 > Step:   10400 Progress: 5.2% Elapsed: 6:59:56 temperature: 1.0 ada_p: No Discriminator_loss: 5.71673 Generator_loss: 5.55152 
    [INFO] 2021-02-21 04:19:13 > Step:   10500 Progress: 5.2% Elapsed: 7:03:43 temperature: 1.0 ada_p: No Discriminator_loss: 5.57823 Generator_loss: 6.02412 
    [INFO] 2021-02-21 04:23:01 > Step:   10600 Progress: 5.3% Elapsed: 7:07:30 temperature: 1.0 ada_p: No Discriminator_loss: 5.74227 Generator_loss: 5.91932 
    [INFO] 2021-02-21 04:26:48 > Step:   10700 Progress: 5.3% Elapsed: 7:11:18 temperature: 1.0 ada_p: No Discriminator_loss: 5.32951 Generator_loss: 6.06626 
    [INFO] 2021-02-21 04:30:36 > Step:   10800 Progress: 5.4% Elapsed: 7:15:06 temperature: 1.0 ada_p: No Discriminator_loss: 5.52856 Generator_loss: 5.99517 
    [INFO] 2021-02-21 04:34:24 > Step:   10900 Progress: 5.5% Elapsed: 7:18:54 temperature: 1.0 ada_p: No Discriminator_loss: 5.88051 Generator_loss: 5.39238 
    [INFO] 2021-02-21 04:38:12 > Step:   11000 Progress: 5.5% Elapsed: 7:22:42 temperature: 1.0 ada_p: No Discriminator_loss: 5.60257 Generator_loss: 5.85679 
    [INFO] 2021-02-21 04:42:00 > Step:   11100 Progress: 5.5% Elapsed: 7:26:29 temperature: 1.0 ada_p: No Discriminator_loss: 5.48181 Generator_loss: 5.90159 
    [INFO] 2021-02-21 04:45:47 > Step:   11200 Progress: 5.6% Elapsed: 7:30:17 temperature: 1.0 ada_p: No Discriminator_loss: 5.52478 Generator_loss: 5.68404 
    [INFO] 2021-02-21 04:49:35 > Step:   11300 Progress: 5.7% Elapsed: 7:34:05 temperature: 1.0 ada_p: No Discriminator_loss: 5.38363 Generator_loss: 5.70954 
    [INFO] 2021-02-21 04:53:51 > Step:   11400 Progress: 5.7% Elapsed: 7:38:21 temperature: 1.0 ada_p: No Discriminator_loss: 5.3916 Generator_loss: 5.77909 
    [INFO] 2021-02-21 04:57:39 > Step:   11500 Progress: 5.8% Elapsed: 7:42:09 temperature: 1.0 ada_p: No Discriminator_loss: 5.45814 Generator_loss: 6.0028 
    [INFO] 2021-02-21 05:01:28 > Step:   11600 Progress: 5.8% Elapsed: 7:45:58 temperature: 1.0 ada_p: No Discriminator_loss: 5.2064 Generator_loss: 6.49754 
    [INFO] 2021-02-21 05:05:16 > Step:   11700 Progress: 5.9% Elapsed: 7:49:46 temperature: 1.0 ada_p: No Discriminator_loss: 5.68127 Generator_loss: 5.98729 
    [INFO] 2021-02-21 05:09:05 > Step:   11800 Progress: 5.9% Elapsed: 7:53:35 temperature: 1.0 ada_p: No Discriminator_loss: 5.53479 Generator_loss: 5.87083 
    [INFO] 2021-02-21 05:12:54 > Step:   11900 Progress: 5.9% Elapsed: 7:57:24 temperature: 1.0 ada_p: No Discriminator_loss: 5.54363 Generator_loss: 5.85797 
    [INFO] 2021-02-21 05:16:42 > Step:   12000 Progress: 6.0% Elapsed: 8:01:12 temperature: 1.0 ada_p: No Discriminator_loss: 5.55418 Generator_loss: 5.78515 
    [INFO] 2021-02-21 05:20:31 > Step:   12100 Progress: 6.0% Elapsed: 8:05:01 temperature: 1.0 ada_p: No Discriminator_loss: 5.51689 Generator_loss: 5.86771 
    [INFO] 2021-02-21 05:24:19 > Step:   12200 Progress: 6.1% Elapsed: 8:08:49 temperature: 1.0 ada_p: No Discriminator_loss: 5.45934 Generator_loss: 5.83361 
    [INFO] 2021-02-21 05:28:08 > Step:   12300 Progress: 6.2% Elapsed: 8:12:38 temperature: 1.0 ada_p: No Discriminator_loss: 5.72423 Generator_loss: 5.92938 
    [INFO] 2021-02-21 05:31:57 > Step:   12400 Progress: 6.2% Elapsed: 8:16:27 temperature: 1.0 ada_p: No Discriminator_loss: 5.38219 Generator_loss: 5.7864 
    [INFO] 2021-02-21 05:35:45 > Step:   12500 Progress: 6.2% Elapsed: 8:20:15 temperature: 1.0 ada_p: No Discriminator_loss: 5.44239 Generator_loss: 5.94229 
    [INFO] 2021-02-21 05:39:39 > Step:   12600 Progress: 6.3% Elapsed: 8:24:09 temperature: 1.0 ada_p: No Discriminator_loss: 5.38677 Generator_loss: 5.96589 
    [INFO] 2021-02-21 05:43:27 > Step:   12700 Progress: 6.3% Elapsed: 8:27:57 temperature: 1.0 ada_p: No Discriminator_loss: 5.23961 Generator_loss: 6.00821 
    [INFO] 2021-02-21 05:47:15 > Step:   12800 Progress: 6.4% Elapsed: 8:31:45 temperature: 1.0 ada_p: No Discriminator_loss: 5.36818 Generator_loss: 6.42155 
    [INFO] 2021-02-21 05:51:04 > Step:   12900 Progress: 6.5% Elapsed: 8:35:34 temperature: 1.0 ada_p: No Discriminator_loss: 5.671 Generator_loss: 5.78141 
    [INFO] 2021-02-21 05:54:52 > Step:   13000 Progress: 6.5% Elapsed: 8:39:22 temperature: 1.0 ada_p: No Discriminator_loss: 5.40353 Generator_loss: 6.13112 
    [INFO] 2021-02-21 05:58:40 > Step:   13100 Progress: 6.6% Elapsed: 8:43:10 temperature: 1.0 ada_p: No Discriminator_loss: 5.66685 Generator_loss: 5.95615 
    [INFO] 2021-02-21 06:02:29 > Step:   13200 Progress: 6.6% Elapsed: 8:46:59 temperature: 1.0 ada_p: No Discriminator_loss: 5.36453 Generator_loss: 5.83562 
    [INFO] 2021-02-21 06:06:18 > Step:   13300 Progress: 6.7% Elapsed: 8:50:48 temperature: 1.0 ada_p: No Discriminator_loss: 5.72367 Generator_loss: 5.69284 
    [INFO] 2021-02-21 06:10:06 > Step:   13400 Progress: 6.7% Elapsed: 8:54:36 temperature: 1.0 ada_p: No Discriminator_loss: 5.36953 Generator_loss: 6.54372 
    [INFO] 2021-02-21 06:13:54 > Step:   13500 Progress: 6.8% Elapsed: 8:58:24 temperature: 1.0 ada_p: No Discriminator_loss: 5.24973 Generator_loss: 6.1977 
    [INFO] 2021-02-21 06:17:43 > Step:   13600 Progress: 6.8% Elapsed: 9:02:13 temperature: 1.0 ada_p: No Discriminator_loss: 5.39923 Generator_loss: 5.81003 
    [INFO] 2021-02-21 06:21:32 > Step:   13700 Progress: 6.9% Elapsed: 9:06:01 temperature: 1.0 ada_p: No Discriminator_loss: 5.54835 Generator_loss: 5.88355 
    [INFO] 2021-02-21 06:25:20 > Step:   13800 Progress: 6.9% Elapsed: 9:09:50 temperature: 1.0 ada_p: No Discriminator_loss: 5.48226 Generator_loss: 5.76783 
    [INFO] 2021-02-21 06:29:09 > Step:   13900 Progress: 7.0% Elapsed: 9:13:39 temperature: 1.0 ada_p: No Discriminator_loss: 5.35625 Generator_loss: 6.02004 
    [INFO] 2021-02-21 06:32:58 > Step:   14000 Progress: 7.0% Elapsed: 9:17:28 temperature: 1.0 ada_p: No Discriminator_loss: 5.32664 Generator_loss: 6.27177 
    [INFO] 2021-02-21 06:36:46 > Step:   14100 Progress: 7.0% Elapsed: 9:21:16 temperature: 1.0 ada_p: No Discriminator_loss: 5.33913 Generator_loss: 6.07489 
    [INFO] 2021-02-21 06:40:35 > Step:   14200 Progress: 7.1% Elapsed: 9:25:05 temperature: 1.0 ada_p: No Discriminator_loss: 5.34456 Generator_loss: 5.96744 
    [INFO] 2021-02-21 06:44:23 > Step:   14300 Progress: 7.1% Elapsed: 9:28:53 temperature: 1.0 ada_p: No Discriminator_loss: 5.46317 Generator_loss: 5.73075 
    [INFO] 2021-02-21 06:48:12 > Step:   14400 Progress: 7.2% Elapsed: 9:32:42 temperature: 1.0 ada_p: No Discriminator_loss: 5.6067 Generator_loss: 5.58613 
    [INFO] 2021-02-21 06:52:00 > Step:   14500 Progress: 7.2% Elapsed: 9:36:30 temperature: 1.0 ada_p: No Discriminator_loss: 5.39823 Generator_loss: 5.86266 
    [INFO] 2021-02-21 06:55:49 > Step:   14600 Progress: 7.3% Elapsed: 9:40:19 temperature: 1.0 ada_p: No Discriminator_loss: 5.39154 Generator_loss: 5.98653 
    [INFO] 2021-02-21 06:59:38 > Step:   14700 Progress: 7.3% Elapsed: 9:44:08 temperature: 1.0 ada_p: No Discriminator_loss: 5.63467 Generator_loss: 5.82919 
    [INFO] 2021-02-21 07:03:26 > Step:   14800 Progress: 7.4% Elapsed: 9:47:56 temperature: 1.0 ada_p: No Discriminator_loss: 5.49369 Generator_loss: 5.90726 
    [INFO] 2021-02-21 07:07:15 > Step:   14900 Progress: 7.4% Elapsed: 9:51:45 temperature: 1.0 ada_p: No Discriminator_loss: 5.39954 Generator_loss: 5.69192 
    [INFO] 2021-02-21 07:11:04 > Step:   15000 Progress: 7.5% Elapsed: 9:55:34 temperature: 1.0 ada_p: No Discriminator_loss: 5.31042 Generator_loss: 6.08007 
    [INFO] 2021-02-21 07:14:56 > Step:   15100 Progress: 7.5% Elapsed: 9:59:26 temperature: 1.0 ada_p: No Discriminator_loss: 5.24395 Generator_loss: 5.94674 
    [INFO] 2021-02-21 07:18:45 > Step:   15200 Progress: 7.6% Elapsed: 10:03:15 temperature: 1.0 ada_p: No Discriminator_loss: 5.31582 Generator_loss: 6.06186 
    [INFO] 2021-02-21 07:22:34 > Step:   15300 Progress: 7.6% Elapsed: 10:07:04 temperature: 1.0 ada_p: No Discriminator_loss: 5.70501 Generator_loss: 5.83366 
    [INFO] 2021-02-21 07:26:22 > Step:   15400 Progress: 7.7% Elapsed: 10:10:52 temperature: 1.0 ada_p: No Discriminator_loss: 5.28351 Generator_loss: 5.99659 
    [INFO] 2021-02-21 07:30:11 > Step:   15500 Progress: 7.8% Elapsed: 10:14:41 temperature: 1.0 ada_p: No Discriminator_loss: 5.45195 Generator_loss: 6.09586 
    [INFO] 2021-02-21 07:34:00 > Step:   15600 Progress: 7.8% Elapsed: 10:18:30 temperature: 1.0 ada_p: No Discriminator_loss: 5.33257 Generator_loss: 5.99524 
    [INFO] 2021-02-21 07:37:49 > Step:   15700 Progress: 7.8% Elapsed: 10:22:19 temperature: 1.0 ada_p: No Discriminator_loss: 5.382 Generator_loss: 5.95735 
    [INFO] 2021-02-21 07:41:38 > Step:   15800 Progress: 7.9% Elapsed: 10:26:08 temperature: 1.0 ada_p: No Discriminator_loss: 5.34306 Generator_loss: 5.89781 
    [INFO] 2021-02-21 07:45:26 > Step:   15900 Progress: 8.0% Elapsed: 10:29:56 temperature: 1.0 ada_p: No Discriminator_loss: 5.13727 Generator_loss: 6.54094 
    [INFO] 2021-02-21 07:49:15 > Step:   16000 Progress: 8.0% Elapsed: 10:33:45 temperature: 1.0 ada_p: No Discriminator_loss: 5.36533 Generator_loss: 5.89231 
    [INFO] 2021-02-21 07:53:04 > Step:   16100 Progress: 8.1% Elapsed: 10:37:34 temperature: 1.0 ada_p: No Discriminator_loss: 5.43185 Generator_loss: 5.95367 
    [INFO] 2021-02-21 07:56:53 > Step:   16200 Progress: 8.1% Elapsed: 10:41:23 temperature: 1.0 ada_p: No Discriminator_loss: 5.32991 Generator_loss: 6.11029 
    [INFO] 2021-02-21 08:00:41 > Step:   16300 Progress: 8.2% Elapsed: 10:45:11 temperature: 1.0 ada_p: No Discriminator_loss: 5.39281 Generator_loss: 5.9911 
    [INFO] 2021-02-21 08:04:30 > Step:   16400 Progress: 8.2% Elapsed: 10:49:00 temperature: 1.0 ada_p: No Discriminator_loss: 5.23903 Generator_loss: 6.10904 
    [INFO] 2021-02-21 08:08:19 > Step:   16500 Progress: 8.2% Elapsed: 10:52:49 temperature: 1.0 ada_p: No Discriminator_loss: 5.24634 Generator_loss: 6.1255 
    [INFO] 2021-02-21 08:12:08 > Step:   16600 Progress: 8.3% Elapsed: 10:56:38 temperature: 1.0 ada_p: No Discriminator_loss: 5.34409 Generator_loss: 6.18421 
    [INFO] 2021-02-21 08:15:56 > Step:   16700 Progress: 8.3% Elapsed: 11:00:26 temperature: 1.0 ada_p: No Discriminator_loss: 5.5438 Generator_loss: 5.73834 
    [INFO] 2021-02-21 08:19:45 > Step:   16800 Progress: 8.4% Elapsed: 11:04:15 temperature: 1.0 ada_p: No Discriminator_loss: 5.69473 Generator_loss: 5.83429 
    [INFO] 2021-02-21 08:23:34 > Step:   16900 Progress: 8.5% Elapsed: 11:08:04 temperature: 1.0 ada_p: No Discriminator_loss: 5.33765 Generator_loss: 6.02617 
    [INFO] 2021-02-21 08:27:23 > Step:   17000 Progress: 8.5% Elapsed: 11:11:53 temperature: 1.0 ada_p: No Discriminator_loss: 5.32373 Generator_loss: 5.92952 
    [INFO] 2021-02-21 08:31:12 > Step:   17100 Progress: 8.6% Elapsed: 11:15:42 temperature: 1.0 ada_p: No Discriminator_loss: 5.51038 Generator_loss: 5.71906 
    [INFO] 2021-02-21 08:35:01 > Step:   17200 Progress: 8.6% Elapsed: 11:19:31 temperature: 1.0 ada_p: No Discriminator_loss: 5.53528 Generator_loss: 5.75337 
    [INFO] 2021-02-21 08:38:49 > Step:   17300 Progress: 8.6% Elapsed: 11:23:19 temperature: 1.0 ada_p: No Discriminator_loss: 5.31306 Generator_loss: 5.9747 
    [INFO] 2021-02-21 08:42:38 > Step:   17400 Progress: 8.7% Elapsed: 11:27:08 temperature: 1.0 ada_p: No Discriminator_loss: 5.29025 Generator_loss: 6.55784 
    [INFO] 2021-02-21 08:46:27 > Step:   17500 Progress: 8.8% Elapsed: 11:30:57 temperature: 1.0 ada_p: No Discriminator_loss: 5.21731 Generator_loss: 6.43052 
    [INFO] 2021-02-21 08:50:19 > Step:   17600 Progress: 8.8% Elapsed: 11:34:49 temperature: 1.0 ada_p: No Discriminator_loss: 5.28761 Generator_loss: 6.15318 
    [INFO] 2021-02-21 08:54:08 > Step:   17700 Progress: 8.8% Elapsed: 11:38:38 temperature: 1.0 ada_p: No Discriminator_loss: 5.18497 Generator_loss: 5.95889 
    [INFO] 2021-02-21 08:57:57 > Step:   17800 Progress: 8.9% Elapsed: 11:42:27 temperature: 1.0 ada_p: No Discriminator_loss: 5.23012 Generator_loss: 5.7805 
    [INFO] 2021-02-21 09:01:45 > Step:   17900 Progress: 8.9% Elapsed: 11:46:15 temperature: 1.0 ada_p: No Discriminator_loss: 5.12742 Generator_loss: 6.34884 
    [INFO] 2021-02-21 09:05:34 > Step:   18000 Progress: 9.0% Elapsed: 11:50:04 temperature: 1.0 ada_p: No Discriminator_loss: 5.15488 Generator_loss: 6.39498 
    [INFO] 2021-02-21 09:09:23 > Step:   18100 Progress: 9.0% Elapsed: 11:53:53 temperature: 1.0 ada_p: No Discriminator_loss: 5.23487 Generator_loss: 5.9891 
    [INFO] 2021-02-21 09:13:12 > Step:   18200 Progress: 9.1% Elapsed: 11:57:42 temperature: 1.0 ada_p: No Discriminator_loss: 5.43528 Generator_loss: 5.78899 
    [INFO] 2021-02-21 09:17:00 > Step:   18300 Progress: 9.2% Elapsed: 12:01:30 temperature: 1.0 ada_p: No Discriminator_loss: 5.28905 Generator_loss: 5.96085 
    [INFO] 2021-02-21 09:20:48 > Step:   18400 Progress: 9.2% Elapsed: 12:05:18 temperature: 1.0 ada_p: No Discriminator_loss: 5.48233 Generator_loss: 5.87019 
    [INFO] 2021-02-21 09:24:37 > Step:   18500 Progress: 9.2% Elapsed: 12:09:07 temperature: 1.0 ada_p: No Discriminator_loss: 5.38186 Generator_loss: 5.92538 
    [INFO] 2021-02-21 09:28:25 > Step:   18600 Progress: 9.3% Elapsed: 12:12:55 temperature: 1.0 ada_p: No Discriminator_loss: 5.20964 Generator_loss: 6.22919 
    [INFO] 2021-02-21 09:32:14 > Step:   18700 Progress: 9.3% Elapsed: 12:16:44 temperature: 1.0 ada_p: No Discriminator_loss: 5.42862 Generator_loss: 5.46972 
    [INFO] 2021-02-21 09:36:02 > Step:   18800 Progress: 9.4% Elapsed: 12:20:32 temperature: 1.0 ada_p: No Discriminator_loss: 5.38264 Generator_loss: 5.87394 
    [INFO] 2021-02-21 09:39:51 > Step:   18900 Progress: 9.4% Elapsed: 12:24:21 temperature: 1.0 ada_p: No Discriminator_loss: 5.37993 Generator_loss: 5.97314 
    [INFO] 2021-02-21 09:43:40 > Step:   19000 Progress: 9.5% Elapsed: 12:28:10 temperature: 1.0 ada_p: No Discriminator_loss: 5.44529 Generator_loss: 5.77881 
    [INFO] 2021-02-21 09:47:29 > Step:   19100 Progress: 9.6% Elapsed: 12:31:58 temperature: 1.0 ada_p: No Discriminator_loss: 5.28966 Generator_loss: 6.26161 
    [INFO] 2021-02-21 09:51:17 > Step:   19200 Progress: 9.6% Elapsed: 12:35:47 temperature: 1.0 ada_p: No Discriminator_loss: 5.28617 Generator_loss: 5.93402 
    [INFO] 2021-02-21 09:55:06 > Step:   19300 Progress: 9.7% Elapsed: 12:39:36 temperature: 1.0 ada_p: No Discriminator_loss: 5.20127 Generator_loss: 6.07594 
    [INFO] 2021-02-21 09:58:54 > Step:   19400 Progress: 9.7% Elapsed: 12:43:24 temperature: 1.0 ada_p: No Discriminator_loss: 5.42907 Generator_loss: 5.90057 
    [INFO] 2021-02-21 10:02:42 > Step:   19500 Progress: 9.8% Elapsed: 12:47:12 temperature: 1.0 ada_p: No Discriminator_loss: 5.44688 Generator_loss: 5.81431 
    [INFO] 2021-02-21 10:06:29 > Step:   19600 Progress: 9.8% Elapsed: 12:50:59 temperature: 1.0 ada_p: No Discriminator_loss: 5.20832 Generator_loss: 6.13596 
    [INFO] 2021-02-21 10:10:17 > Step:   19700 Progress: 9.8% Elapsed: 12:54:47 temperature: 1.0 ada_p: No Discriminator_loss: 5.22939 Generator_loss: 6.06941 
    [INFO] 2021-02-21 10:14:05 > Step:   19800 Progress: 9.9% Elapsed: 12:58:34 temperature: 1.0 ada_p: No Discriminator_loss: 5.12076 Generator_loss: 6.40348 
    [INFO] 2021-02-21 10:17:52 > Step:   19900 Progress: 10.0% Elapsed: 13:02:22 temperature: 1.0 ada_p: No Discriminator_loss: 5.15113 Generator_loss: 6.04587 
    [INFO] 2021-02-21 10:21:40 > Step:   20000 Progress: 10.0% Elapsed: 13:06:09 temperature: 1.0 ada_p: No Discriminator_loss: 5.23642 Generator_loss: 6.12289 
    [INFO] 2021-02-21 10:21:40 > Start Evaluation (20000 Step): ContraGAN256-train-2021_02_20_21_08_34
    [INFO] 2021-02-21 10:21:40 > Start Evaluation FID score ... 
    [INFO] 2021-02-21 10:21:40 > Calculating FID Score....
    [INFO] 2021-02-21 10:22:54 > Start Evaluation IS score ... 
    [INFO] 2021-02-21 10:22:54 > Calculating Inception Score....
    [INFO] 2021-02-21 10:23:55 > Start Evaluation F Beta score ... 
    [INFO] 2021-02-21 10:23:55 > Calculate F_beta Score....
    [INFO] 2021-02-21 10:37:01 > Calculate Accuracies....
    [INFO] 2021-02-21 10:38:17 > Calculate Accuracies....
    [INFO] 2021-02-21 10:44:40 > F_8 score (Step: 20000, Using valid images): 0.5013178702520207
    [INFO] 2021-02-21 10:44:40 > F_1/8 score (Step: 20000, Using valid images): 0.5991395247770677
    [INFO] 2021-02-21 10:44:40 > FID score (Step: 20000, Using valid moments): 86.606241813726
    [INFO] 2021-02-21 10:44:40 > Inception score (Step: 20000, 50000 generated images): 10.576443672180176
    [INFO] 2021-02-21 10:44:40 > Best FID score (Step: 20000, Using valid moments): 86.606241813726
    [INFO] 2021-02-21 10:44:47 > Save model to checkpoints/ContraGAN256-train-2021_02_20_21_08_34
    [INFO] 2021-02-21 10:48:38 > Step:   20100 Progress: 10.1% Elapsed: 13:33:08 temperature: 1.0 ada_p: No Discriminator_loss: 5.2883 Generator_loss: 6.12938 
    [INFO] 2021-02-21 10:52:25 > Step:   20200 Progress: 10.1% Elapsed: 13:36:55 temperature: 1.0 ada_p: No Discriminator_loss: 5.30453 Generator_loss: 6.09166 
    [INFO] 2021-02-21 10:56:12 > Step:   20300 Progress: 10.2% Elapsed: 13:40:42 temperature: 1.0 ada_p: No Discriminator_loss: 5.30201 Generator_loss: 5.95434 
    [INFO] 2021-02-21 11:00:00 > Step:   20400 Progress: 10.2% Elapsed: 13:44:29 temperature: 1.0 ada_p: No Discriminator_loss: 5.73066 Generator_loss: 5.87525 
    [INFO] 2021-02-21 11:03:47 > Step:   20500 Progress: 10.2% Elapsed: 13:48:17 temperature: 1.0 ada_p: No Discriminator_loss: 5.35853 Generator_loss: 5.72359 
    [INFO] 2021-02-21 11:07:34 > Step:   20600 Progress: 10.3% Elapsed: 13:52:04 temperature: 1.0 ada_p: No Discriminator_loss: 5.03984 Generator_loss: 6.49264 
    [INFO] 2021-02-21 11:11:20 > Step:   20700 Progress: 10.3% Elapsed: 13:55:50 temperature: 1.0 ada_p: No Discriminator_loss: 5.18465 Generator_loss: 6.57307 
    [INFO] 2021-02-21 11:15:06 > Step:   20800 Progress: 10.4% Elapsed: 13:59:36 temperature: 1.0 ada_p: No Discriminator_loss: 5.51055 Generator_loss: 6.10052 
    [INFO] 2021-02-21 11:18:52 > Step:   20900 Progress: 10.4% Elapsed: 14:03:22 temperature: 1.0 ada_p: No Discriminator_loss: 5.49472 Generator_loss: 5.90163 
    [INFO] 2021-02-21 11:22:38 > Step:   21000 Progress: 10.5% Elapsed: 14:07:08 temperature: 1.0 ada_p: No Discriminator_loss: 5.13749 Generator_loss: 6.44742 
    [INFO] 2021-02-21 11:26:24 > Step:   21100 Progress: 10.5% Elapsed: 14:10:54 temperature: 1.0 ada_p: No Discriminator_loss: 5.30476 Generator_loss: 6.02995 
    [INFO] 2021-02-21 11:30:11 > Step:   21200 Progress: 10.6% Elapsed: 14:14:41 temperature: 1.0 ada_p: No Discriminator_loss: 5.34746 Generator_loss: 5.99667 
    [INFO] 2021-02-21 11:34:28 > Step:   21300 Progress: 10.7% Elapsed: 14:18:58 temperature: 1.0 ada_p: No Discriminator_loss: 5.14232 Generator_loss: 6.18475 
    [INFO] 2021-02-21 11:38:14 > Step:   21400 Progress: 10.7% Elapsed: 14:22:44 temperature: 1.0 ada_p: No Discriminator_loss: 5.42345 Generator_loss: 5.99887 
    [INFO] 2021-02-21 11:42:01 > Step:   21500 Progress: 10.8% Elapsed: 14:26:31 temperature: 1.0 ada_p: No Discriminator_loss: 5.57803 Generator_loss: 5.5408 
    [INFO] 2021-02-21 11:45:48 > Step:   21600 Progress: 10.8% Elapsed: 14:30:18 temperature: 1.0 ada_p: No Discriminator_loss: 5.33398 Generator_loss: 5.83355 
    [INFO] 2021-02-21 11:49:35 > Step:   21700 Progress: 10.8% Elapsed: 14:34:05 temperature: 1.0 ada_p: No Discriminator_loss: 5.09434 Generator_loss: 6.27549 
    [INFO] 2021-02-21 11:53:22 > Step:   21800 Progress: 10.9% Elapsed: 14:37:52 temperature: 1.0 ada_p: No Discriminator_loss: 5.29219 Generator_loss: 5.82156 
    [INFO] 2021-02-21 11:57:09 > Step:   21900 Progress: 10.9% Elapsed: 14:41:39 temperature: 1.0 ada_p: No Discriminator_loss: 5.35761 Generator_loss: 5.9968 
    [INFO] 2021-02-21 12:00:56 > Step:   22000 Progress: 11.0% Elapsed: 14:45:26 temperature: 1.0 ada_p: No Discriminator_loss: 5.25885 Generator_loss: 5.9278 
    [INFO] 2021-02-21 12:04:43 > Step:   22100 Progress: 11.1% Elapsed: 14:49:13 temperature: 1.0 ada_p: No Discriminator_loss: 5.4543 Generator_loss: 5.9118 
    [INFO] 2021-02-21 12:08:30 > Step:   22200 Progress: 11.1% Elapsed: 14:53:00 temperature: 1.0 ada_p: No Discriminator_loss: 5.22759 Generator_loss: 6.20647 
    [INFO] 2021-02-21 12:12:18 > Step:   22300 Progress: 11.2% Elapsed: 14:56:47 temperature: 1.0 ada_p: No Discriminator_loss: 5.4659 Generator_loss: 5.89839 
    [INFO] 2021-02-21 12:16:04 > Step:   22400 Progress: 11.2% Elapsed: 15:00:34 temperature: 1.0 ada_p: No Discriminator_loss: 5.23793 Generator_loss: 6.00481 
    [INFO] 2021-02-21 12:19:50 > Step:   22500 Progress: 11.2% Elapsed: 15:04:20 temperature: 1.0 ada_p: No Discriminator_loss: 5.45953 Generator_loss: 5.91134 
    [INFO] 2021-02-21 12:23:42 > Step:   22600 Progress: 11.3% Elapsed: 15:08:12 temperature: 1.0 ada_p: No Discriminator_loss: 5.40287 Generator_loss: 5.9602 
    [INFO] 2021-02-21 12:27:30 > Step:   22700 Progress: 11.3% Elapsed: 15:12:00 temperature: 1.0 ada_p: No Discriminator_loss: 5.24328 Generator_loss: 5.97976 
    [INFO] 2021-02-21 12:31:19 > Step:   22800 Progress: 11.4% Elapsed: 15:15:49 temperature: 1.0 ada_p: No Discriminator_loss: 5.21187 Generator_loss: 6.10262 
    [INFO] 2021-02-21 12:35:07 > Step:   22900 Progress: 11.5% Elapsed: 15:19:37 temperature: 1.0 ada_p: No Discriminator_loss: 5.36595 Generator_loss: 6.1337 
    [INFO] 2021-02-21 12:38:54 > Step:   23000 Progress: 11.5% Elapsed: 15:23:24 temperature: 1.0 ada_p: No Discriminator_loss: 5.31028 Generator_loss: 5.9412 
    [INFO] 2021-02-21 12:42:42 > Step:   23100 Progress: 11.6% Elapsed: 15:27:12 temperature: 1.0 ada_p: No Discriminator_loss: 5.33951 Generator_loss: 5.93861 
    [INFO] 2021-02-21 12:46:29 > Step:   23200 Progress: 11.6% Elapsed: 15:30:59 temperature: 1.0 ada_p: No Discriminator_loss: 5.31775 Generator_loss: 6.47206 
    [INFO] 2021-02-21 12:50:17 > Step:   23300 Progress: 11.7% Elapsed: 15:34:47 temperature: 1.0 ada_p: No Discriminator_loss: 5.4954 Generator_loss: 5.80538 
    [INFO] 2021-02-21 12:54:05 > Step:   23400 Progress: 11.7% Elapsed: 15:38:35 temperature: 1.0 ada_p: No Discriminator_loss: 5.08702 Generator_loss: 6.34395 
    [INFO] 2021-02-21 12:57:53 > Step:   23500 Progress: 11.8% Elapsed: 15:42:23 temperature: 1.0 ada_p: No Discriminator_loss: 5.37085 Generator_loss: 5.87989 
    [INFO] 2021-02-21 13:01:40 > Step:   23600 Progress: 11.8% Elapsed: 15:46:10 temperature: 1.0 ada_p: No Discriminator_loss: 5.64066 Generator_loss: 5.79878 
    [INFO] 2021-02-21 13:05:28 > Step:   23700 Progress: 11.8% Elapsed: 15:49:58 temperature: 1.0 ada_p: No Discriminator_loss: 5.24808 Generator_loss: 5.84579 
    [INFO] 2021-02-21 13:09:15 > Step:   23800 Progress: 11.9% Elapsed: 15:53:45 temperature: 1.0 ada_p: No Discriminator_loss: 5.16919 Generator_loss: 6.48627 
    [INFO] 2021-02-21 13:13:03 > Step:   23900 Progress: 11.9% Elapsed: 15:57:33 temperature: 1.0 ada_p: No Discriminator_loss: 5.68281 Generator_loss: 5.59888 
    [INFO] 2021-02-21 13:16:52 > Step:   24000 Progress: 12.0% Elapsed: 16:01:21 temperature: 1.0 ada_p: No Discriminator_loss: 5.36861 Generator_loss: 6.34337 
    [INFO] 2021-02-21 13:20:39 > Step:   24100 Progress: 12.0% Elapsed: 16:05:09 temperature: 1.0 ada_p: No Discriminator_loss: 6.24444 Generator_loss: 6.0074 
    [INFO] 2021-02-21 13:24:27 > Step:   24200 Progress: 12.1% Elapsed: 16:08:57 temperature: 1.0 ada_p: No Discriminator_loss: 5.27877 Generator_loss: 5.72811 
    [INFO] 2021-02-21 13:28:15 > Step:   24300 Progress: 12.2% Elapsed: 16:12:45 temperature: 1.0 ada_p: No Discriminator_loss: 5.23603 Generator_loss: 5.966 
    [INFO] 2021-02-21 13:32:03 > Step:   24400 Progress: 12.2% Elapsed: 16:16:32 temperature: 1.0 ada_p: No Discriminator_loss: 5.11996 Generator_loss: 6.025 
    [INFO] 2021-02-21 13:35:51 > Step:   24500 Progress: 12.2% Elapsed: 16:20:21 temperature: 1.0 ada_p: No Discriminator_loss: 5.35602 Generator_loss: 5.70607 
    [INFO] 2021-02-21 13:39:39 > Step:   24600 Progress: 12.3% Elapsed: 16:24:09 temperature: 1.0 ada_p: No Discriminator_loss: 5.10049 Generator_loss: 6.214 
    [INFO] 2021-02-21 13:43:27 > Step:   24700 Progress: 12.3% Elapsed: 16:27:57 temperature: 1.0 ada_p: No Discriminator_loss: 5.40559 Generator_loss: 5.7541 
    [INFO] 2021-02-21 13:47:14 > Step:   24800 Progress: 12.4% Elapsed: 16:31:44 temperature: 1.0 ada_p: No Discriminator_loss: 5.36092 Generator_loss: 6.01325 
    [INFO] 2021-02-21 13:51:02 > Step:   24900 Progress: 12.4% Elapsed: 16:35:32 temperature: 1.0 ada_p: No Discriminator_loss: 5.21496 Generator_loss: 6.14479 
    [INFO] 2021-02-21 13:54:50 > Step:   25000 Progress: 12.5% Elapsed: 16:39:20 temperature: 1.0 ada_p: No Discriminator_loss: 5.33224 Generator_loss: 5.75272 
    [INFO] 2021-02-21 13:58:42 > Step:   25100 Progress: 12.6% Elapsed: 16:43:12 temperature: 1.0 ada_p: No Discriminator_loss: 5.60206 Generator_loss: 5.52988 
    [INFO] 2021-02-21 14:02:30 > Step:   25200 Progress: 12.6% Elapsed: 16:47:00 temperature: 1.0 ada_p: No Discriminator_loss: 5.61885 Generator_loss: 5.66513 
    [INFO] 2021-02-21 14:06:17 > Step:   25300 Progress: 12.7% Elapsed: 16:50:47 temperature: 1.0 ada_p: No Discriminator_loss: 5.34569 Generator_loss: 5.94746 
    [INFO] 2021-02-21 14:10:05 > Step:   25400 Progress: 12.7% Elapsed: 16:54:35 temperature: 1.0 ada_p: No Discriminator_loss: 5.4149 Generator_loss: 5.80082 
    [INFO] 2021-02-21 14:13:53 > Step:   25500 Progress: 12.8% Elapsed: 16:58:23 temperature: 1.0 ada_p: No Discriminator_loss: 5.47861 Generator_loss: 5.75722 
    [INFO] 2021-02-21 14:17:42 > Step:   25600 Progress: 12.8% Elapsed: 17:02:12 temperature: 1.0 ada_p: No Discriminator_loss: 5.29598 Generator_loss: 5.94232 
    [INFO] 2021-02-21 14:21:30 > Step:   25700 Progress: 12.8% Elapsed: 17:06:00 temperature: 1.0 ada_p: No Discriminator_loss: 5.05562 Generator_loss: 6.2436 
    [INFO] 2021-02-21 14:25:18 > Step:   25800 Progress: 12.9% Elapsed: 17:09:48 temperature: 1.0 ada_p: No Discriminator_loss: 5.17932 Generator_loss: 6.10149 
    [INFO] 2021-02-21 14:29:06 > Step:   25900 Progress: 13.0% Elapsed: 17:13:35 temperature: 1.0 ada_p: No Discriminator_loss: 5.29687 Generator_loss: 6.00762 
    [INFO] 2021-02-21 14:32:53 > Step:   26000 Progress: 13.0% Elapsed: 17:17:23 temperature: 1.0 ada_p: No Discriminator_loss: 5.3674 Generator_loss: 5.76855 
    [INFO] 2021-02-21 14:36:41 > Step:   26100 Progress: 13.1% Elapsed: 17:21:11 temperature: 1.0 ada_p: No Discriminator_loss: 5.38621 Generator_loss: 5.93727 
    [INFO] 2021-02-21 14:40:29 > Step:   26200 Progress: 13.1% Elapsed: 17:24:59 temperature: 1.0 ada_p: No Discriminator_loss: 5.2609 Generator_loss: 6.0648 
    [INFO] 2021-02-21 14:44:17 > Step:   26300 Progress: 13.2% Elapsed: 17:28:47 temperature: 1.0 ada_p: No Discriminator_loss: 5.08737 Generator_loss: 6.19112 
    [INFO] 2021-02-21 14:48:05 > Step:   26400 Progress: 13.2% Elapsed: 17:32:35 temperature: 1.0 ada_p: No Discriminator_loss: 5.25257 Generator_loss: 6.22127 
    [INFO] 2021-02-21 14:51:53 > Step:   26500 Progress: 13.2% Elapsed: 17:36:23 temperature: 1.0 ada_p: No Discriminator_loss: 5.27869 Generator_loss: 5.97233 
    [INFO] 2021-02-21 14:55:40 > Step:   26600 Progress: 13.3% Elapsed: 17:40:10 temperature: 1.0 ada_p: No Discriminator_loss: 5.40338 Generator_loss: 5.74919 
    [INFO] 2021-02-21 14:59:28 > Step:   26700 Progress: 13.4% Elapsed: 17:43:58 temperature: 1.0 ada_p: No Discriminator_loss: 5.35378 Generator_loss: 6.04818 
    [INFO] 2021-02-21 15:03:16 > Step:   26800 Progress: 13.4% Elapsed: 17:47:46 temperature: 1.0 ada_p: No Discriminator_loss: 5.53892 Generator_loss: 5.94349 
    [INFO] 2021-02-21 15:07:04 > Step:   26900 Progress: 13.5% Elapsed: 17:51:34 temperature: 1.0 ada_p: No Discriminator_loss: 6.18354 Generator_loss: 6.3119 
    [INFO] 2021-02-21 15:10:52 > Step:   27000 Progress: 13.5% Elapsed: 17:55:22 temperature: 1.0 ada_p: No Discriminator_loss: 5.14978 Generator_loss: 5.89356 
    [INFO] 2021-02-21 15:14:39 > Step:   27100 Progress: 13.6% Elapsed: 17:59:09 temperature: 1.0 ada_p: No Discriminator_loss: 5.31189 Generator_loss: 5.78471 
    [INFO] 2021-02-21 15:18:27 > Step:   27200 Progress: 13.6% Elapsed: 18:02:57 temperature: 1.0 ada_p: No Discriminator_loss: 5.23154 Generator_loss: 6.01157 
    [INFO] 2021-02-21 15:22:15 > Step:   27300 Progress: 13.7% Elapsed: 18:06:45 temperature: 1.0 ada_p: No Discriminator_loss: 5.08663 Generator_loss: 6.27192 
    [INFO] 2021-02-21 15:26:03 > Step:   27400 Progress: 13.7% Elapsed: 18:10:33 temperature: 1.0 ada_p: No Discriminator_loss: 5.31356 Generator_loss: 5.89537 
    [INFO] 2021-02-21 15:29:50 > Step:   27500 Progress: 13.8% Elapsed: 18:14:20 temperature: 1.0 ada_p: No Discriminator_loss: 5.24365 Generator_loss: 6.04099 
    [INFO] 2021-02-21 15:33:42 > Step:   27600 Progress: 13.8% Elapsed: 18:18:12 temperature: 1.0 ada_p: No Discriminator_loss: 5.28746 Generator_loss: 6.14013 
    [INFO] 2021-02-21 15:37:29 > Step:   27700 Progress: 13.9% Elapsed: 18:21:59 temperature: 1.0 ada_p: No Discriminator_loss: 5.37677 Generator_loss: 5.9664 
    [INFO] 2021-02-21 15:41:18 > Step:   27800 Progress: 13.9% Elapsed: 18:25:48 temperature: 1.0 ada_p: No Discriminator_loss: 5.17099 Generator_loss: 6.02813 
    [INFO] 2021-02-21 15:45:06 > Step:   27900 Progress: 14.0% Elapsed: 18:29:35 temperature: 1.0 ada_p: No Discriminator_loss: 5.26997 Generator_loss: 6.05444 
    [INFO] 2021-02-21 15:48:53 > Step:   28000 Progress: 14.0% Elapsed: 18:33:23 temperature: 1.0 ada_p: No Discriminator_loss: 5.15394 Generator_loss: 6.20182 
    [INFO] 2021-02-21 15:52:41 > Step:   28100 Progress: 14.1% Elapsed: 18:37:11 temperature: 1.0 ada_p: No Discriminator_loss: 5.17112 Generator_loss: 5.96137 
    [INFO] 2021-02-21 15:56:29 > Step:   28200 Progress: 14.1% Elapsed: 18:40:59 temperature: 1.0 ada_p: No Discriminator_loss: 5.27724 Generator_loss: 5.94362 
    [INFO] 2021-02-21 16:00:16 > Step:   28300 Progress: 14.1% Elapsed: 18:44:46 temperature: 1.0 ada_p: No Discriminator_loss: 5.30755 Generator_loss: 5.96189 
    [INFO] 2021-02-21 16:04:05 > Step:   28400 Progress: 14.2% Elapsed: 18:48:35 temperature: 1.0 ada_p: No Discriminator_loss: 5.29154 Generator_loss: 5.83738 
    [INFO] 2021-02-21 16:07:52 > Step:   28500 Progress: 14.2% Elapsed: 18:52:22 temperature: 1.0 ada_p: No Discriminator_loss: 5.09773 Generator_loss: 6.32061 
    [INFO] 2021-02-21 16:11:40 > Step:   28600 Progress: 14.3% Elapsed: 18:56:10 temperature: 1.0 ada_p: No Discriminator_loss: 5.20981 Generator_loss: 6.31188 
    [INFO] 2021-02-21 16:15:27 > Step:   28700 Progress: 14.3% Elapsed: 18:59:57 temperature: 1.0 ada_p: No Discriminator_loss: 5.28109 Generator_loss: 6.04915 
    [INFO] 2021-02-21 16:19:15 > Step:   28800 Progress: 14.4% Elapsed: 19:03:45 temperature: 1.0 ada_p: No Discriminator_loss: 5.38544 Generator_loss: 5.92067 
    [INFO] 2021-02-21 16:23:03 > Step:   28900 Progress: 14.4% Elapsed: 19:07:33 temperature: 1.0 ada_p: No Discriminator_loss: 5.34409 Generator_loss: 5.8407 
    [INFO] 2021-02-21 16:26:51 > Step:   29000 Progress: 14.5% Elapsed: 19:11:21 temperature: 1.0 ada_p: No Discriminator_loss: 5.38391 Generator_loss: 5.95504 
    [INFO] 2021-02-21 16:30:40 > Step:   29100 Progress: 14.5% Elapsed: 19:15:10 temperature: 1.0 ada_p: No Discriminator_loss: 5.28637 Generator_loss: 6.12648 
    [INFO] 2021-02-21 16:34:28 > Step:   29200 Progress: 14.6% Elapsed: 19:18:58 temperature: 1.0 ada_p: No Discriminator_loss: 5.10238 Generator_loss: 6.19373 
    [INFO] 2021-02-21 16:38:15 > Step:   29300 Progress: 14.6% Elapsed: 19:22:45 temperature: 1.0 ada_p: No Discriminator_loss: 5.15797 Generator_loss: 6.44453 
    [INFO] 2021-02-21 16:42:03 > Step:   29400 Progress: 14.7% Elapsed: 19:26:33 temperature: 1.0 ada_p: No Discriminator_loss: 5.67744 Generator_loss: 5.87075 
    [INFO] 2021-02-21 16:45:51 > Step:   29500 Progress: 14.8% Elapsed: 19:30:21 temperature: 1.0 ada_p: No Discriminator_loss: 5.15018 Generator_loss: 6.15653 
    [INFO] 2021-02-21 16:49:39 > Step:   29600 Progress: 14.8% Elapsed: 19:34:09 temperature: 1.0 ada_p: No Discriminator_loss: 5.53126 Generator_loss: 5.85588 
    [INFO] 2021-02-21 16:53:27 > Step:   29700 Progress: 14.8% Elapsed: 19:37:57 temperature: 1.0 ada_p: No Discriminator_loss: 5.04112 Generator_loss: 6.58311 
    [INFO] 2021-02-21 16:57:15 > Step:   29800 Progress: 14.9% Elapsed: 19:41:44 temperature: 1.0 ada_p: No Discriminator_loss: 5.34367 Generator_loss: 5.9395 
    [INFO] 2021-02-21 17:01:03 > Step:   29900 Progress: 14.9% Elapsed: 19:45:32 temperature: 1.0 ada_p: No Discriminator_loss: 5.2917 Generator_loss: 5.91999 
    [INFO] 2021-02-21 17:04:50 > Step:   30000 Progress: 15.0% Elapsed: 19:49:20 temperature: 1.0 ada_p: No Discriminator_loss: 5.89434 Generator_loss: 5.77889 
    [INFO] 2021-02-21 17:04:50 > Start Evaluation (30000 Step): ContraGAN256-train-2021_02_20_21_08_34
    [INFO] 2021-02-21 17:04:50 > Start Evaluation FID score ... 
    [INFO] 2021-02-21 17:04:50 > Calculating FID Score....
    [INFO] 2021-02-21 17:06:05 > Start Evaluation IS score ... 
    [INFO] 2021-02-21 17:06:05 > Calculating Inception Score....
    [INFO] 2021-02-21 17:07:05 > Start Evaluation F Beta score ... 
    [INFO] 2021-02-21 17:07:05 > Calculate F_beta Score....
    [INFO] 2021-02-21 17:20:17 > Calculate Accuracies....
    [INFO] 2021-02-21 17:21:34 > Calculate Accuracies....
    [INFO] 2021-02-21 17:27:53 > F_8 score (Step: 30000, Using valid images): 0.5136544879692874
    [INFO] 2021-02-21 17:27:53 > F_1/8 score (Step: 30000, Using valid images): 0.6867006861692394
    [INFO] 2021-02-21 17:27:53 > FID score (Step: 30000, Using valid moments): 70.19643745527605
    [INFO] 2021-02-21 17:27:53 > Inception score (Step: 30000, 50000 generated images): 12.659643173217773
    [INFO] 2021-02-21 17:27:53 > Best FID score (Step: 30000, Using valid moments): 70.19643745527605
    [INFO] 2021-02-21 17:28:00 > Save model to checkpoints/ContraGAN256-train-2021_02_20_21_08_34
    [INFO] 2021-02-21 17:31:52 > Step:   30100 Progress: 15.0% Elapsed: 20:16:22 temperature: 1.0 ada_p: No Discriminator_loss: 5.05155 Generator_loss: 6.20151 
    [INFO] 2021-02-21 17:35:39 > Step:   30200 Progress: 15.1% Elapsed: 20:20:09 temperature: 1.0 ada_p: No Discriminator_loss: 5.47695 Generator_loss: 5.79782 
    [INFO] 2021-02-21 17:39:27 > Step:   30300 Progress: 15.2% Elapsed: 20:23:57 temperature: 1.0 ada_p: No Discriminator_loss: 5.02515 Generator_loss: 6.40599 
    [INFO] 2021-02-21 17:43:33 > Step:   30400 Progress: 15.2% Elapsed: 20:28:03 temperature: 1.0 ada_p: No Discriminator_loss: 5.36418 Generator_loss: 5.97274 
    [INFO] 2021-02-21 17:47:21 > Step:   30500 Progress: 15.2% Elapsed: 20:31:51 temperature: 1.0 ada_p: No Discriminator_loss: 5.19326 Generator_loss: 6.03908 
    [INFO] 2021-02-21 17:51:09 > Step:   30600 Progress: 15.3% Elapsed: 20:35:39 temperature: 1.0 ada_p: No Discriminator_loss: 5.15559 Generator_loss: 6.38265 
    [INFO] 2021-02-21 17:54:57 > Step:   30700 Progress: 15.3% Elapsed: 20:39:27 temperature: 1.0 ada_p: No Discriminator_loss: 5.12741 Generator_loss: 6.59032 
    [INFO] 2021-02-21 17:58:44 > Step:   30800 Progress: 15.4% Elapsed: 20:43:14 temperature: 1.0 ada_p: No Discriminator_loss: 5.27531 Generator_loss: 6.49408 
    [INFO] 2021-02-21 18:02:32 > Step:   30900 Progress: 15.4% Elapsed: 20:47:02 temperature: 1.0 ada_p: No Discriminator_loss: 5.28838 Generator_loss: 6.12299 
    [INFO] 2021-02-21 18:06:20 > Step:   31000 Progress: 15.5% Elapsed: 20:50:50 temperature: 1.0 ada_p: No Discriminator_loss: 5.28347 Generator_loss: 6.15947 
    [INFO] 2021-02-21 18:10:08 > Step:   31100 Progress: 15.6% Elapsed: 20:54:38 temperature: 1.0 ada_p: No Discriminator_loss: 5.12922 Generator_loss: 6.23659 
    [INFO] 2021-02-21 18:13:56 > Step:   31200 Progress: 15.6% Elapsed: 20:58:26 temperature: 1.0 ada_p: No Discriminator_loss: 5.36957 Generator_loss: 6.7195 
    [INFO] 2021-02-21 18:17:43 > Step:   31300 Progress: 15.7% Elapsed: 21:02:13 temperature: 1.0 ada_p: No Discriminator_loss: 5.44033 Generator_loss: 5.97841 
    [INFO] 2021-02-21 18:21:31 > Step:   31400 Progress: 15.7% Elapsed: 21:06:01 temperature: 1.0 ada_p: No Discriminator_loss: 5.12908 Generator_loss: 6.32009 
    [INFO] 2021-02-21 18:25:20 > Step:   31500 Progress: 15.8% Elapsed: 21:09:50 temperature: 1.0 ada_p: No Discriminator_loss: 5.0962 Generator_loss: 5.99568 
    [INFO] 2021-02-21 18:29:08 > Step:   31600 Progress: 15.8% Elapsed: 21:13:38 temperature: 1.0 ada_p: No Discriminator_loss: 5.08896 Generator_loss: 6.20694 
    [INFO] 2021-02-21 18:32:56 > Step:   31700 Progress: 15.8% Elapsed: 21:17:26 temperature: 1.0 ada_p: No Discriminator_loss: 5.25247 Generator_loss: 5.81306 
    [INFO] 2021-02-21 18:36:44 > Step:   31800 Progress: 15.9% Elapsed: 21:21:14 temperature: 1.0 ada_p: No Discriminator_loss: 5.17325 Generator_loss: 6.35757 
    [INFO] 2021-02-21 18:40:32 > Step:   31900 Progress: 16.0% Elapsed: 21:25:02 temperature: 1.0 ada_p: No Discriminator_loss: 5.21564 Generator_loss: 5.96355 
    [INFO] 2021-02-21 18:44:21 > Step:   32000 Progress: 16.0% Elapsed: 21:28:51 temperature: 1.0 ada_p: No Discriminator_loss: 5.03802 Generator_loss: 6.34164 
    [INFO] 2021-02-21 18:48:09 > Step:   32100 Progress: 16.1% Elapsed: 21:32:39 temperature: 1.0 ada_p: No Discriminator_loss: 5.18154 Generator_loss: 5.96444 
    [INFO] 2021-02-21 18:51:58 > Step:   32200 Progress: 16.1% Elapsed: 21:36:28 temperature: 1.0 ada_p: No Discriminator_loss: 5.18579 Generator_loss: 6.13574 
    [INFO] 2021-02-21 18:55:46 > Step:   32300 Progress: 16.2% Elapsed: 21:40:16 temperature: 1.0 ada_p: No Discriminator_loss: 5.35457 Generator_loss: 6.05285 
    [INFO] 2021-02-21 18:59:34 > Step:   32400 Progress: 16.2% Elapsed: 21:44:04 temperature: 1.0 ada_p: No Discriminator_loss: 5.16543 Generator_loss: 6.12303 
    [INFO] 2021-02-21 19:03:22 > Step:   32500 Progress: 16.2% Elapsed: 21:47:52 temperature: 1.0 ada_p: No Discriminator_loss: 5.10238 Generator_loss: 6.20554 
    [INFO] 2021-02-21 19:07:15 > Step:   32600 Progress: 16.3% Elapsed: 21:51:45 temperature: 1.0 ada_p: No Discriminator_loss: 5.0999 Generator_loss: 6.07568 
    [INFO] 2021-02-21 19:11:04 > Step:   32700 Progress: 16.4% Elapsed: 21:55:34 temperature: 1.0 ada_p: No Discriminator_loss: 5.13936 Generator_loss: 6.131 
    [INFO] 2021-02-21 19:14:52 > Step:   32800 Progress: 16.4% Elapsed: 21:59:22 temperature: 1.0 ada_p: No Discriminator_loss: 5.20239 Generator_loss: 6.01947 
    [INFO] 2021-02-21 19:18:41 > Step:   32900 Progress: 16.4% Elapsed: 22:03:11 temperature: 1.0 ada_p: No Discriminator_loss: 5.32076 Generator_loss: 5.94062 
    [INFO] 2021-02-21 19:22:29 > Step:   33000 Progress: 16.5% Elapsed: 22:06:59 temperature: 1.0 ada_p: No Discriminator_loss: 5.28047 Generator_loss: 6.04741 
    [INFO] 2021-02-21 19:26:17 > Step:   33100 Progress: 16.6% Elapsed: 22:10:47 temperature: 1.0 ada_p: No Discriminator_loss: 5.3496 Generator_loss: 5.94781 
    [INFO] 2021-02-21 19:30:05 > Step:   33200 Progress: 16.6% Elapsed: 22:14:35 temperature: 1.0 ada_p: No Discriminator_loss: 5.20924 Generator_loss: 5.92055 
    [INFO] 2021-02-21 19:33:54 > Step:   33300 Progress: 16.7% Elapsed: 22:18:24 temperature: 1.0 ada_p: No Discriminator_loss: 5.21588 Generator_loss: 6.19786 
    [INFO] 2021-02-21 19:37:42 > Step:   33400 Progress: 16.7% Elapsed: 22:22:12 temperature: 1.0 ada_p: No Discriminator_loss: 5.27875 Generator_loss: 6.02719 
    [INFO] 2021-02-21 19:41:30 > Step:   33500 Progress: 16.8% Elapsed: 22:26:00 temperature: 1.0 ada_p: No Discriminator_loss: 5.10643 Generator_loss: 6.5457 
    [INFO] 2021-02-21 19:45:19 > Step:   33600 Progress: 16.8% Elapsed: 22:29:49 temperature: 1.0 ada_p: No Discriminator_loss: 5.26658 Generator_loss: 6.03168 
    [INFO] 2021-02-21 19:49:07 > Step:   33700 Progress: 16.9% Elapsed: 22:33:37 temperature: 1.0 ada_p: No Discriminator_loss: 5.0798 Generator_loss: 6.3885 
    [INFO] 2021-02-21 19:52:56 > Step:   33800 Progress: 16.9% Elapsed: 22:37:26 temperature: 1.0 ada_p: No Discriminator_loss: 5.39332 Generator_loss: 6.03879 
    [INFO] 2021-02-21 19:56:44 > Step:   33900 Progress: 17.0% Elapsed: 22:41:14 temperature: 1.0 ada_p: No Discriminator_loss: 5.14021 Generator_loss: 6.20322 
    [INFO] 2021-02-21 20:00:32 > Step:   34000 Progress: 17.0% Elapsed: 22:45:02 temperature: 1.0 ada_p: No Discriminator_loss: 5.09532 Generator_loss: 6.47567 
    [INFO] 2021-02-21 20:04:21 > Step:   34100 Progress: 17.1% Elapsed: 22:48:51 temperature: 1.0 ada_p: No Discriminator_loss: 5.13878 Generator_loss: 6.3769 
    [INFO] 2021-02-21 20:08:09 > Step:   34200 Progress: 17.1% Elapsed: 22:52:39 temperature: 1.0 ada_p: No Discriminator_loss: 5.29317 Generator_loss: 5.9889 
    [INFO] 2021-02-21 20:11:58 > Step:   34300 Progress: 17.2% Elapsed: 22:56:28 temperature: 1.0 ada_p: No Discriminator_loss: 5.09485 Generator_loss: 6.31649 
    [INFO] 2021-02-21 20:15:46 > Step:   34400 Progress: 17.2% Elapsed: 23:00:16 temperature: 1.0 ada_p: No Discriminator_loss: 5.09831 Generator_loss: 6.31191 
    [INFO] 2021-02-21 20:19:34 > Step:   34500 Progress: 17.2% Elapsed: 23:04:04 temperature: 1.0 ada_p: No Discriminator_loss: 5.20673 Generator_loss: 5.98918 
    [INFO] 2021-02-21 20:23:22 > Step:   34600 Progress: 17.3% Elapsed: 23:07:52 temperature: 1.0 ada_p: No Discriminator_loss: 5.56494 Generator_loss: 5.93731 
    [INFO] 2021-02-21 20:27:10 > Step:   34700 Progress: 17.3% Elapsed: 23:11:40 temperature: 1.0 ada_p: No Discriminator_loss: 5.03194 Generator_loss: 6.49179 
    [INFO] 2021-02-21 20:30:58 > Step:   34800 Progress: 17.4% Elapsed: 23:15:28 temperature: 1.0 ada_p: No Discriminator_loss: 5.38373 Generator_loss: 5.96955 
    [INFO] 2021-02-21 20:34:47 > Step:   34900 Progress: 17.4% Elapsed: 23:19:17 temperature: 1.0 ada_p: No Discriminator_loss: 5.15426 Generator_loss: 6.23223 
    [INFO] 2021-02-21 20:38:35 > Step:   35000 Progress: 17.5% Elapsed: 23:23:05 temperature: 1.0 ada_p: No Discriminator_loss: 5.07036 Generator_loss: 6.38467 
    [INFO] 2021-02-21 20:42:26 > Step:   35100 Progress: 17.5% Elapsed: 23:26:56 temperature: 1.0 ada_p: No Discriminator_loss: 5.11514 Generator_loss: 6.68027 
    [INFO] 2021-02-21 20:46:15 > Step:   35200 Progress: 17.6% Elapsed: 23:30:45 temperature: 1.0 ada_p: No Discriminator_loss: 5.16605 Generator_loss: 6.10298 
    [INFO] 2021-02-21 20:50:03 > Step:   35300 Progress: 17.6% Elapsed: 23:34:33 temperature: 1.0 ada_p: No Discriminator_loss: 5.29572 Generator_loss: 6.45219 
    [INFO] 2021-02-21 20:53:50 > Step:   35400 Progress: 17.7% Elapsed: 23:38:20 temperature: 1.0 ada_p: No Discriminator_loss: 5.14585 Generator_loss: 6.62538 
    [INFO] 2021-02-21 20:57:39 > Step:   35500 Progress: 17.8% Elapsed: 23:42:09 temperature: 1.0 ada_p: No Discriminator_loss: 5.15845 Generator_loss: 5.94217 
    [INFO] 2021-02-21 21:01:27 > Step:   35600 Progress: 17.8% Elapsed: 23:45:57 temperature: 1.0 ada_p: No Discriminator_loss: 5.06746 Generator_loss: 6.28141 
    [INFO] 2021-02-21 21:05:15 > Step:   35700 Progress: 17.8% Elapsed: 23:49:45 temperature: 1.0 ada_p: No Discriminator_loss: 5.30422 Generator_loss: 6.02137 
    [INFO] 2021-02-21 21:09:04 > Step:   35800 Progress: 17.9% Elapsed: 23:53:34 temperature: 1.0 ada_p: No Discriminator_loss: 5.24561 Generator_loss: 5.98001 
    [INFO] 2021-02-21 21:12:52 > Step:   35900 Progress: 17.9% Elapsed: 23:57:22 temperature: 1.0 ada_p: No Discriminator_loss: 5.0775 Generator_loss: 6.40101 
    [INFO] 2021-02-21 21:16:39 > Step:   36000 Progress: 18.0% Elapsed: 1 day, 0:01:09 temperature: 1.0 ada_p: No Discriminator_loss: 5.27999 Generator_loss: 6.18003 
    [INFO] 2021-02-21 21:20:27 > Step:   36100 Progress: 18.1% Elapsed: 1 day, 0:04:57 temperature: 1.0 ada_p: No Discriminator_loss: 5.04727 Generator_loss: 6.28657 
    [INFO] 2021-02-21 21:24:15 > Step:   36200 Progress: 18.1% Elapsed: 1 day, 0:08:45 temperature: 1.0 ada_p: No Discriminator_loss: 5.08966 Generator_loss: 6.14284 
    [INFO] 2021-02-21 21:28:03 > Step:   36300 Progress: 18.1% Elapsed: 1 day, 0:12:33 temperature: 1.0 ada_p: No Discriminator_loss: 5.25269 Generator_loss: 6.20099 
    [INFO] 2021-02-21 21:31:51 > Step:   36400 Progress: 18.2% Elapsed: 1 day, 0:16:21 temperature: 1.0 ada_p: No Discriminator_loss: 5.15676 Generator_loss: 6.12451 
    [INFO] 2021-02-21 21:35:39 > Step:   36500 Progress: 18.2% Elapsed: 1 day, 0:20:09 temperature: 1.0 ada_p: No Discriminator_loss: 5.09109 Generator_loss: 6.43937 
    [INFO] 2021-02-21 21:39:27 > Step:   36600 Progress: 18.3% Elapsed: 1 day, 0:23:57 temperature: 1.0 ada_p: No Discriminator_loss: 5.79462 Generator_loss: 5.80056 
    [INFO] 2021-02-21 21:43:15 > Step:   36700 Progress: 18.4% Elapsed: 1 day, 0:27:45 temperature: 1.0 ada_p: No Discriminator_loss: 5.41256 Generator_loss: 5.96121 
    [INFO] 2021-02-21 21:47:03 > Step:   36800 Progress: 18.4% Elapsed: 1 day, 0:31:33 temperature: 1.0 ada_p: No Discriminator_loss: 5.05075 Generator_loss: 6.36022 
    [INFO] 2021-02-21 21:50:51 > Step:   36900 Progress: 18.4% Elapsed: 1 day, 0:35:21 temperature: 1.0 ada_p: No Discriminator_loss: 5.13126 Generator_loss: 6.08407 
    [INFO] 2021-02-21 21:54:39 > Step:   37000 Progress: 18.5% Elapsed: 1 day, 0:39:09 temperature: 1.0 ada_p: No Discriminator_loss: 5.01868 Generator_loss: 6.18073 
    [INFO] 2021-02-21 21:58:27 > Step:   37100 Progress: 18.6% Elapsed: 1 day, 0:42:57 temperature: 1.0 ada_p: No Discriminator_loss: 5.26267 Generator_loss: 5.94035 
    [INFO] 2021-02-21 22:02:15 > Step:   37200 Progress: 18.6% Elapsed: 1 day, 0:46:45 temperature: 1.0 ada_p: No Discriminator_loss: 5.11207 Generator_loss: 6.33365 
    [INFO] 2021-02-21 22:06:04 > Step:   37300 Progress: 18.6% Elapsed: 1 day, 0:50:34 temperature: 1.0 ada_p: No Discriminator_loss: 5.2997 Generator_loss: 5.82954 
    [INFO] 2021-02-21 22:09:52 > Step:   37400 Progress: 18.7% Elapsed: 1 day, 0:54:22 temperature: 1.0 ada_p: No Discriminator_loss: 5.18945 Generator_loss: 6.2154 
    [INFO] 2021-02-21 22:13:40 > Step:   37500 Progress: 18.8% Elapsed: 1 day, 0:58:10 temperature: 1.0 ada_p: No Discriminator_loss: 5.35028 Generator_loss: 6.19939 
    [INFO] 2021-02-21 22:17:32 > Step:   37600 Progress: 18.8% Elapsed: 1 day, 1:02:01 temperature: 1.0 ada_p: No Discriminator_loss: 5.20258 Generator_loss: 6.01062 
    [INFO] 2021-02-21 22:21:20 > Step:   37700 Progress: 18.9% Elapsed: 1 day, 1:05:50 temperature: 1.0 ada_p: No Discriminator_loss: 4.98773 Generator_loss: 6.27408 
    [INFO] 2021-02-21 22:25:07 > Step:   37800 Progress: 18.9% Elapsed: 1 day, 1:09:37 temperature: 1.0 ada_p: No Discriminator_loss: 5.28536 Generator_loss: 6.07578 
    [INFO] 2021-02-21 22:28:55 > Step:   37900 Progress: 18.9% Elapsed: 1 day, 1:13:25 temperature: 1.0 ada_p: No Discriminator_loss: 5.0776 Generator_loss: 6.63166 
    [INFO] 2021-02-21 22:32:43 > Step:   38000 Progress: 19.0% Elapsed: 1 day, 1:17:13 temperature: 1.0 ada_p: No Discriminator_loss: 4.99677 Generator_loss: 6.09053 
    [INFO] 2021-02-21 22:36:31 > Step:   38100 Progress: 19.1% Elapsed: 1 day, 1:21:01 temperature: 1.0 ada_p: No Discriminator_loss: 5.24304 Generator_loss: 6.1091 
    [INFO] 2021-02-21 22:40:19 > Step:   38200 Progress: 19.1% Elapsed: 1 day, 1:24:49 temperature: 1.0 ada_p: No Discriminator_loss: 5.20614 Generator_loss: 5.94099 
    [INFO] 2021-02-21 22:44:07 > Step:   38300 Progress: 19.1% Elapsed: 1 day, 1:28:37 temperature: 1.0 ada_p: No Discriminator_loss: 5.15014 Generator_loss: 6.3009 
    [INFO] 2021-02-21 22:47:55 > Step:   38400 Progress: 19.2% Elapsed: 1 day, 1:32:25 temperature: 1.0 ada_p: No Discriminator_loss: 5.1578 Generator_loss: 5.98558 
    [INFO] 2021-02-21 22:51:43 > Step:   38500 Progress: 19.2% Elapsed: 1 day, 1:36:13 temperature: 1.0 ada_p: No Discriminator_loss: 5.07948 Generator_loss: 6.17212 
    [INFO] 2021-02-21 22:55:31 > Step:   38600 Progress: 19.3% Elapsed: 1 day, 1:40:00 temperature: 1.0 ada_p: No Discriminator_loss: 5.10417 Generator_loss: 6.22869 
    [INFO] 2021-02-21 22:59:19 > Step:   38700 Progress: 19.4% Elapsed: 1 day, 1:43:49 temperature: 1.0 ada_p: No Discriminator_loss: 5.07529 Generator_loss: 6.64336 
    [INFO] 2021-02-21 23:03:07 > Step:   38800 Progress: 19.4% Elapsed: 1 day, 1:47:37 temperature: 1.0 ada_p: No Discriminator_loss: 5.25276 Generator_loss: 6.13442 
    [INFO] 2021-02-21 23:06:55 > Step:   38900 Progress: 19.4% Elapsed: 1 day, 1:51:25 temperature: 1.0 ada_p: No Discriminator_loss: 5.11438 Generator_loss: 6.10336 
    [INFO] 2021-02-21 23:10:43 > Step:   39000 Progress: 19.5% Elapsed: 1 day, 1:55:13 temperature: 1.0 ada_p: No Discriminator_loss: 5.17888 Generator_loss: 6.39192 
    [INFO] 2021-02-21 23:14:31 > Step:   39100 Progress: 19.6% Elapsed: 1 day, 1:59:01 temperature: 1.0 ada_p: No Discriminator_loss: 5.369 Generator_loss: 5.84534 
    [INFO] 2021-02-21 23:18:19 > Step:   39200 Progress: 19.6% Elapsed: 1 day, 2:02:49 temperature: 1.0 ada_p: No Discriminator_loss: 5.27153 Generator_loss: 6.18144 
    [INFO] 2021-02-21 23:22:07 > Step:   39300 Progress: 19.7% Elapsed: 1 day, 2:06:37 temperature: 1.0 ada_p: No Discriminator_loss: 5.10056 Generator_loss: 6.16416 
    [INFO] 2021-02-21 23:25:55 > Step:   39400 Progress: 19.7% Elapsed: 1 day, 2:10:25 temperature: 1.0 ada_p: No Discriminator_loss: 5.12949 Generator_loss: 6.20841 
    [INFO] 2021-02-21 23:29:43 > Step:   39500 Progress: 19.8% Elapsed: 1 day, 2:14:13 temperature: 1.0 ada_p: No Discriminator_loss: 5.0778 Generator_loss: 6.14455 
    [INFO] 2021-02-21 23:33:31 > Step:   39600 Progress: 19.8% Elapsed: 1 day, 2:18:01 temperature: 1.0 ada_p: No Discriminator_loss: 5.20052 Generator_loss: 5.99851 
    [INFO] 2021-02-21 23:37:20 > Step:   39700 Progress: 19.9% Elapsed: 1 day, 2:21:50 temperature: 1.0 ada_p: No Discriminator_loss: 5.08284 Generator_loss: 6.01853 
    [INFO] 2021-02-21 23:41:08 > Step:   39800 Progress: 19.9% Elapsed: 1 day, 2:25:38 temperature: 1.0 ada_p: No Discriminator_loss: 5.34302 Generator_loss: 5.85601 
    [INFO] 2021-02-21 23:44:58 > Step:   39900 Progress: 20.0% Elapsed: 1 day, 2:29:28 temperature: 1.0 ada_p: No Discriminator_loss: 5.14532 Generator_loss: 6.19045 
    [INFO] 2021-02-21 23:48:47 > Step:   40000 Progress: 20.0% Elapsed: 1 day, 2:33:17 temperature: 1.0 ada_p: No Discriminator_loss: 5.13558 Generator_loss: 6.52147 
    [INFO] 2021-02-21 23:48:47 > Start Evaluation (40000 Step): ContraGAN256-train-2021_02_20_21_08_34
    [INFO] 2021-02-21 23:48:47 > Start Evaluation FID score ... 
    [INFO] 2021-02-21 23:48:47 > Calculating FID Score....
    [INFO] 2021-02-21 23:50:03 > Start Evaluation IS score ... 
    [INFO] 2021-02-21 23:50:03 > Calculating Inception Score....
    [INFO] 2021-02-21 23:51:04 > Start Evaluation F Beta score ... 
    [INFO] 2021-02-21 23:51:04 > Calculate F_beta Score....
    [INFO] 2021-02-22 00:04:11 > Calculate Accuracies....
    [INFO] 2021-02-22 00:05:28 > Calculate Accuracies....
    [INFO] 2021-02-22 00:11:48 > F_8 score (Step: 40000, Using valid images): 0.544327256184011
    [INFO] 2021-02-22 00:11:48 > F_1/8 score (Step: 40000, Using valid images): 0.7119455113812765
    [INFO] 2021-02-22 00:11:48 > FID score (Step: 40000, Using valid moments): 64.90893037449274
    [INFO] 2021-02-22 00:11:48 > Inception score (Step: 40000, 50000 generated images): 13.746240615844727
    [INFO] 2021-02-22 00:11:48 > Best FID score (Step: 40000, Using valid moments): 64.90893037449274
    [INFO] 2021-02-22 00:11:55 > Save model to checkpoints/ContraGAN256-train-2021_02_20_21_08_34
    [INFO] 2021-02-22 00:15:45 > Step:   40100 Progress: 20.1% Elapsed: 1 day, 3:00:15 temperature: 1.0 ada_p: No Discriminator_loss: 5.16077 Generator_loss: 6.14215 
    [INFO] 2021-02-22 00:19:33 > Step:   40200 Progress: 20.1% Elapsed: 1 day, 3:04:03 temperature: 1.0 ada_p: No Discriminator_loss: 5.00183 Generator_loss: 6.72474 
    [INFO] 2021-02-22 00:23:21 > Step:   40300 Progress: 20.2% Elapsed: 1 day, 3:07:51 temperature: 1.0 ada_p: No Discriminator_loss: 5.26301 Generator_loss: 5.91268 
    [INFO] 2021-02-22 00:27:10 > Step:   40400 Progress: 20.2% Elapsed: 1 day, 3:11:39 temperature: 1.0 ada_p: No Discriminator_loss: 5.21824 Generator_loss: 5.79094 
    [INFO] 2021-02-22 00:30:57 > Step:   40500 Progress: 20.2% Elapsed: 1 day, 3:15:27 temperature: 1.0 ada_p: No Discriminator_loss: 5.14668 Generator_loss: 6.08706 
    [INFO] 2021-02-22 00:34:45 > Step:   40600 Progress: 20.3% Elapsed: 1 day, 3:19:15 temperature: 1.0 ada_p: No Discriminator_loss: 5.01606 Generator_loss: 6.50676 
    [INFO] 2021-02-22 00:38:52 > Step:   40700 Progress: 20.3% Elapsed: 1 day, 3:23:22 temperature: 1.0 ada_p: No Discriminator_loss: 4.97037 Generator_loss: 6.62912 
    [INFO] 2021-02-22 00:42:40 > Step:   40800 Progress: 20.4% Elapsed: 1 day, 3:27:10 temperature: 1.0 ada_p: No Discriminator_loss: 5.01534 Generator_loss: 6.62869 
    [INFO] 2021-02-22 00:46:29 > Step:   40900 Progress: 20.4% Elapsed: 1 day, 3:30:59 temperature: 1.0 ada_p: No Discriminator_loss: 5.15178 Generator_loss: 6.09659 
    [INFO] 2021-02-22 00:50:18 > Step:   41000 Progress: 20.5% Elapsed: 1 day, 3:34:48 temperature: 1.0 ada_p: No Discriminator_loss: 5.1372 Generator_loss: 5.86216 
    [INFO] 2021-02-22 00:54:06 > Step:   41100 Progress: 20.5% Elapsed: 1 day, 3:38:36 temperature: 1.0 ada_p: No Discriminator_loss: 5.67514 Generator_loss: 5.77892 
    [INFO] 2021-02-22 00:57:55 > Step:   41200 Progress: 20.6% Elapsed: 1 day, 3:42:25 temperature: 1.0 ada_p: No Discriminator_loss: 5.17361 Generator_loss: 6.1235 
    [INFO] 2021-02-22 01:01:43 > Step:   41300 Progress: 20.6% Elapsed: 1 day, 3:46:13 temperature: 1.0 ada_p: No Discriminator_loss: 5.10543 Generator_loss: 6.0512 
    [INFO] 2021-02-22 01:05:32 > Step:   41400 Progress: 20.7% Elapsed: 1 day, 3:50:01 temperature: 1.0 ada_p: No Discriminator_loss: 5.19599 Generator_loss: 6.07129 
    [INFO] 2021-02-22 01:09:20 > Step:   41500 Progress: 20.8% Elapsed: 1 day, 3:53:50 temperature: 1.0 ada_p: No Discriminator_loss: 5.16993 Generator_loss: 5.90154 
    [INFO] 2021-02-22 01:13:08 > Step:   41600 Progress: 20.8% Elapsed: 1 day, 3:57:38 temperature: 1.0 ada_p: No Discriminator_loss: 5.352 Generator_loss: 5.25947 
    [INFO] 2021-02-22 01:16:56 > Step:   41700 Progress: 20.8% Elapsed: 1 day, 4:01:26 temperature: 1.0 ada_p: No Discriminator_loss: 5.35937 Generator_loss: 5.97781 
    [INFO] 2021-02-22 01:20:44 > Step:   41800 Progress: 20.9% Elapsed: 1 day, 4:05:14 temperature: 1.0 ada_p: No Discriminator_loss: 5.15923 Generator_loss: 6.03984 
    [INFO] 2021-02-22 01:24:33 > Step:   41900 Progress: 20.9% Elapsed: 1 day, 4:09:03 temperature: 1.0 ada_p: No Discriminator_loss: 5.26952 Generator_loss: 6.07482 
    [INFO] 2021-02-22 01:28:21 > Step:   42000 Progress: 21.0% Elapsed: 1 day, 4:12:51 temperature: 1.0 ada_p: No Discriminator_loss: 5.10792 Generator_loss: 6.15056 
    [INFO] 2021-02-22 01:32:09 > Step:   42100 Progress: 21.1% Elapsed: 1 day, 4:16:39 temperature: 1.0 ada_p: No Discriminator_loss: 5.06296 Generator_loss: 6.23929 
    [INFO] 2021-02-22 01:35:57 > Step:   42200 Progress: 21.1% Elapsed: 1 day, 4:20:27 temperature: 1.0 ada_p: No Discriminator_loss: 5.0585 Generator_loss: 6.26239 
    [INFO] 2021-02-22 01:39:46 > Step:   42300 Progress: 21.1% Elapsed: 1 day, 4:24:16 temperature: 1.0 ada_p: No Discriminator_loss: 5.13536 Generator_loss: 5.82622 
    [INFO] 2021-02-22 01:43:34 > Step:   42400 Progress: 21.2% Elapsed: 1 day, 4:28:04 temperature: 1.0 ada_p: No Discriminator_loss: 5.25286 Generator_loss: 5.90151 
    [INFO] 2021-02-22 01:47:23 > Step:   42500 Progress: 21.2% Elapsed: 1 day, 4:31:53 temperature: 1.0 ada_p: No Discriminator_loss: 5.18182 Generator_loss: 6.19503 
    [INFO] 2021-02-22 01:51:16 > Step:   42600 Progress: 21.3% Elapsed: 1 day, 4:35:46 temperature: 1.0 ada_p: No Discriminator_loss: 5.57954 Generator_loss: 5.21755 
    [INFO] 2021-02-22 01:55:05 > Step:   42700 Progress: 21.3% Elapsed: 1 day, 4:39:35 temperature: 1.0 ada_p: No Discriminator_loss: 5.09183 Generator_loss: 6.14043 
    [INFO] 2021-02-22 01:58:54 > Step:   42800 Progress: 21.4% Elapsed: 1 day, 4:43:23 temperature: 1.0 ada_p: No Discriminator_loss: 5.10217 Generator_loss: 6.24005 
    [INFO] 2021-02-22 02:02:42 > Step:   42900 Progress: 21.4% Elapsed: 1 day, 4:47:12 temperature: 1.0 ada_p: No Discriminator_loss: 5.22101 Generator_loss: 6.10067 
    [INFO] 2021-02-22 02:06:31 > Step:   43000 Progress: 21.5% Elapsed: 1 day, 4:51:01 temperature: 1.0 ada_p: No Discriminator_loss: 5.27179 Generator_loss: 6.03899 
    [INFO] 2021-02-22 02:10:20 > Step:   43100 Progress: 21.6% Elapsed: 1 day, 4:54:50 temperature: 1.0 ada_p: No Discriminator_loss: 5.25459 Generator_loss: 6.118 
    [INFO] 2021-02-22 02:14:09 > Step:   43200 Progress: 21.6% Elapsed: 1 day, 4:58:39 temperature: 1.0 ada_p: No Discriminator_loss: 5.13134 Generator_loss: 6.02594 
    [INFO] 2021-02-22 02:17:58 > Step:   43300 Progress: 21.6% Elapsed: 1 day, 5:02:28 temperature: 1.0 ada_p: No Discriminator_loss: 5.01272 Generator_loss: 6.16095 
    [INFO] 2021-02-22 02:21:48 > Step:   43400 Progress: 21.7% Elapsed: 1 day, 5:06:18 temperature: 1.0 ada_p: No Discriminator_loss: 5.11851 Generator_loss: 6.13422 
    [INFO] 2021-02-22 02:25:37 > Step:   43500 Progress: 21.8% Elapsed: 1 day, 5:10:06 temperature: 1.0 ada_p: No Discriminator_loss: 5.01842 Generator_loss: 6.63731 
    [INFO] 2021-02-22 02:29:26 > Step:   43600 Progress: 21.8% Elapsed: 1 day, 5:13:56 temperature: 1.0 ada_p: No Discriminator_loss: 5.10707 Generator_loss: 6.46618 
    [INFO] 2021-02-22 02:33:14 > Step:   43700 Progress: 21.9% Elapsed: 1 day, 5:17:44 temperature: 1.0 ada_p: No Discriminator_loss: 5.42861 Generator_loss: 5.81585 
    [INFO] 2021-02-22 02:37:03 > Step:   43800 Progress: 21.9% Elapsed: 1 day, 5:21:33 temperature: 1.0 ada_p: No Discriminator_loss: 5.17771 Generator_loss: 5.99894 
    [INFO] 2021-02-22 02:40:53 > Step:   43900 Progress: 21.9% Elapsed: 1 day, 5:25:23 temperature: 1.0 ada_p: No Discriminator_loss: 5.07584 Generator_loss: 6.17814 
    [INFO] 2021-02-22 02:44:41 > Step:   44000 Progress: 22.0% Elapsed: 1 day, 5:29:11 temperature: 1.0 ada_p: No Discriminator_loss: 5.14576 Generator_loss: 6.06104 
    [INFO] 2021-02-22 02:48:30 > Step:   44100 Progress: 22.1% Elapsed: 1 day, 5:33:00 temperature: 1.0 ada_p: No Discriminator_loss: 5.20795 Generator_loss: 6.05659 
    [INFO] 2021-02-22 02:52:20 > Step:   44200 Progress: 22.1% Elapsed: 1 day, 5:36:50 temperature: 1.0 ada_p: No Discriminator_loss: 5.23801 Generator_loss: 6.07551 
    [INFO] 2021-02-22 02:56:08 > Step:   44300 Progress: 22.1% Elapsed: 1 day, 5:40:38 temperature: 1.0 ada_p: No Discriminator_loss: 5.00476 Generator_loss: 6.1404 
    [INFO] 2021-02-22 02:59:57 > Step:   44400 Progress: 22.2% Elapsed: 1 day, 5:44:27 temperature: 1.0 ada_p: No Discriminator_loss: 5.08951 Generator_loss: 6.2208 
    [INFO] 2021-02-22 03:03:46 > Step:   44500 Progress: 22.2% Elapsed: 1 day, 5:48:16 temperature: 1.0 ada_p: No Discriminator_loss: 5.13701 Generator_loss: 5.96101 
    [INFO] 2021-02-22 03:07:35 > Step:   44600 Progress: 22.3% Elapsed: 1 day, 5:52:05 temperature: 1.0 ada_p: No Discriminator_loss: 5.1607 Generator_loss: 6.26319 
    [INFO] 2021-02-22 03:11:24 > Step:   44700 Progress: 22.4% Elapsed: 1 day, 5:55:54 temperature: 1.0 ada_p: No Discriminator_loss: 5.11869 Generator_loss: 6.16975 
    [INFO] 2021-02-22 03:15:13 > Step:   44800 Progress: 22.4% Elapsed: 1 day, 5:59:43 temperature: 1.0 ada_p: No Discriminator_loss: 5.03285 Generator_loss: 6.29793 
    [INFO] 2021-02-22 03:19:02 > Step:   44900 Progress: 22.4% Elapsed: 1 day, 6:03:32 temperature: 1.0 ada_p: No Discriminator_loss: 5.31757 Generator_loss: 6.05101 
    [INFO] 2021-02-22 03:22:51 > Step:   45000 Progress: 22.5% Elapsed: 1 day, 6:07:21 temperature: 1.0 ada_p: No Discriminator_loss: 5.23647 Generator_loss: 5.96617 
    [INFO] 2021-02-22 03:26:43 > Step:   45100 Progress: 22.6% Elapsed: 1 day, 6:11:13 temperature: 1.0 ada_p: No Discriminator_loss: 5.07422 Generator_loss: 6.21632 
    [INFO] 2021-02-22 03:30:32 > Step:   45200 Progress: 22.6% Elapsed: 1 day, 6:15:02 temperature: 1.0 ada_p: No Discriminator_loss: 5.12074 Generator_loss: 6.28864 
    [INFO] 2021-02-22 03:34:21 > Step:   45300 Progress: 22.7% Elapsed: 1 day, 6:18:51 temperature: 1.0 ada_p: No Discriminator_loss: 5.13611 Generator_loss: 6.22546 
    [INFO] 2021-02-22 03:38:10 > Step:   45400 Progress: 22.7% Elapsed: 1 day, 6:22:40 temperature: 1.0 ada_p: No Discriminator_loss: 5.12673 Generator_loss: 6.13617 
    [INFO] 2021-02-22 03:41:59 > Step:   45500 Progress: 22.8% Elapsed: 1 day, 6:26:29 temperature: 1.0 ada_p: No Discriminator_loss: 5.12738 Generator_loss: 6.16564 
    [INFO] 2021-02-22 03:45:48 > Step:   45600 Progress: 22.8% Elapsed: 1 day, 6:30:18 temperature: 1.0 ada_p: No Discriminator_loss: 5.01929 Generator_loss: 6.05605 
    [INFO] 2021-02-22 03:49:37 > Step:   45700 Progress: 22.9% Elapsed: 1 day, 6:34:07 temperature: 1.0 ada_p: No Discriminator_loss: 5.01947 Generator_loss: 6.40309 
    [INFO] 2021-02-22 03:53:26 > Step:   45800 Progress: 22.9% Elapsed: 1 day, 6:37:56 temperature: 1.0 ada_p: No Discriminator_loss: 5.16297 Generator_loss: 6.72509 
    [INFO] 2021-02-22 03:57:15 > Step:   45900 Progress: 22.9% Elapsed: 1 day, 6:41:45 temperature: 1.0 ada_p: No Discriminator_loss: 5.04441 Generator_loss: 6.22261 
    [INFO] 2021-02-22 04:01:04 > Step:   46000 Progress: 23.0% Elapsed: 1 day, 6:45:34 temperature: 1.0 ada_p: No Discriminator_loss: 5.03553 Generator_loss: 6.21187 
    [INFO] 2021-02-22 04:04:52 > Step:   46100 Progress: 23.1% Elapsed: 1 day, 6:49:22 temperature: 1.0 ada_p: No Discriminator_loss: 5.11314 Generator_loss: 6.17019 
    [INFO] 2021-02-22 04:08:41 > Step:   46200 Progress: 23.1% Elapsed: 1 day, 6:53:11 temperature: 1.0 ada_p: No Discriminator_loss: 5.22485 Generator_loss: 5.9085 
    [INFO] 2021-02-22 04:12:31 > Step:   46300 Progress: 23.2% Elapsed: 1 day, 6:57:01 temperature: 1.0 ada_p: No Discriminator_loss: 5.10517 Generator_loss: 6.28221 
    [INFO] 2021-02-22 04:16:19 > Step:   46400 Progress: 23.2% Elapsed: 1 day, 7:00:49 temperature: 1.0 ada_p: No Discriminator_loss: 5.06092 Generator_loss: 6.25033 
    [INFO] 2021-02-22 04:20:08 > Step:   46500 Progress: 23.2% Elapsed: 1 day, 7:04:38 temperature: 1.0 ada_p: No Discriminator_loss: 5.13497 Generator_loss: 6.07487 
    [INFO] 2021-02-22 04:23:57 > Step:   46600 Progress: 23.3% Elapsed: 1 day, 7:08:27 temperature: 1.0 ada_p: No Discriminator_loss: 5.00144 Generator_loss: 6.13049 
    [INFO] 2021-02-22 04:27:45 > Step:   46700 Progress: 23.4% Elapsed: 1 day, 7:12:15 temperature: 1.0 ada_p: No Discriminator_loss: 5.18526 Generator_loss: 5.99132 
    [INFO] 2021-02-22 04:31:34 > Step:   46800 Progress: 23.4% Elapsed: 1 day, 7:16:04 temperature: 1.0 ada_p: No Discriminator_loss: 5.06406 Generator_loss: 6.06167 
    [INFO] 2021-02-22 04:35:23 > Step:   46900 Progress: 23.4% Elapsed: 1 day, 7:19:53 temperature: 1.0 ada_p: No Discriminator_loss: 4.94992 Generator_loss: 6.21247 
    [INFO] 2021-02-22 04:39:12 > Step:   47000 Progress: 23.5% Elapsed: 1 day, 7:23:42 temperature: 1.0 ada_p: No Discriminator_loss: 5.45032 Generator_loss: 5.76139 
    [INFO] 2021-02-22 04:43:01 > Step:   47100 Progress: 23.5% Elapsed: 1 day, 7:27:31 temperature: 1.0 ada_p: No Discriminator_loss: 5.02759 Generator_loss: 6.03233 
    [INFO] 2021-02-22 04:46:50 > Step:   47200 Progress: 23.6% Elapsed: 1 day, 7:31:20 temperature: 1.0 ada_p: No Discriminator_loss: 5.20869 Generator_loss: 6.04563 
    [INFO] 2021-02-22 04:50:39 > Step:   47300 Progress: 23.6% Elapsed: 1 day, 7:35:09 temperature: 1.0 ada_p: No Discriminator_loss: 5.25778 Generator_loss: 5.70551 
    [INFO] 2021-02-22 04:54:28 > Step:   47400 Progress: 23.7% Elapsed: 1 day, 7:38:58 temperature: 1.0 ada_p: No Discriminator_loss: 5.23976 Generator_loss: 5.94907 
    [INFO] 2021-02-22 04:58:17 > Step:   47500 Progress: 23.8% Elapsed: 1 day, 7:42:47 temperature: 1.0 ada_p: No Discriminator_loss: 4.99506 Generator_loss: 6.22019 
    [INFO] 2021-02-22 05:02:09 > Step:   47600 Progress: 23.8% Elapsed: 1 day, 7:46:39 temperature: 1.0 ada_p: No Discriminator_loss: 5.00467 Generator_loss: 6.22136 
    [INFO] 2021-02-22 05:05:58 > Step:   47700 Progress: 23.8% Elapsed: 1 day, 7:50:28 temperature: 1.0 ada_p: No Discriminator_loss: 5.19981 Generator_loss: 5.81114 
    [INFO] 2021-02-22 05:09:47 > Step:   47800 Progress: 23.9% Elapsed: 1 day, 7:54:17 temperature: 1.0 ada_p: No Discriminator_loss: 5.07491 Generator_loss: 6.43727 
    [INFO] 2021-02-22 05:13:36 > Step:   47900 Progress: 23.9% Elapsed: 1 day, 7:58:06 temperature: 1.0 ada_p: No Discriminator_loss: 5.03015 Generator_loss: 6.5055 
    [INFO] 2021-02-22 05:17:25 > Step:   48000 Progress: 24.0% Elapsed: 1 day, 8:01:55 temperature: 1.0 ada_p: No Discriminator_loss: 5.03534 Generator_loss: 6.09113 
    [INFO] 2021-02-22 05:21:13 > Step:   48100 Progress: 24.1% Elapsed: 1 day, 8:05:43 temperature: 1.0 ada_p: No Discriminator_loss: 5.03744 Generator_loss: 6.22275 
    [INFO] 2021-02-22 05:25:02 > Step:   48200 Progress: 24.1% Elapsed: 1 day, 8:09:32 temperature: 1.0 ada_p: No Discriminator_loss: 5.03949 Generator_loss: 6.43559 
    [INFO] 2021-02-22 05:28:51 > Step:   48300 Progress: 24.1% Elapsed: 1 day, 8:13:21 temperature: 1.0 ada_p: No Discriminator_loss: 5.16778 Generator_loss: 6.07853 
    [INFO] 2021-02-22 05:32:40 > Step:   48400 Progress: 24.2% Elapsed: 1 day, 8:17:10 temperature: 1.0 ada_p: No Discriminator_loss: 5.01304 Generator_loss: 6.24566 
    [INFO] 2021-02-22 05:36:29 > Step:   48500 Progress: 24.2% Elapsed: 1 day, 8:20:59 temperature: 1.0 ada_p: No Discriminator_loss: 5.12084 Generator_loss: 6.51318 
    [INFO] 2021-02-22 05:40:18 > Step:   48600 Progress: 24.3% Elapsed: 1 day, 8:24:48 temperature: 1.0 ada_p: No Discriminator_loss: 5.02575 Generator_loss: 6.17282 
    [INFO] 2021-02-22 05:44:06 > Step:   48700 Progress: 24.3% Elapsed: 1 day, 8:28:36 temperature: 1.0 ada_p: No Discriminator_loss: 5.06592 Generator_loss: 6.16143 
    [INFO] 2021-02-22 05:47:55 > Step:   48800 Progress: 24.4% Elapsed: 1 day, 8:32:25 temperature: 1.0 ada_p: No Discriminator_loss: 4.99627 Generator_loss: 6.18451 
    [INFO] 2021-02-22 05:51:44 > Step:   48900 Progress: 24.4% Elapsed: 1 day, 8:36:14 temperature: 1.0 ada_p: No Discriminator_loss: 4.95894 Generator_loss: 6.52805 
    [INFO] 2021-02-22 05:55:32 > Step:   49000 Progress: 24.5% Elapsed: 1 day, 8:40:02 temperature: 1.0 ada_p: No Discriminator_loss: 5.26015 Generator_loss: 5.94535 
    [INFO] 2021-02-22 05:59:21 > Step:   49100 Progress: 24.6% Elapsed: 1 day, 8:43:51 temperature: 1.0 ada_p: No Discriminator_loss: 5.006 Generator_loss: 6.63337 
    [INFO] 2021-02-22 06:03:10 > Step:   49200 Progress: 24.6% Elapsed: 1 day, 8:47:40 temperature: 1.0 ada_p: No Discriminator_loss: 4.95642 Generator_loss: 6.58352 
    [INFO] 2021-02-22 06:06:59 > Step:   49300 Progress: 24.6% Elapsed: 1 day, 8:51:29 temperature: 1.0 ada_p: No Discriminator_loss: 5.16746 Generator_loss: 5.98424 
    [INFO] 2021-02-22 06:10:48 > Step:   49400 Progress: 24.7% Elapsed: 1 day, 8:55:18 temperature: 1.0 ada_p: No Discriminator_loss: 5.30615 Generator_loss: 5.73819 
    [INFO] 2021-02-22 06:14:37 > Step:   49500 Progress: 24.8% Elapsed: 1 day, 8:59:07 temperature: 1.0 ada_p: No Discriminator_loss: 5.08546 Generator_loss: 6.24706 
    [INFO] 2021-02-22 06:18:26 > Step:   49600 Progress: 24.8% Elapsed: 1 day, 9:02:56 temperature: 1.0 ada_p: No Discriminator_loss: 5.00104 Generator_loss: 6.2795 
    [INFO] 2021-02-22 06:22:15 > Step:   49700 Progress: 24.9% Elapsed: 1 day, 9:06:45 temperature: 1.0 ada_p: No Discriminator_loss: 5.02014 Generator_loss: 6.3654 
    [INFO] 2021-02-22 06:26:04 > Step:   49800 Progress: 24.9% Elapsed: 1 day, 9:10:34 temperature: 1.0 ada_p: No Discriminator_loss: 5.19441 Generator_loss: 5.91212 
    [INFO] 2021-02-22 06:29:53 > Step:   49900 Progress: 24.9% Elapsed: 1 day, 9:14:23 temperature: 1.0 ada_p: No Discriminator_loss: 5.06459 Generator_loss: 6.01172 
    [INFO] 2021-02-22 06:33:41 > Step:   50000 Progress: 25.0% Elapsed: 1 day, 9:18:11 temperature: 1.0 ada_p: No Discriminator_loss: 5.06186 Generator_loss: 6.59504 
    [INFO] 2021-02-22 06:33:41 > Start Evaluation (50000 Step): ContraGAN256-train-2021_02_20_21_08_34
    [INFO] 2021-02-22 06:33:41 > Start Evaluation FID score ... 
    [INFO] 2021-02-22 06:33:41 > Calculating FID Score....
    [INFO] 2021-02-22 06:34:56 > Start Evaluation IS score ... 
    [INFO] 2021-02-22 06:34:56 > Calculating Inception Score....
    [INFO] 2021-02-22 06:35:58 > Start Evaluation F Beta score ... 
    [INFO] 2021-02-22 06:35:58 > Calculate F_beta Score....
    [INFO] 2021-02-22 06:49:08 > Calculate Accuracies....
    [INFO] 2021-02-22 06:50:25 > Calculate Accuracies....
    [INFO] 2021-02-22 06:56:46 > F_8 score (Step: 50000, Using valid images): 0.5321817457212351
    [INFO] 2021-02-22 06:56:46 > F_1/8 score (Step: 50000, Using valid images): 0.7190109961397075
    [INFO] 2021-02-22 06:56:46 > FID score (Step: 50000, Using valid moments): 62.25328218970208
    [INFO] 2021-02-22 06:56:46 > Inception score (Step: 50000, 50000 generated images): 14.74788761138916
    [INFO] 2021-02-22 06:56:46 > Best FID score (Step: 50000, Using valid moments): 62.25328218970208
    [INFO] 2021-02-22 06:56:53 > Save model to checkpoints/ContraGAN256-train-2021_02_20_21_08_34
    [INFO] 2021-02-22 07:00:46 > Step:   50100 Progress: 25.1% Elapsed: 1 day, 9:45:16 temperature: 1.0 ada_p: No Discriminator_loss: 5.01133 Generator_loss: 6.29488 
    [INFO] 2021-02-22 07:04:34 > Step:   50200 Progress: 25.1% Elapsed: 1 day, 9:49:04 temperature: 1.0 ada_p: No Discriminator_loss: 5.16964 Generator_loss: 5.982 
    [INFO] 2021-02-22 07:08:22 > Step:   50300 Progress: 25.1% Elapsed: 1 day, 9:52:52 temperature: 1.0 ada_p: No Discriminator_loss: 5.09118 Generator_loss: 6.21525 
    [INFO] 2021-02-22 07:12:10 > Step:   50400 Progress: 25.2% Elapsed: 1 day, 9:56:40 temperature: 1.0 ada_p: No Discriminator_loss: 5.12363 Generator_loss: 6.11715 
    [INFO] 2021-02-22 07:15:58 > Step:   50500 Progress: 25.2% Elapsed: 1 day, 10:00:28 temperature: 1.0 ada_p: No Discriminator_loss: 5.03652 Generator_loss: 6.70519 
    [INFO] 2021-02-22 07:19:46 > Step:   50600 Progress: 25.3% Elapsed: 1 day, 10:04:16 temperature: 1.0 ada_p: No Discriminator_loss: 5.09509 Generator_loss: 6.32008 
    [INFO] 2021-02-22 07:23:35 > Step:   50700 Progress: 25.4% Elapsed: 1 day, 10:08:05 temperature: 1.0 ada_p: No Discriminator_loss: 5.06899 Generator_loss: 6.22663 
    [INFO] 2021-02-22 07:27:23 > Step:   50800 Progress: 25.4% Elapsed: 1 day, 10:11:53 temperature: 1.0 ada_p: No Discriminator_loss: 5.03466 Generator_loss: 6.46442 
    [INFO] 2021-02-22 07:31:11 > Step:   50900 Progress: 25.4% Elapsed: 1 day, 10:15:41 temperature: 1.0 ada_p: No Discriminator_loss: 5.11307 Generator_loss: 6.65875 
    [INFO] 2021-02-22 07:34:59 > Step:   51000 Progress: 25.5% Elapsed: 1 day, 10:19:29 temperature: 1.0 ada_p: No Discriminator_loss: 5.13903 Generator_loss: 5.98846 
    [INFO] 2021-02-22 07:38:47 > Step:   51100 Progress: 25.6% Elapsed: 1 day, 10:23:17 temperature: 1.0 ada_p: No Discriminator_loss: 5.05935 Generator_loss: 6.17622 
    [INFO] 2021-02-22 07:42:36 > Step:   51200 Progress: 25.6% Elapsed: 1 day, 10:27:06 temperature: 1.0 ada_p: No Discriminator_loss: 4.9901 Generator_loss: 6.18684 
    [INFO] 2021-02-22 07:46:43 > Step:   51300 Progress: 25.7% Elapsed: 1 day, 10:31:13 temperature: 1.0 ada_p: No Discriminator_loss: 4.96325 Generator_loss: 6.29019 
    [INFO] 2021-02-22 07:50:31 > Step:   51400 Progress: 25.7% Elapsed: 1 day, 10:35:01 temperature: 1.0 ada_p: No Discriminator_loss: 4.97326 Generator_loss: 6.42697 
    [INFO] 2021-02-22 07:54:20 > Step:   51500 Progress: 25.8% Elapsed: 1 day, 10:38:50 temperature: 1.0 ada_p: No Discriminator_loss: 5.13018 Generator_loss: 6.19051 
    [INFO] 2021-02-22 07:58:09 > Step:   51600 Progress: 25.8% Elapsed: 1 day, 10:42:39 temperature: 1.0 ada_p: No Discriminator_loss: 5.08723 Generator_loss: 6.6899 
    [INFO] 2021-02-22 08:01:57 > Step:   51700 Progress: 25.9% Elapsed: 1 day, 10:46:27 temperature: 1.0 ada_p: No Discriminator_loss: 5.29569 Generator_loss: 6.02585 
    [INFO] 2021-02-22 08:05:46 > Step:   51800 Progress: 25.9% Elapsed: 1 day, 10:50:16 temperature: 1.0 ada_p: No Discriminator_loss: 5.03785 Generator_loss: 6.56231 
    [INFO] 2021-02-22 08:09:35 > Step:   51900 Progress: 25.9% Elapsed: 1 day, 10:54:05 temperature: 1.0 ada_p: No Discriminator_loss: 5.30661 Generator_loss: 5.80565 
    [INFO] 2021-02-22 08:13:24 > Step:   52000 Progress: 26.0% Elapsed: 1 day, 10:57:54 temperature: 1.0 ada_p: No Discriminator_loss: 4.93423 Generator_loss: 6.46937 
    [INFO] 2021-02-22 08:17:13 > Step:   52100 Progress: 26.1% Elapsed: 1 day, 11:01:43 temperature: 1.0 ada_p: No Discriminator_loss: 5.05965 Generator_loss: 6.157 
    [INFO] 2021-02-22 08:21:02 > Step:   52200 Progress: 26.1% Elapsed: 1 day, 11:05:32 temperature: 1.0 ada_p: No Discriminator_loss: 5.17729 Generator_loss: 6.14334 
    [INFO] 2021-02-22 08:24:51 > Step:   52300 Progress: 26.2% Elapsed: 1 day, 11:09:21 temperature: 1.0 ada_p: No Discriminator_loss: 5.12172 Generator_loss: 5.83164 
    [INFO] 2021-02-22 08:28:40 > Step:   52400 Progress: 26.2% Elapsed: 1 day, 11:13:10 temperature: 1.0 ada_p: No Discriminator_loss: 5.06141 Generator_loss: 6.00181 
    [INFO] 2021-02-22 08:32:29 > Step:   52500 Progress: 26.2% Elapsed: 1 day, 11:16:59 temperature: 1.0 ada_p: No Discriminator_loss: 5.32602 Generator_loss: 5.81275 
    [INFO] 2021-02-22 08:36:22 > Step:   52600 Progress: 26.3% Elapsed: 1 day, 11:20:52 temperature: 1.0 ada_p: No Discriminator_loss: 5.19646 Generator_loss: 6.06332 
    [INFO] 2021-02-22 08:40:11 > Step:   52700 Progress: 26.4% Elapsed: 1 day, 11:24:41 temperature: 1.0 ada_p: No Discriminator_loss: 4.92909 Generator_loss: 6.43028 
    [INFO] 2021-02-22 08:44:00 > Step:   52800 Progress: 26.4% Elapsed: 1 day, 11:28:30 temperature: 1.0 ada_p: No Discriminator_loss: 5.1626 Generator_loss: 6.0889 
    [INFO] 2021-02-22 08:47:49 > Step:   52900 Progress: 26.5% Elapsed: 1 day, 11:32:19 temperature: 1.0 ada_p: No Discriminator_loss: 4.97494 Generator_loss: 6.50095 
    [INFO] 2021-02-22 08:51:38 > Step:   53000 Progress: 26.5% Elapsed: 1 day, 11:36:08 temperature: 1.0 ada_p: No Discriminator_loss: 5.04943 Generator_loss: 6.50215 
    [INFO] 2021-02-22 08:55:27 > Step:   53100 Progress: 26.6% Elapsed: 1 day, 11:39:57 temperature: 1.0 ada_p: No Discriminator_loss: 5.02833 Generator_loss: 6.74061 
    [INFO] 2021-02-22 08:59:16 > Step:   53200 Progress: 26.6% Elapsed: 1 day, 11:43:46 temperature: 1.0 ada_p: No Discriminator_loss: 5.33408 Generator_loss: 5.37649 
    [INFO] 2021-02-22 09:03:05 > Step:   53300 Progress: 26.7% Elapsed: 1 day, 11:47:35 temperature: 1.0 ada_p: No Discriminator_loss: 5.07977 Generator_loss: 6.51638 
    [INFO] 2021-02-22 09:06:54 > Step:   53400 Progress: 26.7% Elapsed: 1 day, 11:51:24 temperature: 1.0 ada_p: No Discriminator_loss: 5.17007 Generator_loss: 6.17957 
    [INFO] 2021-02-22 09:10:43 > Step:   53500 Progress: 26.8% Elapsed: 1 day, 11:55:13 temperature: 1.0 ada_p: No Discriminator_loss: 5.15196 Generator_loss: 5.97452 
    [INFO] 2021-02-22 09:14:32 > Step:   53600 Progress: 26.8% Elapsed: 1 day, 11:59:01 temperature: 1.0 ada_p: No Discriminator_loss: 5.20089 Generator_loss: 5.94851 
    [INFO] 2021-02-22 09:18:20 > Step:   53700 Progress: 26.9% Elapsed: 1 day, 12:02:50 temperature: 1.0 ada_p: No Discriminator_loss: 4.98512 Generator_loss: 6.47826 
    [INFO] 2021-02-22 09:22:09 > Step:   53800 Progress: 26.9% Elapsed: 1 day, 12:06:39 temperature: 1.0 ada_p: No Discriminator_loss: 4.97317 Generator_loss: 6.27499 
    [INFO] 2021-02-22 09:25:58 > Step:   53900 Progress: 27.0% Elapsed: 1 day, 12:10:28 temperature: 1.0 ada_p: No Discriminator_loss: 5.04463 Generator_loss: 6.4179 
    [INFO] 2021-02-22 09:29:47 > Step:   54000 Progress: 27.0% Elapsed: 1 day, 12:14:17 temperature: 1.0 ada_p: No Discriminator_loss: 5.19266 Generator_loss: 6.05346 
    [INFO] 2021-02-22 09:33:36 > Step:   54100 Progress: 27.1% Elapsed: 1 day, 12:18:06 temperature: 1.0 ada_p: No Discriminator_loss: 5.27231 Generator_loss: 5.99652 
    [INFO] 2021-02-22 09:37:25 > Step:   54200 Progress: 27.1% Elapsed: 1 day, 12:21:55 temperature: 1.0 ada_p: No Discriminator_loss: 4.9704 Generator_loss: 6.43523 
    [INFO] 2021-02-22 09:41:14 > Step:   54300 Progress: 27.2% Elapsed: 1 day, 12:25:44 temperature: 1.0 ada_p: No Discriminator_loss: 5.27341 Generator_loss: 5.97461 
    [INFO] 2021-02-22 09:45:03 > Step:   54400 Progress: 27.2% Elapsed: 1 day, 12:29:33 temperature: 1.0 ada_p: No Discriminator_loss: 5.11706 Generator_loss: 6.07529 
    [INFO] 2021-02-22 09:48:52 > Step:   54500 Progress: 27.3% Elapsed: 1 day, 12:33:22 temperature: 1.0 ada_p: No Discriminator_loss: 5.1506 Generator_loss: 6.00594 
    [INFO] 2021-02-22 09:52:41 > Step:   54600 Progress: 27.3% Elapsed: 1 day, 12:37:11 temperature: 1.0 ada_p: No Discriminator_loss: 5.15432 Generator_loss: 5.97685 
    [INFO] 2021-02-22 09:56:29 > Step:   54700 Progress: 27.4% Elapsed: 1 day, 12:40:59 temperature: 1.0 ada_p: No Discriminator_loss: 4.99249 Generator_loss: 6.33237 
    [INFO] 2021-02-22 10:00:18 > Step:   54800 Progress: 27.4% Elapsed: 1 day, 12:44:48 temperature: 1.0 ada_p: No Discriminator_loss: 5.0133 Generator_loss: 6.43999 
    [INFO] 2021-02-22 10:04:07 > Step:   54900 Progress: 27.5% Elapsed: 1 day, 12:48:37 temperature: 1.0 ada_p: No Discriminator_loss: 5.095 Generator_loss: 6.24829 
    [INFO] 2021-02-22 10:07:56 > Step:   55000 Progress: 27.5% Elapsed: 1 day, 12:52:26 temperature: 1.0 ada_p: No Discriminator_loss: 5.36692 Generator_loss: 5.99982 
    [INFO] 2021-02-22 10:11:48 > Step:   55100 Progress: 27.6% Elapsed: 1 day, 12:56:18 temperature: 1.0 ada_p: No Discriminator_loss: 5.06502 Generator_loss: 6.08699 
    [INFO] 2021-02-22 10:15:37 > Step:   55200 Progress: 27.6% Elapsed: 1 day, 13:00:07 temperature: 1.0 ada_p: No Discriminator_loss: 5.24713 Generator_loss: 5.93189 
    [INFO] 2021-02-22 10:19:26 > Step:   55300 Progress: 27.7% Elapsed: 1 day, 13:03:56 temperature: 1.0 ada_p: No Discriminator_loss: 5.02183 Generator_loss: 6.47812 
    [INFO] 2021-02-22 10:23:15 > Step:   55400 Progress: 27.7% Elapsed: 1 day, 13:07:45 temperature: 1.0 ada_p: No Discriminator_loss: 5.13189 Generator_loss: 6.03888 
    [INFO] 2021-02-22 10:27:04 > Step:   55500 Progress: 27.8% Elapsed: 1 day, 13:11:34 temperature: 1.0 ada_p: No Discriminator_loss: 5.04327 Generator_loss: 6.22004 
    [INFO] 2021-02-22 10:30:53 > Step:   55600 Progress: 27.8% Elapsed: 1 day, 13:15:23 temperature: 1.0 ada_p: No Discriminator_loss: 5.08427 Generator_loss: 5.99502 
    [INFO] 2021-02-22 10:34:41 > Step:   55700 Progress: 27.9% Elapsed: 1 day, 13:19:11 temperature: 1.0 ada_p: No Discriminator_loss: 5.03427 Generator_loss: 6.0784 
    [INFO] 2021-02-22 10:38:30 > Step:   55800 Progress: 27.9% Elapsed: 1 day, 13:23:00 temperature: 1.0 ada_p: No Discriminator_loss: 4.98526 Generator_loss: 6.46442 
    [INFO] 2021-02-22 10:42:18 > Step:   55900 Progress: 28.0% Elapsed: 1 day, 13:26:48 temperature: 1.0 ada_p: No Discriminator_loss: 5.03764 Generator_loss: 6.13631 
    [INFO] 2021-02-22 10:46:07 > Step:   56000 Progress: 28.0% Elapsed: 1 day, 13:30:37 temperature: 1.0 ada_p: No Discriminator_loss: 4.94919 Generator_loss: 6.49842 
    [INFO] 2021-02-22 10:49:56 > Step:   56100 Progress: 28.1% Elapsed: 1 day, 13:34:26 temperature: 1.0 ada_p: No Discriminator_loss: 5.00146 Generator_loss: 6.19433 
    [INFO] 2021-02-22 10:53:44 > Step:   56200 Progress: 28.1% Elapsed: 1 day, 13:38:14 temperature: 1.0 ada_p: No Discriminator_loss: 5.40062 Generator_loss: 6.53825 
    [INFO] 2021-02-22 10:57:33 > Step:   56300 Progress: 28.1% Elapsed: 1 day, 13:42:03 temperature: 1.0 ada_p: No Discriminator_loss: 5.14335 Generator_loss: 6.08294 
    [INFO] 2021-02-22 11:01:21 > Step:   56400 Progress: 28.2% Elapsed: 1 day, 13:45:51 temperature: 1.0 ada_p: No Discriminator_loss: 5.12197 Generator_loss: 6.08191 
    [INFO] 2021-02-22 11:05:12 > Step:   56500 Progress: 28.2% Elapsed: 1 day, 13:49:42 temperature: 1.0 ada_p: No Discriminator_loss: 5.06901 Generator_loss: 6.03618 
    [INFO] 2021-02-22 11:09:02 > Step:   56600 Progress: 28.3% Elapsed: 1 day, 13:53:32 temperature: 1.0 ada_p: No Discriminator_loss: 5.06672 Generator_loss: 6.2388 
    [INFO] 2021-02-22 11:12:53 > Step:   56700 Progress: 28.3% Elapsed: 1 day, 13:57:23 temperature: 1.0 ada_p: No Discriminator_loss: 5.09537 Generator_loss: 6.03419 
    [INFO] 2021-02-22 11:16:44 > Step:   56800 Progress: 28.4% Elapsed: 1 day, 14:01:14 temperature: 1.0 ada_p: No Discriminator_loss: 5.1206 Generator_loss: 5.9855 
    [INFO] 2021-02-22 11:20:35 > Step:   56900 Progress: 28.4% Elapsed: 1 day, 14:05:05 temperature: 1.0 ada_p: No Discriminator_loss: 5.0056 Generator_loss: 6.48507 
    [INFO] 2021-02-22 11:24:27 > Step:   57000 Progress: 28.5% Elapsed: 1 day, 14:08:57 temperature: 1.0 ada_p: No Discriminator_loss: 5.01101 Generator_loss: 6.22598 
    [INFO] 2021-02-22 11:28:18 > Step:   57100 Progress: 28.5% Elapsed: 1 day, 14:12:48 temperature: 1.0 ada_p: No Discriminator_loss: 5.08638 Generator_loss: 6.2052 
    [INFO] 2021-02-22 11:32:10 > Step:   57200 Progress: 28.6% Elapsed: 1 day, 14:16:40 temperature: 1.0 ada_p: No Discriminator_loss: 5.18087 Generator_loss: 5.92536 
    [INFO] 2021-02-22 11:36:03 > Step:   57300 Progress: 28.6% Elapsed: 1 day, 14:20:33 temperature: 1.0 ada_p: No Discriminator_loss: 5.0591 Generator_loss: 6.34448 
    [INFO] 2021-02-22 11:39:55 > Step:   57400 Progress: 28.7% Elapsed: 1 day, 14:24:25 temperature: 1.0 ada_p: No Discriminator_loss: 4.95947 Generator_loss: 6.56313 
    [INFO] 2021-02-22 11:43:47 > Step:   57500 Progress: 28.7% Elapsed: 1 day, 14:28:17 temperature: 1.0 ada_p: No Discriminator_loss: 5.18985 Generator_loss: 5.9419 
    [INFO] 2021-02-22 11:47:43 > Step:   57600 Progress: 28.8% Elapsed: 1 day, 14:32:13 temperature: 1.0 ada_p: No Discriminator_loss: 5.07028 Generator_loss: 6.04666 
    [INFO] 2021-02-22 11:51:36 > Step:   57700 Progress: 28.8% Elapsed: 1 day, 14:36:06 temperature: 1.0 ada_p: No Discriminator_loss: 5.04481 Generator_loss: 6.16732 
    [INFO] 2021-02-22 11:55:27 > Step:   57800 Progress: 28.9% Elapsed: 1 day, 14:39:57 temperature: 1.0 ada_p: No Discriminator_loss: 5.03923 Generator_loss: 6.63125 
    [INFO] 2021-02-22 11:59:19 > Step:   57900 Progress: 28.9% Elapsed: 1 day, 14:43:49 temperature: 1.0 ada_p: No Discriminator_loss: 5.11313 Generator_loss: 6.11078 
    [INFO] 2021-02-22 12:03:12 > Step:   58000 Progress: 29.0% Elapsed: 1 day, 14:47:42 temperature: 1.0 ada_p: No Discriminator_loss: 5.31138 Generator_loss: 6.07956 
    [INFO] 2021-02-22 12:07:04 > Step:   58100 Progress: 29.0% Elapsed: 1 day, 14:51:34 temperature: 1.0 ada_p: No Discriminator_loss: 5.0702 Generator_loss: 6.04716 
    [INFO] 2021-02-22 12:10:56 > Step:   58200 Progress: 29.1% Elapsed: 1 day, 14:55:25 temperature: 1.0 ada_p: No Discriminator_loss: 5.02917 Generator_loss: 5.92949 
    [INFO] 2021-02-22 12:14:48 > Step:   58300 Progress: 29.1% Elapsed: 1 day, 14:59:18 temperature: 1.0 ada_p: No Discriminator_loss: 4.93617 Generator_loss: 6.58612 
    [INFO] 2021-02-22 12:18:40 > Step:   58400 Progress: 29.2% Elapsed: 1 day, 15:03:10 temperature: 1.0 ada_p: No Discriminator_loss: 5.04221 Generator_loss: 6.63136 
    [INFO] 2021-02-22 12:22:32 > Step:   58500 Progress: 29.2% Elapsed: 1 day, 15:07:02 temperature: 1.0 ada_p: No Discriminator_loss: 5.0054 Generator_loss: 6.38062 
    [INFO] 2021-02-22 12:26:24 > Step:   58600 Progress: 29.3% Elapsed: 1 day, 15:10:54 temperature: 1.0 ada_p: No Discriminator_loss: 5.14732 Generator_loss: 6.57346 
    [INFO] 2021-02-22 12:30:16 > Step:   58700 Progress: 29.3% Elapsed: 1 day, 15:14:46 temperature: 1.0 ada_p: No Discriminator_loss: 5.02654 Generator_loss: 6.37865 
    [INFO] 2021-02-22 12:34:09 > Step:   58800 Progress: 29.4% Elapsed: 1 day, 15:18:39 temperature: 1.0 ada_p: No Discriminator_loss: 5.24925 Generator_loss: 6.03098 
    [INFO] 2021-02-22 12:38:01 > Step:   58900 Progress: 29.4% Elapsed: 1 day, 15:22:31 temperature: 1.0 ada_p: No Discriminator_loss: 4.95723 Generator_loss: 6.6306 
    [INFO] 2021-02-22 12:41:53 > Step:   59000 Progress: 29.5% Elapsed: 1 day, 15:26:23 temperature: 1.0 ada_p: No Discriminator_loss: 5.36617 Generator_loss: 5.85673 
    [INFO] 2021-02-22 12:45:45 > Step:   59100 Progress: 29.5% Elapsed: 1 day, 15:30:15 temperature: 1.0 ada_p: No Discriminator_loss: 4.94819 Generator_loss: 6.32601 
    [INFO] 2021-02-22 12:49:37 > Step:   59200 Progress: 29.6% Elapsed: 1 day, 15:34:07 temperature: 1.0 ada_p: No Discriminator_loss: 5.1071 Generator_loss: 6.04245 
    [INFO] 2021-02-22 12:53:30 > Step:   59300 Progress: 29.6% Elapsed: 1 day, 15:38:00 temperature: 1.0 ada_p: No Discriminator_loss: 5.24542 Generator_loss: 5.81296 
    [INFO] 2021-02-22 12:57:23 > Step:   59400 Progress: 29.7% Elapsed: 1 day, 15:41:53 temperature: 1.0 ada_p: No Discriminator_loss: 5.00985 Generator_loss: 6.36312 
    [INFO] 2021-02-22 13:01:15 > Step:   59500 Progress: 29.8% Elapsed: 1 day, 15:45:45 temperature: 1.0 ada_p: No Discriminator_loss: 5.11849 Generator_loss: 6.1307 
    [INFO] 2021-02-22 13:05:07 > Step:   59600 Progress: 29.8% Elapsed: 1 day, 15:49:37 temperature: 1.0 ada_p: No Discriminator_loss: 5.02994 Generator_loss: 6.1588 
    [INFO] 2021-02-22 13:08:59 > Step:   59700 Progress: 29.8% Elapsed: 1 day, 15:53:29 temperature: 1.0 ada_p: No Discriminator_loss: 5.08636 Generator_loss: 6.27762 
    [INFO] 2021-02-22 13:12:52 > Step:   59800 Progress: 29.9% Elapsed: 1 day, 15:57:22 temperature: 1.0 ada_p: No Discriminator_loss: 5.10781 Generator_loss: 6.00397 
    [INFO] 2021-02-22 13:16:44 > Step:   59900 Progress: 29.9% Elapsed: 1 day, 16:01:14 temperature: 1.0 ada_p: No Discriminator_loss: 5.01234 Generator_loss: 6.31747 
    [INFO] 2021-02-22 13:20:35 > Step:   60000 Progress: 30.0% Elapsed: 1 day, 16:05:05 temperature: 1.0 ada_p: No Discriminator_loss: 5.01462 Generator_loss: 6.19957 
    [INFO] 2021-02-22 13:20:35 > Start Evaluation (60000 Step): ContraGAN256-train-2021_02_20_21_08_34
    [INFO] 2021-02-22 13:20:35 > Start Evaluation FID score ... 
    [INFO] 2021-02-22 13:20:35 > Calculating FID Score....
    [INFO] 2021-02-22 13:21:54 > Start Evaluation IS score ... 
    [INFO] 2021-02-22 13:21:54 > Calculating Inception Score....
    [INFO] 2021-02-22 13:22:57 > Start Evaluation F Beta score ... 
    [INFO] 2021-02-22 13:22:57 > Calculate F_beta Score....
    [INFO] 2021-02-22 13:36:21 > Calculate Accuracies....
    [INFO] 2021-02-22 13:37:41 > Calculate Accuracies....
    [INFO] 2021-02-22 13:44:10 > F_8 score (Step: 60000, Using valid images): 0.580602963511551
    [INFO] 2021-02-22 13:44:10 > F_1/8 score (Step: 60000, Using valid images): 0.7150814486269845
    [INFO] 2021-02-22 13:44:10 > FID score (Step: 60000, Using valid moments): 60.38544475604181
    [INFO] 2021-02-22 13:44:10 > Inception score (Step: 60000, 50000 generated images): 15.345391273498535
    [INFO] 2021-02-22 13:44:10 > Best FID score (Step: 60000, Using valid moments): 60.38544475604181
    [INFO] 2021-02-22 13:44:19 > Save model to checkpoints/ContraGAN256-train-2021_02_20_21_08_34
    [INFO] 2021-02-22 13:48:14 > Step:   60100 Progress: 30.0% Elapsed: 1 day, 16:32:44 temperature: 1.0 ada_p: No Discriminator_loss: 5.02087 Generator_loss: 5.93529 
    [INFO] 2021-02-22 13:52:05 > Step:   60200 Progress: 30.1% Elapsed: 1 day, 16:36:35 temperature: 1.0 ada_p: No Discriminator_loss: 5.19664 Generator_loss: 6.0798 
    [INFO] 2021-02-22 13:55:57 > Step:   60300 Progress: 30.1% Elapsed: 1 day, 16:40:27 temperature: 1.0 ada_p: No Discriminator_loss: 5.17831 Generator_loss: 6.01606 
    [INFO] 2021-02-22 13:59:48 > Step:   60400 Progress: 30.2% Elapsed: 1 day, 16:44:18 temperature: 1.0 ada_p: No Discriminator_loss: 5.07307 Generator_loss: 6.2076 
    [INFO] 2021-02-22 14:03:59 > Step:   60500 Progress: 30.2% Elapsed: 1 day, 16:48:29 temperature: 1.0 ada_p: No Discriminator_loss: 5.06138 Generator_loss: 6.17557 
    [INFO] 2021-02-22 14:07:51 > Step:   60600 Progress: 30.3% Elapsed: 1 day, 16:52:21 temperature: 1.0 ada_p: No Discriminator_loss: 5.04362 Generator_loss: 6.40416 
    [INFO] 2021-02-22 14:11:44 > Step:   60700 Progress: 30.3% Elapsed: 1 day, 16:56:14 temperature: 1.0 ada_p: No Discriminator_loss: 5.24281 Generator_loss: 5.8953 
    [INFO] 2021-02-22 14:15:36 > Step:   60800 Progress: 30.4% Elapsed: 1 day, 17:00:06 temperature: 1.0 ada_p: No Discriminator_loss: 5.02365 Generator_loss: 6.13938 
    [INFO] 2021-02-22 14:19:28 > Step:   60900 Progress: 30.4% Elapsed: 1 day, 17:03:58 temperature: 1.0 ada_p: No Discriminator_loss: 5.2079 Generator_loss: 5.79653 
    [INFO] 2021-02-22 14:23:20 > Step:   61000 Progress: 30.5% Elapsed: 1 day, 17:07:50 temperature: 1.0 ada_p: No Discriminator_loss: 5.0742 Generator_loss: 6.01185 
    [INFO] 2021-02-22 14:27:13 > Step:   61100 Progress: 30.6% Elapsed: 1 day, 17:11:43 temperature: 1.0 ada_p: No Discriminator_loss: 5.06362 Generator_loss: 6.25461 
    [INFO] 2021-02-22 14:31:05 > Step:   61200 Progress: 30.6% Elapsed: 1 day, 17:15:35 temperature: 1.0 ada_p: No Discriminator_loss: 5.0767 Generator_loss: 6.13494 
    [INFO] 2021-02-22 14:34:58 > Step:   61300 Progress: 30.6% Elapsed: 1 day, 17:19:28 temperature: 1.0 ada_p: No Discriminator_loss: 5.02651 Generator_loss: 6.14857 
    [INFO] 2021-02-22 14:38:50 > Step:   61400 Progress: 30.7% Elapsed: 1 day, 17:23:20 temperature: 1.0 ada_p: No Discriminator_loss: 5.03478 Generator_loss: 6.51512 
    [INFO] 2021-02-22 14:42:42 > Step:   61500 Progress: 30.8% Elapsed: 1 day, 17:27:12 temperature: 1.0 ada_p: No Discriminator_loss: 5.1415 Generator_loss: 6.14445 
    [INFO] 2021-02-22 14:46:35 > Step:   61600 Progress: 30.8% Elapsed: 1 day, 17:31:05 temperature: 1.0 ada_p: No Discriminator_loss: 5.01882 Generator_loss: 6.15265 
    [INFO] 2021-02-22 14:50:27 > Step:   61700 Progress: 30.9% Elapsed: 1 day, 17:34:57 temperature: 1.0 ada_p: No Discriminator_loss: 7.16094 Generator_loss: 5.91486 
    [INFO] 2021-02-22 14:54:19 > Step:   61800 Progress: 30.9% Elapsed: 1 day, 17:38:48 temperature: 1.0 ada_p: No Discriminator_loss: 5.04549 Generator_loss: 6.2683 
    [INFO] 2021-02-22 14:58:10 > Step:   61900 Progress: 30.9% Elapsed: 1 day, 17:42:40 temperature: 1.0 ada_p: No Discriminator_loss: 4.98412 Generator_loss: 6.48826 
    [INFO] 2021-02-22 15:02:02 > Step:   62000 Progress: 31.0% Elapsed: 1 day, 17:46:32 temperature: 1.0 ada_p: No Discriminator_loss: 5.11212 Generator_loss: 6.06465 
    [INFO] 2021-02-22 15:05:53 > Step:   62100 Progress: 31.1% Elapsed: 1 day, 17:50:23 temperature: 1.0 ada_p: No Discriminator_loss: 4.95372 Generator_loss: 6.42268 
    [INFO] 2021-02-22 15:09:44 > Step:   62200 Progress: 31.1% Elapsed: 1 day, 17:54:14 temperature: 1.0 ada_p: No Discriminator_loss: 5.13628 Generator_loss: 6.01807 
    [INFO] 2021-02-22 15:13:38 > Step:   62300 Progress: 31.1% Elapsed: 1 day, 17:58:08 temperature: 1.0 ada_p: No Discriminator_loss: 5.05635 Generator_loss: 6.26319 
    [INFO] 2021-02-22 15:17:30 > Step:   62400 Progress: 31.2% Elapsed: 1 day, 18:02:00 temperature: 1.0 ada_p: No Discriminator_loss: 5.01339 Generator_loss: 6.23822 
    [INFO] 2021-02-22 15:21:22 > Step:   62500 Progress: 31.2% Elapsed: 1 day, 18:05:52 temperature: 1.0 ada_p: No Discriminator_loss: 5.00103 Generator_loss: 6.39918 
    [INFO] 2021-02-22 15:25:18 > Step:   62600 Progress: 31.3% Elapsed: 1 day, 18:09:48 temperature: 1.0 ada_p: No Discriminator_loss: 4.97998 Generator_loss: 6.26209 
    [INFO] 2021-02-22 15:29:10 > Step:   62700 Progress: 31.4% Elapsed: 1 day, 18:13:40 temperature: 1.0 ada_p: No Discriminator_loss: 5.00511 Generator_loss: 6.53149 
    [INFO] 2021-02-22 15:33:02 > Step:   62800 Progress: 31.4% Elapsed: 1 day, 18:17:32 temperature: 1.0 ada_p: No Discriminator_loss: 5.10841 Generator_loss: 6.3155 
    [INFO] 2021-02-22 15:36:54 > Step:   62900 Progress: 31.4% Elapsed: 1 day, 18:21:24 temperature: 1.0 ada_p: No Discriminator_loss: 4.95776 Generator_loss: 6.27106 
    [INFO] 2021-02-22 15:40:45 > Step:   63000 Progress: 31.5% Elapsed: 1 day, 18:25:15 temperature: 1.0 ada_p: No Discriminator_loss: 5.1131 Generator_loss: 6.06904 
    [INFO] 2021-02-22 15:44:36 > Step:   63100 Progress: 31.6% Elapsed: 1 day, 18:29:06 temperature: 1.0 ada_p: No Discriminator_loss: 5.1849 Generator_loss: 6.02514 
    [INFO] 2021-02-22 15:48:27 > Step:   63200 Progress: 31.6% Elapsed: 1 day, 18:32:57 temperature: 1.0 ada_p: No Discriminator_loss: 4.9453 Generator_loss: 6.26532 
    [INFO] 2021-02-22 15:52:18 > Step:   63300 Progress: 31.6% Elapsed: 1 day, 18:36:48 temperature: 1.0 ada_p: No Discriminator_loss: 5.05653 Generator_loss: 6.18351 
    [INFO] 2021-02-22 15:56:10 > Step:   63400 Progress: 31.7% Elapsed: 1 day, 18:40:40 temperature: 1.0 ada_p: No Discriminator_loss: 5.10892 Generator_loss: 6.04262 
    [INFO] 2021-02-22 16:00:01 > Step:   63500 Progress: 31.8% Elapsed: 1 day, 18:44:31 temperature: 1.0 ada_p: No Discriminator_loss: 5.09856 Generator_loss: 6.44507 
    [INFO] 2021-02-22 16:03:53 > Step:   63600 Progress: 31.8% Elapsed: 1 day, 18:48:23 temperature: 1.0 ada_p: No Discriminator_loss: 4.94793 Generator_loss: 6.21656 
    [INFO] 2021-02-22 16:07:46 > Step:   63700 Progress: 31.9% Elapsed: 1 day, 18:52:15 temperature: 1.0 ada_p: No Discriminator_loss: 5.09282 Generator_loss: 6.05934 
    [INFO] 2021-02-22 16:11:38 > Step:   63800 Progress: 31.9% Elapsed: 1 day, 18:56:08 temperature: 1.0 ada_p: No Discriminator_loss: 5.01985 Generator_loss: 6.16901 
    [INFO] 2021-02-22 16:15:30 > Step:   63900 Progress: 31.9% Elapsed: 1 day, 19:00:00 temperature: 1.0 ada_p: No Discriminator_loss: 5.06293 Generator_loss: 6.18355 
    [INFO] 2021-02-22 16:19:21 > Step:   64000 Progress: 32.0% Elapsed: 1 day, 19:03:51 temperature: 1.0 ada_p: No Discriminator_loss: 5.01043 Generator_loss: 6.34307 
    [INFO] 2021-02-22 16:23:12 > Step:   64100 Progress: 32.0% Elapsed: 1 day, 19:07:42 temperature: 1.0 ada_p: No Discriminator_loss: 4.98594 Generator_loss: 6.44038 
    [INFO] 2021-02-22 16:27:03 > Step:   64200 Progress: 32.1% Elapsed: 1 day, 19:11:33 temperature: 1.0 ada_p: No Discriminator_loss: 5.0333 Generator_loss: 6.15523 
    [INFO] 2021-02-22 16:30:54 > Step:   64300 Progress: 32.1% Elapsed: 1 day, 19:15:24 temperature: 1.0 ada_p: No Discriminator_loss: 4.92508 Generator_loss: 6.23601 
    [INFO] 2021-02-22 16:34:46 > Step:   64400 Progress: 32.2% Elapsed: 1 day, 19:19:16 temperature: 1.0 ada_p: No Discriminator_loss: 5.01746 Generator_loss: 5.84882 
    [INFO] 2021-02-22 16:38:37 > Step:   64500 Progress: 32.2% Elapsed: 1 day, 19:23:07 temperature: 1.0 ada_p: No Discriminator_loss: 5.14181 Generator_loss: 6.16386 
    [INFO] 2021-02-22 16:42:29 > Step:   64600 Progress: 32.3% Elapsed: 1 day, 19:26:59 temperature: 1.0 ada_p: No Discriminator_loss: 5.23365 Generator_loss: 5.57016 
    [INFO] 2021-02-22 16:46:21 > Step:   64700 Progress: 32.4% Elapsed: 1 day, 19:30:51 temperature: 1.0 ada_p: No Discriminator_loss: 5.04717 Generator_loss: 6.17993 
    [INFO] 2021-02-22 16:50:13 > Step:   64800 Progress: 32.4% Elapsed: 1 day, 19:34:43 temperature: 1.0 ada_p: No Discriminator_loss: 5.04249 Generator_loss: 5.89343 
    [INFO] 2021-02-22 16:54:05 > Step:   64900 Progress: 32.5% Elapsed: 1 day, 19:38:35 temperature: 1.0 ada_p: No Discriminator_loss: 5.04389 Generator_loss: 6.19408 
    [INFO] 2021-02-22 16:57:57 > Step:   65000 Progress: 32.5% Elapsed: 1 day, 19:42:27 temperature: 1.0 ada_p: No Discriminator_loss: 5.42393 Generator_loss: 5.84119 
    [INFO] 2021-02-22 17:01:53 > Step:   65100 Progress: 32.6% Elapsed: 1 day, 19:46:23 temperature: 1.0 ada_p: No Discriminator_loss: 4.98968 Generator_loss: 6.41098 
    [INFO] 2021-02-22 17:05:44 > Step:   65200 Progress: 32.6% Elapsed: 1 day, 19:50:14 temperature: 1.0 ada_p: No Discriminator_loss: 4.95085 Generator_loss: 6.23883 
    [INFO] 2021-02-22 17:09:35 > Step:   65300 Progress: 32.6% Elapsed: 1 day, 19:54:05 temperature: 1.0 ada_p: No Discriminator_loss: 5.166 Generator_loss: 6.07125 
    [INFO] 2021-02-22 17:13:27 > Step:   65400 Progress: 32.7% Elapsed: 1 day, 19:57:57 temperature: 1.0 ada_p: No Discriminator_loss: 5.08408 Generator_loss: 5.94584 
    [INFO] 2021-02-22 17:17:19 > Step:   65500 Progress: 32.8% Elapsed: 1 day, 20:01:48 temperature: 1.0 ada_p: No Discriminator_loss: 5.1091 Generator_loss: 6.09569 
    [INFO] 2021-02-22 17:21:11 > Step:   65600 Progress: 32.8% Elapsed: 1 day, 20:05:41 temperature: 1.0 ada_p: No Discriminator_loss: 4.91542 Generator_loss: 6.48114 
    [INFO] 2021-02-22 17:25:03 > Step:   65700 Progress: 32.9% Elapsed: 1 day, 20:09:33 temperature: 1.0 ada_p: No Discriminator_loss: 4.9874 Generator_loss: 6.39021 
    [INFO] 2021-02-22 17:28:55 > Step:   65800 Progress: 32.9% Elapsed: 1 day, 20:13:25 temperature: 1.0 ada_p: No Discriminator_loss: 4.9616 Generator_loss: 6.31673 
    [INFO] 2021-02-22 17:32:47 > Step:   65900 Progress: 33.0% Elapsed: 1 day, 20:17:17 temperature: 1.0 ada_p: No Discriminator_loss: 5.13341 Generator_loss: 6.02269 
    [INFO] 2021-02-22 17:36:39 > Step:   66000 Progress: 33.0% Elapsed: 1 day, 20:21:09 temperature: 1.0 ada_p: No Discriminator_loss: 5.01062 Generator_loss: 6.07063 
    [INFO] 2021-02-22 17:40:31 > Step:   66100 Progress: 33.1% Elapsed: 1 day, 20:25:01 temperature: 1.0 ada_p: No Discriminator_loss: 5.17103 Generator_loss: 6.02726 
    [INFO] 2021-02-22 17:44:23 > Step:   66200 Progress: 33.1% Elapsed: 1 day, 20:28:53 temperature: 1.0 ada_p: No Discriminator_loss: 4.94602 Generator_loss: 6.54384 
    [INFO] 2021-02-22 17:48:15 > Step:   66300 Progress: 33.1% Elapsed: 1 day, 20:32:45 temperature: 1.0 ada_p: No Discriminator_loss: 5.22002 Generator_loss: 5.81684 
    [INFO] 2021-02-22 17:52:05 > Step:   66400 Progress: 33.2% Elapsed: 1 day, 20:36:35 temperature: 1.0 ada_p: No Discriminator_loss: 5.00967 Generator_loss: 6.16887 
    [INFO] 2021-02-22 17:55:57 > Step:   66500 Progress: 33.2% Elapsed: 1 day, 20:40:27 temperature: 1.0 ada_p: No Discriminator_loss: 5.01106 Generator_loss: 6.27861 
    [INFO] 2021-02-22 17:59:49 > Step:   66600 Progress: 33.3% Elapsed: 1 day, 20:44:19 temperature: 1.0 ada_p: No Discriminator_loss: 4.92923 Generator_loss: 6.45368 
    [INFO] 2021-02-22 18:03:41 > Step:   66700 Progress: 33.4% Elapsed: 1 day, 20:48:11 temperature: 1.0 ada_p: No Discriminator_loss: 5.04997 Generator_loss: 5.87841 
    [INFO] 2021-02-22 18:07:33 > Step:   66800 Progress: 33.4% Elapsed: 1 day, 20:52:03 temperature: 1.0 ada_p: No Discriminator_loss: 5.25908 Generator_loss: 5.99957 
    [INFO] 2021-02-22 18:11:25 > Step:   66900 Progress: 33.5% Elapsed: 1 day, 20:55:55 temperature: 1.0 ada_p: No Discriminator_loss: 5.0271 Generator_loss: 6.13902 
    [INFO] 2021-02-22 18:15:15 > Step:   67000 Progress: 33.5% Elapsed: 1 day, 20:59:45 temperature: 1.0 ada_p: No Discriminator_loss: 5.15851 Generator_loss: 6.01697 
    [INFO] 2021-02-22 18:19:05 > Step:   67100 Progress: 33.6% Elapsed: 1 day, 21:03:35 temperature: 1.0 ada_p: No Discriminator_loss: 5.03582 Generator_loss: 6.37253 
    [INFO] 2021-02-22 18:22:57 > Step:   67200 Progress: 33.6% Elapsed: 1 day, 21:07:27 temperature: 1.0 ada_p: No Discriminator_loss: 5.15196 Generator_loss: 6.04485 
    [INFO] 2021-02-22 18:26:49 > Step:   67300 Progress: 33.7% Elapsed: 1 day, 21:11:19 temperature: 1.0 ada_p: No Discriminator_loss: 6.27865 Generator_loss: 5.83469 
    [INFO] 2021-02-22 18:30:42 > Step:   67400 Progress: 33.7% Elapsed: 1 day, 21:15:12 temperature: 1.0 ada_p: No Discriminator_loss: 5.06861 Generator_loss: 6.11417 
    [INFO] 2021-02-22 18:34:33 > Step:   67500 Progress: 33.8% Elapsed: 1 day, 21:19:03 temperature: 1.0 ada_p: No Discriminator_loss: 5.13569 Generator_loss: 5.98584 
    [INFO] 2021-02-22 18:38:29 > Step:   67600 Progress: 33.8% Elapsed: 1 day, 21:22:59 temperature: 1.0 ada_p: No Discriminator_loss: 4.96016 Generator_loss: 6.65143 
    [INFO] 2021-02-22 18:42:18 > Step:   67700 Progress: 33.9% Elapsed: 1 day, 21:26:48 temperature: 1.0 ada_p: No Discriminator_loss: 5.14996 Generator_loss: 6.09203 
    [INFO] 2021-02-22 18:46:11 > Step:   67800 Progress: 33.9% Elapsed: 1 day, 21:30:41 temperature: 1.0 ada_p: No Discriminator_loss: 5.02492 Generator_loss: 6.28857 
    [INFO] 2021-02-22 18:50:03 > Step:   67900 Progress: 34.0% Elapsed: 1 day, 21:34:33 temperature: 1.0 ada_p: No Discriminator_loss: 5.074 Generator_loss: 5.98781 
    [INFO] 2021-02-22 18:53:55 > Step:   68000 Progress: 34.0% Elapsed: 1 day, 21:38:25 temperature: 1.0 ada_p: No Discriminator_loss: 4.97398 Generator_loss: 6.32286 
    [INFO] 2021-02-22 18:57:47 > Step:   68100 Progress: 34.1% Elapsed: 1 day, 21:42:17 temperature: 1.0 ada_p: No Discriminator_loss: 5.30424 Generator_loss: 5.92322 
    [INFO] 2021-02-22 19:01:39 > Step:   68200 Progress: 34.1% Elapsed: 1 day, 21:46:09 temperature: 1.0 ada_p: No Discriminator_loss: 5.09036 Generator_loss: 6.12119 
    [INFO] 2021-02-22 19:05:28 > Step:   68300 Progress: 34.2% Elapsed: 1 day, 21:49:58 temperature: 1.0 ada_p: No Discriminator_loss: 5.0027 Generator_loss: 6.07366 
    [INFO] 2021-02-22 19:09:18 > Step:   68400 Progress: 34.2% Elapsed: 1 day, 21:53:48 temperature: 1.0 ada_p: No Discriminator_loss: 5.00306 Generator_loss: 6.3909 
    [INFO] 2021-02-22 19:13:10 > Step:   68500 Progress: 34.2% Elapsed: 1 day, 21:57:40 temperature: 1.0 ada_p: No Discriminator_loss: 5.07798 Generator_loss: 6.09009 
    [INFO] 2021-02-22 19:17:01 > Step:   68600 Progress: 34.3% Elapsed: 1 day, 22:01:31 temperature: 1.0 ada_p: No Discriminator_loss: 5.05154 Generator_loss: 6.063 
    [INFO] 2021-02-22 19:20:52 > Step:   68700 Progress: 34.4% Elapsed: 1 day, 22:05:22 temperature: 1.0 ada_p: No Discriminator_loss: 5.01097 Generator_loss: 6.21314 
    [INFO] 2021-02-22 19:24:44 > Step:   68800 Progress: 34.4% Elapsed: 1 day, 22:09:14 temperature: 1.0 ada_p: No Discriminator_loss: 5.09892 Generator_loss: 5.97032 
    [INFO] 2021-02-22 19:28:36 > Step:   68900 Progress: 34.4% Elapsed: 1 day, 22:13:06 temperature: 1.0 ada_p: No Discriminator_loss: 5.05553 Generator_loss: 6.26956 
    [INFO] 2021-02-22 19:32:27 > Step:   69000 Progress: 34.5% Elapsed: 1 day, 22:16:57 temperature: 1.0 ada_p: No Discriminator_loss: 4.98984 Generator_loss: 6.39778 
    [INFO] 2021-02-22 19:36:17 > Step:   69100 Progress: 34.5% Elapsed: 1 day, 22:20:47 temperature: 1.0 ada_p: No Discriminator_loss: 4.97432 Generator_loss: 6.27515 
    [INFO] 2021-02-22 19:40:08 > Step:   69200 Progress: 34.6% Elapsed: 1 day, 22:24:38 temperature: 1.0 ada_p: No Discriminator_loss: 4.94753 Generator_loss: 6.1465 
    [INFO] 2021-02-22 19:44:00 > Step:   69300 Progress: 34.6% Elapsed: 1 day, 22:28:30 temperature: 1.0 ada_p: No Discriminator_loss: 5.28189 Generator_loss: 5.6204 
    [INFO] 2021-02-22 19:47:52 > Step:   69400 Progress: 34.7% Elapsed: 1 day, 22:32:22 temperature: 1.0 ada_p: No Discriminator_loss: 5.01257 Generator_loss: 6.31947 
    [INFO] 2021-02-22 19:51:45 > Step:   69500 Progress: 34.8% Elapsed: 1 day, 22:36:15 temperature: 1.0 ada_p: No Discriminator_loss: 5.2111 Generator_loss: 5.9306 
    [INFO] 2021-02-22 19:55:37 > Step:   69600 Progress: 34.8% Elapsed: 1 day, 22:40:07 temperature: 1.0 ada_p: No Discriminator_loss: 4.98052 Generator_loss: 6.11901 
    [INFO] 2021-02-22 19:59:28 > Step:   69700 Progress: 34.8% Elapsed: 1 day, 22:43:58 temperature: 1.0 ada_p: No Discriminator_loss: 5.12434 Generator_loss: 5.98364 
    [INFO] 2021-02-22 20:03:18 > Step:   69800 Progress: 34.9% Elapsed: 1 day, 22:47:48 temperature: 1.0 ada_p: No Discriminator_loss: 5.04282 Generator_loss: 5.99922 
    [INFO] 2021-02-22 20:07:10 > Step:   69900 Progress: 34.9% Elapsed: 1 day, 22:51:40 temperature: 1.0 ada_p: No Discriminator_loss: 4.98907 Generator_loss: 6.18781 
    [INFO] 2021-02-22 20:11:02 > Step:   70000 Progress: 35.0% Elapsed: 1 day, 22:55:32 temperature: 1.0 ada_p: No Discriminator_loss: 5.07796 Generator_loss: 6.19389 
    [INFO] 2021-02-22 20:11:02 > Start Evaluation (70000 Step): ContraGAN256-train-2021_02_20_21_08_34
    [INFO] 2021-02-22 20:11:02 > Start Evaluation FID score ... 
    [INFO] 2021-02-22 20:11:02 > Calculating FID Score....
    [INFO] 2021-02-22 20:12:20 > Start Evaluation IS score ... 
    [INFO] 2021-02-22 20:12:20 > Calculating Inception Score....
    [INFO] 2021-02-22 20:13:23 > Start Evaluation F Beta score ... 
    [INFO] 2021-02-22 20:13:23 > Calculate F_beta Score....
    [INFO] 2021-02-22 20:27:29 > Calculate Accuracies....
    [INFO] 2021-02-22 20:28:48 > Calculate Accuracies....
    [INFO] 2021-02-22 20:35:21 > F_8 score (Step: 70000, Using valid images): 0.5493300340333513
    [INFO] 2021-02-22 20:35:21 > F_1/8 score (Step: 70000, Using valid images): 0.7079148330028179
    [INFO] 2021-02-22 20:35:21 > FID score (Step: 70000, Using valid moments): 62.75621204121188
    [INFO] 2021-02-22 20:35:21 > Inception score (Step: 70000, 50000 generated images): 15.645279884338379
    [INFO] 2021-02-22 20:35:21 > Best FID score (Step: 60000, Using valid moments): 60.38544475604181
    [INFO] 2021-02-22 20:35:25 > Save model to checkpoints/ContraGAN256-train-2021_02_20_21_08_34
    [INFO] 2021-02-22 20:39:20 > Step:   70100 Progress: 35.0% Elapsed: 1 day, 23:23:50 temperature: 1.0 ada_p: No Discriminator_loss: 5.01168 Generator_loss: 6.28082 
    [INFO] 2021-02-22 20:43:11 > Step:   70200 Progress: 35.1% Elapsed: 1 day, 23:27:41 temperature: 1.0 ada_p: No Discriminator_loss: 5.01508 Generator_loss: 6.2193 
    [INFO] 2021-02-22 20:47:01 > Step:   70300 Progress: 35.1% Elapsed: 1 day, 23:31:31 temperature: 1.0 ada_p: No Discriminator_loss: 5.03012 Generator_loss: 6.14338 
    [INFO] 2021-02-22 20:50:48 > Step:   70400 Progress: 35.2% Elapsed: 1 day, 23:35:18 temperature: 1.0 ada_p: No Discriminator_loss: 4.90132 Generator_loss: 6.41045 
    [INFO] 2021-02-22 20:54:39 > Step:   70500 Progress: 35.2% Elapsed: 1 day, 23:39:08 temperature: 1.0 ada_p: No Discriminator_loss: 5.18983 Generator_loss: 6.03834 
    [INFO] 2021-02-22 20:58:29 > Step:   70600 Progress: 35.3% Elapsed: 1 day, 23:42:59 temperature: 1.0 ada_p: No Discriminator_loss: 4.93996 Generator_loss: 6.30489 
    [INFO] 2021-02-22 21:02:40 > Step:   70700 Progress: 35.4% Elapsed: 1 day, 23:47:10 temperature: 1.0 ada_p: No Discriminator_loss: 4.92518 Generator_loss: 6.36177 
    [INFO] 2021-02-22 21:06:31 > Step:   70800 Progress: 35.4% Elapsed: 1 day, 23:51:01 temperature: 1.0 ada_p: No Discriminator_loss: 4.96478 Generator_loss: 6.32473 
    [INFO] 2021-02-22 21:10:23 > Step:   70900 Progress: 35.4% Elapsed: 1 day, 23:54:53 temperature: 1.0 ada_p: No Discriminator_loss: 4.98691 Generator_loss: 6.32561 
    [INFO] 2021-02-22 21:14:12 > Step:   71000 Progress: 35.5% Elapsed: 1 day, 23:58:42 temperature: 1.0 ada_p: No Discriminator_loss: 5.0385 Generator_loss: 6.34438 
    [INFO] 2021-02-22 21:18:02 > Step:   71100 Progress: 35.5% Elapsed: 2 days, 0:02:32 temperature: 1.0 ada_p: No Discriminator_loss: 5.01648 Generator_loss: 6.18221 
    [INFO] 2021-02-22 21:21:53 > Step:   71200 Progress: 35.6% Elapsed: 2 days, 0:06:23 temperature: 1.0 ada_p: No Discriminator_loss: 5.14308 Generator_loss: 5.9936 
    [INFO] 2021-02-22 21:25:44 > Step:   71300 Progress: 35.6% Elapsed: 2 days, 0:10:14 temperature: 1.0 ada_p: No Discriminator_loss: 4.90273 Generator_loss: 6.42052 
    [INFO] 2021-02-22 21:29:35 > Step:   71400 Progress: 35.7% Elapsed: 2 days, 0:14:05 temperature: 1.0 ada_p: No Discriminator_loss: 5.13237 Generator_loss: 5.90916 
    [INFO] 2021-02-22 21:33:26 > Step:   71500 Progress: 35.8% Elapsed: 2 days, 0:17:56 temperature: 1.0 ada_p: No Discriminator_loss: 5.00511 Generator_loss: 6.40525 
    [INFO] 2021-02-22 21:37:14 > Step:   71600 Progress: 35.8% Elapsed: 2 days, 0:21:44 temperature: 1.0 ada_p: No Discriminator_loss: 5.05125 Generator_loss: 6.12683 
    [INFO] 2021-02-22 21:41:06 > Step:   71700 Progress: 35.9% Elapsed: 2 days, 0:25:35 temperature: 1.0 ada_p: No Discriminator_loss: 5.02803 Generator_loss: 6.30907 
    [INFO] 2021-02-22 21:44:56 > Step:   71800 Progress: 35.9% Elapsed: 2 days, 0:29:26 temperature: 1.0 ada_p: No Discriminator_loss: 5.04658 Generator_loss: 6.0927 
    [INFO] 2021-02-22 21:48:47 > Step:   71900 Progress: 35.9% Elapsed: 2 days, 0:33:17 temperature: 1.0 ada_p: No Discriminator_loss: 5.11583 Generator_loss: 6.13016 
    [INFO] 2021-02-22 21:52:38 > Step:   72000 Progress: 36.0% Elapsed: 2 days, 0:37:07 temperature: 1.0 ada_p: No Discriminator_loss: 5.01369 Generator_loss: 6.13418 
    [INFO] 2021-02-22 21:56:28 > Step:   72100 Progress: 36.0% Elapsed: 2 days, 0:40:58 temperature: 1.0 ada_p: No Discriminator_loss: 4.97082 Generator_loss: 6.22012 
    [INFO] 2021-02-22 22:00:17 > Step:   72200 Progress: 36.1% Elapsed: 2 days, 0:44:47 temperature: 1.0 ada_p: No Discriminator_loss: 4.96939 Generator_loss: 6.31906 
    [INFO] 2021-02-22 22:04:10 > Step:   72300 Progress: 36.1% Elapsed: 2 days, 0:48:40 temperature: 1.0 ada_p: No Discriminator_loss: 5.02017 Generator_loss: 6.06522 
    [INFO] 2021-02-22 22:08:02 > Step:   72400 Progress: 36.2% Elapsed: 2 days, 0:52:32 temperature: 1.0 ada_p: No Discriminator_loss: 4.93595 Generator_loss: 6.3322 
    [INFO] 2021-02-22 22:11:54 > Step:   72500 Progress: 36.2% Elapsed: 2 days, 0:56:24 temperature: 1.0 ada_p: No Discriminator_loss: 5.02258 Generator_loss: 6.02204 
    [INFO] 2021-02-22 22:15:51 > Step:   72600 Progress: 36.3% Elapsed: 2 days, 1:00:21 temperature: 1.0 ada_p: No Discriminator_loss: 4.85364 Generator_loss: 6.53545 
    [INFO] 2021-02-22 22:19:43 > Step:   72700 Progress: 36.4% Elapsed: 2 days, 1:04:13 temperature: 1.0 ada_p: No Discriminator_loss: 5.20369 Generator_loss: 5.83993 
    [INFO] 2021-02-22 22:23:35 > Step:   72800 Progress: 36.4% Elapsed: 2 days, 1:08:04 temperature: 1.0 ada_p: No Discriminator_loss: 4.90157 Generator_loss: 6.31382 
    [INFO] 2021-02-22 22:27:27 > Step:   72900 Progress: 36.4% Elapsed: 2 days, 1:11:57 temperature: 1.0 ada_p: No Discriminator_loss: 4.93695 Generator_loss: 6.21095 
    [INFO] 2021-02-22 22:31:19 > Step:   73000 Progress: 36.5% Elapsed: 2 days, 1:15:49 temperature: 1.0 ada_p: No Discriminator_loss: 4.9514 Generator_loss: 6.42915 
    [INFO] 2021-02-22 22:35:12 > Step:   73100 Progress: 36.5% Elapsed: 2 days, 1:19:42 temperature: 1.0 ada_p: No Discriminator_loss: 4.90313 Generator_loss: 6.2669 
    [INFO] 2021-02-22 22:39:04 > Step:   73200 Progress: 36.6% Elapsed: 2 days, 1:23:34 temperature: 1.0 ada_p: No Discriminator_loss: 5.28876 Generator_loss: 5.96186 
    [INFO] 2021-02-22 22:42:55 > Step:   73300 Progress: 36.6% Elapsed: 2 days, 1:27:25 temperature: 1.0 ada_p: No Discriminator_loss: 5.29704 Generator_loss: 5.63542 
    [INFO] 2021-02-22 22:46:47 > Step:   73400 Progress: 36.7% Elapsed: 2 days, 1:31:17 temperature: 1.0 ada_p: No Discriminator_loss: 5.15202 Generator_loss: 5.95966 
    [INFO] 2021-02-22 22:50:40 > Step:   73500 Progress: 36.8% Elapsed: 2 days, 1:35:10 temperature: 1.0 ada_p: No Discriminator_loss: 5.02155 Generator_loss: 6.16077 
    [INFO] 2021-02-22 22:54:32 > Step:   73600 Progress: 36.8% Elapsed: 2 days, 1:39:02 temperature: 1.0 ada_p: No Discriminator_loss: 5.08617 Generator_loss: 6.10033 
    [INFO] 2021-02-22 22:58:25 > Step:   73700 Progress: 36.9% Elapsed: 2 days, 1:42:55 temperature: 1.0 ada_p: No Discriminator_loss: 4.99281 Generator_loss: 6.26982 
    [INFO] 2021-02-22 23:02:17 > Step:   73800 Progress: 36.9% Elapsed: 2 days, 1:46:47 temperature: 1.0 ada_p: No Discriminator_loss: 5.01369 Generator_loss: 6.07752 
    [INFO] 2021-02-22 23:06:09 > Step:   73900 Progress: 37.0% Elapsed: 2 days, 1:50:38 temperature: 1.0 ada_p: No Discriminator_loss: 5.05151 Generator_loss: 6.09776 
    [INFO] 2021-02-22 23:10:00 > Step:   74000 Progress: 37.0% Elapsed: 2 days, 1:54:30 temperature: 1.0 ada_p: No Discriminator_loss: 4.95993 Generator_loss: 6.21148 
    [INFO] 2021-02-22 23:13:52 > Step:   74100 Progress: 37.0% Elapsed: 2 days, 1:58:22 temperature: 1.0 ada_p: No Discriminator_loss: 5.41162 Generator_loss: 5.85084 
    [INFO] 2021-02-22 23:17:44 > Step:   74200 Progress: 37.1% Elapsed: 2 days, 2:02:13 temperature: 1.0 ada_p: No Discriminator_loss: 5.0156 Generator_loss: 5.89862 
    [INFO] 2021-02-22 23:21:35 > Step:   74300 Progress: 37.1% Elapsed: 2 days, 2:06:05 temperature: 1.0 ada_p: No Discriminator_loss: 5.07035 Generator_loss: 6.17651 
    [INFO] 2021-02-22 23:25:26 > Step:   74400 Progress: 37.2% Elapsed: 2 days, 2:09:56 temperature: 1.0 ada_p: No Discriminator_loss: 5.06915 Generator_loss: 6.48861 
    [INFO] 2021-02-22 23:29:18 > Step:   74500 Progress: 37.2% Elapsed: 2 days, 2:13:48 temperature: 1.0 ada_p: No Discriminator_loss: 5.02812 Generator_loss: 6.1536 
    [INFO] 2021-02-22 23:33:10 > Step:   74600 Progress: 37.3% Elapsed: 2 days, 2:17:40 temperature: 1.0 ada_p: No Discriminator_loss: 4.97814 Generator_loss: 6.60488 
    [INFO] 2021-02-22 23:37:02 > Step:   74700 Progress: 37.4% Elapsed: 2 days, 2:21:32 temperature: 1.0 ada_p: No Discriminator_loss: 4.8989 Generator_loss: 6.20975 
    [INFO] 2021-02-22 23:40:53 > Step:   74800 Progress: 37.4% Elapsed: 2 days, 2:25:23 temperature: 1.0 ada_p: No Discriminator_loss: 4.93609 Generator_loss: 6.25196 
    [INFO] 2021-02-22 23:44:44 > Step:   74900 Progress: 37.5% Elapsed: 2 days, 2:29:14 temperature: 1.0 ada_p: No Discriminator_loss: 5.01392 Generator_loss: 6.10067 
    [INFO] 2021-02-22 23:48:35 > Step:   75000 Progress: 37.5% Elapsed: 2 days, 2:33:05 temperature: 1.0 ada_p: No Discriminator_loss: 5.00554 Generator_loss: 6.13543 
    [INFO] 2021-02-22 23:52:29 > Step:   75100 Progress: 37.5% Elapsed: 2 days, 2:36:59 temperature: 1.0 ada_p: No Discriminator_loss: 5.02503 Generator_loss: 6.4211 
    [INFO] 2021-02-22 23:56:21 > Step:   75200 Progress: 37.6% Elapsed: 2 days, 2:40:51 temperature: 1.0 ada_p: No Discriminator_loss: 4.94365 Generator_loss: 6.07166 
    [INFO] 2021-02-23 00:00:12 > Step:   75300 Progress: 37.6% Elapsed: 2 days, 2:44:42 temperature: 1.0 ada_p: No Discriminator_loss: 4.95317 Generator_loss: 6.28445 
    [INFO] 2021-02-23 00:04:04 > Step:   75400 Progress: 37.7% Elapsed: 2 days, 2:48:34 temperature: 1.0 ada_p: No Discriminator_loss: 5.0052 Generator_loss: 6.05634 
    [INFO] 2021-02-23 00:07:56 > Step:   75500 Progress: 37.8% Elapsed: 2 days, 2:52:26 temperature: 1.0 ada_p: No Discriminator_loss: 5.00374 Generator_loss: 6.1575 
    [INFO] 2021-02-23 00:11:48 > Step:   75600 Progress: 37.8% Elapsed: 2 days, 2:56:18 temperature: 1.0 ada_p: No Discriminator_loss: 4.97284 Generator_loss: 6.3913 
    [INFO] 2021-02-23 00:15:40 > Step:   75700 Progress: 37.9% Elapsed: 2 days, 3:00:10 temperature: 1.0 ada_p: No Discriminator_loss: 5.16414 Generator_loss: 5.9308 
    [INFO] 2021-02-23 00:19:32 > Step:   75800 Progress: 37.9% Elapsed: 2 days, 3:04:02 temperature: 1.0 ada_p: No Discriminator_loss: 4.92423 Generator_loss: 6.30936 
    [INFO] 2021-02-23 00:23:24 > Step:   75900 Progress: 38.0% Elapsed: 2 days, 3:07:54 temperature: 1.0 ada_p: No Discriminator_loss: 5.1584 Generator_loss: 6.32483 
    [INFO] 2021-02-23 00:27:17 > Step:   76000 Progress: 38.0% Elapsed: 2 days, 3:11:47 temperature: 1.0 ada_p: No Discriminator_loss: 4.93453 Generator_loss: 6.55047 
    [INFO] 2021-02-23 00:31:09 > Step:   76100 Progress: 38.0% Elapsed: 2 days, 3:15:39 temperature: 1.0 ada_p: No Discriminator_loss: 5.26382 Generator_loss: 5.88568 
    [INFO] 2021-02-23 00:35:01 > Step:   76200 Progress: 38.1% Elapsed: 2 days, 3:19:31 temperature: 1.0 ada_p: No Discriminator_loss: 4.96302 Generator_loss: 6.00606 
    [INFO] 2021-02-23 00:38:54 > Step:   76300 Progress: 38.1% Elapsed: 2 days, 3:23:24 temperature: 1.0 ada_p: No Discriminator_loss: 5.05491 Generator_loss: 6.05819 
    [INFO] 2021-02-23 00:42:46 > Step:   76400 Progress: 38.2% Elapsed: 2 days, 3:27:16 temperature: 1.0 ada_p: No Discriminator_loss: 4.93503 Generator_loss: 6.22977 
    [INFO] 2021-02-23 00:46:38 > Step:   76500 Progress: 38.2% Elapsed: 2 days, 3:31:08 temperature: 1.0 ada_p: No Discriminator_loss: 5.09206 Generator_loss: 6.00908 
    [INFO] 2021-02-23 00:50:31 > Step:   76600 Progress: 38.3% Elapsed: 2 days, 3:35:00 temperature: 1.0 ada_p: No Discriminator_loss: 4.97503 Generator_loss: 6.22783 
    [INFO] 2021-02-23 00:54:23 > Step:   76700 Progress: 38.4% Elapsed: 2 days, 3:38:53 temperature: 1.0 ada_p: No Discriminator_loss: 4.92132 Generator_loss: 6.56514 
    [INFO] 2021-02-23 00:58:15 > Step:   76800 Progress: 38.4% Elapsed: 2 days, 3:42:45 temperature: 1.0 ada_p: No Discriminator_loss: 5.06744 Generator_loss: 5.88638 
    [INFO] 2021-02-23 01:02:06 > Step:   76900 Progress: 38.5% Elapsed: 2 days, 3:46:36 temperature: 1.0 ada_p: No Discriminator_loss: 5.04441 Generator_loss: 6.26089 
    [INFO] 2021-02-23 01:05:58 > Step:   77000 Progress: 38.5% Elapsed: 2 days, 3:50:28 temperature: 1.0 ada_p: No Discriminator_loss: 4.99399 Generator_loss: 6.31561 
    [INFO] 2021-02-23 01:09:48 > Step:   77100 Progress: 38.6% Elapsed: 2 days, 3:54:18 temperature: 1.0 ada_p: No Discriminator_loss: 5.1192 Generator_loss: 6.12228 
    [INFO] 2021-02-23 01:13:40 > Step:   77200 Progress: 38.6% Elapsed: 2 days, 3:58:10 temperature: 1.0 ada_p: No Discriminator_loss: 4.99583 Generator_loss: 6.26678 
    [INFO] 2021-02-23 01:17:32 > Step:   77300 Progress: 38.6% Elapsed: 2 days, 4:02:02 temperature: 1.0 ada_p: No Discriminator_loss: 5.05303 Generator_loss: 6.08813 
    [INFO] 2021-02-23 01:21:24 > Step:   77400 Progress: 38.7% Elapsed: 2 days, 4:05:54 temperature: 1.0 ada_p: No Discriminator_loss: 4.98712 Generator_loss: 6.39833 
    [INFO] 2021-02-23 01:25:16 > Step:   77500 Progress: 38.8% Elapsed: 2 days, 4:09:46 temperature: 1.0 ada_p: No Discriminator_loss: 4.91082 Generator_loss: 6.35902 
    [INFO] 2021-02-23 01:29:12 > Step:   77600 Progress: 38.8% Elapsed: 2 days, 4:13:42 temperature: 1.0 ada_p: No Discriminator_loss: 5.28513 Generator_loss: 5.73728 
    [INFO] 2021-02-23 01:33:04 > Step:   77700 Progress: 38.9% Elapsed: 2 days, 4:17:34 temperature: 1.0 ada_p: No Discriminator_loss: 4.99686 Generator_loss: 6.14862 
    [INFO] 2021-02-23 01:36:56 > Step:   77800 Progress: 38.9% Elapsed: 2 days, 4:21:26 temperature: 1.0 ada_p: No Discriminator_loss: 4.93689 Generator_loss: 6.20302 
    [INFO] 2021-02-23 01:40:48 > Step:   77900 Progress: 39.0% Elapsed: 2 days, 4:25:18 temperature: 1.0 ada_p: No Discriminator_loss: 4.98237 Generator_loss: 5.87737 
    [INFO] 2021-02-23 01:44:40 > Step:   78000 Progress: 39.0% Elapsed: 2 days, 4:29:10 temperature: 1.0 ada_p: No Discriminator_loss: 4.96725 Generator_loss: 6.43768 
    [INFO] 2021-02-23 01:48:32 > Step:   78100 Progress: 39.1% Elapsed: 2 days, 4:33:02 temperature: 1.0 ada_p: No Discriminator_loss: 4.9972 Generator_loss: 6.14867 
    [INFO] 2021-02-23 01:52:25 > Step:   78200 Progress: 39.1% Elapsed: 2 days, 4:36:54 temperature: 1.0 ada_p: No Discriminator_loss: 5.01012 Generator_loss: 6.35257 
    [INFO] 2021-02-23 01:56:17 > Step:   78300 Progress: 39.1% Elapsed: 2 days, 4:40:46 temperature: 1.0 ada_p: No Discriminator_loss: 4.9827 Generator_loss: 6.1059 
    [INFO] 2021-02-23 02:00:09 > Step:   78400 Progress: 39.2% Elapsed: 2 days, 4:44:39 temperature: 1.0 ada_p: No Discriminator_loss: 5.05167 Generator_loss: 6.13607 
    [INFO] 2021-02-23 02:04:01 > Step:   78500 Progress: 39.2% Elapsed: 2 days, 4:48:31 temperature: 1.0 ada_p: No Discriminator_loss: 5.07889 Generator_loss: 5.91437 
    [INFO] 2021-02-23 02:07:53 > Step:   78600 Progress: 39.3% Elapsed: 2 days, 4:52:23 temperature: 1.0 ada_p: No Discriminator_loss: 5.12428 Generator_loss: 6.00526 
    [INFO] 2021-02-23 02:11:45 > Step:   78700 Progress: 39.4% Elapsed: 2 days, 4:56:15 temperature: 1.0 ada_p: No Discriminator_loss: 5.05424 Generator_loss: 6.15237 
    [INFO] 2021-02-23 02:15:37 > Step:   78800 Progress: 39.4% Elapsed: 2 days, 5:00:07 temperature: 1.0 ada_p: No Discriminator_loss: 4.93702 Generator_loss: 6.29626 
    [INFO] 2021-02-23 02:19:29 > Step:   78900 Progress: 39.5% Elapsed: 2 days, 5:03:59 temperature: 1.0 ada_p: No Discriminator_loss: 4.98731 Generator_loss: 6.17601 
    [INFO] 2021-02-23 02:23:22 > Step:   79000 Progress: 39.5% Elapsed: 2 days, 5:07:52 temperature: 1.0 ada_p: No Discriminator_loss: 5.0343 Generator_loss: 6.04482 
    [INFO] 2021-02-23 02:27:13 > Step:   79100 Progress: 39.6% Elapsed: 2 days, 5:11:43 temperature: 1.0 ada_p: No Discriminator_loss: 4.97674 Generator_loss: 6.12165 
    [INFO] 2021-02-23 02:31:06 > Step:   79200 Progress: 39.6% Elapsed: 2 days, 5:15:35 temperature: 1.0 ada_p: No Discriminator_loss: 5.01402 Generator_loss: 6.21979 
    [INFO] 2021-02-23 02:34:58 > Step:   79300 Progress: 39.6% Elapsed: 2 days, 5:19:28 temperature: 1.0 ada_p: No Discriminator_loss: 5.18312 Generator_loss: 5.95122 
    [INFO] 2021-02-23 02:38:50 > Step:   79400 Progress: 39.7% Elapsed: 2 days, 5:23:19 temperature: 1.0 ada_p: No Discriminator_loss: 4.99477 Generator_loss: 6.4003 
    [INFO] 2021-02-23 02:42:42 > Step:   79500 Progress: 39.8% Elapsed: 2 days, 5:27:12 temperature: 1.0 ada_p: No Discriminator_loss: 4.93962 Generator_loss: 6.54064 
    [INFO] 2021-02-23 02:46:33 > Step:   79600 Progress: 39.8% Elapsed: 2 days, 5:31:03 temperature: 1.0 ada_p: No Discriminator_loss: 5.14351 Generator_loss: 5.99372 
    [INFO] 2021-02-23 02:50:25 > Step:   79700 Progress: 39.9% Elapsed: 2 days, 5:34:55 temperature: 1.0 ada_p: No Discriminator_loss: 5.03227 Generator_loss: 6.07628 
    [INFO] 2021-02-23 02:54:17 > Step:   79800 Progress: 39.9% Elapsed: 2 days, 5:38:47 temperature: 1.0 ada_p: No Discriminator_loss: 5.09841 Generator_loss: 5.76655 
    [INFO] 2021-02-23 02:58:09 > Step:   79900 Progress: 40.0% Elapsed: 2 days, 5:42:39 temperature: 1.0 ada_p: No Discriminator_loss: 4.92225 Generator_loss: 6.46483 
    [INFO] 2021-02-23 03:02:01 > Step:   80000 Progress: 40.0% Elapsed: 2 days, 5:46:31 temperature: 1.0 ada_p: No Discriminator_loss: 4.93722 Generator_loss: 6.45335 
    [INFO] 2021-02-23 03:02:01 > Start Evaluation (80000 Step): ContraGAN256-train-2021_02_20_21_08_34
    [INFO] 2021-02-23 03:02:01 > Start Evaluation FID score ... 
    [INFO] 2021-02-23 03:02:01 > Calculating FID Score....
    [INFO] 2021-02-23 03:03:19 > Start Evaluation IS score ... 
    [INFO] 2021-02-23 03:03:19 > Calculating Inception Score....
    [INFO] 2021-02-23 03:04:21 > Start Evaluation F Beta score ... 
    [INFO] 2021-02-23 03:04:21 > Calculate F_beta Score....
    [INFO] 2021-02-23 03:18:59 > Calculate Accuracies....
    [INFO] 2021-02-23 03:20:19 > Calculate Accuracies....
    [INFO] 2021-02-23 03:26:48 > F_8 score (Step: 80000, Using valid images): 0.590745360974391
    [INFO] 2021-02-23 03:26:48 > F_1/8 score (Step: 80000, Using valid images): 0.6629399337189412
    [INFO] 2021-02-23 03:26:48 > FID score (Step: 80000, Using valid moments): 70.24895783259228
    [INFO] 2021-02-23 03:26:48 > Inception score (Step: 80000, 50000 generated images): 15.23951530456543
    [INFO] 2021-02-23 03:26:48 > Best FID score (Step: 60000, Using valid moments): 60.38544475604181
    [INFO] 2021-02-23 03:26:53 > Save model to checkpoints/ContraGAN256-train-2021_02_20_21_08_34
    [INFO] 2021-02-23 03:30:47 > Step:   80100 Progress: 40.1% Elapsed: 2 days, 6:15:17 temperature: 1.0 ada_p: No Discriminator_loss: 5.12802 Generator_loss: 5.85555 
    [INFO] 2021-02-23 03:34:38 > Step:   80200 Progress: 40.1% Elapsed: 2 days, 6:19:08 temperature: 1.0 ada_p: No Discriminator_loss: 5.03082 Generator_loss: 6.17064 
    [INFO] 2021-02-23 03:38:30 > Step:   80300 Progress: 40.2% Elapsed: 2 days, 6:22:59 temperature: 1.0 ada_p: No Discriminator_loss: 5.02789 Generator_loss: 6.19102 
    [INFO] 2021-02-23 03:42:50 > Step:   80400 Progress: 40.2% Elapsed: 2 days, 6:27:20 temperature: 1.0 ada_p: No Discriminator_loss: 5.03155 Generator_loss: 5.96624 
    [INFO] 2021-02-23 03:46:42 > Step:   80500 Progress: 40.2% Elapsed: 2 days, 6:31:12 temperature: 1.0 ada_p: No Discriminator_loss: 5.04514 Generator_loss: 6.05144 
    [INFO] 2021-02-23 03:50:33 > Step:   80600 Progress: 40.3% Elapsed: 2 days, 6:35:03 temperature: 1.0 ada_p: No Discriminator_loss: 5.09615 Generator_loss: 5.92148 
    [INFO] 2021-02-23 03:54:25 > Step:   80700 Progress: 40.4% Elapsed: 2 days, 6:38:55 temperature: 1.0 ada_p: No Discriminator_loss: 5.08664 Generator_loss: 5.97665 
    [INFO] 2021-02-23 03:58:17 > Step:   80800 Progress: 40.4% Elapsed: 2 days, 6:42:47 temperature: 1.0 ada_p: No Discriminator_loss: 4.96753 Generator_loss: 6.06437 
    [INFO] 2021-02-23 04:02:09 > Step:   80900 Progress: 40.5% Elapsed: 2 days, 6:46:39 temperature: 1.0 ada_p: No Discriminator_loss: 4.91152 Generator_loss: 6.17383 
    [INFO] 2021-02-23 04:06:01 > Step:   81000 Progress: 40.5% Elapsed: 2 days, 6:50:30 temperature: 1.0 ada_p: No Discriminator_loss: 5.38659 Generator_loss: 5.89584 
    [INFO] 2021-02-23 04:09:52 > Step:   81100 Progress: 40.6% Elapsed: 2 days, 6:54:22 temperature: 1.0 ada_p: No Discriminator_loss: 4.99123 Generator_loss: 6.30572 
    [INFO] 2021-02-23 04:13:44 > Step:   81200 Progress: 40.6% Elapsed: 2 days, 6:58:14 temperature: 1.0 ada_p: No Discriminator_loss: 4.89921 Generator_loss: 6.43385 
    [INFO] 2021-02-23 04:17:36 > Step:   81300 Progress: 40.6% Elapsed: 2 days, 7:02:06 temperature: 1.0 ada_p: No Discriminator_loss: 5.04368 Generator_loss: 6.26267 
    [INFO] 2021-02-23 04:21:28 > Step:   81400 Progress: 40.7% Elapsed: 2 days, 7:05:58 temperature: 1.0 ada_p: No Discriminator_loss: 4.95699 Generator_loss: 5.9941 
    [INFO] 2021-02-23 04:25:19 > Step:   81500 Progress: 40.8% Elapsed: 2 days, 7:09:49 temperature: 1.0 ada_p: No Discriminator_loss: 4.98642 Generator_loss: 6.01332 
    [INFO] 2021-02-23 04:29:11 > Step:   81600 Progress: 40.8% Elapsed: 2 days, 7:13:41 temperature: 1.0 ada_p: No Discriminator_loss: 4.91008 Generator_loss: 5.96699 
    [INFO] 2021-02-23 04:33:03 > Step:   81700 Progress: 40.8% Elapsed: 2 days, 7:17:33 temperature: 1.0 ada_p: No Discriminator_loss: 7.05937 Generator_loss: 7.30047 
    [INFO] 2021-02-23 04:36:56 > Step:   81800 Progress: 40.9% Elapsed: 2 days, 7:21:26 temperature: 1.0 ada_p: No Discriminator_loss: 4.98307 Generator_loss: 6.04989 
    [INFO] 2021-02-23 04:40:48 > Step:   81900 Progress: 40.9% Elapsed: 2 days, 7:25:18 temperature: 1.0 ada_p: No Discriminator_loss: 4.8993 Generator_loss: 6.24911 
    [INFO] 2021-02-23 04:44:40 > Step:   82000 Progress: 41.0% Elapsed: 2 days, 7:29:10 temperature: 1.0 ada_p: No Discriminator_loss: 4.91936 Generator_loss: 6.32828 
    [INFO] 2021-02-23 04:48:32 > Step:   82100 Progress: 41.0% Elapsed: 2 days, 7:33:02 temperature: 1.0 ada_p: No Discriminator_loss: 4.90865 Generator_loss: 6.25694 
    [INFO] 2021-02-23 04:52:24 > Step:   82200 Progress: 41.1% Elapsed: 2 days, 7:36:54 temperature: 1.0 ada_p: No Discriminator_loss: 4.86995 Generator_loss: 6.34107 
    [INFO] 2021-02-23 04:56:16 > Step:   82300 Progress: 41.1% Elapsed: 2 days, 7:40:46 temperature: 1.0 ada_p: No Discriminator_loss: 4.92027 Generator_loss: 6.32026 
    [INFO] 2021-02-23 05:00:08 > Step:   82400 Progress: 41.2% Elapsed: 2 days, 7:44:38 temperature: 1.0 ada_p: No Discriminator_loss: 5.27168 Generator_loss: 5.76058 
    [INFO] 2021-02-23 05:04:00 > Step:   82500 Progress: 41.2% Elapsed: 2 days, 7:48:30 temperature: 1.0 ada_p: No Discriminator_loss: 5.09888 Generator_loss: 5.80072 
    [INFO] 2021-02-23 05:07:56 > Step:   82600 Progress: 41.3% Elapsed: 2 days, 7:52:26 temperature: 1.0 ada_p: No Discriminator_loss: 4.96612 Generator_loss: 5.9952 
    [INFO] 2021-02-23 05:11:49 > Step:   82700 Progress: 41.3% Elapsed: 2 days, 7:56:19 temperature: 1.0 ada_p: No Discriminator_loss: 4.9517 Generator_loss: 6.13932 
    [INFO] 2021-02-23 05:15:41 > Step:   82800 Progress: 41.4% Elapsed: 2 days, 8:00:11 temperature: 1.0 ada_p: No Discriminator_loss: 4.94959 Generator_loss: 6.38071 
    [INFO] 2021-02-23 05:19:33 > Step:   82900 Progress: 41.4% Elapsed: 2 days, 8:04:03 temperature: 1.0 ada_p: No Discriminator_loss: 4.92685 Generator_loss: 6.13424 
    [INFO] 2021-02-23 05:23:25 > Step:   83000 Progress: 41.5% Elapsed: 2 days, 8:07:55 temperature: 1.0 ada_p: No Discriminator_loss: 4.89432 Generator_loss: 6.28593 
    [INFO] 2021-02-23 05:27:18 > Step:   83100 Progress: 41.5% Elapsed: 2 days, 8:11:48 temperature: 1.0 ada_p: No Discriminator_loss: 4.91877 Generator_loss: 6.05528 
    [INFO] 2021-02-23 05:31:10 > Step:   83200 Progress: 41.6% Elapsed: 2 days, 8:15:40 temperature: 1.0 ada_p: No Discriminator_loss: 4.9419 Generator_loss: 6.19452 
    [INFO] 2021-02-23 05:35:02 > Step:   83300 Progress: 41.6% Elapsed: 2 days, 8:19:32 temperature: 1.0 ada_p: No Discriminator_loss: 5.06151 Generator_loss: 6.23643 
    [INFO] 2021-02-23 05:38:54 > Step:   83400 Progress: 41.7% Elapsed: 2 days, 8:23:24 temperature: 1.0 ada_p: No Discriminator_loss: 5.02092 Generator_loss: 6.09857 
    [INFO] 2021-02-23 05:42:46 > Step:   83500 Progress: 41.8% Elapsed: 2 days, 8:27:16 temperature: 1.0 ada_p: No Discriminator_loss: 4.95719 Generator_loss: 6.11144 
    [INFO] 2021-02-23 05:46:38 > Step:   83600 Progress: 41.8% Elapsed: 2 days, 8:31:08 temperature: 1.0 ada_p: No Discriminator_loss: 5.02355 Generator_loss: 6.10075 
    [INFO] 2021-02-23 05:50:31 > Step:   83700 Progress: 41.9% Elapsed: 2 days, 8:35:01 temperature: 1.0 ada_p: No Discriminator_loss: 4.96006 Generator_loss: 6.11531 
    [INFO] 2021-02-23 05:54:23 > Step:   83800 Progress: 41.9% Elapsed: 2 days, 8:38:53 temperature: 1.0 ada_p: No Discriminator_loss: 4.93593 Generator_loss: 6.1885 
    [INFO] 2021-02-23 05:58:15 > Step:   83900 Progress: 41.9% Elapsed: 2 days, 8:42:45 temperature: 1.0 ada_p: No Discriminator_loss: 5.12085 Generator_loss: 6.17301 
    [INFO] 2021-02-23 06:02:07 > Step:   84000 Progress: 42.0% Elapsed: 2 days, 8:46:37 temperature: 1.0 ada_p: No Discriminator_loss: 4.88247 Generator_loss: 6.14279 
    [INFO] 2021-02-23 06:05:59 > Step:   84100 Progress: 42.0% Elapsed: 2 days, 8:50:29 temperature: 1.0 ada_p: No Discriminator_loss: 4.92109 Generator_loss: 6.18184 
    [INFO] 2021-02-23 06:09:51 > Step:   84200 Progress: 42.1% Elapsed: 2 days, 8:54:21 temperature: 1.0 ada_p: No Discriminator_loss: 4.99584 Generator_loss: 6.09069 
    [INFO] 2021-02-23 06:13:43 > Step:   84300 Progress: 42.1% Elapsed: 2 days, 8:58:13 temperature: 1.0 ada_p: No Discriminator_loss: 4.95446 Generator_loss: 6.1585 
    [INFO] 2021-02-23 06:17:35 > Step:   84400 Progress: 42.2% Elapsed: 2 days, 9:02:05 temperature: 1.0 ada_p: No Discriminator_loss: 4.99586 Generator_loss: 6.06261 
    [INFO] 2021-02-23 06:21:28 > Step:   84500 Progress: 42.2% Elapsed: 2 days, 9:05:58 temperature: 1.0 ada_p: No Discriminator_loss: 4.97253 Generator_loss: 5.97281 
    [INFO] 2021-02-23 06:25:19 > Step:   84600 Progress: 42.3% Elapsed: 2 days, 9:09:49 temperature: 1.0 ada_p: No Discriminator_loss: 4.94804 Generator_loss: 6.32452 
    [INFO] 2021-02-23 06:29:11 > Step:   84700 Progress: 42.4% Elapsed: 2 days, 9:13:41 temperature: 1.0 ada_p: No Discriminator_loss: 4.96267 Generator_loss: 6.12388 
    [INFO] 2021-02-23 06:33:03 > Step:   84800 Progress: 42.4% Elapsed: 2 days, 9:17:33 temperature: 1.0 ada_p: No Discriminator_loss: 4.9246 Generator_loss: 6.14888 
    [INFO] 2021-02-23 06:36:55 > Step:   84900 Progress: 42.4% Elapsed: 2 days, 9:21:25 temperature: 1.0 ada_p: No Discriminator_loss: 5.0094 Generator_loss: 5.92977 
    [INFO] 2021-02-23 06:40:47 > Step:   85000 Progress: 42.5% Elapsed: 2 days, 9:25:17 temperature: 1.0 ada_p: No Discriminator_loss: 4.88913 Generator_loss: 6.21602 
    [INFO] 2021-02-23 06:44:44 > Step:   85100 Progress: 42.5% Elapsed: 2 days, 9:29:14 temperature: 1.0 ada_p: No Discriminator_loss: 5.01578 Generator_loss: 5.96113 
    [INFO] 2021-02-23 06:48:36 > Step:   85200 Progress: 42.6% Elapsed: 2 days, 9:33:06 temperature: 1.0 ada_p: No Discriminator_loss: 4.88819 Generator_loss: 6.34579 
    [INFO] 2021-02-23 06:52:28 > Step:   85300 Progress: 42.6% Elapsed: 2 days, 9:36:58 temperature: 1.0 ada_p: No Discriminator_loss: 5.2255 Generator_loss: 5.81584 
    [INFO] 2021-02-23 06:56:20 > Step:   85400 Progress: 42.7% Elapsed: 2 days, 9:40:50 temperature: 1.0 ada_p: No Discriminator_loss: 4.89642 Generator_loss: 6.087 
    [INFO] 2021-02-23 07:00:13 > Step:   85500 Progress: 42.8% Elapsed: 2 days, 9:44:43 temperature: 1.0 ada_p: No Discriminator_loss: 4.99426 Generator_loss: 6.01796 
    [INFO] 2021-02-23 07:04:05 > Step:   85600 Progress: 42.8% Elapsed: 2 days, 9:48:35 temperature: 1.0 ada_p: No Discriminator_loss: 4.90401 Generator_loss: 6.34513 
    [INFO] 2021-02-23 07:07:56 > Step:   85700 Progress: 42.9% Elapsed: 2 days, 9:52:26 temperature: 1.0 ada_p: No Discriminator_loss: 4.88898 Generator_loss: 6.17619 
    [INFO] 2021-02-23 07:11:48 > Step:   85800 Progress: 42.9% Elapsed: 2 days, 9:56:18 temperature: 1.0 ada_p: No Discriminator_loss: 4.98845 Generator_loss: 6.31489 
    [INFO] 2021-02-23 07:15:40 > Step:   85900 Progress: 43.0% Elapsed: 2 days, 10:00:10 temperature: 1.0 ada_p: No Discriminator_loss: 5.05083 Generator_loss: 6.10117 
    [INFO] 2021-02-23 07:19:32 > Step:   86000 Progress: 43.0% Elapsed: 2 days, 10:04:02 temperature: 1.0 ada_p: No Discriminator_loss: 4.9708 Generator_loss: 6.31987 
    [INFO] 2021-02-23 07:23:24 > Step:   86100 Progress: 43.0% Elapsed: 2 days, 10:07:54 temperature: 1.0 ada_p: No Discriminator_loss: 4.93352 Generator_loss: 6.38631 
    [INFO] 2021-02-23 07:27:16 > Step:   86200 Progress: 43.1% Elapsed: 2 days, 10:11:46 temperature: 1.0 ada_p: No Discriminator_loss: 4.8922 Generator_loss: 6.24443 
    [INFO] 2021-02-23 07:31:08 > Step:   86300 Progress: 43.1% Elapsed: 2 days, 10:15:38 temperature: 1.0 ada_p: No Discriminator_loss: 5.03252 Generator_loss: 6.10018 
    [INFO] 2021-02-23 07:35:00 > Step:   86400 Progress: 43.2% Elapsed: 2 days, 10:19:30 temperature: 1.0 ada_p: No Discriminator_loss: 4.99069 Generator_loss: 6.10178 
    [INFO] 2021-02-23 07:38:52 > Step:   86500 Progress: 43.2% Elapsed: 2 days, 10:23:22 temperature: 1.0 ada_p: No Discriminator_loss: 4.98235 Generator_loss: 6.03568 
    [INFO] 2021-02-23 07:42:43 > Step:   86600 Progress: 43.3% Elapsed: 2 days, 10:27:13 temperature: 1.0 ada_p: No Discriminator_loss: 4.97119 Generator_loss: 6.01332 
    [INFO] 2021-02-23 07:46:34 > Step:   86700 Progress: 43.4% Elapsed: 2 days, 10:31:04 temperature: 1.0 ada_p: No Discriminator_loss: 5.10057 Generator_loss: 5.9905 
    [INFO] 2021-02-23 07:50:25 > Step:   86800 Progress: 43.4% Elapsed: 2 days, 10:34:55 temperature: 1.0 ada_p: No Discriminator_loss: 4.98186 Generator_loss: 6.09732 
    [INFO] 2021-02-23 07:54:16 > Step:   86900 Progress: 43.5% Elapsed: 2 days, 10:38:46 temperature: 1.0 ada_p: No Discriminator_loss: 4.91195 Generator_loss: 6.07098 
    [INFO] 2021-02-23 07:58:07 > Step:   87000 Progress: 43.5% Elapsed: 2 days, 10:42:37 temperature: 1.0 ada_p: No Discriminator_loss: 4.86943 Generator_loss: 6.3823 
    [INFO] 2021-02-23 08:01:58 > Step:   87100 Progress: 43.5% Elapsed: 2 days, 10:46:28 temperature: 1.0 ada_p: No Discriminator_loss: 4.98351 Generator_loss: 5.95181 
    [INFO] 2021-02-23 08:05:50 > Step:   87200 Progress: 43.6% Elapsed: 2 days, 10:50:20 temperature: 1.0 ada_p: No Discriminator_loss: 4.91358 Generator_loss: 6.14978 
    [INFO] 2021-02-23 08:09:43 > Step:   87300 Progress: 43.6% Elapsed: 2 days, 10:54:13 temperature: 1.0 ada_p: No Discriminator_loss: 4.86473 Generator_loss: 6.18694 
    [INFO] 2021-02-23 08:13:35 > Step:   87400 Progress: 43.7% Elapsed: 2 days, 10:58:05 temperature: 1.0 ada_p: No Discriminator_loss: 4.92522 Generator_loss: 6.14904 
    [INFO] 2021-02-23 08:17:27 > Step:   87500 Progress: 43.8% Elapsed: 2 days, 11:01:57 temperature: 1.0 ada_p: No Discriminator_loss: 4.9944 Generator_loss: 6.40017 
    [INFO] 2021-02-23 08:21:22 > Step:   87600 Progress: 43.8% Elapsed: 2 days, 11:05:52 temperature: 1.0 ada_p: No Discriminator_loss: 5.00698 Generator_loss: 6.01792 
    [INFO] 2021-02-23 08:25:15 > Step:   87700 Progress: 43.9% Elapsed: 2 days, 11:09:45 temperature: 1.0 ada_p: No Discriminator_loss: 4.89839 Generator_loss: 6.03368 
    [INFO] 2021-02-23 08:29:07 > Step:   87800 Progress: 43.9% Elapsed: 2 days, 11:13:37 temperature: 1.0 ada_p: No Discriminator_loss: 4.9016 Generator_loss: 6.37994 
    [INFO] 2021-02-23 08:32:59 > Step:   87900 Progress: 44.0% Elapsed: 2 days, 11:17:29 temperature: 1.0 ada_p: No Discriminator_loss: 4.88697 Generator_loss: 6.06352 
    [INFO] 2021-02-23 08:36:51 > Step:   88000 Progress: 44.0% Elapsed: 2 days, 11:21:21 temperature: 1.0 ada_p: No Discriminator_loss: 4.9155 Generator_loss: 6.20347 
    [INFO] 2021-02-23 08:40:43 > Step:   88100 Progress: 44.0% Elapsed: 2 days, 11:25:13 temperature: 1.0 ada_p: No Discriminator_loss: 4.94417 Generator_loss: 6.18961 
    [INFO] 2021-02-23 08:44:36 > Step:   88200 Progress: 44.1% Elapsed: 2 days, 11:29:06 temperature: 1.0 ada_p: No Discriminator_loss: 5.01876 Generator_loss: 5.92301 
    [INFO] 2021-02-23 08:48:28 > Step:   88300 Progress: 44.1% Elapsed: 2 days, 11:32:58 temperature: 1.0 ada_p: No Discriminator_loss: 4.85899 Generator_loss: 6.26162 
    [INFO] 2021-02-23 08:52:20 > Step:   88400 Progress: 44.2% Elapsed: 2 days, 11:36:50 temperature: 1.0 ada_p: No Discriminator_loss: 5.045 Generator_loss: 5.92607 
    [INFO] 2021-02-23 08:56:12 > Step:   88500 Progress: 44.2% Elapsed: 2 days, 11:40:42 temperature: 1.0 ada_p: No Discriminator_loss: 4.88602 Generator_loss: 6.05301 
    [INFO] 2021-02-23 09:00:02 > Step:   88600 Progress: 44.3% Elapsed: 2 days, 11:44:32 temperature: 1.0 ada_p: No Discriminator_loss: 4.89507 Generator_loss: 6.40207 
    [INFO] 2021-02-23 09:03:54 > Step:   88700 Progress: 44.4% Elapsed: 2 days, 11:48:24 temperature: 1.0 ada_p: No Discriminator_loss: 4.83741 Generator_loss: 6.22803 
    [INFO] 2021-02-23 09:07:45 > Step:   88800 Progress: 44.4% Elapsed: 2 days, 11:52:15 temperature: 1.0 ada_p: No Discriminator_loss: 4.97426 Generator_loss: 6.20987 
    [INFO] 2021-02-23 09:11:36 > Step:   88900 Progress: 44.5% Elapsed: 2 days, 11:56:05 temperature: 1.0 ada_p: No Discriminator_loss: 4.8976 Generator_loss: 6.26204 
    [INFO] 2021-02-23 09:15:27 > Step:   89000 Progress: 44.5% Elapsed: 2 days, 11:59:57 temperature: 1.0 ada_p: No Discriminator_loss: 4.92757 Generator_loss: 6.13161 
    [INFO] 2021-02-23 09:19:19 > Step:   89100 Progress: 44.5% Elapsed: 2 days, 12:03:49 temperature: 1.0 ada_p: No Discriminator_loss: 4.96441 Generator_loss: 6.05318 
    [INFO] 2021-02-23 09:23:11 > Step:   89200 Progress: 44.6% Elapsed: 2 days, 12:07:41 temperature: 1.0 ada_p: No Discriminator_loss: 4.99622 Generator_loss: 6.0512 
    [INFO] 2021-02-23 09:27:03 > Step:   89300 Progress: 44.6% Elapsed: 2 days, 12:11:33 temperature: 1.0 ada_p: No Discriminator_loss: 4.86266 Generator_loss: 6.1296 
    [INFO] 2021-02-23 09:30:55 > Step:   89400 Progress: 44.7% Elapsed: 2 days, 12:15:25 temperature: 1.0 ada_p: No Discriminator_loss: 4.86114 Generator_loss: 6.21185 
    [INFO] 2021-02-23 09:34:48 > Step:   89500 Progress: 44.8% Elapsed: 2 days, 12:19:18 temperature: 1.0 ada_p: No Discriminator_loss: 4.94148 Generator_loss: 5.97761 
    [INFO] 2021-02-23 09:38:40 > Step:   89600 Progress: 44.8% Elapsed: 2 days, 12:23:10 temperature: 1.0 ada_p: No Discriminator_loss: 4.88568 Generator_loss: 6.18203 
    [INFO] 2021-02-23 09:42:33 > Step:   89700 Progress: 44.9% Elapsed: 2 days, 12:27:03 temperature: 1.0 ada_p: No Discriminator_loss: 4.93721 Generator_loss: 6.13719 
    [INFO] 2021-02-23 09:46:25 > Step:   89800 Progress: 44.9% Elapsed: 2 days, 12:30:55 temperature: 1.0 ada_p: No Discriminator_loss: 4.88539 Generator_loss: 6.20645 
    [INFO] 2021-02-23 09:50:17 > Step:   89900 Progress: 45.0% Elapsed: 2 days, 12:34:47 temperature: 1.0 ada_p: No Discriminator_loss: 4.92238 Generator_loss: 6.0885 
    [INFO] 2021-02-23 09:54:09 > Step:   90000 Progress: 45.0% Elapsed: 2 days, 12:38:39 temperature: 1.0 ada_p: No Discriminator_loss: 4.84563 Generator_loss: 6.06762 
    [INFO] 2021-02-23 09:54:09 > Start Evaluation (90000 Step): ContraGAN256-train-2021_02_20_21_08_34
    [INFO] 2021-02-23 09:54:09 > Start Evaluation FID score ... 
    [INFO] 2021-02-23 09:54:09 > Calculating FID Score....
    [INFO] 2021-02-23 09:55:29 > Start Evaluation IS score ... 
    [INFO] 2021-02-23 09:55:29 > Calculating Inception Score....
    [INFO] 2021-02-23 09:56:31 > Start Evaluation F Beta score ... 
    [INFO] 2021-02-23 09:56:31 > Calculate F_beta Score....
    [INFO] 2021-02-23 10:10:48 > Calculate Accuracies....
    [INFO] 2021-02-23 10:12:08 > Calculate Accuracies....
    [INFO] 2021-02-23 10:18:43 > F_8 score (Step: 90000, Using valid images): 0.5547686321634491
    [INFO] 2021-02-23 10:18:43 > F_1/8 score (Step: 90000, Using valid images): 0.5658219612974544
    [INFO] 2021-02-23 10:18:43 > FID score (Step: 90000, Using valid moments): 83.86185976555566
    [INFO] 2021-02-23 10:18:43 > Inception score (Step: 90000, 50000 generated images): 15.04889965057373
    [INFO] 2021-02-23 10:18:43 > Best FID score (Step: 60000, Using valid moments): 60.38544475604181
    [INFO] 2021-02-23 10:18:47 > Save model to checkpoints/ContraGAN256-train-2021_02_20_21_08_34
    [INFO] 2021-02-23 10:22:41 > Step:   90100 Progress: 45.1% Elapsed: 2 days, 13:07:11 temperature: 1.0 ada_p: No Discriminator_loss: 4.81646 Generator_loss: 6.19829 
    [INFO] 2021-02-23 10:26:32 > Step:   90200 Progress: 45.1% Elapsed: 2 days, 13:11:02 temperature: 1.0 ada_p: No Discriminator_loss: 4.84779 Generator_loss: 6.20781 
    [INFO] 2021-02-23 10:30:23 > Step:   90300 Progress: 45.1% Elapsed: 2 days, 13:14:53 temperature: 1.0 ada_p: No Discriminator_loss: 4.94972 Generator_loss: 6.11874 
    [INFO] 2021-02-23 10:34:13 > Step:   90400 Progress: 45.2% Elapsed: 2 days, 13:18:43 temperature: 1.0 ada_p: No Discriminator_loss: 4.9034 Generator_loss: 6.33127 
    [INFO] 2021-02-23 10:38:04 > Step:   90500 Progress: 45.2% Elapsed: 2 days, 13:22:34 temperature: 1.0 ada_p: No Discriminator_loss: 4.97582 Generator_loss: 6.1413 
    [INFO] 2021-02-23 10:41:54 > Step:   90600 Progress: 45.3% Elapsed: 2 days, 13:26:24 temperature: 1.0 ada_p: No Discriminator_loss: 4.95446 Generator_loss: 6.31812 
    [INFO] 2021-02-23 10:45:45 > Step:   90700 Progress: 45.4% Elapsed: 2 days, 13:30:15 temperature: 1.0 ada_p: No Discriminator_loss: 4.88549 Generator_loss: 6.26355 
    [INFO] 2021-02-23 10:49:36 > Step:   90800 Progress: 45.4% Elapsed: 2 days, 13:34:06 temperature: 1.0 ada_p: No Discriminator_loss: 4.90624 Generator_loss: 6.31631 
    [INFO] 2021-02-23 10:53:26 > Step:   90900 Progress: 45.5% Elapsed: 2 days, 13:37:56 temperature: 1.0 ada_p: No Discriminator_loss: 4.87333 Generator_loss: 6.22297 
    [INFO] 2021-02-23 10:57:18 > Step:   91000 Progress: 45.5% Elapsed: 2 days, 13:41:48 temperature: 1.0 ada_p: No Discriminator_loss: 4.90147 Generator_loss: 6.14487 
    [INFO] 2021-02-23 11:01:09 > Step:   91100 Progress: 45.6% Elapsed: 2 days, 13:45:39 temperature: 1.0 ada_p: No Discriminator_loss: 4.86412 Generator_loss: 6.55526 
    [INFO] 2021-02-23 11:05:00 > Step:   91200 Progress: 45.6% Elapsed: 2 days, 13:49:30 temperature: 1.0 ada_p: No Discriminator_loss: 4.8981 Generator_loss: 6.38319 
    [INFO] 2021-02-23 11:08:49 > Step:   91300 Progress: 45.6% Elapsed: 2 days, 13:53:19 temperature: 1.0 ada_p: No Discriminator_loss: 4.92991 Generator_loss: 6.42892 
    [INFO] 2021-02-23 11:12:39 > Step:   91400 Progress: 45.7% Elapsed: 2 days, 13:57:09 temperature: 1.0 ada_p: No Discriminator_loss: 4.9356 Generator_loss: 6.26071 
    [INFO] 2021-02-23 11:16:28 > Step:   91500 Progress: 45.8% Elapsed: 2 days, 14:00:58 temperature: 1.0 ada_p: No Discriminator_loss: 4.96373 Generator_loss: 6.02793 
    [INFO] 2021-02-23 11:20:17 > Step:   91600 Progress: 45.8% Elapsed: 2 days, 14:04:47 temperature: 1.0 ada_p: No Discriminator_loss: 4.95108 Generator_loss: 6.14298 
    [INFO] 2021-02-23 11:24:07 > Step:   91700 Progress: 45.9% Elapsed: 2 days, 14:08:37 temperature: 1.0 ada_p: No Discriminator_loss: 4.9046 Generator_loss: 6.23153 
    [INFO] 2021-02-23 11:28:24 > Step:   91800 Progress: 45.9% Elapsed: 2 days, 14:12:54 temperature: 1.0 ada_p: No Discriminator_loss: 4.95753 Generator_loss: 6.16701 
    [INFO] 2021-02-23 11:32:16 > Step:   91900 Progress: 46.0% Elapsed: 2 days, 14:16:46 temperature: 1.0 ada_p: No Discriminator_loss: 4.89926 Generator_loss: 6.30553 
    [INFO] 2021-02-23 11:36:07 > Step:   92000 Progress: 46.0% Elapsed: 2 days, 14:20:37 temperature: 1.0 ada_p: No Discriminator_loss: 5.00258 Generator_loss: 6.0485 
    [INFO] 2021-02-23 11:39:58 > Step:   92100 Progress: 46.1% Elapsed: 2 days, 14:24:28 temperature: 1.0 ada_p: No Discriminator_loss: 4.90375 Generator_loss: 6.44369 
    [INFO] 2021-02-23 11:43:50 > Step:   92200 Progress: 46.1% Elapsed: 2 days, 14:28:20 temperature: 1.0 ada_p: No Discriminator_loss: 4.87471 Generator_loss: 6.15186 
    [INFO] 2021-02-23 11:47:42 > Step:   92300 Progress: 46.2% Elapsed: 2 days, 14:32:12 temperature: 1.0 ada_p: No Discriminator_loss: 4.86764 Generator_loss: 6.1878 
    [INFO] 2021-02-23 11:51:34 > Step:   92400 Progress: 46.2% Elapsed: 2 days, 14:36:04 temperature: 1.0 ada_p: No Discriminator_loss: 4.8478 Generator_loss: 6.13844 
    [INFO] 2021-02-23 11:55:26 > Step:   92500 Progress: 46.2% Elapsed: 2 days, 14:39:56 temperature: 1.0 ada_p: No Discriminator_loss: 4.89801 Generator_loss: 6.35556 
    [INFO] 2021-02-23 11:59:22 > Step:   92600 Progress: 46.3% Elapsed: 2 days, 14:43:52 temperature: 1.0 ada_p: No Discriminator_loss: 4.89809 Generator_loss: 6.21341 
    [INFO] 2021-02-23 12:03:14 > Step:   92700 Progress: 46.4% Elapsed: 2 days, 14:47:44 temperature: 1.0 ada_p: No Discriminator_loss: 4.99707 Generator_loss: 5.96754 
    [INFO] 2021-02-23 12:07:04 > Step:   92800 Progress: 46.4% Elapsed: 2 days, 14:51:34 temperature: 1.0 ada_p: No Discriminator_loss: 5.08925 Generator_loss: 5.67382 
    [INFO] 2021-02-23 12:10:55 > Step:   92900 Progress: 46.5% Elapsed: 2 days, 14:55:25 temperature: 1.0 ada_p: No Discriminator_loss: 4.87846 Generator_loss: 6.22993 
    [INFO] 2021-02-23 12:14:46 > Step:   93000 Progress: 46.5% Elapsed: 2 days, 14:59:16 temperature: 1.0 ada_p: No Discriminator_loss: 4.89065 Generator_loss: 6.25527 
    [INFO] 2021-02-23 12:18:37 > Step:   93100 Progress: 46.6% Elapsed: 2 days, 15:03:07 temperature: 1.0 ada_p: No Discriminator_loss: 5.03761 Generator_loss: 5.97115 
    [INFO] 2021-02-23 12:22:30 > Step:   93200 Progress: 46.6% Elapsed: 2 days, 15:06:59 temperature: 1.0 ada_p: No Discriminator_loss: 4.89161 Generator_loss: 6.37566 
    [INFO] 2021-02-23 12:26:22 > Step:   93300 Progress: 46.7% Elapsed: 2 days, 15:10:52 temperature: 1.0 ada_p: No Discriminator_loss: 4.89254 Generator_loss: 6.34416 
    [INFO] 2021-02-23 12:30:16 > Step:   93400 Progress: 46.7% Elapsed: 2 days, 15:14:46 temperature: 1.0 ada_p: No Discriminator_loss: 4.92871 Generator_loss: 6.13108 
    [INFO] 2021-02-23 12:34:09 > Step:   93500 Progress: 46.8% Elapsed: 2 days, 15:18:39 temperature: 1.0 ada_p: No Discriminator_loss: 4.90882 Generator_loss: 6.18076 
    [INFO] 2021-02-23 12:38:02 > Step:   93600 Progress: 46.8% Elapsed: 2 days, 15:22:32 temperature: 1.0 ada_p: No Discriminator_loss: 4.8687 Generator_loss: 6.37932 
    [INFO] 2021-02-23 12:41:55 > Step:   93700 Progress: 46.9% Elapsed: 2 days, 15:26:25 temperature: 1.0 ada_p: No Discriminator_loss: 4.97786 Generator_loss: 5.9902 
    [INFO] 2021-02-23 12:45:49 > Step:   93800 Progress: 46.9% Elapsed: 2 days, 15:30:19 temperature: 1.0 ada_p: No Discriminator_loss: 4.8377 Generator_loss: 6.24218 
    [INFO] 2021-02-23 12:49:42 > Step:   93900 Progress: 46.9% Elapsed: 2 days, 15:34:12 temperature: 1.0 ada_p: No Discriminator_loss: 4.87488 Generator_loss: 6.09507 
    [INFO] 2021-02-23 12:53:35 > Step:   94000 Progress: 47.0% Elapsed: 2 days, 15:38:05 temperature: 1.0 ada_p: No Discriminator_loss: 4.8348 Generator_loss: 6.19201 
    [INFO] 2021-02-23 12:57:26 > Step:   94100 Progress: 47.0% Elapsed: 2 days, 15:41:56 temperature: 1.0 ada_p: No Discriminator_loss: 4.89825 Generator_loss: 6.3333 
    [INFO] 2021-02-23 13:01:18 > Step:   94200 Progress: 47.1% Elapsed: 2 days, 15:45:48 temperature: 1.0 ada_p: No Discriminator_loss: 4.83876 Generator_loss: 6.29212 
    [INFO] 2021-02-23 13:05:12 > Step:   94300 Progress: 47.1% Elapsed: 2 days, 15:49:42 temperature: 1.0 ada_p: No Discriminator_loss: 4.85521 Generator_loss: 6.34778 
    [INFO] 2021-02-23 13:09:05 > Step:   94400 Progress: 47.2% Elapsed: 2 days, 15:53:35 temperature: 1.0 ada_p: No Discriminator_loss: 4.87075 Generator_loss: 6.21915 
    [INFO] 2021-02-23 13:12:58 > Step:   94500 Progress: 47.2% Elapsed: 2 days, 15:57:28 temperature: 1.0 ada_p: No Discriminator_loss: 4.89867 Generator_loss: 5.99695 
    [INFO] 2021-02-23 13:16:52 > Step:   94600 Progress: 47.3% Elapsed: 2 days, 16:01:22 temperature: 1.0 ada_p: No Discriminator_loss: 4.91652 Generator_loss: 6.42719 
    [INFO] 2021-02-23 13:20:44 > Step:   94700 Progress: 47.3% Elapsed: 2 days, 16:05:14 temperature: 1.0 ada_p: No Discriminator_loss: 4.96695 Generator_loss: 6.06062 
    [INFO] 2021-02-23 13:24:38 > Step:   94800 Progress: 47.4% Elapsed: 2 days, 16:09:08 temperature: 1.0 ada_p: No Discriminator_loss: 4.83751 Generator_loss: 6.25906 
    [INFO] 2021-02-23 13:28:31 > Step:   94900 Progress: 47.4% Elapsed: 2 days, 16:13:01 temperature: 1.0 ada_p: No Discriminator_loss: 4.90003 Generator_loss: 6.27466 
    [INFO] 2021-02-23 13:32:24 > Step:   95000 Progress: 47.5% Elapsed: 2 days, 16:16:54 temperature: 1.0 ada_p: No Discriminator_loss: 4.90893 Generator_loss: 6.23104 
    [INFO] 2021-02-23 13:36:19 > Step:   95100 Progress: 47.5% Elapsed: 2 days, 16:20:49 temperature: 1.0 ada_p: No Discriminator_loss: 4.90713 Generator_loss: 6.06244 
    [INFO] 2021-02-23 13:40:10 > Step:   95200 Progress: 47.6% Elapsed: 2 days, 16:24:40 temperature: 1.0 ada_p: No Discriminator_loss: 4.84558 Generator_loss: 6.2109 
    [INFO] 2021-02-23 13:44:01 > Step:   95300 Progress: 47.6% Elapsed: 2 days, 16:28:31 temperature: 1.0 ada_p: No Discriminator_loss: 4.85144 Generator_loss: 6.11725 
    [INFO] 2021-02-23 13:47:54 > Step:   95400 Progress: 47.7% Elapsed: 2 days, 16:32:24 temperature: 1.0 ada_p: No Discriminator_loss: 4.86369 Generator_loss: 6.18554 
    [INFO] 2021-02-23 13:51:46 > Step:   95500 Progress: 47.8% Elapsed: 2 days, 16:36:16 temperature: 1.0 ada_p: No Discriminator_loss: 4.88224 Generator_loss: 6.09301 
    [INFO] 2021-02-23 13:55:39 > Step:   95600 Progress: 47.8% Elapsed: 2 days, 16:40:09 temperature: 1.0 ada_p: No Discriminator_loss: 4.88658 Generator_loss: 6.08692 
    [INFO] 2021-02-23 13:59:32 > Step:   95700 Progress: 47.9% Elapsed: 2 days, 16:44:02 temperature: 1.0 ada_p: No Discriminator_loss: 4.8426 Generator_loss: 6.17932 
    [INFO] 2021-02-23 14:03:24 > Step:   95800 Progress: 47.9% Elapsed: 2 days, 16:47:54 temperature: 1.0 ada_p: No Discriminator_loss: 4.93287 Generator_loss: 6.14917 
    [INFO] 2021-02-23 14:07:17 > Step:   95900 Progress: 47.9% Elapsed: 2 days, 16:51:47 temperature: 1.0 ada_p: No Discriminator_loss: 4.89767 Generator_loss: 6.02798 
    [INFO] 2021-02-23 14:11:10 > Step:   96000 Progress: 48.0% Elapsed: 2 days, 16:55:40 temperature: 1.0 ada_p: No Discriminator_loss: 4.8379 Generator_loss: 6.32044 
    [INFO] 2021-02-23 14:15:03 > Step:   96100 Progress: 48.0% Elapsed: 2 days, 16:59:33 temperature: 1.0 ada_p: No Discriminator_loss: 4.87194 Generator_loss: 6.14097 
    [INFO] 2021-02-23 14:18:54 > Step:   96200 Progress: 48.1% Elapsed: 2 days, 17:03:24 temperature: 1.0 ada_p: No Discriminator_loss: 4.8922 Generator_loss: 6.09684 
    [INFO] 2021-02-23 14:22:46 > Step:   96300 Progress: 48.1% Elapsed: 2 days, 17:07:16 temperature: 1.0 ada_p: No Discriminator_loss: 4.89233 Generator_loss: 6.01682 
    [INFO] 2021-02-23 14:26:38 > Step:   96400 Progress: 48.2% Elapsed: 2 days, 17:11:08 temperature: 1.0 ada_p: No Discriminator_loss: 4.84357 Generator_loss: 6.09488 
    [INFO] 2021-02-23 14:30:31 > Step:   96500 Progress: 48.2% Elapsed: 2 days, 17:15:01 temperature: 1.0 ada_p: No Discriminator_loss: 4.87313 Generator_loss: 6.1209 
    [INFO] 2021-02-23 14:34:23 > Step:   96600 Progress: 48.3% Elapsed: 2 days, 17:18:53 temperature: 1.0 ada_p: No Discriminator_loss: 4.90247 Generator_loss: 6.02566 
    [INFO] 2021-02-23 14:38:16 > Step:   96700 Progress: 48.4% Elapsed: 2 days, 17:22:46 temperature: 1.0 ada_p: No Discriminator_loss: 4.86648 Generator_loss: 6.15333 
    [INFO] 2021-02-23 14:42:09 > Step:   96800 Progress: 48.4% Elapsed: 2 days, 17:26:39 temperature: 1.0 ada_p: No Discriminator_loss: 4.83088 Generator_loss: 6.18407 
    [INFO] 2021-02-23 14:46:01 > Step:   96900 Progress: 48.4% Elapsed: 2 days, 17:30:31 temperature: 1.0 ada_p: No Discriminator_loss: 4.86981 Generator_loss: 6.18506 
    [INFO] 2021-02-23 14:49:53 > Step:   97000 Progress: 48.5% Elapsed: 2 days, 17:34:23 temperature: 1.0 ada_p: No Discriminator_loss: 4.93113 Generator_loss: 6.2457 
    [INFO] 2021-02-23 14:53:45 > Step:   97100 Progress: 48.5% Elapsed: 2 days, 17:38:15 temperature: 1.0 ada_p: No Discriminator_loss: 4.85358 Generator_loss: 6.14057 
    [INFO] 2021-02-23 14:57:37 > Step:   97200 Progress: 48.6% Elapsed: 2 days, 17:42:07 temperature: 1.0 ada_p: No Discriminator_loss: 4.86477 Generator_loss: 6.19888 
    [INFO] 2021-02-23 15:01:29 > Step:   97300 Progress: 48.6% Elapsed: 2 days, 17:45:59 temperature: 1.0 ada_p: No Discriminator_loss: 4.85981 Generator_loss: 6.10489 
    [INFO] 2021-02-23 15:05:21 > Step:   97400 Progress: 48.7% Elapsed: 2 days, 17:49:51 temperature: 1.0 ada_p: No Discriminator_loss: 4.90024 Generator_loss: 6.14645 
    [INFO] 2021-02-23 15:09:13 > Step:   97500 Progress: 48.8% Elapsed: 2 days, 17:53:43 temperature: 1.0 ada_p: No Discriminator_loss: 4.88928 Generator_loss: 6.12331 
    [INFO] 2021-02-23 15:13:09 > Step:   97600 Progress: 48.8% Elapsed: 2 days, 17:57:39 temperature: 1.0 ada_p: No Discriminator_loss: 4.97929 Generator_loss: 6.15232 
    [INFO] 2021-02-23 15:17:00 > Step:   97700 Progress: 48.9% Elapsed: 2 days, 18:01:30 temperature: 1.0 ada_p: No Discriminator_loss: 4.89975 Generator_loss: 6.07438 
    [INFO] 2021-02-23 15:20:51 > Step:   97800 Progress: 48.9% Elapsed: 2 days, 18:05:21 temperature: 1.0 ada_p: No Discriminator_loss: 4.91863 Generator_loss: 6.25845 
    [INFO] 2021-02-23 15:24:42 > Step:   97900 Progress: 48.9% Elapsed: 2 days, 18:09:12 temperature: 1.0 ada_p: No Discriminator_loss: 5.04714 Generator_loss: 5.96617 
    [INFO] 2021-02-23 15:28:33 > Step:   98000 Progress: 49.0% Elapsed: 2 days, 18:13:03 temperature: 1.0 ada_p: No Discriminator_loss: 4.94111 Generator_loss: 6.17885 
    [INFO] 2021-02-23 15:32:25 > Step:   98100 Progress: 49.0% Elapsed: 2 days, 18:16:55 temperature: 1.0 ada_p: No Discriminator_loss: 4.83203 Generator_loss: 6.31808 
    [INFO] 2021-02-23 15:36:17 > Step:   98200 Progress: 49.1% Elapsed: 2 days, 18:20:47 temperature: 1.0 ada_p: No Discriminator_loss: 4.91035 Generator_loss: 6.26432 
    [INFO] 2021-02-23 15:40:10 > Step:   98300 Progress: 49.1% Elapsed: 2 days, 18:24:39 temperature: 1.0 ada_p: No Discriminator_loss: 4.9617 Generator_loss: 6.28282 
    [INFO] 2021-02-23 15:44:01 > Step:   98400 Progress: 49.2% Elapsed: 2 days, 18:28:31 temperature: 1.0 ada_p: No Discriminator_loss: 4.88654 Generator_loss: 6.26701 
    [INFO] 2021-02-23 15:47:52 > Step:   98500 Progress: 49.2% Elapsed: 2 days, 18:32:22 temperature: 1.0 ada_p: No Discriminator_loss: 4.83235 Generator_loss: 6.18709 
    [INFO] 2021-02-23 15:51:43 > Step:   98600 Progress: 49.3% Elapsed: 2 days, 18:36:13 temperature: 1.0 ada_p: No Discriminator_loss: 4.86933 Generator_loss: 6.3286 
    [INFO] 2021-02-23 15:55:34 > Step:   98700 Progress: 49.4% Elapsed: 2 days, 18:40:04 temperature: 1.0 ada_p: No Discriminator_loss: 4.84194 Generator_loss: 6.069 
    [INFO] 2021-02-23 15:59:26 > Step:   98800 Progress: 49.4% Elapsed: 2 days, 18:43:56 temperature: 1.0 ada_p: No Discriminator_loss: 4.89277 Generator_loss: 6.1405 
    [INFO] 2021-02-23 16:03:19 > Step:   98900 Progress: 49.5% Elapsed: 2 days, 18:47:49 temperature: 1.0 ada_p: No Discriminator_loss: 4.86988 Generator_loss: 6.1396 
    [INFO] 2021-02-23 16:07:11 > Step:   99000 Progress: 49.5% Elapsed: 2 days, 18:51:41 temperature: 1.0 ada_p: No Discriminator_loss: 4.85899 Generator_loss: 6.23855 
    [INFO] 2021-02-23 16:11:04 > Step:   99100 Progress: 49.5% Elapsed: 2 days, 18:55:34 temperature: 1.0 ada_p: No Discriminator_loss: 4.89566 Generator_loss: 6.32946 
    [INFO] 2021-02-23 16:14:57 > Step:   99200 Progress: 49.6% Elapsed: 2 days, 18:59:27 temperature: 1.0 ada_p: No Discriminator_loss: 4.84965 Generator_loss: 6.29854 
    [INFO] 2021-02-23 16:18:50 > Step:   99300 Progress: 49.6% Elapsed: 2 days, 19:03:20 temperature: 1.0 ada_p: No Discriminator_loss: 4.89316 Generator_loss: 6.09008 
    [INFO] 2021-02-23 16:22:44 > Step:   99400 Progress: 49.7% Elapsed: 2 days, 19:07:14 temperature: 1.0 ada_p: No Discriminator_loss: 4.83282 Generator_loss: 6.13388 
    [INFO] 2021-02-23 16:26:36 > Step:   99500 Progress: 49.8% Elapsed: 2 days, 19:11:06 temperature: 1.0 ada_p: No Discriminator_loss: 4.87184 Generator_loss: 6.20809 
    [INFO] 2021-02-23 16:30:28 > Step:   99600 Progress: 49.8% Elapsed: 2 days, 19:14:58 temperature: 1.0 ada_p: No Discriminator_loss: 4.86616 Generator_loss: 6.17352 
    [INFO] 2021-02-23 16:34:20 > Step:   99700 Progress: 49.9% Elapsed: 2 days, 19:18:50 temperature: 1.0 ada_p: No Discriminator_loss: 4.91773 Generator_loss: 6.1536 
    [INFO] 2021-02-23 16:38:13 > Step:   99800 Progress: 49.9% Elapsed: 2 days, 19:22:42 temperature: 1.0 ada_p: No Discriminator_loss: 4.87546 Generator_loss: 6.25496 
    [INFO] 2021-02-23 16:42:05 > Step:   99900 Progress: 50.0% Elapsed: 2 days, 19:26:35 temperature: 1.0 ada_p: No Discriminator_loss: 4.93619 Generator_loss: 5.91653 
    [INFO] 2021-02-23 16:45:58 > Step:  100000 Progress: 50.0% Elapsed: 2 days, 19:30:28 temperature: 1.0 ada_p: No Discriminator_loss: 4.88613 Generator_loss: 6.24134 
    [INFO] 2021-02-23 16:45:58 > Start Evaluation (100000 Step): ContraGAN256-train-2021_02_20_21_08_34
    [INFO] 2021-02-23 16:45:58 > Start Evaluation FID score ... 
    [INFO] 2021-02-23 16:45:58 > Calculating FID Score....
    [INFO] 2021-02-23 16:47:18 > Start Evaluation IS score ... 
    [INFO] 2021-02-23 16:47:18 > Calculating Inception Score....
    [INFO] 2021-02-23 16:48:21 > Start Evaluation F Beta score ... 
    [INFO] 2021-02-23 16:48:21 > Calculate F_beta Score....
    [INFO] 2021-02-23 17:02:10 > Calculate Accuracies....
    [INFO] 2021-02-23 17:03:30 > Calculate Accuracies....
    [INFO] 2021-02-23 17:09:58 > F_8 score (Step: 100000, Using valid images): 0.5647498118120066
    [INFO] 2021-02-23 17:09:58 > F_1/8 score (Step: 100000, Using valid images): 0.41189299143070984
    [INFO] 2021-02-23 17:09:58 > FID score (Step: 100000, Using valid moments): 106.83836304696894
    [INFO] 2021-02-23 17:09:58 > Inception score (Step: 100000, 50000 generated images): 12.470245361328125
    [INFO] 2021-02-23 17:09:58 > Best FID score (Step: 60000, Using valid moments): 60.38544475604181
    [INFO] 2021-02-23 17:10:01 > Save model to checkpoints/ContraGAN256-train-2021_02_20_21_08_34
    [INFO] 2021-02-23 17:13:56 > Step:  100100 Progress: 50.0% Elapsed: 2 days, 19:58:25 temperature: 1.0 ada_p: No Discriminator_loss: 4.87462 Generator_loss: 6.22005 
    [INFO] 2021-02-23 17:17:47 > Step:  100200 Progress: 50.1% Elapsed: 2 days, 20:02:17 temperature: 1.0 ada_p: No Discriminator_loss: 4.86102 Generator_loss: 6.13267 
    [INFO] 2021-02-23 17:21:39 > Step:  100300 Progress: 50.1% Elapsed: 2 days, 20:06:09 temperature: 1.0 ada_p: No Discriminator_loss: 4.87268 Generator_loss: 6.22166 
    [INFO] 2021-02-23 17:25:31 > Step:  100400 Progress: 50.2% Elapsed: 2 days, 20:10:01 temperature: 1.0 ada_p: No Discriminator_loss: 4.8299 Generator_loss: 6.1568 
    [INFO] 2021-02-23 17:29:23 > Step:  100500 Progress: 50.2% Elapsed: 2 days, 20:13:53 temperature: 1.0 ada_p: No Discriminator_loss: 4.87357 Generator_loss: 6.29569 
    [INFO] 2021-02-23 17:33:15 > Step:  100600 Progress: 50.3% Elapsed: 2 days, 20:17:45 temperature: 1.0 ada_p: No Discriminator_loss: 4.81752 Generator_loss: 6.24262 
    [INFO] 2021-02-23 17:37:07 > Step:  100700 Progress: 50.3% Elapsed: 2 days, 20:21:37 temperature: 1.0 ada_p: No Discriminator_loss: 4.91245 Generator_loss: 6.25011 
    [INFO] 2021-02-23 17:40:59 > Step:  100800 Progress: 50.4% Elapsed: 2 days, 20:25:29 temperature: 1.0 ada_p: No Discriminator_loss: 4.8524 Generator_loss: 6.1212 
    [INFO] 2021-02-23 17:44:52 > Step:  100900 Progress: 50.4% Elapsed: 2 days, 20:29:22 temperature: 1.0 ada_p: No Discriminator_loss: 4.84331 Generator_loss: 6.15091 
    [INFO] 2021-02-23 17:48:43 > Step:  101000 Progress: 50.5% Elapsed: 2 days, 20:33:13 temperature: 1.0 ada_p: No Discriminator_loss: 4.85256 Generator_loss: 6.31234 
    [INFO] 2021-02-23 17:52:33 > Step:  101100 Progress: 50.5% Elapsed: 2 days, 20:37:03 temperature: 1.0 ada_p: No Discriminator_loss: 5.00685 Generator_loss: 5.99287 
    [INFO] 2021-02-23 17:56:24 > Step:  101200 Progress: 50.6% Elapsed: 2 days, 20:40:54 temperature: 1.0 ada_p: No Discriminator_loss: 4.93036 Generator_loss: 6.1156 
    [INFO] 2021-02-23 18:00:17 > Step:  101300 Progress: 50.6% Elapsed: 2 days, 20:44:47 temperature: 1.0 ada_p: No Discriminator_loss: 4.84466 Generator_loss: 6.19562 
    [INFO] 2021-02-23 18:04:09 > Step:  101400 Progress: 50.7% Elapsed: 2 days, 20:48:39 temperature: 1.0 ada_p: No Discriminator_loss: 4.86936 Generator_loss: 6.19606 
    [INFO] 2021-02-23 18:08:01 > Step:  101500 Progress: 50.7% Elapsed: 2 days, 20:52:31 temperature: 1.0 ada_p: No Discriminator_loss: 4.83002 Generator_loss: 6.10993 
    [INFO] 2021-02-23 18:11:53 > Step:  101600 Progress: 50.8% Elapsed: 2 days, 20:56:23 temperature: 1.0 ada_p: No Discriminator_loss: 4.9262 Generator_loss: 6.13906 
    [INFO] 2021-02-23 18:15:46 > Step:  101700 Progress: 50.8% Elapsed: 2 days, 21:00:16 temperature: 1.0 ada_p: No Discriminator_loss: 5.05316 Generator_loss: 6.20794 
    [INFO] 2021-02-23 18:19:38 > Step:  101800 Progress: 50.9% Elapsed: 2 days, 21:04:08 temperature: 1.0 ada_p: No Discriminator_loss: 4.83387 Generator_loss: 6.19561 
    [INFO] 2021-02-23 18:23:30 > Step:  101900 Progress: 50.9% Elapsed: 2 days, 21:08:00 temperature: 1.0 ada_p: No Discriminator_loss: 4.84121 Generator_loss: 6.04899 
    [INFO] 2021-02-23 18:27:43 > Step:  102000 Progress: 51.0% Elapsed: 2 days, 21:12:13 temperature: 1.0 ada_p: No Discriminator_loss: 4.85181 Generator_loss: 6.14338 
    [INFO] 2021-02-23 18:31:34 > Step:  102100 Progress: 51.0% Elapsed: 2 days, 21:16:04 temperature: 1.0 ada_p: No Discriminator_loss: 4.98565 Generator_loss: 6.18699 
    [INFO] 2021-02-23 18:35:25 > Step:  102200 Progress: 51.1% Elapsed: 2 days, 21:19:55 temperature: 1.0 ada_p: No Discriminator_loss: 4.83879 Generator_loss: 6.28453 
    [INFO] 2021-02-23 18:39:18 > Step:  102300 Progress: 51.1% Elapsed: 2 days, 21:23:47 temperature: 1.0 ada_p: No Discriminator_loss: 4.89303 Generator_loss: 6.16461 
    [INFO] 2021-02-23 18:43:11 > Step:  102400 Progress: 51.2% Elapsed: 2 days, 21:27:41 temperature: 1.0 ada_p: No Discriminator_loss: 4.90148 Generator_loss: 6.36757 
    [INFO] 2021-02-23 18:47:03 > Step:  102500 Progress: 51.2% Elapsed: 2 days, 21:31:33 temperature: 1.0 ada_p: No Discriminator_loss: 4.87893 Generator_loss: 6.24281 
    [INFO] 2021-02-23 18:51:00 > Step:  102600 Progress: 51.3% Elapsed: 2 days, 21:35:30 temperature: 1.0 ada_p: No Discriminator_loss: 4.82468 Generator_loss: 6.12198 
    [INFO] 2021-02-23 18:54:52 > Step:  102700 Progress: 51.3% Elapsed: 2 days, 21:39:22 temperature: 1.0 ada_p: No Discriminator_loss: 4.82019 Generator_loss: 6.06188 
    [INFO] 2021-02-23 18:58:45 > Step:  102800 Progress: 51.4% Elapsed: 2 days, 21:43:15 temperature: 1.0 ada_p: No Discriminator_loss: 4.87603 Generator_loss: 6.1209 
    [INFO] 2021-02-23 19:02:37 > Step:  102900 Progress: 51.4% Elapsed: 2 days, 21:47:07 temperature: 1.0 ada_p: No Discriminator_loss: 4.8846 Generator_loss: 6.03209 
    [INFO] 2021-02-23 19:06:29 > Step:  103000 Progress: 51.5% Elapsed: 2 days, 21:50:59 temperature: 1.0 ada_p: No Discriminator_loss: 4.8987 Generator_loss: 6.15334 
    [INFO] 2021-02-23 19:10:21 > Step:  103100 Progress: 51.5% Elapsed: 2 days, 21:54:50 temperature: 1.0 ada_p: No Discriminator_loss: 4.84103 Generator_loss: 5.97288 
    [INFO] 2021-02-23 19:14:11 > Step:  103200 Progress: 51.6% Elapsed: 2 days, 21:58:41 temperature: 1.0 ada_p: No Discriminator_loss: 4.8428 Generator_loss: 6.17706 
    [INFO] 2021-02-23 19:18:00 > Step:  103300 Progress: 51.6% Elapsed: 2 days, 22:02:30 temperature: 1.0 ada_p: No Discriminator_loss: 4.98181 Generator_loss: 6.26323 
    [INFO] 2021-02-23 19:21:52 > Step:  103400 Progress: 51.7% Elapsed: 2 days, 22:06:21 temperature: 1.0 ada_p: No Discriminator_loss: 4.81991 Generator_loss: 6.08122 
    [INFO] 2021-02-23 19:25:43 > Step:  103500 Progress: 51.7% Elapsed: 2 days, 22:10:13 temperature: 1.0 ada_p: No Discriminator_loss: 5.02066 Generator_loss: 6.00079 
    [INFO] 2021-02-23 19:29:35 > Step:  103600 Progress: 51.8% Elapsed: 2 days, 22:14:05 temperature: 1.0 ada_p: No Discriminator_loss: 4.84939 Generator_loss: 6.17011 
    [INFO] 2021-02-23 19:33:28 > Step:  103700 Progress: 51.8% Elapsed: 2 days, 22:17:58 temperature: 1.0 ada_p: No Discriminator_loss: 5.3072 Generator_loss: 5.97429 
    [INFO] 2021-02-23 19:37:20 > Step:  103800 Progress: 51.9% Elapsed: 2 days, 22:21:50 temperature: 1.0 ada_p: No Discriminator_loss: 4.93098 Generator_loss: 5.99354 
    [INFO] 2021-02-23 19:41:13 > Step:  103900 Progress: 51.9% Elapsed: 2 days, 22:25:43 temperature: 1.0 ada_p: No Discriminator_loss: 4.84383 Generator_loss: 6.11023 
    [INFO] 2021-02-23 19:45:05 > Step:  104000 Progress: 52.0% Elapsed: 2 days, 22:29:35 temperature: 1.0 ada_p: No Discriminator_loss: 4.90621 Generator_loss: 6.17702 
    [INFO] 2021-02-23 19:48:57 > Step:  104100 Progress: 52.0% Elapsed: 2 days, 22:33:27 temperature: 1.0 ada_p: No Discriminator_loss: 4.8937 Generator_loss: 6.25937 
    [INFO] 2021-02-23 19:52:50 > Step:  104200 Progress: 52.1% Elapsed: 2 days, 22:37:20 temperature: 1.0 ada_p: No Discriminator_loss: 4.83056 Generator_loss: 6.15484 
    [INFO] 2021-02-23 19:56:42 > Step:  104300 Progress: 52.1% Elapsed: 2 days, 22:41:12 temperature: 1.0 ada_p: No Discriminator_loss: 4.92858 Generator_loss: 5.97964 
    [INFO] 2021-02-23 20:00:32 > Step:  104400 Progress: 52.2% Elapsed: 2 days, 22:45:02 temperature: 1.0 ada_p: No Discriminator_loss: 4.88575 Generator_loss: 6.19241 
    [INFO] 2021-02-23 20:04:24 > Step:  104500 Progress: 52.2% Elapsed: 2 days, 22:48:54 temperature: 1.0 ada_p: No Discriminator_loss: 4.88334 Generator_loss: 6.29097 
    [INFO] 2021-02-23 20:08:16 > Step:  104600 Progress: 52.3% Elapsed: 2 days, 22:52:46 temperature: 1.0 ada_p: No Discriminator_loss: 4.92377 Generator_loss: 6.23504 
    
    
    </details>
    
    opened by youjiangxu 6
  • Failed in installing environments

    Failed in installing environments

    截屏2021-09-21 下午12 38 01

    My system is Ubuntu20.10, and I have installed Nvidia-docker2, but when I use docker to run the code, it shows numpy haven't been installed yet.

    And I have tried to install by Conda, it failed too.

    截屏2021-09-21 下午1 33 45

    Could you show me how to fix it? I will appreciate it. :)

    opened by uniartisan 5
  • ContraGAN trained on Cifar10

    ContraGAN trained on Cifar10

    the trained ContraGAN on CIFAR10 seems to permute the classes. the order of CIFAR10 classes is: airplane  0      cars          1     
    birds        2     
    cats          3     
    deer        4     
    dog          5     
    frog          6     
    horse      7        ship        8        truck.    9   

    but the generated classes using ContraGAN are: airplane 0 cars 1 deer 2 bird 3 horse 4 frog 5 truck 6 dog 7 ship 8 cats     9

    any suggestions? I used the default setting to train ContraGAN on Cifar10.

    opened by mitralz 2
  • RUN THE CODE

    RUN THE CODE

    has anyone run the code successfully? i cannot set the right environment

    opened by wwjwy 7
  • How can I get the same result as you?

    How can I get the same result as you?

    Hi,

    I run the ContraGAN on CIFAR10 using the config file you provided and without any modification. However, I got a worse result with 10.435 FID while your result is 8.065. I wonder what's the different between us. In addition, you get a much better result than the figure reported in the paper(10.597±0.273), how did you do that?

    If anyone faces the same issue, please tell me. Thank you!

    opened by DylanTao94 6
  • How to generate images?

    How to generate images?

    Hello :)

    Am I missing the function that generates images, after the model training is completed? If not, how do I approach it best?

    opened by ChristofJugel 1
  • Modifications in ACGAN

    Modifications in ACGAN

    Hi, Thanks for your awesome work!

    Can you please specify the modifications which have been made in ACGAN-Mod which increase the performance of ACGAN? Also can you please let us know how much is improvement that you get with those modifications?

    Thanks Harsh

    opened by rangwani-harsh 1
  • About DCGAN Loss

    About DCGAN Loss

    Hi,

    It's a nice and helpful work, implementing a collections of useful tricks of GAN. I feel confused about the implementation of loss_dcgan in src/utils./losses.py. Is the loss_dcgan just a reproduction of BCELoss? If so, I think dis_loss = -torch.mean(nn.LogSigmoid()(dis_out_real) + (ones - dis_out_fake.sigmoid()).log() may be more reasonable, or a safer one dis_loss = -torch.mean(nn.LogSigmoid()(dis_out_real) + (ones - dis_out_fake.sigmoid() + 1e-6).log()

    Best, MTandHJ

    opened by MTandHJ 1
  • Plan to add StyleGAN2 baseline for experiments

    Plan to add StyleGAN2 baseline for experiments

    Hi Minguk,

    Any plans for implementing the StyleGAN2 generator architecture in the library? Are there some issues which are making it hard to implement in this framework?

    Thanks for the great library.

    opened by rangwani-harsh 1
Releases(v0.2.0)
  • v0.2.0(Feb 23, 2021)

    Second release of StudioGAN with following features

    • Fix minor bugs (slow convergence of training GAN + ADA models, tracking bn statistics during evaluation, etc.)
    • Add multi-node DistributedDataParallel (DDP) training.
    • Comprehensive benchmarks on CIFAR10, Tiny_ImageNet, and ImageNet datasets.
    • Provide pre-trained models and log files for the future research.
    • Add LARS optimizer and TSNE analysis.
    Source code(tar.gz)
    Source code(zip)
  • v0.1.0(Dec 7, 2020)

    First StudioGAN release with following features

    • Extensive GAN implementations for Pytorch: From DCGAN to ADAGAN
    • Comprehensive benchmark of GANs using CIFAR10 dataset
    • Better performance and lower memory consumption than original implementations
    • Providing pre-trained models that are fully compatible with up-to-date PyTorch environment
    • Support Multi-GPU(both DP and DDP), Mixed precision, Synchronized Batch Normalization, and Tensorboard Visualization
    Source code(tar.gz)
    Source code(zip)
Collection of generative models in Pytorch version.

pytorch-generative-model-collections Original : [Tensorflow version] Pytorch implementation of various GANs. This repository was re-implemented with r

Hyeonwoo Kang 2.3k Oct 18, 2021
Annotated, understandable, and visually interpretable PyTorch implementations of: VAE, BIRVAE, NSGAN, MMGAN, WGAN, WGANGP, LSGAN, DRAGAN, BEGAN, RaGAN, InfoGAN, fGAN, FisherGAN

Overview PyTorch 0.4.1 | Python 3.6.5 Annotated implementations with comparative introductions for minimax, non-saturating, wasserstein, wasserstein g

Shayne O'Brien 454 Oct 4, 2021
The Incredible PyTorch: a curated list of tutorials, papers, projects, communities and more relating to PyTorch.

This is a curated list of tutorials, projects, libraries, videos, papers, books and anything related to the incredible PyTorch. Feel free to make a pu

Ritchie Ng 8.5k Oct 22, 2021
A collection of resources on GAN Inversion.

This repo is a collection of resources on GAN inversion, as a supplement for our survey

null 357 Oct 16, 2021
A comprehensive list of published machine learning applications to cosmology

ml-in-cosmology This github attempts to maintain a comprehensive list of published machine learning applications to cosmology, organized by subject ma

George Stein 211 Oct 15, 2021
Finding an Unsupervised Image Segmenter in each of your Deep Generative Models

Finding an Unsupervised Image Segmenter in each of your Deep Generative Models Description Recent research has shown that numerous human-interpretable

Luke Melas-Kyriazi 40 Oct 4, 2021
Adversarial Robustness Toolbox (ART) - Python Library for Machine Learning Security - Evasion, Poisoning, Extraction, Inference - Red and Blue Teams

Adversarial Robustness Toolbox (ART) is a Python library for Machine Learning Security. ART provides tools that enable developers and researchers to defend and evaluate Machine Learning models and applications against the adversarial threats of Evasion, Poisoning, Extraction, and Inference. ART supports all popular machine learning frameworks (TensorFlow, Keras, PyTorch, MXNet, scikit-learn, XGBoost, LightGBM, CatBoost, GPy, etc.), all data types (images, tables, audio, video, etc.) and machine learning tasks (classification, object detection, speech recognition, generation, certification, etc.).

null 2.5k Oct 22, 2021
Official repository for "On Generating Transferable Targeted Perturbations" (ICCV 2021)

On Generating Transferable Targeted Perturbations (ICCV'21) Muzammal Naseer, Salman Khan, Munawar Hayat, Fahad Shahbaz Khan, and Fatih Porikli Paper:

Muzammal Naseer 29 Sep 15, 2021
Based on the paper "Geometry-aware Instance-reweighted Adversarial Training" ICLR 2021 oral

Geometry-aware Instance-reweighted Adversarial Training This repository provides codes for Geometry-aware Instance-reweighted Adversarial Training (ht

Jingfeng 34 Sep 8, 2021
Code for ICE-BeeM paper - NeurIPS 2020

ICE-BeeM: Identifiable Conditional Energy-Based Deep Models Based on Nonlinear ICA This repository contains code to run and reproduce the experiments

Ilyes Khemakhem 41 Oct 13, 2021
Advanced Deep Learning with TensorFlow 2 and Keras (Updated for 2nd Edition)

Advanced Deep Learning with TensorFlow 2 and Keras (Updated for 2nd Edition)

Packt 1.1k Oct 21, 2021
Alias-Free Generative Adversarial Networks (StyleGAN3) Official PyTorch implementation

Alias-Free Generative Adversarial Networks (StyleGAN3) Official PyTorch implementation

NVIDIA Research Projects 2.6k Oct 22, 2021
GNN4Traffic - This is the repository for the collection of Graph Neural Network for Traffic Forecasting

GNN4Traffic - This is the repository for the collection of Graph Neural Network for Traffic Forecasting

null 215 Oct 18, 2021
StyleGAN2-ADA - Official PyTorch implementation

Abstract: Training generative adversarial networks (GAN) using too little data typically leads to discriminator overfitting, causing training to diverge. We propose an adaptive discriminator augmentation mechanism that significantly stabilizes training in limited data regimes.

NVIDIA Research Projects 2k Oct 22, 2021
StyleGAN2-ADA - Official PyTorch implementation

Need Help? If you’re new to StyleGAN2-ADA and looking to get started, please check out this video series from a course Lia Coleman and I taught in Oct

Derrick Schultz 126 Oct 22, 2021
Fast and Easy Infinite Neural Networks in Python

Neural Tangents ICLR 2020 Video | Paper | Quickstart | Install guide | Reference docs | Release notes Overview Neural Tangents is a high-level neural

Google 1.6k Oct 24, 2021
Image-to-Image Translation in PyTorch

CycleGAN and pix2pix in PyTorch New: Please check out contrastive-unpaired-translation (CUT), our new unpaired image-to-image translation model that e

Jun-Yan Zhu 16k Oct 23, 2021
Code for the paper: Adversarial Training Against Location-Optimized Adversarial Patches. ECCV-W 2020.

Adversarial Training Against Location-Optimized Adversarial Patches arXiv | Paper | Code | Video | Slides Code for the paper: Sukrut Rao, David Stutz,

Sukrut Rao 19 Sep 23, 2021
[CVPR 2020] Interpreting the Latent Space of GANs for Semantic Face Editing

InterFaceGAN - Interpreting the Latent Space of GANs for Semantic Face Editing Figure: High-quality facial attributes editing results with InterFaceGA

GenForce: May Generative Force Be with You 1k Oct 20, 2021