Dear author: Thanks for your job. In your model_search cifa and 256 size code ,I do not find where the patchsize use in the Generator but only in the Discriminator. Can you tell me where it is in your code? Thank you very much.

I've tried using the function train in functions.py, and training seems to fail:

  File "functional.py", line 86, in adam
    exp_avg.mul_(beta1).add_(grad, alpha=1 - beta1)
RuntimeError: output with shape [768] doesn't match the broadcast shape [3, 48, 1, 768]
``

excuse me, sorry to disturb you. When I run the train_derived.py, the terminal appeared the "UnboundLocalError: local variable 'gen_net' referenced before assignment". The sentences of if , elif and else are used too much, I think. How can I fix it. Thanks.

Hello, I was wondering if you had available anywhere some samples of the outputs of the GAN a different stages of training: like before training, after N epochs, etc... To get an idea of how the model is reaching its goal, and what to expect while training to see if I'm way out of the path. I have been trying to make something very similar with limited success, and always have to fall back to some convolutional layers (like putting a Conv2D after every attention layer) to get any relevant results...

Thanks!

Hi, thanks for the work. Follow the training code in functions.py, it seems that you did not freeze D when training G. When running dis_optimizer.step(), the gradient from G training will also be used to update D's parameters. So I wonder whether if I missed something, or it was a bug here. Thanks a lot!

As I understand, your paper wants to completely remove convolution layers. But in code, for the linear unflatten layer (to obtain RGB image), I see you use conv2d. Why do you use this? Is there anyway that we can get RGB image without conv2d?

Thanks for your work！

When I'm looking at the code of model/Celeba64_TransGAN.py , I notice that the function “get_attn_mask” seems to play a role in the training process.Can you point out the specific role of this function?

Thank you~~

Before iteration , i meet follow error,what should i do

and the cfg.py

import argparse

def str2bool(v): if v.lower() in ('yes', 'true', 't', 'y', '1'): return True elif v.lower() in ('no', 'false', 'f', 'n', '0'): return False else: raise argparse.ArgumentTypeError('Boolean value expected.')

def parse_args(): parser = argparse.ArgumentParser() parser.add_argument('--world-size', default=-1, type=int, help='number of nodes for distributed training') parser.add_argument('--rank', default=-1, type=int, help='node rank for distributed training') parser.add_argument('--loca_rank', default=-1, type=int, help='node rank for distributed training') parser.add_argument('--dist-url', default='tcp://224.66.41.62:23456', type=str, help='url used to set up distributed training') parser.add_argument('--dist-backend', default='nccl', type=str, help='distributed backend') parser.add_argument('--seed', default=12345, type=int, help='seed for initializing training. ') parser.add_argument('--gpu', default=0, type=int, help='GPU id to use.') parser.add_argument('--multiprocessing-distributed', action='store_true', help='Use multi-processing distributed training to launch ' 'N processes per node, which has N GPUs. This is the ' 'fastest way to use PyTorch for either single node or ' 'multi node data parallel training') parser.add_argument( '--max_epoch', type=int, default=200, help='number of epochs of training') parser.add_argument( '--max_iter', type=int, default=10, help='set the max iteration number') parser.add_argument( '-gen_bs', '--gen_batch_size', type=int, default=4, help='size of the batches') parser.add_argument( '-dis_bs', '--dis_batch_size', type=int, default=4, help='size of the batches') parser.add_argument( '--g_lr', type=float, default=0.0002, help='adam: gen learning rate') parser.add_argument( '--wd', type=float, default=0, help='adamw: gen weight decay') parser.add_argument( '--d_lr', type=float, default=0.0002, help='adam: disc learning rate') parser.add_argument( '--ctrl_lr', type=float, default=3.5e-4, help='adam: ctrl learning rate') parser.add_argument( '--lr_decay', action='store_true', help='learning rate decay or not') parser.add_argument( '--beta1', type=float, default=0.0, help='adam: decay of first order momentum of gradient') parser.add_argument( '--beta2', type=float, default=0.9, help='adam: decay of first order momentum of gradient') parser.add_argument( '--num_workers', type=int, default=0, help='number of cpu threads to use during batch generation') parser.add_argument( '--latent_dim', type=int, default=128, help='dimensionality of the latent space') parser.add_argument( '--img_size', type=int, default=256, help='size of each image dimension') parser.add_argument( '--channels', type=int, default=3, help='number of image channels') parser.add_argument( '--n_critic', type=int, default=1, help='number of training steps for discriminator per iter') parser.add_argument( '--val_freq', type=int, default=20, help='interval between each validation') parser.add_argument( '--print_freq', type=int, default=100, help='interval between each verbose') parser.add_argument( '--load_path', type=str, help='The reload model path') parser.add_argument( '--exp_name', type=str, default='Test', help='The name of exp') parser.add_argument( '--d_spectral_norm', type=str2bool, default=False, help='add spectral_norm on discriminator?') parser.add_argument( '--g_spectral_norm', type=str2bool, default=False, help='add spectral_norm on generator?') parser.add_argument( '--dataset', type=str, default='cifar10', help='dataset type') parser.add_argument( '--data_path', type=str, default='./data', help='The path of data set') parser.add_argument('--init_type', type=str, default='normal', choices=['normal', 'orth', 'xavier_uniform', 'false'], help='The init type') parser.add_argument('--gf_dim', type=int, default=64, help='The base channel num of gen') parser.add_argument('--df_dim', type=int, default=64, help='The base channel num of disc') parser.add_argument( '--gen_model', type=str, default='ViT_custom_rp', help='path of gen model') parser.add_argument( '--dis_model', type=str, default='ViT_custom_rp', help='path of dis model') parser.add_argument( '--controller', type=str, default='controller', help='path of controller') parser.add_argument('--eval_batch_size', type=int, default=100) parser.add_argument('--num_eval_imgs', type=int, default=50000) parser.add_argument( '--bottom_width', type=int, default=4, help="the base resolution of the GAN") parser.add_argument('--random_seed', type=int, default=12345)

# search
parser.add_argument('--shared_epoch', type=int, default=15,
                    help='the number of epoch to train the shared gan at each search iteration')
parser.add_argument('--grow_step1', type=int, default=25,
                    help='which iteration to grow the image size from 8 to 16')
parser.add_argument('--grow_step2', type=int, default=55,
                    help='which iteration to grow the image size from 16 to 32')
parser.add_argument('--max_search_iter', type=int, default=90,
                    help='max search iterations of this algorithm')
parser.add_argument('--ctrl_step', type=int, default=30,
                    help='number of steps to train the controller at each search iteration')
parser.add_argument('--ctrl_sample_batch', type=int, default=1,
                    help='sample size of controller of each step')
parser.add_argument('--hid_size', type=int, default=100,
                    help='the size of hidden vector')
parser.add_argument('--baseline_decay', type=float, default=0.9,
                    help='baseline decay rate in RL')
parser.add_argument('--rl_num_eval_img', type=int, default=5000,
                    help='number of images to be sampled in order to get the reward')
parser.add_argument('--num_candidate', type=int, default=10,
                    help='number of candidate architectures to be sampled')
parser.add_argument('--topk', type=int, default=5,
                    help='preserve topk models architectures after each stage' )
parser.add_argument('--entropy_coeff', type=float, default=1e-3,
                    help='to encourage the exploration')
parser.add_argument('--dynamic_reset_threshold', type=float, default=1e-3,
                    help='var threshold')
parser.add_argument('--dynamic_reset_window', type=int, default=500,
                    help='the window size')
parser.add_argument('--arch', nargs='+', type=int,
                    help='the vector of a discovered architecture')
parser.add_argument('--optimizer', type=str, default="adam",
                    help='optimizer')
parser.add_argument('--loss', type=str, default="hinge",
                    help='loss function')
parser.add_argument('--n_classes', type=int, default=0,
                    help='classes')
parser.add_argument('--phi', type=float, default=1,
                    help='wgan-gp phi')
parser.add_argument('--grow_steps', nargs='+', type=int,default=[50,100,150],
                    help='the vector of a discovered architecture')
parser.add_argument('--D_downsample', type=str, default="avg",
                    help='downsampling type')
parser.add_argument('--fade_in', type=float, default=1,
                    help='fade in step')
parser.add_argument('--d_depth', type=int, default=7,
                    help='Discriminator Depth')
parser.add_argument('--g_depth', type=str, default="5,4,2",
                    help='Generator Depth')
parser.add_argument('--g_norm', type=str, default="ln",
                    help='Generator Normalization')
parser.add_argument('--d_norm', type=str, default="ln",
                    help='Discriminator Normalization')
parser.add_argument('--g_act', type=str, default="gelu",
                    help='Generator activation Layer')
parser.add_argument('--d_act', type=str, default="gelu",
                    help='Discriminator activation layer')
parser.add_argument('--patch_size', type=int, default=4,
                    help='Discriminator Depth')
parser.add_argument('--fid_stat', type=str, default="./fid_stat/fid_camera.npz",
                    help='Discriminator Depth')
parser.add_argument('--diff_aug', type=str, default="None",
                    help='differentiable augmentation type')
parser.add_argument('--accumulated_times', type=int, default=1,
                    help='gradient accumulation')
parser.add_argument('--g_accumulated_times', type=int, default=1,
                    help='gradient accumulation')
parser.add_argument('--num_landmarks', type=int, default=64,
                    help='number of landmarks')
parser.add_argument('--d_heads', type=int, default=4,
                    help='number of heads')
parser.add_argument('--dropout', type=float, default=0.,
                    help='dropout ratio')
parser.add_argument('--ema', type=float, default=0.995,
                    help='ema')
parser.add_argument('--ema_warmup', type=float, default=0.,
                    help='ema warm up')
parser.add_argument('--ema_kimg', type=int, default=500,
                    help='ema thousand images')
parser.add_argument('--latent_norm',action='store_true',
    help='latent vector normalization')
parser.add_argument('--ministd',action='store_true',
    help='mini batch std')
parser.add_argument('--g_mlp', type=int, default=4,
                    help='generator mlp ratio')
parser.add_argument('--d_mlp', type=int, default=4,
                    help='discriminator mlp ratio')
parser.add_argument('--g_window_size', type=int, default=8,
                    help='generator mlp ratio')
parser.add_argument('--d_window_size', type=int, default=8,
                    help='discriminator mlp ratio')
parser.add_argument('--show', action='store_true',
                help='show')

opt = parser.parse_args()

return opt

Hi,thank u for your great work.But I got some problems when testing your provided model from https://drive.google.com/file/d/1Td9baoNua6jNtVvsnJczW1u4QWa1w9sl/view?usp=sharing. It seemed that the network of trained model is different from that in model_search.

Due to the patch-wise generation of TransGAN, I found some blocking artifacts in your generations. I think the authors had already known these phenomena. Are there any tricks to eliminate these artifacts?

Can you explain more about using Conv2d as a patch-splitting image? (in the Discriminator) Since the image will change when applying the Conv and it didn't split the image into parts of a grid as the Vision-Transformer do. And I saw some other implement uses Unfold to split the image.

        self.fRGB_1 = nn.Conv2d(3, embed_dim//4, kernel_size=6, stride=4, padding=1)
        self.fRGB_2 = nn.Conv2d(3, embed_dim//4, kernel_size=10, stride=8, padding=1)
        self.fRGB_3 = nn.Conv2d(3, embed_dim//2, kernel_size=18, stride=16, padding=1)

Now I have experimented with a framework in which the generator is transgan and the discriminator is autogan, but it doesn't seem to converge? Epoch is 320, and the experimental result FID is 130. What tricks did you use in the experiment?Thank you

Could you provide 256x256 CelebA pretrained model? Thank you so much. By the way, how long did it take you to train a 256 * 256 celeba model. How many GPUs and what model are used？

UnboundLocalError: local variable 'gen_net' referenced before assignment 请问 全局变量和局部变量冲突了？ is the global variable conflict with the local variable...

I train this model and download cifar_checkpoint,but: => calculate fid score 0%| | 0/3125 [00:02<?, ?it/s] Traceback (most recent call last): File "/root/miniconda3/lib/python3.8/site-packages/tensorflow/python/client/session.py", line 1377, in _do_call return fn(*args) File "/root/miniconda3/lib/python3.8/site-packages/tensorflow/python/client/session.py", line 1360, in _run_fn return self._call_tf_sessionrun(options, feed_dict, fetch_list, File "/root/miniconda3/lib/python3.8/site-packages/tensorflow/python/client/session.py", line 1453, in _call_tf_sessionrun return tf_session.TF_SessionRun_wrapper(self._session, options, feed_dict, tensorflow.python.framework.errors_impl.UnimplementedError: 2 root error(s) found. (0) UNIMPLEMENTED: DNN library is not found. [[{{node FID_Inception_Net/conv/Conv2D}}]] [[FID_Inception_Net/pool_3/_3]] (1) UNIMPLEMENTED: DNN library is not found. [[{{node FID_Inception_Net/conv/Conv2D}}]] 0 successful operations. 0 derived errors ignored. I try to reduce the batchsize,but it still no work

RuntimeError: CUDA out of memory. Tried to allocate 1024.00 MiB (GPU 0; 7.94 GiB total capacity; 7.03 GiB already allocated; 144.81 MiB free; 7.12 GiB reserved in total by PyTorch)

I meet this question after i print "python exps/cifar_train" in Terminal ,and it apears after "path:logs/cifar_train_2022_03_22_19_29_36 0%| |0/1563 [00:00<?, it/s]" . I know this means the CUDA is out of memory, but i only run this one program ,and the image has not loaded. Did author or someone also meet this question and how did you deal with it?