Hello, Exciting project. Could you please share the minimum resource requirements to train this model? I am getting memory errors training 500,000 256x256 images on 4 40GB A100 GPU's. Also my CPU memory is 128GB.

I tensorflow/stream_executor/stream.cc:1990] [stream=0x5570d9fd4600,impl=0x5570d9fd2050] did not wait for [stream=0x5570d9fd40a0,impl=0x5570d9fd2940] 2021-05-26 01:09:18.033272: I tensorflow/stream_executor/stream.cc:4925] [stream=0x5570d9fd4600,impl=0x5570d9fd2050] did not memcpy device-to-host; source: 0x2ac3e5d41a00 2021-05-26 01:09:18.033313: F tensorflow/core/common_runtime/gpu/gpu_util.cc:293] GPU->CPU Memcpy failed /hpc/users/marxg01/.lsbatch/1622001386.34151325.shell: line 23: 389547 Aborted (core dumped)

Hello! I would first like to thank you for sharing codes for such wonderful work. I have some questions on intermediate output dimension of feature maps in the generator.

According to Figure 16, I presumed output dimension of f (φ1) should be (1024, height, width), as the first 1x1 convolution is noted as DeConv(2^(11-i), 1, 1). To my understanding, it would not match noise input dimension of (512,4,4).

Then on examining your module.py code, the output dimension was controlled by nd() function, which has upper limit of 512 unlike noted output dimension in figure 16. I presumed such function was introduced to solve the dimension mismatch problem mentioned above.

In addition, i noticed a outlier in dimension of orthonormal basis. To my understanding, dimensions of orthonormal basis uij ∈ R^(Hi×Wi×Ci) should double as it moves up one layer, as height and width values are doubled whereas Ci value is halved. However, as nd(height) is a parameter determining dimension of U on module.py, I noticed dimension quadruples on moving from layer 1 to layer 2 (8192->32768), unlike expected behavior mentioned above.

Regarding aforementioned issues, I would like to ask questions below.

1. I would like to ask whether nd() function was introduced just to match the dimension, or there is other training gains of using such output dimensions.
2. I am wondering if there is any test results on setting initial noise dimension to 1024, i.e eps_dim = 1024 so the generator architecture would work as given on figure 16.
3. I am curious whether I have understood correctly about dimension of orthonormal basis.

If there is anything which I have misunderstood, please kindly point it out. Thank you for your kind attention.

Attached below is toy code i used to estimate dimensions of intermediate feature map

zs = [tf.random.normal([64, z_dim]) for z_dim in [6] * 6]
h = tf.random.normal([64, 4 * 4 * 512])
h = tf.reshape(h, [-1, 4, 4, 512])

nd = lambda size: min(int(2**(12 - np.log2(size))), 512)
# nd = lambda size : 2048//size

print(f'Noise shape : {h[0].shape}')

for i, z in enumerate(zs):
    height = width = 4 * 2 ** i
#     print(i,height,z.shape[-1])
    U = tf.compat.v1.get_variable(f'U_{i}',initializer=tf.initializers.orthogonal(),shape=[height, width, nd(height), z.shape[-1]])
    L = tf.compat.v1.get_variable(f'L_{i}',shape=[z.shape[-1]],initializer=tf.initializers.constant([3 * i for i in range(z.shape[-1], 0, -1)]))
    mu = tf.compat.v1.get_variable(f'mu_{i}',shape=[height, width, nd(height)],initializer=tf.initializers.zeros())
    print(f'basis dimension : {tf.reshape(U[:,:,:,0],-1).shape}')
    
    h_ = tf.reduce_sum(U[None, ...] * (L[None, :] * z)[:, None, None, None, :], axis=-1) + mu[None, ...]
    h1 = transposed_convolution2d(h_, num_outputs = nd(height), kernel_size = 1) #deconv
    h2 = transposed_convolution2d(h_, num_outputs = nd(height*2), kernel_size = 3, stride = 2)
    
    
    h0 = transposed_convolution2d((h+h1),nd(height * 2), 3, 2)
    print(h1.shape, h2.shape, h0.shape)
    h = transposed_convolution2d((h0+h2),nd(height * 2), 3, 1)
#     print(h_.shape, h1.shape,h2.shape)
    print(f'output shape of layer {i+1} : {h[0].shape}')
print(f'final result :{convolution(h,num_outputs=3,kernel_size = 7)[0].shape}') #conv

Hello!

I was wondering if there is a planned functionality to get the latent variables (U,L,z etc) of a real image that is similar to the dataset the GAN was trained on, essentially a way to recreate the metrics in Table 1 of the publication...