IJCAI2020 & IJCV 2020 :city_sunrise: Unsupervised Scene Adaptation with Memory Regularization in vivo

Hi, thank you for your great work on domain adaptation! The origin input image is 1024512, but now I want to train on higher resolution, like 19201080. How should I change the class balance parameters like max_value and 6464n. In my opinion, it will be better to use ratio threshold instead of a hard assigned number like 64 like this: for i in range(self.num_classes): count[i] = torch.sum(labels==i) if count[i]/(nhw) < 1/128.0: #small classes, original train size is 1024512 weight[i] = self.max_value and 6464/(5121024) = 1/128 But I get worse results than default settings trained with input size (1024512). Is there anything that I missed? Any help will be appreciated.

Hi, I have some problems 1.for your release model: stage 1 model (SE_GN_batchsize2_1024x512_pp_ms_me0_classbalance7_kl0.1_lr2_drop0.1_seg0.5), I test this model using your code, the mIoU is only 38.07, much lower than 45.5(stage 1 miou in MRNet paper). stage2 model(1280x640_restore_ft_GN_batchsize9_512x256_pp_ms_me0_classbalance7_kl0_lr1_drop0.2_seg0.5_BN_80_255_0.8_Noaug) is 50.34 , which is same as the report result in second paper. 2. train result I use your released stage 1 model to generate pseudo label, then train stage 2, However, the performance is low: 43.89 for 25000.pth, 42.69 for 50000.pth, 41.98 for 100000.pth, much lower than the result in your paper. And I do not change any code except variable DATA_DIRECTORY

Nice work! I am a novice in the domain adaptation field. I have a question about stage2. In my understanding about your paper, uncertainty estimation is used for target domain images. But in your code trainer_ms_variance.py, it seems to be calculated on the image of the source domain.

    def gen_update(self, images, images_t, labels, labels_t, i_iter):
            self.gen_opt.zero_grad()

            pred1, pred2 = self.G(images)
            pred1 = self.interp(pred1)
            pred2 = self.interp(pred2)

            if self.class_balance:            
                self.seg_loss = self.update_class_criterion(labels)

            loss_seg1 = self.update_variance(labels, pred1, pred2) 
            loss_seg2 = self.update_variance(labels, pred2, pred1)
 
            loss = loss_seg2 + self.lambda_seg * loss_seg1

Hi, thanks for your great job! Recently I've been reading your code and I have a question about stage 2(rectifying). You set lambda_adv_target1 and lambda_adv_target2 to 0 which means there is no adv training in stage 2(Right?), but you keep training generator with false instruction from discriminator(the weight of discriminators is not loaded in stage 3), you annotated here which means you keep training G, but here you never updated D, is this the right behavior or maybe I misunderstood sth?

When conducting experiments of stage 1 training, state dicts of self.G model is inconsistent with the pretrained model in 'http://vllab.ucmerced.edu/ytsai/CVPR18/DeepLab_resnet_pretrained_init-f81d91e8.pth'. For example, key of state dict of self.G is 'layer5.conv2d_list.0.0.weight' and 'layer6.bottleneck.0.se.0.weight', but key of the pretrained model is 'layer5.conv2d_list.0.weight' and there is no module 'layer6.bottleneck.0.se.0.weight'. Should I set the strict as False in load_state_dict()?

Hi, thanks for sharing your implementation. I have two questions about it:

1. Does it also work on tabular data?
2. Is it possible to identify the noisy instances (return the noisy IDs or the clean set)?

Thanks!

Hi, thanks for the nice code~ I have some questions about the pseudo-labels training part of the code,

this is the pseudo labeling loss of the source data as Eq.11 in paper: loss_seg1 = self.update_variance(labels, pred1, pred2), in which the labels come from the source domain.

the target domain does not use the pseudo-labels but the entropy minimization: loss_kl = ( self.kl_loss(self.log_sm(pred_target2) , mean_pred) + self.kl_loss(self.log_sm(pred_target1) , mean_pred))/(nhw)

Hello, thank for sharing the great work. I have a quick question. I find that ''random scale'' and ''mirror'' are used for preprocessing data, which is a little different from previous works.

• So, for what reason did you adopt these strategies?

• Have you conducted experiments to verify how much performance improvement they bring?

Thanks!

Hi, thanks for the code! I was wondering why in stage 2 the loss weight lambda-kl-target is set to 0. From my understanding, the rectification is done by using this term. Thanks in advance.

If I want to run the code without rectifying the pseudo label, I am confused that why in stage 2 the loss weight lambda-kl-target is set to 0? (From README) From Eq.9 of the IJCAI2020 paper, I understand the KL-divergence loss is should be used. Thanks

Thank you for sharing this code! I have a small question, when I run the train_ms_robot. py file, I always get an error, AttributeError: 'Namespace' object has no attribute 'sync_bn'.

Hello,

could you maybe explain the difference between prediction variance and prediction Confidence? In the paper, Figure 5 visualizes both, but I'm not sure how do you calculate them. I assume the KL term divergence is used as variance, then how to get the confidence?

Thanks in advance.

Hi @layumi , I noticed that in your paper, the uncertainty is obtained by computing both MC-Dropout and difference between two classifiers, but I cannot find the code used for MC-Dropout, could you please kindly tell me?

Sorry to bother you, when I downloaded the SYNTHIA dataset and decompressed it. I found there was no folder with "ground truth"(GT). I want to know if someone else is in this situation.