Why do you use validation dataset (500 images Cityscapes) to select best model? Valid dataset (500 images Cityscapes) is only used for evaluation, not for model selection (for saving best model), this paper is having a big problem!

Can you share how to train warm-up stage? I use this script to train warm-up stage but the result is quite fail and can't get 43.3 mIoU like pre-trained. python train.py --name gta2citylabv2_warmup --stage warm_up --student_init simclr --no_resume --lr 0.0001

We rerun your code following the default setting. The result is quite lower than reported(57.5%). our result is as follows: Overall Acc: 0.8863829080331662 Mean Acc : 0.6676762265179861 FreqW Acc : 0.8026627310122789 Mean IoU : 0.5486842157620068

Hi, congratulations on your great work and acceptance in CVPR '21. Thanks for releasing the code and model weights.

In the paper, you mention using DeepLabv2 with ResNet101 backbone. However, your code actually makes use of a modified ASPP module (ClassifierModule2 in models/deeplabv2.py) while actually ClassifierModule has to be used for DeepLabv2. Similar issues were raised here and here which mention that this type of ASPP module is used in DeepLabv3+ which has a much better performance compared to DeepLabv2 (both issues were raised in Jan. 2020). Could you please confirm this point and if you have also performed experiments with the original DeepLabv2 model, could you report those results for a fair comparison with prior arts?

I am wondering whether there is any motivation behind the fact you excluded log() whiling computing kl_div loss in distillation stage? (in line 315 of adaptation_modelv2.py). Usually people use loss_kd = F.kl_div(student.log(), teacher), but here you are giving loss_kd = F.kl_div(student, teacher), right?

Hi, I am trying to reproduce the results for the training process adapting GTA V to Cityscapes. I have downloaded the warm-up model and generated the soft pseudo-labels successfully, and also calculated the prototypes. But after running the script to train the stage 1, the highest mIoU that I get is 52.9.

The parameters that I have used for my current set-up (2 Nvidia RTV 2080 with 24GB each) are:

batch size = 2 learning_rate = 0.0001/2 epochs = 84 train_iters = 90000*2 I couldn't run the script with the default configuration (bs=4, lr=0.0001 and train_iters = 90000) due to out of memory. Any thoughts on how I can achieve the results with my hardware configuration?

How long will each stage last based on the paper setting? I am training the stage1 on 2 Tesla V100-DGXS-32GB, and it takes 2 hours to train 1000 iters(about 1.3 epoch), it's a little slow, is this to be expected?

hello~Thanks for your excellent work, but now I meet a problem when I try to follow the steps in "Inference Using Pretrained Model"-"2) GTA -> Cityscapes". It throws a error: RuntimeError: Error(s) in loading state_dict for ResNet101: Missing key(s) in state_dict: "bn_pretrain.weight", "bn_pretrain.bias", "bn_pretrain.running_mean", "bn_pretrain.running_var". if anyone meet the same error?

Dear author， Thanks for your great work. Recently I do the code reproduction and fine a small issue which I'm not sure. When I finish the stage1 training and start to train the stage2 I found that the loss will be negative in some cases. After checking the loss code part I thought it will not be negative theoretically. May you know what the problem is or it is not a problem due to my poor knowledge.

Hi. Thank you to share your perfect code.

I am running your code. And I met one issue that can be relatived with the problem in your paper, that ProDA's "train class mIoU score" was Zero.

1. Adding this to your code

ProDA/models/adaptation_modelsv2.py this is for source only training

class CustomModel():
    def step_source(self, source_x, source_label, source_imageS=None, source_params=None):
        self.BaseOpti.zero_grad()
        
        if self.opt.S_pseudo_src > 0: # Always [opt.S_pseudo_src == 0]
            source_output = self.BaseNet_DP(source_imageS)
            source_label_d4 = F.interpolate(source_label.unsqueeze(1).float(), size=source_output['out'].size()[2:])
            source_labelS = self.label_strong_T(source_label_d4.clone().float(), source_params, padding=250, scale=4).to(torch.int64)
            loss_ = cross_entropy2d(input=source_output['out'], target=source_labelS.squeeze(1))
            loss_GTA = loss_ * self.opt.S_pseudo_src
            source_outputUp = F.interpolate(source_output['out'], size=source_x.size()[2:], mode='bilinear', align_corners=True)
        else: # Runing part
            source_output = self.BaseNet_DP(source_x, ssl=True)
            source_outputUp = F.interpolate(source_output['out'], size=source_x.size()[2:], mode='bilinear', align_corners=True)
            loss_GTA = cross_entropy2d(input=source_outputUp, target=source_label, size_average=True, reduction='mean')

        loss_GTA.backward()
        self.BaseOpti.step()

        return loss_GTA.item()

2. Result to train with source only

But I got this result.

3. the main issue

I've never seen a few classes score 0. But If we see the result above, we can see Zero score. How do you think about this??

Thank you.

PS: Runging config

init_parameter = 'imagenet'

# gta5 / synthia / cityscapes
src_dataset = 'gta5' 
tgt_dataset = 'gta5'
# GTA5 / Synthia / Cityscapes
src_rootpath = 'Dataset/GTA5'
tgt_rootpath = 'Dataset/GTA5'
n_class = 19
num_workers = 8
bs = 8

name = 'gta5_src'
seed = 1337

## model
model_name = 'deeplabv2'
lr = 0.0005
freeze_bn = False
epochs = 84
train_iters = 90000
bn = 'sync_bn'
no_resume = False
stage = 'src_only' # warm_up / stage1 / stage2 or stage3 / src_only 
finetune = False
bn_clr = False
train_thred = 0
used_save_pseudo = False
no_droplast = False

resize = 2200
rcrop = [896, 512]
hflip = 0.5

S_pseudo = 0.0

noshuffle = False
noaug = False

Hi, I am trying to reproduce the full pipeline from the very beginning, i.e. starting from training the model only with the source dataset, then doing the warm-up stage and so on.

Could you provide more information about the amount of epochs or iterations that you trained the model for the following stages:

1. Source only
2. Warm-up

Thanks for the help.

Good job!! I would like to ask which pictures of Cityscapes dataset are use to make qualitative results of semantic segmentation, 500 val images? thanks!

why the softmax output "teacher" will equal to 250 (ignore index) here?

https://github.com/microsoft/ProDA/blob/9ba80c7dbbd23ba1a126e3f4003a72f27d121a1f/models/adaptation_modelv2.py#L316

Is it a bug? Is it should be "threshold_arg"?

when I run tran.py

 python train.py --name gta2citylabv2_stage1Denoise --used_save_pseudo --ema --proto_rectify --moving_prototype --path_soft Pseudo/gta2citylabv2_warmup_soft --resume_path ./pretrained/gta2citylabv2_warmup/from_gta5_to_cityscapes_on_deeplabv2_best_model.pkl --proto_consistW 10 --rce --regular_w 0.1

Traceback (most recent call last):
  File "train2.py", line 217, in <module>
    train(opt, logger)
  File "train2.py", line 88, in train
    target_lpsoft, target_image_full, target_weak_params)
  File "/media/ailab/data/yy/ProDA/models/adaptation_modelv2.py", line 209, in step
    weights = self.get_prototype_weight(ema_out['feat'], target_weak_params=target_weak_params)
  File "/media/ailab/data/yy/ProDA/models/adaptation_modelv2.py", line 350, in get_prototype_weight
    feat_proto_distance = self.feat_prototype_distance(feat)
  File "/media/ailab/data/yy/ProDA/models/adaptation_modelv2.py", line 345, in feat_prototype_distance
    feat_proto_distance[:, i, :, :] = torch.norm(self.objective_vectors[i].reshape(-1,1,1).expand(-1, H, W) - feat, 2, dim=1,)
RuntimeError: Expected all tensors to be on the same device, but found at least two devices, cuda:0 and cpu!

I only want to use two gpu, So I change the code only in here

 if opt.model_name == 'deeplabv2':
        model = adaptation_modelv2.CustomModel(opt, logger)
        model = torch.nn.DataParallel(model, device_ids=device_ids)

How many classes have you used in the warmup training of Synthia - > cityscapes? Can you provide the warmup training command of Synthia - > cityscapes as an example? thank you very much indeed

Bumps numpy from 1.18 to 1.22.0.

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

The warmup pretrained model was supposed to be 43.3 mIoU, but when I test on cityscapes valid dataset, I've got a really lower one.I'm using the runningSocre class in your Repo and my metric function with the same result. I'm really confused with the result, could you tell me how to get the correct mIoU about the pretrained model?