if args_parser.seed is not None:
	random.seed(args_parser.seed)
	torch.manual_seed(args_parser.seed)

Each gpu is set to the same seed.

# fix the seed for reproducibility
if args_parser.seed is not None:
	from lib.utils.distributed import get_rank()
	seed = args_parser.seed + get_rank()
	torch.manual_seed(seed)
	np.random.seed(seed)
	random.seed(seed)

Reference

Hi, I downloaded the checkpoint, prepared the data and ran the evaluation script :

bash scripts/cityscapes/hrnet/run_h_48_d_4_proto.sh val hrnet_proto_80k

I had to include a tiny fix label_img_ = Image.fromarray(label_img_) instead of label_img_ = Image.fromarray(label_img_, 'P') in tester.py because the labels were all black in the output directory. If I then execute the above line, I end up with an mIoU of 85.7, much better than the 81.1 reported in your paper in Table 2 for HRNet. This is the output:

classes          IoU      nIoU
--------------------------------
road          : 0.978194      nan
sidewalk      : 0.817676      nan
building      : 0.966470      nan
wall          : 0.586037      nan
fence         : 0.650901      nan
pole          : 0.858629      nan
traffic light : 0.865478      nan
traffic sign  : 0.904490      nan
vegetation    : 0.977709      nan
terrain       : 0.664823      nan
sky           : 0.973433      nan
person        : 0.950269    0.000000
rider         : 0.802941    0.000000
car           : 0.985926    0.000000
truck         : 0.818478    0.000000
bus           : 0.939901    0.000000
train         : 0.811736    0.000000
motorcycle    : 0.806717    0.000000
bicycle       : 0.919648    0.000000
--------------------------------
Score Average : 0.856814    0.000000
--------------------------------

I also used my own evaluation script on the generated labels that are in the label directory and I get exactly the same results. Could you check?

Thanks for the great work ! I want to ask a question that confuses me. In the Paper, Table 4 shows that the parameter numbers of the entire model is not increased. However, i see that in the code the prototype is inceased by class_num as follow: self.prototypes = nn.Parameter(torch.zeros(self.num_classes, self.num_prototype, in_channels), requires_grad=True) If class_num is increased, the parameter numbers of prototypes are also increased. Did I get it wrong？

Hi, thanks for the impressive work.

After reading the paper, I have a question that how the micro-prototypes are initialized? They seem to be properly initialized so as for a reasonable solution in eq.(10).

Cheers

Thanks for your great work! I notice you use layernorm for the final features before the classifier and also for the predictions. I think it is quite uncommon in prototype learning (correct me if i am wrong).

Could you please provide some explanation for this? And if removing the two layernorm, will the performance be degraded?

https://github.com/tfzhou/ProtoSeg/blob/1c4a7784bbce96c06fe72d55255af15e6cf1ca96/lib/models/nets/hrnet.py#L81

Hi, I'm interested in your work. I wonder if there are k prototypes of some classes that become similar after training? For example, after visualization according to Figure 3 in the paper, it will be found that the activation area of each prototype is roughly the same. I found this problem while running your code. I suspect it's caused by some classes of my dataset that don't have meaningful parts.

Hope to receive your reply, thanks!

Hi, I'm interested in your work. After reading the paper, I'm confused that the goal of Within-Class Online Clustering is to map the pixels Ic to the K prototypes of class c. But how to know if pixels Ic belongs to class c? Did you use Ground Truth in this step? So how do you set it up when testing?

Hope to receive your reply, thanks!

Dear author, Thank you so much for your work and code. I have a question about the number of model parameters.

As I understand the paper, pixels are classified as the closest prototype among CK prototypes at inference time. In the end, we have to store CK prototypes, then I wonder why we don't interpret them as model parameters. Also, the number of prototypes to be stored is proportional to the number of classes. Is it just convention?

Thank you.

Hi, I'm interested in your work. After reading the paper, I'm confused that the PPC loss is achieved by contrastive learning strategy in your paper. But according to the code, the PPC loss is using cross entropy loss. Hope to receive your reply, thanks.