OpenMatch: Open-set Consistency Regularization for Semi-supervised Learning with Outliers (NeurIPS 2021)

Hi, nice work!

When I ran experiments on cifar10, I got the error, it seemed like that the files of LSUN_fix. I was a freshman on the task, Would you please give me some advice?

Best wished to you!

First of all, thank you very much for your excellent work. I have a question about which accuracy rate was chosen in your paper to calculate the error? Closed acc or Overall acc? I look forward to your answer. I will be very grateful！

Hello, thanks for your interesting research! I was looking for the experimental settings of data for open-set semi-supervised learning.

I have a question about splitting labeled and unlabeled data (function x_u_split(args, labels)).

As long as I understood, in open-set semi-supervised learning, the unlabeled data may contain new categories which are not present in labeled data. For example, the categories of labeled data are 0, 1, 2, 3, 4, 5 while the categories of unlabeled data are 2, 3, 4, 5, 6, 7 (6 and 7 are new). However, in line 170 of cifar.py, because args.num_classes for cifar10 is 6, and the data samples of each class are assigned in the same for loop, both labeled data and unlabeled data have identical categories: 0, 1, 2, 3, 4, and 5. Then, the categories of each datasets become as follows: labeled: 0, 1, 2, 3, 4, 5 unlabeled: 0, 1, 2, 3, 4, 5 test: 0, 1, 2, 3, 4, 5, 6, 7, 8, 9

I think I might miss something but couldn't figure it out.

Thanks for your help in advance.

Dear Author

thanks for providing the code. I have a question and hope you can help me understand it a bit.

1. Is it possible that during some iterations, the exclude_dataset function selected no unlabeled data to be inliner, how does the code handle this edge case? https://github.com/VisionLearningGroup/OP_Match/blob/fd55de2e1913ffbcb3fd413bd9f024afce44c024/utils/misc.py#L167

2. In the x_u_split function, why do you need to manually repeat the labeled set? Would the code not work without manually repeating the labeled set?

https://github.com/VisionLearningGroup/OP_Match/blob/fd55de2e1913ffbcb3fd413bd9f024afce44c024/dataset/cifar.py#L183

Hope to hear from you, thanks!

Thanks for your code. It seems that the threshold for pseudo labeling is 0 in utils/parser.py. Usually, the threshold is 0.95 for FixMatch-based methods. Why the threshold is so low? Thanks!

Thank you for sharing the code~ You split the overall training dataset into labeled set/ val set/ unlabeled set, and then applying the val set to select the best ACC model, thus the extra val dataset leads to the unlabeled dataset is less than fixmatch. could you explain the rule of splitting

Hello, thanks for your interesting research! https://github.com/VisionLearningGroup/OP_Match/blob/fd55de2e1913ffbcb3fd413bd9f024afce44c024/utils/misc.py#L67 On line 67, I think unknown_size = len(ind) should be replaced by unknown_size = ind.sum(). len(ind) is the batch size.

Thanks for your help in advance.

Thanks for the excellent work. I have followed the readme to run python main.py --dataset cifar100 --num-labeled 50 --out Results --num-super 10 --arch wideresnet --lambda_oem 0.1 --lambda_socr 1.0 --batch-size 64 --lr 0.03 --expand-labels --seed 0 --mu 2 The final performance after epoch 512 is Does the Table 1 include the best performance on test set or the mean of last 20 epochs? If run with the amp, the result is Is it normal?

Thanks for your great work!

I met some problems when I reimplemented the Fixmatch results in your table. The error rate of Fixmatch is 7.4% on CIFAR10(Known/Unknown 6/4, number of the labeled image is 50), which is lower than your reported results. I have not figured out why. Do you have any ideas?

I utilize the Fixmatch code in https://github.com/TorchSSL/TorchSSL