Hi, I am interested in your work. research was very interesting. I want to know if 'Flexmatch' in the corresponding code reproduces the results of the paper. I tried CIFAR100_400 case, but the top-1 score doesn't go up.

I look at the code and have a question related to it. Is it right that classwise_acc all starts at 0.? Doesn't this mean that the threshold of each class is zero? This seems to be a factor in lowering the score by breaking the model. Is it the intention of the paper to raise the threshold from zero? (I'm sorry if I didn't understand.)

Is there any other way to train the model properly?

Best Regards, Harim

Dear authors,

Thanks for the impressive work FlexMatch and the awesome codebase!

Recently, I am trying to conduct some experiments on CIFAR100 based on this repo and I just found that FlexMatch training may cost around 50min for 5k iterations (3 * NVIDIA RTX2080Ti) which may take 7 days for the whole training. It is quite a long period.

I wonder if it is possible for you to release the training log or tensorboard for FlexMatch method on CIFAR100 to provide me a reference which would help me a lot. Many many thanks~

Cheers, Haiming

Hello, and many thanks for creating this repo!

I've run several experiments, the log gives the expected results but when i run tensorboard I consistently get "No dashboards active". Am I doing something wrong?

Firstly, thanks a lot for open source this code base which will help the development of semi-supervised field. I have questions about multi-gpu training and look forward to your reply: As far as I understand, this code supports multi-gpu training and I want to know have you ever tested different parameters in multi-gpu environment, for example, increasing batch size when using more gpu. And how did you set the parameters in .yaml when you have multiple gpus in one machine?

Hi,

Thanks for the great work. I tried to reproduce FlexMatch number on CIFAR100 with 400 labels. I followed the instructions to create a conda env and ran python flexmatch.py --c config/flexmatch/flexmatch_cifar100_400_0.yaml with 3 Tesla V100.

Unfortunately, the best top-1 accuracy I got was 60.65, which is 6.91% lower than the reported number in the paper. There seems to be a sharp performance drop at around iteration 600K. But I couldn't pinpoint the issue from the training statistics. I wonder if you also have observed similar behaviors. It would be great if you could offer some insights here. Thanks!

Here is the tensorboard file: tf_logs.zip

Besides, the curve of the mask ratio looks a bit strange to me. Because 1.0 - mask.detach() is actually logged in the code, so, shouldn't it start from 1 and then decrease? Any intuition why it starts from 0 and increases at the very beginning? Thanks!

Thanks for your great work!

Just one quick question, how many GPUs were used to obtain the results in the paper? I didn't seem to find the specification on this.

Best

When I run python flexmatch.py --c ./config/flexmatch/flexmatch_cifar100_400_1.yaml, I always get the following error. Can you help me to fix the mistake?

I put the custom data set using the same data loader as Imagenet, and organize my data as: "imagenet"/{train or val}/class name/*.jpg

But this situation still appears, how should I solve it?

and there is the [isic.yaml.] setting. isic.txt

Hi, I just discovered your repo, its great ! However, I was wondering if it was possible to modify for example a supervised training loop that i have to do semi-supervised learning ? How would one adapt and use your repo to achieve such task ? for example if i am currently using detectron2 module to do training, is there a way i can modify train.py to do semi-supervised learning ?

Thank you so much !

Thanks for your great work!

But it seems there is no validation set sampling part in this repository.

As far as I know, in general, 5,000 images are subtracted from the training set and used as validation. [1] Official FixMatch, validation set size options, fixmatch [2] A. Oliver, et al., "Realistic Evaluation of Deep Semi-Supervised Learning Algorithms"

I tried to find it in the datasets/ssl_dataset.py, but I couldn't. May I know where the validation set part, or why it's not there? Thanks!

Hi, Thanks a lot for your contribution.

I wanted to ask about the warmup thing. In your flexmatch training code there is args.warmup inside the if condition if max(pseudo_counter.values()) < len(self.ulb_dset):

Will it not everytime go in this if condition and args.warmup if condition? Because I printed the if statement and through the training it goes in args.warmup if condition.

So ACC to paper the warmup should get over after some time? Is there a issue in a code or am I understanding it in a wrong way?

Thanks a lot, Shreejal

I think there's a small bug when saving the model.

The model is saved before the self.it is updated, so when training is resumed, the model starts with the same self.it. However, it should be self.it + 1

https://github.com/TorchSSL/TorchSSL/blob/f2f46076cbea1b6f6c9b3c1c45609502c6576250/models/fixmatch/fixmatch.py#L191-L196

https://github.com/TorchSSL/TorchSSL/blob/f2f46076cbea1b6f6c9b3c1c45609502c6576250/models/fixmatch/fixmatch.py#L220

In my workaround, take note that you can only use save_model() after updating the model and before updating self.it.

I am running this code on a server that has a time limit of one day per job. So, I need to resume the code. I see a drop in accuracy when the training is resuming. Could you please comment on what might be causing such drops?

Hello, I read the code you open source to Github and found that the size of selected_label is the size of the entire unlabeled dataset. When updating selected_label later, the index used is the current batch size index, for example, batch is 64, X_ULb_idx is the index range of 0-63. Or the index of the original dataset, I don't know if you can understand this description?

Thanks for your excellent work. I found that the results in cifar100 are different in different versions of paper. I noticed that your code also changed five months ago. I downloaded your code on 2021.10.19 and experimented on it. Is the code of 2021.10.19 different from the current one? Do I need to redo the experiment on the current code?