I'm trying to reproduce the paper, but cann't reach 0.85 ACC runing this code on GoogleNews-S. All hyper-parameters are set to same values in paper, and the data is enhanced with contextual argumentation. The running result shows, not the model but the representation with K-Means performances better with 0.75 acc , and the model just reaches 0.62 acc. When I increase the clustering head lr, the result with model still remains 0.62 level. What should I do to improve this？

I am trying to reproduce the original paper, but I am not able to find the twitter dataset and the link (Yin and Wang, 2016) in the paper does not direct to the dataset.

@Dejiao2018 Hy, the idea in the paper is quite good. But I am not able to get good results with a different dataset. For eg. Banking Dataset (77 clusters) (https://arxiv.org/pdf/2003.04807.pdf)

Convergence Criteria: Percentage Change in Prediction in Subsequent Iterations < 0.1 % Using the same available code.

Initial Results Before Training: Kmeans : ACC: 42.4 NMI: 60.95 ARI: 27.49

After Training: Kmeans : ACC: 13.0 NMI: 30.32 ARI: 5.2
After Training: Cluster Model: ACC: 11.4 NMI: 26.12 ARI: 4.7

Over subsequent iterations, the results get worse.

In the training.py: cluster_loss = self.cluster_loss((output+1e-08).log(), target)/output.shape[0]

Why is .log() used here, which is inconsistent with the paper.

https://pytorch.org/docs/stable/generated/torch.nn.KLDivLoss.html?highlight=kldivloss#torch.nn.KLDivLoss

Did you divide the dataset into training dataset and test dataset? I didn't find any content related to dataset partitioning in the paper. In the code corresponding to the paper, I found that there are data files named 'test' and 'train', but it seems that the main program file 'main.py' uses the same file specified by argument 'dataname' when training and testing.

Hi, I am trying to reproduce the paper. But I am not sure how the augmented pairs generated by the contextual augmenter. The paper shows that the data is augmented by Bertbase and Roberta via word substitution Does it mean that you augment the input sentence once by bertbase and roberta correspondingly and combine these two sentences the augmented pair, or it means that during the experiment, you just use one single pre-trained model (e.g., roberta) and use it to augment the sentence twice to get the augmented pair? I try the latter one way, i.e, use the roberta to generate the two augmented sentences for one sentence, but it did not work.

train_data = pd.read_csv(os.path.join(args.data_path, args.dataname)) train_text = train_data['text'].fillna('.').values train_text1 = train_data['text1'].fillna('.').values train_text2 = train_data['text2'].fillna('.').values train_label = train_data['label'].astype(int).values

What is the difference between text, text1 and text2 here？

I have a small doubt in the file dataloader.py under the dataloader directory. In the function augment_loader, the train_label makes sense i.e., if train_text1 and train_text2 belong to same parent, then it is 1, otherwise it is -1. But, in the function train_unshuffle_loader, I don't get the use of train_label. Can you please explain that?

Hi, I'm reproducing paper and having some issues. I used below SCCL with virtual augmentation code. ''' python3 main.py
--resdir $path-to-store-your-results
--use_pretrain SBERT
--bert distilbert
--datapath $path-to-your-data
--dataname searchsnippets
--num_classes 8
--text text
--label label
--objective SCCL
--augtype virtual
--temperature 0.5
--eta 10
--lr 1e-05
--lr_scale 100
--max_length 32
--batch_size 400
--max_iter 1000
--print_freq 100
--gpuid 1 & ''' but Acc of Representation with Kmeans is higher than Acc of model. I have no idea what the problem is.. Thank you for your help.

Issue #, if available:

Description of changes:

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

In section3.1 of the paper , contrastive loss = -log(pos/neg).but in code,contrastive loss = -log(pos/(neg+pos)).Why the denominator needs to add pos in the code?

Can you do early stopping whenever the losses stop decreases or the prediction stops changing much?

Originally posted by @Dejiao2018 in https://github.com/amazon-research/sccl/issues/22#issuecomment-1175308185

Sometimes, I can get a comparative score in a certain step, however, it is always convergent to a bad score. I don't know what is wrong, Can somebody helps?

I think both the place want to get the representation of sentences, why don't use the same function to get the same representation?Is there any benefit from doing this?

It seems a bug in sccl.utils.metric.Confusion's add function. Inself._conf_flat.index_add_(0, indices, ones), the indices's index comes error: IndexError: index out of range in self. Why do add operation : indices = (target*self.conf.stride(0) + pred.squeeze_().type_as(target)).type_as(self.conf) indices = (target*self.conf.stride(0) + pred.squeeze_().type_as(target)).type_as(self.conf)