Excuse me, do you test the PC Softmax on the same experiment setting of LADE about batch size and learning rate? I find that PC Softmax even get higher accuracy in some datasets such as ImageNet-LT, Places-LT and iNaturalist 2018. inital learning rate is 0.05

Hi, I'm very interested in your paper.

As seen from the paper, PC Softmax seems to be a strong baseline.

Can you provide the implementation of the PC softmax (mainly about how to calculate the p_{s}(y) and p_{t}(y) )?

Hi, thanks for your inspiring work. I have a small question about the PC Softmax after reading the paper.

In this paper, the logits of PC softmax is logit - logS + logT (Eq.4, was writen a bit casual, hope you can understand it). Thus it should be trained with standard CE, and add -logS + logT during inference. For the proposed LADE (or balanced softmax when alpha=0), the model was trained like: CrossEntropy(logit + logS).

I compared balanced softmax bellow. taken the class-balanced test set as example (where logT = const):

| method | train logits | test logits | train - test offset | | ---------------- | ------------ | ------------------- | --| | balanced softmax | logit + logS | logit + logT | - logS + logT | | PC softmax | logit | logit - logS + logT | -logS + logT |

In this table, Balanced-Softmax and PC-Softmax should be equivallent. In my opinion, Balanced Softmax learns logit + logS and PC-Softmax learns logit, and they should equal. The logS in Balanced Softmax seems like a "residual connection".

However, there is some performance difference between them as shown in paper. My experiments on CIFAR100-LT and ImageNet-LT (ResNet50) also shows the difference. Balanced Softmax is 0.9% higher on ImageNet-LT but PC Softmax is 1.3% higher on CIFAR100-LT. That confused me... Could you please share some opinion on the difference of the two kind of Softmax? Looking forward to your reply~

Thanks for the impressive work! I'd like to know whether the proposed method is compatible with margin loss (e.g. arc margin, cos margin, etc) and label smoothing? If yes, in which stage can we insert these techniques?

Hi, thanks for sharing the code of your great work. I am interested in your work and inspired a lot by it. When reproducing the PC softmax result on Cifar100-LT with imbalance factor 100, a gap appeared between result I reproduced and those reported in your paper.

I implemented PC softmax by using the PriorCEloss.py for training and it got an acc of 41.80. Then I changed to train by CE loss and directly add -log p_s(y) to the logits when predicting, and the acc got better to 42.3 but still lower than the result of 45.3 in the paper. I want to ask if you used other methods together with PC softmax or if I misunderstood the PC softmax to implement it wrongly.Or is there other reasons to the gap?

I will appreciate it getting some advice or hints from you. Thanks for sharing your awesome work again.

Thanks for your research, I got a lot of inspiration from yours. I'm trying to adapt the PC softmax strategy at the different domains (1d signal), but it's not working well as I thought.

My question is, I'm not sure my implementation of the PC softmax you suggested in the paper well.

For example, suppose that there is a model that gives the logit result value. During the training phase, we just use vanilla softmax. And on the inference phase, we will post-compensate (in other words, fix the logit value or something?) the logit value. For this, first, apply softmax on the model logit value and minus the log value of source data prior and plus the target data prior.

Describe it with the pseudo-code, (on the inference phase) output = argmax( softmax(logit) - logS + logT )

Now I'm implementing it with this code, and strangely, other losses work well (even LADE), but only PC softmax does not seem to work well.

If you check this implementation, it will be of great help to me.

Hi, I was really impressed with your research. Your code really helped my toy problem experiment and it works well. But I'm a bit confused about matching your paper and code.

I understood that PC softmax term (LADE-CE loss) is implemented in "loss/PriorCELoss.py". In the paper, equation 4 said that PC Strategy considers source and target probability. ( -log p_s(y) + log p_t(y) ) But in the "loss/PriorCELoss.py", it only considers prior (source probability) implemented with the function "calculate_prior".

If I misunderstood, it would be of great help to me if you let me know what was wrong.

Thank you!

Hello Sir,

I could solve my issues for LADE and balanced-softmax at your previous response.

But, I couldn't solve my issues completely.

I can understand your comment for LADE and balanced-softmax. (Just replacing loss function, I can try it. I can understand) Thanks...

But, still I didn't solve Causal-Norm in your code. Should I do create new network using feature-extraction layers and your classifier for other networks ??

Thanks, Edward Cho.

Hello Sir,

I checked DotProductClassifier.py and CausualNormClassifier.py in "models" folder of your source code.

If I use in other networks (ex: Inception_v1), for LADE or Balanced softmax or Causal Norm, How to use this function in other networks?? (Should I do create new network using feature-extraction layers and your classifiers? ) (at Training and at Inference)

Thanks, Edward Cho

Hi, thanks for your great work. I have a question related to random seeds. First, I tried the seed=1 and default hyperparameters, and the result of LADE is 52.2, which is actually lower than 53 in the original paper. I have no idea what is the difference. Then, I tried the seed=2 and default hyperparameters, and the result of LADE is 48.9. Does that mean this method is unstable? Thanks very much if you can reply.

Interesting and impressive work. Recently i am trying to reproduce the results given in the paper. However, i find different versions of pytorch would produce different results on several data sets. Could you give the specific versions of pytorch. Thank you