第一次在cos_forward函数里，if self.KEEP_FG会做一次softmax https://github.com/KaihuaTang/Long-Tailed-Recognition.pytorch/blob/90c8b2c0b66d17f78b67263861bc9d858fe20128/lvis1.0/mmdet/models/roi_heads/bbox_heads/convfc_bbox_head.py#L241

第二次在基类bbox_head的get_bboxes函数里 https://github.com/KaihuaTang/Long-Tailed-Recognition.pytorch/blob/90c8b2c0b66d17f78b67263861bc9d858fe20128/lvis1.0/mmdet/models/roi_heads/bbox_heads/bbox_head.py#L197 请问是我理解错了还是就是这样设计的哇？

尊敬的作者，您好，感谢您的工作并开源代码 我尝试使用您的方案训练自己的数据集（5类，训练集和测试集都是长尾分布，且同分布）

训练和模型参数

# default num_head = 2
criterions:
  PerformanceLoss:
    def_file: ./loss/SoftmaxLoss.py
    loss_params: {}
    optim_params: null
    weight: 1.0
last: false
# apply incremental pca to remove main components
apply_ipca: false
num_components: 512
model_dir: null
tuning_memory: false
networks:
  classifier:
    def_file: ./models/CausalNormClassifier.py
    optim_params: {lr: 0.001, momentum: 0.9, weight_decay: 0}
    scheduler_params: {coslr: false, endlr: 0.0, gamma: 0.1, step_size: 30, warmup: true, lr_step: [60, 80], lr_factor: 0.1, warm_epoch: 5}
    params: {dataset: GCAssCls, feat_dim: 128, num_classes: 5, stage1_weights: false, use_effect: true, num_head: 2, tau: 16.0, alpha: 1.0, gamma: 0.03125}
  feat_model:
    def_file: ./models/ResNet18Feature.py
    fix: false
    optim_params: {lr: 0.001, momentum: 0.9, weight_decay: 0}
    scheduler_params: {coslr: false, endlr: 0.0, gamma: 0.1, step_size: 30, warmup: true, lr_step: [60, 80], lr_factor: 0.1, warm_epoch: 5}
    params: {dataset: GCAssCls, dropout: 0.5, stage1_weights: false, use_fc: True, fc_channel: 128, pretrained: True}
shuffle: false
training_opt:
  backbone: resnet18
  batch_size: 128
  dataset: GCAssCls
  display_step: 10
  display_grad: False
  display_grad_step: 10
  feature_dim: 128
  log_dir: ./logs/GCAssCls/models/resnet18_e100_C5_warmup_causal_norm_lr1e-3_adam
  log_root: /logs/GCAssCls
  num_classes: 5
  num_epochs: 100
  num_freeze_epochs: 6
  num_workers: 12
  open_threshold: 0.1
  sampler: null
  sub_dir: models
  optimizer: adam

我使用了ResNet18作为backbone，并使用了ImageNet上的pretrain model，训练出来的结果，并在backbone连接了fc和dropout，迭代100个epoch，在训练集上的准确率基本到99.9%，此时获得模型在测试集上表现很差 classifice 中 use_effect 置为 false 是测试 不含 TDE 的结果。

测试在长尾测试集上的结果

几点疑问

• 添加了TDE后，many-shot类准确率大幅下降，few-shot类的召回率有所提升，请问这正常吗？
• 论文分类任务中test数据集都是类别均衡分布的数据集，如 ImageNet-LT 每个类有50个样本，cifar-10/100只在训练时进行非均衡采样，而实际生产场景中的分类任务很多本身也是长尾分布的，在收集的训练集基本同应用场景同分布，在与实际场景不同分布的情况下去评估模型的性能有违初衷？

def update_embed(self, targets, gt_label): if self.training: # remove background with torch.no_grad(): fg_target = targets[gt_label > 0].clone().detach().mean(0, keepdim=True) self.causal_embed = self.MU * self.causal_embed + fg_target return

新版mmdet中0-(num_classes-1)才是正样本，上面gt_label > 0是不是错了？

Hello, i have added CausalNormClassifier to my own project, and recorded embed mean. However, i meet the problem that the embed mean grow to inf. I print torch.sum(embed_mean)

tensor(1902.6863, device='cuda:0')
Train:   2%|                                   | 1/45 [00:07<05:36,  7.65s/it]
tensor(8754.8096, device='cuda:0')
Train:   4%|                                   | 2/45 [00:06<03:03,  4.27s/it]
tensor(33422.0312, device='cuda:0')
Train:   7%|                                  | 3/45 [00:08<02:47,  3.99s/it]
tensor(122225.3984, device='cuda:0')

the embed_mean is updated as follows:

self.embed_mean = torch.zeros(int(self.training_opt['feature_dim'])).numpy()
self.embed_mean = self.mu * self.embed_mean + self.features.detach().mean(0).view(-1).cpu().numpy()

During the train process, the gradient will be small and small, so the the velocity will not grow to inf. But the feature which generated by model may not be small and small, so it seems to grow to inf. So i can't store this variable. How can i solve this problem? Or anything i miss?

What does training_opt: {open_threshold: 0.1} do? It looks like, it is pointing to the theta below but it is not being used at all.

def F_measure(preds, labels, theta=None):
    # Regular f1 score
    return f1_score(labels.detach().cpu().numpy(), preds.detach().cpu().numpy(), average='macro')

In the code the moving average is like this, https://github.com/KaihuaTang/Long-Tailed-Recognition.pytorch/blob/90c8b2c0b66d17f78b67263861bc9d858fe20128/classification/run_networks.py#L215

In the above scenario, more importance is being given to newer epochs, in that case, why can't we just use the final epoch's model? Or is there any other rationale behind it?

Should the line have been like the following instead ? self.embed_mean = self.mu * self.embed_mean + (1-self.mu)*self.features.detach().mean(0).view(-1).cpu().numpy()

where the self.mu = 0.9.

Why is there a moving average of the embedding out of the feature before it goes into the fully connected network? What is it used for? https://github.com/KaihuaTang/Long-Tailed-Recognition.pytorch/blob/540cae6da251f49b1b021630e219dde7c5867ce2/classification/run_networks.py#L215

1. Is feat_uniform.yaml just normal training?
2. For ResNet32Feature.py, why are you using the BBN_ResNet_Cifar class instead of using the parent class ResNetFeature ?

Hey @KaihuaTang, I am an NLPer. Thanks for your interesting work. I've observed that you got the same cofigs regarding the parameters in class Causal_Norm_Classifier, namely num_head, tau, alpha and gamma, for both CIFAR and ImagNet datasets. How come? Could you also please explain the idea of that kinda assginment? Thanks a lot.

作者你好，最近才看到这篇关于长尾数据分类的论文。有一些地方不是很明白。 1)论文指出验证或测试时才需要消除头部偏移的影响，那么消除头部偏移影响后，模型在训练集上的表现如何？ 2)该偏移可以看作零输入的特征输出，那么能否将网络中所有卷积和BatchNormalization的bias均设置为否？ 3) alpha的含义是什么，为什么可以大于1

谢谢

您好，首先非常感谢您solid的工作。

但在我看具体代码时，有一些问题想要请教：

在 lvis1.0\mmdet\models\roi_heads\bbox_heads\convfc_bbox_head.py 文件中的 update_embed 函数里：

        if self.training:
            # remove background
            with torch.no_grad():
                fg_target = targets[gt_label > 0].clone().detach().mean(0, keepdim=True)   
                self.causal_embed = self.MU * self.causal_embed + fg_target    
        return

就我理解而言，这里是给滑动平均值加上了特征图的（fg_target）的数值，而非梯度值，与下图式中的 g_t 不符。请问是我对代码的理解有误吗？可否指正一下，非常感谢！

Hello. I found in ImbalanceCIFAR.py, the validation set and the test set seem to be the same. Is it better to use a balanced training set (50000 frames) as the validation set instead of using the 10000 frame test set for validation? Thanks!

Thank you for your excellent work. I want to try to migrate the classifier (CausalNormClassifier) in this work to the task of multi label long tailel recognition. However, the following problems appeared. I believe the model has not learned any knowledge at all, perhaps the output is smoothed out in the inference stage. Because the loss change during training seems normal. Can you give me some advice or tell me the possible problems?

感谢你开源这么优秀的工作，我想尝试把这个工作中的分类器迁移到多标签长尾数据分类任务中去，但是却出现了如下的问题，感觉模型完全没有学习到任何知识，或者好像是在推理的阶段输出被平滑抵消掉了？因为在训练过程中的loss变化看起来是正常的。能否给我一些建议或者告诉我可能出现的问题？