Hello, Yu Lei, I would like to ask whether there should be two teacher models for the CSS part, and then predict the mixing. It seems that I don't see this part of the code in the code. Maybe I didn't notice. Can you tell me what this part of the code is.

Hello Yulei, thank you for your paper and code, which benefit me a lot. When I tried to run cfvqa, I may have encountered some problems due to my incorrect usage. I hope that you could answer them. After training the teacher model, the student model encountered the following error when loading the Optimizer:

[I 2022-12-04 22:48:30] ...fvqa/engines/engine.py.87: Loading last checkpoint
[I 2022-12-04 22:48:30] ...fvqa/engines/engine.py.395: Loading model...
[I 2022-12-04 22:48:30] ...fvqa/engines/engine.py.401: Loading optimizer...
[I 2022-12-04 22:48:30] ...qa/cfint1/cfvqa/run.py.120: Traceback (most recent call last):
  File "/mnt/home/lxpvqa/cfint1/cfvqa/run.py", line 113, in main
    run(path_opts=path_opts)
  File "/mnt/home/lxpvqa/cfint1/cfvqa/run.py", line 92, in run
    engine.resume()
  File "/mnt/home/lxpvqa/cfint1/cfvqa/cfvqa/engines/engine.py", line 91, in resume
    map_location=map_location)
  File "/mnt/home/lxpvqa/cfint1/cfvqa/cfvqa/engines/engine.py", line 403, in load
    optimizer.load_state_dict(optimizer_state)
  File "/usr/local/lib/python3.6/dist-packages/block/optimizers/lr_scheduler.py", line 123, in load_state_dict
    self.optimizer.load_state_dict(state['optimizer'])
  File "/usr/local/lib/python3.6/dist-packages/torch/optim/optimizer.py", line 124, in load_state_dict
    raise ValueError("loaded state dict contains a parameter group "
ValueError: loaded state dict contains a parameter group that doesn't match the size of optimizer's group

The run commands I used are as follows:

python -m bootstrap.run -o cfvqa/options/vqa2/smrl_cfvqa_sum.yaml
#mkdir ./logs/vqa2/smrl_cfvqaintrod_sum/
cp -r ./logs/vqa2/smrl_cfvqa_sum/ ./logs/vqa2/smrl_cfvqaintrod_sum/
python -m run -o ./cfvqa/options/vqa2/smrl_cfvqaintrod_sum.yaml

The following shows the contents of the optimizer in smrl_cfvqaintrod_sum:

optimizer:
  import: cfvqa.optimizers.factory
  name: Adam
  lr: 0.0003
  gradual_warmup_steps: [0.5, 2.0, 7.0] #torch.linspace
  gradual_warmup_steps_mm: [0.5, 2.0, 7.0] #torch.linspace
  lr_decay_epochs: [14, 24, 2] #range
  lr_decay_rate: .25

Could you please tell me how to solve this problem? Thanks a lot.