Can Action-Net run as an online inference version(frame by frame input) like TSM(https://github.com/mit-han-lab/temporal-shift-module/tree/master/online_demo)?

Hi, I'm a newbie. I would like to ask what is the code on the Jester dataset, something-something V2 dataset and generating the training, testing, and validating PKL files?

Hi, I have downloaded your jester-resnet pretrained model and jester annotations. I find the lables in test.pkl are all 0. When I run the test_jester_8f.sh, I get top1 =0, top5 = 100 witch mode in test.py is 'test' and use the jester_annotation test.pkl.

Hello, I would like to know what is the basis for your fine-tuning of the learning rate ? The experiment ? Each adjustment of a parameter, an experiment is performed ？

return [
            {'params': first_conv_weight, 'lr_mult': 1, 'decay_mult': 1, 'name': "first_conv_weight"},
            {'params': first_conv_bias, 'lr_mult': 2, 'decay_mult': 0, 'name': "first_conv_bias"},
            {'params': normal_weight, 'lr_mult': 1, 'decay_mult': 1, 'name': "normal_weight"},
            {'params': normal_bias, 'lr_mult': 2, 'decay_mult': 0, 'name': "normal_bias"},
            {'params': bn, 'lr_mult': 1, 'decay_mult': 0, 'name': "BN scale/shift"},
            {'params': custom_weight, 'lr_mult': 1, 'decay_mult': 1, 'name': "custom_weight"},
            {'params': custom_bn, 'lr_mult': 1, 'decay_mult': 0, 'name': "custom_bn"},
            # for fc
            {'params': lr5_weight, 'lr_mult': 5, 'decay_mult': 1, 'name': "lr5_weight"},
            {'params': lr10_bias, 'lr_mult': 10, 'decay_mult': 0, 'name': "lr10_bias"},
        ]

`

`

Hello! I want to ask a question about the paper.

Do you directly use the result of C3D:Resnext101 in their paper or train the model by yourself?

Thanks!

Hello, I follow your suggestion and this is my hyperparameters of 16 frames training.

python3 train.py --is_train --is_shift \ --dataset EgoGesture --clip_len 16 \ --shift_div 8 --wd 1e-5 --dropout 0 \ --cuda_id $cuda_id --batch_size 16 --lr_steps 10 15 20 \ --lr 25e-4 --base_model resnet50 --epochs 25

The val accuracy is quiet lower than 8 frames result,which is 91.7 and 8 frames is 94.4. I really want to reproduce the result of 16 frames in Egogesture. Is there any hyperparameter I need to change?

In Jester dataset, TSM result in Table 1 of your paper is 94.4 (ResNet50 8frame 3crop 10clip), But TSM result in origin paper(Table 1) is 97.0 (ResNet50 8frame full-resolution 2clips ). I test the TSM by myself from the TSM official public code and it can also reach 97.0. How cause this Gap between 94.4 and 97.0?

Thank you very much for providing the source code of this project, I want to use this project to experiment on the Something-Something V1, but the annotation file of the Something-Something V1 is in csv format, I am having difficulty with how to generate the pkl-file. Can you provide the code to generate the pkl-file or the pkl-file about the Something-Something V1. Thanks.

Hi, thank you for your great work. I tried to reproduce your work, but the machine was shut down. So, I tried to resume the training process, but the epoch started at 0. Is there any way to resume my training process from the checkpoint? (the epoch should start at the previous state)

code path is models/action.py, line63-72. line 63 I could konw x.size is nt, c, h, w. After x forward from line 63 to line 72, could get x_shift and thex_shift.size is nt, c,h,w. The x_shift.size is euqal to x.size. why don't make x_shift = x? I want to konw what function of these code.

Hello, after reading your code and paper, I am very interested in the visualization part of your model(Figure 1.). Your input is five-dimensional data which include time dimension. How do you call the CAM module? Or could open source the code about CAM you have adjusted, thanks a lot.

Thank you for your work. After I checked the source code, I have some questions. In model.py, I see that you use the shift operation of TSM model, but it is not explained in the paper. Is the test result obtained by adding the ACTION　module you proposed to the TSM model? Thanks for your reply.