Thank you very much for your help on the datasets! Can I run it on two 3090ti GPUs(24GB)? I changed args.num_gpus and args.num_workers to 2, 1 respectively in run.py, but it failed. Did I miss anything else?

I RUN THIS :python run.py -c checkpoint_dir --query_per_class 4 --shot 5 --way 5 --trans_linear_out_dim 1152 --tasks_per_batch 16 --test_iters 75000 --dataset ssv2 --split 3 -lr 0.001 --method resnet50 --img_size 224

Options: Namespace(checkpoint_dir='checkpoint_dir', dataset='ssv2', debug_loader=False, img_size=224, learning_rate=0.001, method='resnet50', num_gpus=2, num_test_tasks=10000, num_workers=1, opt='sgd',

/trx-main/model.py", line 94, in forward class_k = torch.index_select(mh_support_set_ks, 0, self._extract_class_indices(support_labels, c)) RuntimeError: Input, output and indices must be on the current device

Best wishes, Han

Thanks a lot for making your code public. I am trying to replicate your results on SSv2 and HMDB51. This was my first foray into video action recognition. Some problems happen when i work with datasets. 
1. I get SSv2 from TwentyBN  https://20bn.com/datasets/download, but when i use cat 20bn-something-something-v2-?? | tar zx, it brings a mistake:
    localhost dataset]$ cat 20bn-something-something-v2-?? | tar zx
    gzip: stdin: not in gzip format
    tar: Child died with signal 13
    tar: Error is not recoverable: exiting now
2. As you said, "Download your chosen dataset, and extract frames to be of the form dataset/class/video/frame-number.jpg", does this mean that I should put all the extracted frames into the ZIP file ? (args.path = os.path.join(args.scratch, "video_datasets/data/hmdb51.zip") or this zip file should consist of all the videos ? 
3. How can i decompose the video into frames? I cannot see the way ARN dealt with.
It would be greatly appreciated.

Best wishes!

Thanks for sharing the code. Could you please tell us how we could use validation set at the code? The code only reads train and test lists. I would like to select best model using the validation set and then test the best model in the test list. It would be greatly appreciated.

Thanks a lot for making your code public. I am trying to replicate your results on UCF-101 and HMDB. I have the following queries regarding them and the code in general:

1. I have a similar GPU configuration as you have reported on your Github(4 GPUs of 11GB) each but I am not able to run the code for either HMDB or UCF-101. When I use resnet34 instead of resnet50, the sizes have decreased and I am able to run properly. But, does the accuracy get affected by doing this? My code for training is:

python3 run.py -c checkpoint_dir --query_per_class 5 --shot 5 --way 5 --trans_linear_out_dim 1152 --test_iters 75000 --dataset ucf --split 3 -lr 0.001 --img_size 224 --scratch new --num_gpus 4 --method resnet34 -r --print_freq 1 --save_freq 1000

2. In the script, you have used confidence:

https://github.com/tobyperrett/trx/blob/390c20aa8ed7e309ae4949464fdcf015c4009329/run.py#L225

Why have you used 196.0. I checked the paper but couldn't find the reason behind this. It will be great if you can explain the same. What is the reason behind the confidence metric?

3. A general doubt, when can I say that model has started overfitting in this context of the few-shot video classification? I know that evaluation is performed for 10000 tasks but in training, the default number of tasks is 100K. Should I stop after I see that the losses have started approaching values of 0.0 decimal?

Thanks for clasrifying the validation part

I use your code as suggestion(replace dataload with mine,use random sample and randomw flip as augmentation,size of image is 3112112),but accuracy(5 way 5 shot) only achieve 78.5% after 10000 meta-train, and 76.9% after 50000 meta-train. It's far away from 85.9%.

Are there any other importance process in your dataload?

ps: loss function in code seems to be wrong(will increase to Nan as iteration),so I replace it with a common CrossEntropy Loss.

In the few-shot learning setting, a dataset is split into training, validation, and testing splits. However, all models are trained on training splits, and the validation set is used for validation, and the test set is ONLY used for computing the final result (because we do not know the specific distribution on the test set). Common evaluation: Select the best model (higher accuracy) on the val set when training, and employ this model on the test set (only once). And some works select the N episodes model, but the hyperparameter N is obtained by val set.

However, your implementation does NOT use the val set, and meanwhile, the test set is used repeatedly.

And it is unfair to compare to previous works.

Thanks for the code again. When you are are changing the parameters for lets say 20-way 1-shot are you changing the parameters of query_per_class, query_per_test? Or should we just change --way and --shot parameters and leave the others as default? changing; parser.add_argument("--way", type=int, default=5, help="Way of each task.") -> 20 parser.add_argument("--shot", type=int, default=5, help="Shots per class.") -> 1 not changing; parser.add_argument("--query_per_class", type=int, default=5, help="Target samples (i.e. queries) per class used for training.") parser.add_argument("--query_per_class_test", type=int, default=1, help="Target samples (i.e. queries) per class used for testing.") Thanks for your help!

Thanks for your published work! When I reproduce your results on the Kinetics-100 dataset (with a total of 100k iterations), I find that the accuracy of the model will drop later. When will the performance of the model reach the best in the training process？ In addition, can you give me more details about the number of training iterations and the steps to drop the learning rate on both SSV2 and Kinetics-100 datasets. Thank you again for your help！

Thanks for sharing your code. can you please provide the scrips that tell us the value of the arguments that you used for training/testing. It would be greatly appreciated.

Hello, I would like to ask how many frame rates are set to capture a picture for the 4 different datasets of video data when building the dataset. Thank you very much!

Hello, thanks for the amazing work! It would be great if you could share the trained checkpoint of the model. Could you provide a google drive link or something like that?

Thanks

Hey there. In your paper, you guys have stated CrossTransformers' potential weakness is that relative spatial information is not encoded. (in Related Work)

What is the rationale behind this? Have you guys done some experiments proving this or…?

Thanks for sharing your code. I use your code to train on UCF101 with the suggested hyper-parameters (i.e., lr=0.001, trans_linear_out_dim=1152， img_size=224, tasks_per_batch =16, num_test_tasks=10000) and the same data loader. However, the 5-way 5-shot accuracy can only attain 94.8%, which is lower than your reported 96.1%. Are there any other tricks you used to improve the performance? Or I use the wrong hyper-parameters?

Hi, I was able to only acquire 92% of the total kinetics-100dataset in this link（https://github.com/ffmpbgrnn/CMN）, the rest were not available.

Could you please send me the download link for the split

Much thanks!