Can't run the train_predict.py, after i download the whole file and the datasets, then when i tried to run train_predict.py it stopped. Is there anything that i do it wrong? Thanks 👍

While training, during loading of batches, in the 3rd batch I'm facing this error. I tried to modify the random seed but it's stuck at 3rd batch specifically.

File "/lustre/fs0/home/akumar/self/prp/datasets/predict_dataset.py", line 77, in getitem videodata, sample_step_label = self.loadcvvideo_Finsert(index, recon_rate=recon_rate, sample_step=None) File "/lustre/fs0/home/akumar/self/prp/datasets/predict_dataset.py", line 179, in loadcvvideo_Finsert buffer, sample_step_label = self.loadcvvideo_Finsert(index, recon_rate, sample_step) File "/lustre/fs0/home/akumar/self/prp/datasets/predict_dataset.py", line 132, in loadcvvideo_Finsert sample_step_proposal = self.sample_retrieval[recon_rate] KeyError: 1

recon_rate is 1 I tried try and except to skip the video causing the error but then again new error pops up. The video path is correct but there's some error with the buffer length I'm facing. retaining:False buffer_len:106 sample_len:128

If I skip this video, then at this line https://github.com/yuanyao366/PRP/blob/58a301d92a540c915296de0d60a1cbaa304f0819/datasets/predict_dataset.py#L71

it's showing me NoneType object error.

If I change the sample step from 1,2,4,8 to 1,2,4 or 1,2, do I need to modify some lines in pat_region.py. If so, can you point it out? I changed the sample step list and again the loss nan error starts coming in each epoch at some iteration over a batch. Using kinetics dataset for pre-training.

I have no problems in code just have some doubts about weight loading

1. Why is there a difference in weight loading in train_predict.py and ft_classfy.py?
2. Did you use nn.Dataparallel even with 1 GPU to add a "module" in the starting of parameters and save it like that to maintain consistency of each checkpoint?
3. If we use 2 networks, one successive to the other, the previous network weights are saved as module.base_network?
4. Does it mean we modify the base network (c3d, r21d, r3d) parameters according to the pretext task as in whole? and then use only those weights for finetuning on the downstream task dataset?
5. In ft_classfy load_pretrained _weights function why did you modify it to module.base_network and +14:. I mean since the strict is False, so it will upload only those weights which are common to both the networks? I mean only base network? What's the problem in this?

def load_pretrained_weights(ckpt_path):
    """load pretrained weights and adjust params name."""
    adjusted_weights = {}
    pretrained_weights = torch.load(ckpt_path)
    for name, params in pretrained_weights.items():
        if 'base_network' in name:
            name = name[name.find('.')+1:]
            adjusted_weights[name] = params
    return adjusted_weights

Thanks.

Hi, I am facing an issue trying to run the finetuning task for PRP.

Exact Command used: python ft_classfy.py --gpu 0 --model_name c3d --pre_path 0 --split 1

The error is added as image attachment

Steps before running this command:

1. The environment contains the more recent versions of packages
2. Modified dataset locations throughout the project according to my folder structure
3. With these changes, training on pretext task worked fine. Ran command: python3 train_predict.py --gpu 0 --epoch 300 --model_name c3d (Due to resource restriction, terminated after 147 epochs)
4. Added the path to the best model in file ft_classfy.py in pretrain_path0. From the output text file, it seems that this is best model location is being picked up as the model.

Solutions attempted:

1. To make sure the issue is not due to train_predict terminating before completing 300 epochs, I ran it for 2 epochs and tried with the best model obtained there too.
2. Similar error first occurred with the mode attribute when the code hit the part "if self.mode == 'train'" in the class ClassifyDataSet(data.Dataset) in predict_dataset.py. This got resolved once I moved the self.mode=mode above the said condition check.
3. To me, the attribute error regarding dataset looked like an issue of child class not having the attributes of the parent, so I also tried super().init() and super().init(*args, **kwargs) without any success.

Can you please advise regarding this, specially if I have missed any step after running the train_predict? Is there any other area of the project which is to be modifed to add the pretrained model path apart from pretrain_path0 in ft_classfy.py? I have also added a snippet with the last part of the output file (separate from the error file)

Please let me know if any other information would be helpful. Thank you in advance.

Hi,

I was running your code and after few epochs, nan loss started appearing. I'm sharing it from epoch 99 but it started appearing with epoch 5 or so.

Epoch:[99][200/278] data_time:0.128,batch time:1.571 loss:nan loss_recon:nan loss_class:nan accuracy:27.125 [TRAIN] loss_cls: nan, acc: 0.266 tensor([2367., 0., 0., 0.]) tensor([2367., 2146., 2168., 2215.]) tensor([1., 0., 0., 0.]) 33%|3 | 99/300 [27:19:23<56:29:50, 1011.89s/it][VAL] loss_cls: nan, acc: 0.292 tensor([467., 0., 0., 0.]) tensor([467., 363., 385., 385.]) tensor([1., 0., 0., 0.]) WARNING:root:NaN or Inf found in input tensor. WARNING:root:NaN or Inf found in input tensor. WARNING:root:NaN or Inf found in input tensor. WARNING:root:NaN or Inf found in input tensor. WARNING:root:NaN or Inf found in input tensor. WARNING:root:NaN or Inf found in input tensor. WARNING:root:NaN or Inf found in input tensor. WARNING:root:NaN or Inf found in input tensor. WARNING:root:NaN or Inf found in input tensor. WARNING:root:NaN or Inf found in input tensor. WARNING:root:NaN or Inf found in input tensor. WARNING:root:NaN or Inf found in input tensor.

conv_lr:0.001 fc8_lr:0.010000000000000002 Epoch:[100][100/278] data_time:0.126,batch time:1.425 loss:nan loss_recon:nan loss_class:nan accuracy:26.969

I'm unable to figure out where did I went wrong? What should I modify? I'm working on the Kinetics dataset

When I trained on UCF 101 dataset, this didn't happen. I checked for 90 epochs and the pretext task accuracy also increased, here it's stuck at 26%.

hi,
when training in self-supervised and fine-tune stage. your work dont use softmax after linear layer, which some other related work is used. So,my question is,: In training, what is the difference when softmax is applied or not. Best wishes.

I think the logic is not strict.

In the predict_dataset.py, I list several lines in the following.

count_need=16
frame_count = int(capture.get(cv2.CAP_PROP_FRAME_COUNT))
start = np.random.randint(0, frame_count - count_need + 1)

There is no garantee that the frame_count is larger than the default setting 16, and it will report an error here.

File "/raid/home/taoli/exp/prp/datasets/predict_dataset.py", line 222, in loadcvvideo
    start = np.random.randint(0, frame_count - count_need + 1);
  File "mtrand.pyx", line 993, in mtrand.RandomState.randint
ValueError: low >= high

When I add the following codes

if frame_count - count_need + 1 < 1:
    start = 0
else:
    start = np.random.randint(0, frame_count - count_need + 1)

It will print too many reload msg in the command line.

I am facing an error when loading the state_dict. It states that there are missing keys ("linear.weight", "linear.bias") as well as unexpected keys. See attached image for error details.

Please note that this particular screenshot is from when I try C3D. Similar error happens when I try to load R3D or R(2+1)D. Please advise. Thank you in advance.

Based on your repo, it seems that you are using the same decoder structure for all of the backbones(c3d, r3d, r(2+1)d). But in your paper, it seems you used different decoder structure based on C3D-block, R2D-block and R21D-block.

We cannot reproduce the result reported in the paper base on current code. Could you also provide your decoder implementation of r3d, r(2+1)d?