Hello,

Thank you for your amazing work !

I was wondering if there is any particular reason for imposing a batch size of 1 in model.py: https://github.com/ChinaYi/ASFormer/blob/89e72d840a3d3eb8f16c270adb5031d648d9fb73/model.py#L138

In my testing, ASFormer learns fine with bigger batch sizes.

您好，您提到的层次注意力是不是指的是band attention（如下图所示），只不过随着层数增加，窗口大小指数递增。这样的话model.py里这个函数里的那个for循环内容，是不是应该改为window_mask[:, i, i:i+self.bl] = 1

def construct_window_mask(self):
    window_mask = torch.zeros((1, self.bl, self.bl + 2* (self.bl //2)))
    for i in range(self.bl):
        window_mask[:, :, i:i+self.bl] = 1
    return window_mask.to(device)

Hi, can you provide more informations about the feature extraction? I would like to use this fantastic model on my dataset but I don't know how to extract the features to feed to the encoder.

Hi thanks for your work. I was able to train and test the model and achieve similar performance as mentioned in the paper when I use both enc and dec. However, when I don't use the decoder, the results are much worse than what is mentioned in the table 5 (first row). I was wondering if I need to do any changes to the setting to get the same performance (specially for Acc)? I notice that without using the decoder the acc drops lower than 80.

Hi,

Thanks for sharing the code. I noticed the bg_class in the evaluation code is not properly set.

The default name of background class is set to background, which is true in GTEA yet need to be changed to SIL for breakfast and action_start and action_end for 50salads. It seems they are not changed for the results in the paper.

With the correct class name and the released model, I obtained a lower result | | [email protected] | [email protected] | [email protected]| |-|---------|----------|---------| |Breakfast|70.9 |67.5 |56.7 | |50salads|83.7|81.8|73.7|

Hi, Thank you for your work. When I try to increase the batch size, the index drops a lot. What do you think are the possible reasons

the GPU is A100 40g.

train with default setting, only split 1

(s1)[83.40807175 81.16591928 72.19730942] 75.934108 83.2241

just change batch size to 8 lr 0.001 and then train

(s1)[68.94977169 67.57990868 55.25114155] 63.931922 72.0049

Hi ! Firstly, thanks for sharing this repo ! I'm struggling to download the model (3. Download the pre-trained models at (https://pan.baidu.com/s/1zf-d-7eYqK-IxroBKTxDfg)) Indeed, the site says that you need to create an account to download the file. The thing is I cannot create an account with a french phone number 😅 Any other way to download the pretrained model ? Many thanks !

Hello,

I am adapting your code for my own dataset which usually train relatively fast when using only ASRF, but when using your model with the transformer it's taking approximately 10x times longer. Do you have a similar behaviour with Salad/breakfast/gtea datasets ?

Thank you :)

I installed the environment as you asked: Pytorch == 1.1.0, torchvision == 0.3.0, python == 3.6, CUDA=10.1

It is certain that the model is loaded because the model size is printed: Model Size: 1130860

But the problem is: Traceback (most recent call last): File "main.py", line 99, in trainer.predict(model_dir, results_dir, features_path, batch_gen_tst, num_epochs, actions_dict, sample_rate) File "/home/cpslabrtx3090/zjb/projects/ASFormer/model.py", line 399, in predict self.model.load_state_dict(torch.load(model_dir + "/epoch-" + str(epoch) + ".model")) File "/home/cpslabrtx3090/anaconda3/envs/ASFormer/lib/python3.6/site-packages/torch/serialization.py", line 387, in load return _load(f, map_location, pickle_module, **pickle_load_args) File "/home/cpslabrtx3090/anaconda3/envs/ASFormer/lib/python3.6/site-packages/torch/serialization.py", line 560, in _load raise RuntimeError("{} is a zip archive (did you mean to use torch.jit.load()?)".format(f.name)) RuntimeError: ./models/gtea/split_1/epoch-120.model is a zip archive (did you mean to use torch.jit.load()?)

Traceback (most recent call last): File "/home/cpslabrtx3090/anaconda3/envs/ASFormer/lib/python3.6/tarfile.py", line 189, in nti n = int(s.strip() or "0", 8) ValueError: invalid literal for int() with base 8: 'ld_tenso'

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "/home/cpslabrtx3090/anaconda3/envs/ASFormer/lib/python3.6/tarfile.py", line 2299, in next tarinfo = self.tarinfo.fromtarfile(self) File "/home/cpslabrtx3090/anaconda3/envs/ASFormer/lib/python3.6/tarfile.py", line 1093, in fromtarfile obj = cls.frombuf(buf, tarfile.encoding, tarfile.errors) File "/home/cpslabrtx3090/anaconda3/envs/ASFormer/lib/python3.6/tarfile.py", line 1035, in frombuf chksum = nti(buf[148:156]) File "/home/cpslabrtx3090/anaconda3/envs/ASFormer/lib/python3.6/tarfile.py", line 191, in nti raise InvalidHeaderError("invalid header") tarfile.InvalidHeaderError: invalid header

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "/home/cpslabrtx3090/anaconda3/envs/ASFormer/lib/python3.6/site-packages/torch/serialization.py", line 556, in _load return legacy_load(f) File "/home/cpslabrtx3090/anaconda3/envs/ASFormer/lib/python3.6/site-packages/torch/serialization.py", line 467, in legacy_load with closing(tarfile.open(fileobj=f, mode='r:', format=tarfile.PAX_FORMAT)) as tar,
File "/home/cpslabrtx3090/anaconda3/envs/ASFormer/lib/python3.6/tarfile.py", line 1591, in open return func(name, filemode, fileobj, **kwargs) File "/home/cpslabrtx3090/anaconda3/envs/ASFormer/lib/python3.6/tarfile.py", line 1621, in taropen return cls(name, mode, fileobj, **kwargs) File "/home/cpslabrtx3090/anaconda3/envs/ASFormer/lib/python3.6/tarfile.py", line 1484, in init self.firstmember = self.next() File "/home/cpslabrtx3090/anaconda3/envs/ASFormer/lib/python3.6/tarfile.py", line 2311, in next raise ReadError(str(e)) tarfile.ReadError: invalid header

Hi, Thank you for your code https://github.com/ChinaYi/ASFormer/blob/3940443a7dca336f879c43a42bcf91c5bf7c790f/model.py?_pjax=%23js-repo-pjax-container%2C%20div%5Bitemtype%3D%22http%3A%2F%2Fschema.org%2FSoftwareSourceCode%22%5D%20main%2C%20%5Bdata-pjax-container%5D#L370 When I change the test interval from 10 to 20, 30, etc., different results(such as training loss ) are obtained under the same seed. What do you think is the reason？ best regards

Thanks for you nice work, meanwhile, may I confirm one thing? By using your features and pre-trained models (epoch=120), the obtained scores are lower than your BMVC paper for three datasets. For instance, the edit and F1@10 of gtea can only reach 84.0 and 88.9, which are lower than 84.6 and 90.1 in your paper. Same for another two datasets. 50salads edit=75.7, F1@10=83.4.