It seems like the validation is on test_loader while not vali_loader, which is unfair to some extent and would make the results a little bit different.

https://github.com/thuml/Anomaly-Transformer/blob/72a71e5f0847bd14ba0253de899f7b0d5ba6ee97/solver.py#L196

Moreover, directly using thre_loader to find thresholds would cause test datasets leakage, since thre_loader is built on test_data while not valid_data.

https://github.com/thuml/Anomaly-Transformer/blob/72a71e5f0847bd14ba0253de899f7b0d5ba6ee97/solver.py#L254

https://github.com/thuml/Anomaly-Transformer/blob/72a71e5f0847bd14ba0253de899f7b0d5ba6ee97/data_factory/data_loader.py#L66-L69

Thanks for submitting such a great job, but I have some questions with prior-association of the code

https://github.com/thuml/Anomaly-Transformer/blob/72a71e5f0847bd14ba0253de899f7b0d5ba6ee97/model/attn.py#L48-L49

In this part, some mathematical processing is done on the sigma Could you please explain some reasons for doing this? I can not find any reference in your paper.

Hello, looking at other questions, I found that you mentioned that your environment is 3090, but I found that the cuda version supported by 3090 is above 11. How did you solve it?

Hi ! Thx for your attention. In your paper, I found some results inconsistent with the original paper of other methods, like "OmniAnomaly" and "InterFusion". Is there some thing different in experiment detail?

Hi, this is an amazing job. Here I come across a small problem. On MSL dataset, the model performed good, looks like: ======================TEST MODE====================== Threshold : 0.0017330783803481142 pred: (73700,) gt: (73700,) pred: (73700,) gt: (73700,) Accuracy : 0.9853, Precision : 0.9161, Recall : 0.9473, F-score : 0.9314

But after I annotated the "detection adjustment" code, the score was poorly, looks like: ======================TEST MODE====================== Threshold : 0.0017330783803481142 pred: (73700,) gt: (73700,) pred: (73700,) gt: (73700,) Accuracy : 0.8866, Precision : 0.1120, Recall : 0.0109, F-score : 0.0199

And I'm sure only the "detection adjustment" code was annotated.

Can you help me out of this problem? thx.

https://github.com/thuml/Anomaly-Transformer/blob/bfe075e4f3a0be789f168b2aeee7a4ce30482ce5/solver.py#L189-L192 When the first optimizer.step() execute all the gradient relate to loss1 will update，but some variable in loss2 are common in loss1.So this maybe cause some problem.

运行时会在实例化solver卡住，具体是在 self.build_model() -> self.model = AnomalyTransformer(win_size=self.win_size, enc_in=self.input_c, c_out=self.output_c, e_layers=3) -> self.encoder = Encoder(...) 并没有报错，只是运行到这里时就卡住了,不知道是什么原因导致的？ python 3.6, cuda 10.1, pytorch 1.4.0 GPU型号A100

Hello, When I run bash ./scripts/MSL.sh, I get the following error: Variable._execution_engine.run_backward( # Calls into the C++ engine to run the backward pass RuntimeError: one of the variables needed for gradient computation has been modified by an inplace operation: [torch.cuda.FloatTensor [512, 55]], which is output 0 of AsStridedBackward0, is at version 2; expected version 1 instead. Hint: enable anomaly detection to find the operation that failed to compute its gradient, with torch.autograd.set_detect_anomaly(True).

Thanks for the open source code! And there is a problem confuing me. When i running the code with [python main.py],there is a Pytorch error: RuntimeError: one of the variables needed for gradient computation has been modified by an inplace operation: [torch.cuda.FloatTensor [512, 38]], which is output 0 of AsStridedBackward0, is at version 2; expected version 1 instead. Hint: enable anomaly detection to find the operation that failed to compute its gradient, with torch.autograd.set_detect_anomaly(True).

Code error is located in solver.py,line 188 //Minimax strategy loss1.backward(retain_graph=True) self.optimizer.step() loss2.backward() self.optimizer.step()

when i change the code to loss1.backward(retain_graph=True) //self.optimizer.step() loss2.backward() self.optimizer.step()

It worked! I wonder if the first [self.optimizer.step()] should be annotated,if not, how to resolve the error. Thank you.

good work,i have a simple question. solver.py line 338 # detection adjustment in the detection adjustment stage, the code use gt to adjust pred result.But gt can't be used before auc test. Why?Is that ok to do this?

I want to know the details about what you experimented using NeurIPS-TS dataset. According to the paper, It includes univariate time series which have total length of 50,000. I couldn't find any information about how to generate or sample that train, test samples. Could you explain more detailed design of your experiment using this dataset? Is there any future plans to update the related codes?

hello, when i trained the model with the PSM data, i found that it is difficult to escape from early stop. is it not easy to get the best vali loss1 score and vali loss2 score at the same time so that training process stop so early? vali loss1 score= -rec_loss + AssDiss vali loss2 score= -rec_loss - AssDiss

I want to reproduce the performance of Anomaly-Transformer on the SWaT dataset, but I don't know how the SWaT dataset is preprocessed. Can you help me? Thank you!

Hello, I've encountered the exactly same issue as previous issue, and want to ask your opinion.

To provide more, Along with the code without point adjustment (commenting out the PA part as in previous issue), I personally got the following result:

| Dataset | F1-PA | F1-PA (paper) | F1 | | ----------- | ----------- | ----------- | ----------- | | MSL | 0.9500 | 0.9359 | 0.0209 | | PSM | 0.9750 | 0.9789 | 0.0217 | | SMAP | 0.9636 | 0.9669 | 0.0189| | SMD | 0.8944 | 0.9233 | 0.0201|

Although I agree that the F1-PA algorithm has practical justification (abnormal time point will cause an alert and further make the whole segment noticed in real-world applications.), (1) F1 seems to aggravate too much, and (2) AAAI paper raises concern about F1-PA metrics: even random guessing can achieive high F1-PA depending on data distribution.

I want to ask your opinion on these results. Thanks in advance.

Many thanks for your excellent work!

I would like to know about the details of the baselines and SOTA method codes. I can not find any official code for the LSTM, LSTM-VAE, BeatGAN, and THOC methods. Can you provide some links and code repo for us to reproduce?

Thanks again! Look forward to your kind reply!!

Hello,

I have a question regarding your work.

1. How validation set are selected here? from dataloader code, validation set seems to be test set. Did I get the codes right?
2. Anomaly Transformer's thresholding mechanism. Is the model using test dataset rather than validation set to set the threshold? From Appendix H, validation set (which is consisted of normal data only) is used to pick an appropriate threshold. However, the code seems to utilize test dataset for thresholding.
   Did I miss some points?

Thanks a lot for your answer in advance.