When I run the test phase of this project(mvtec/carpet)，I got this error:

train.py", line 333, in test_step anomaly_map = score_patches[:,0].reshape((28,28)) ValueError: cannot reshape array of size 1024 into shape (28,28) Testing: 0%| | 0/117 [00:09<?, ?it/s] Any ideas? thank you !

Hi,I find that the detection time grows longer as the training set grows,Is that reasonable? The problem seems to related to this line of code, https://github.com/hcw-00/PatchCore_anomaly_detection/blob/5680042fd114abbfbef820edef1f61d566a128fd/train.py#L280 Or is that a problem?Thanks

Hi, I am trying to train the model in dataset, however I find that the speed is getting more and more slowly. I noticed that the embedding size is getting bigger, so whether we can change it? Thanks.

and it needs more small gpu-memory.

There is a memory overload in torch.pow(x-y,p).sum() When I try with this code, I already reserve 8GB and try to allocate another 8GB.

I have a RTX 3080 GPU with 10GB memory. And when I input big image (3,896,896), distance matrix needs 14.36GB but torch.cdist works well.

It means we can use more large sampling rates by using torch.cdist function. I can run sampling rate = 0.01 with torch.cdist but I got CUDA out of memory with distance_matrix.

Actual GPU memory usages with MVTec AD. https://github.com/hcw-00/PatchCore_anomaly_detection/issues/19#issuecomment-962706455

Thanks for this great implementation. In the paper the authors claim in "Evaluation on other benchmarks":

As the detection context is much closer to that of natural image data available in ImageNet and images are larger, we make use of deeper network feature maps at hierarchy levels 3 and 4, but otherwise do not perform any hyperparameter tuning for PatchCore.

I tried to simply swap layer2 and 3 to 3 and 4 and had to adapt the reshaping later on. But the results were a complete failure. The heatmap is always blank.

Has someone tried to use layer4 successfully?

Hi I run the train.py under following env:

1. Ubuntu 16.04, docker env: pytorch 1.8.1 with GPU support
2. Machine Memory: 32G, swap area size: 12GB; GPU: GF RTX 2080 Ti 12 GB
3. Num of epochs:50, batch_size:1, load_size: 256, input_size:224, with other parameters default/unchanged
4. Trainning dataset consists of 150 non-defect images with 1300*2300 resolution
5. Test dataset consists of 100 defect and non-defect images, with ground truth images where a pure black for non-defect images and write musk for the defect areas for defect images

Problem: After epoch 11, the machine memory (not GPU memory) takes up to 40GB, and at epoch 15, the usage of machine memory would be over 50GB, then the system would kill the training process.

Any ideas why this happen, and how to fix?

As described in title, the data from real world without GT_mask, but almost every algorithms need gt_mask to get roc_auc_score. Do you try to decide the threshold of anomaly score which higher than the threshold will be thought as anomaly ? Other than obtained the optim threshold from GT ? The problem confiused me all the time.

Hello, I have try to run your code by your sample cmd

python train.py --phase train or test --dataset_path .../mvtec_anomaly_detection --category carpet --project_root_path path/to/save/results --coreset_sampling_ratio 0.01 --n_neighbors 9

But I found coreset_sampling_ratio 0.01 will make error so I try use default parameters 0.001 and this is correctly work.

Error code

Error report is : Traceback (most recent call last): File "train.py", line 441, in <module> trainer.fit(model) File "/home/neaf-3090/.pyenv/versions/3.6.8/lib/python3.6/site-packages/pytorch_lightning/trainer/trainer.py", line 458, in fit self._run(model) File "/home/neaf-3090/.pyenv/versions/3.6.8/lib/python3.6/site-packages/pytorch_lightning/trainer/trainer.py", line 756, in _run self.dispatch() File "/home/neaf-3090/.pyenv/versions/3.6.8/lib/python3.6/site-packages/pytorch_lightning/trainer/trainer.py", line 797, in dispatch self.accelerator.start_training(self) File "/home/neaf-3090/.pyenv/versions/3.6.8/lib/python3.6/site-packages/pytorch_lightning/accelerators/accelerator.py", line 96, in start_training self.training_type_plugin.start_training(trainer) File "/home/neaf-3090/.pyenv/versions/3.6.8/lib/python3.6/site-packages/pytorch_lightning/plugins/training_type/training_type_plugin.py", line 144, in start_training self._results = trainer.run_stage() File "/home/neaf-3090/.pyenv/versions/3.6.8/lib/python3.6/site-packages/pytorch_lightning/trainer/trainer.py", line 807, in run_stage return self.run_train() File "/home/neaf-3090/.pyenv/versions/3.6.8/lib/python3.6/site-packages/pytorch_lightning/trainer/trainer.py", line 869, in run_train self.train_loop.run_training_epoch() File "/home/neaf-3090/.pyenv/versions/3.6.8/lib/python3.6/site-packages/pytorch_lightning/trainer/training_loop.py", line 556, in run_training_epoch self.on_train_epoch_end(epoch_output) File "/home/neaf-3090/.pyenv/versions/3.6.8/lib/python3.6/site-packages/pytorch_lightning/trainer/training_loop.py", line 596, in on_train_epoch_end training_epoch_end_output = model.training_epoch_end(processed_epoch_output) File "train.py", line 345, in training_epoch_end selected_idx = selector.select_batch(model=self.randomprojector, already_selected=[], N=int(total_embeddings.shape[0]*args.coreset_sampling_ratio)) ValueError: invalid literal for int() with base 10: '0.010.010.010.010.010.010.010.010.010.010.010.010.010.010.010.010.010.010.010.010.010.010.010.010.010.010.010.010.010.010.010.010.010.010.010.010.010.010.010.010.010.010.010.010.010.010.010.010.010.0

Look like the problem is happened in line 345: selected_idx = selector.select_batch(model=self.randomprojector, already_selected=[], N=int(total_embeddings.shape[0]*args.coreset_sampling_ratio))

System and python interpreter

• Ubuntu 20
• Python 3.8
• pip 21.2.4
• torch 1.9.0
• torchvision 0.10

Can you tell me why coreset_sampling_ratio set 0.01 will make this bug ?

thx~

I tried to run the code with the carpet class of the MVTec AD, but the above error was yielded. I followed the installation process but, even so, this error appeared. Any idea why?

hey there, first of all thanks for the code!

regarding the final anomaly map computed: shouldn't the finale rescaling of the scores be done to all patch features? and not only to the highest one? thus creating a different anomaly map for the whole image?

any thoughts? thanks!

https://github.com/hcw-00/PatchCore_anomaly_detection/blob/ded157b8546bd71338b89a47d6ee5a872f382b14/train.py#L295

AdaptiveAvgPool is applied to each feature map on the same scale due to feature[0].shape[-2:] This means that nothing happens to the feature maps.

If you want to rescale the second feature maps to the first, I guess you could use features[0].shape[-2:]

Furthermore, even if you would use the suggestion above, you still don't apply any aggregation function f_agg to the first feature maps (as described in the paper). Shouldn't you use AvgPool2d in order to actually aggregate information from nearby patches?

Hello Sir.

I have interesting anomaly detection. I found your code in paperwithcode, I want to try to test your code.

I think that patchCore does not have training-phase (Neural Networks) as PaDiM. But your code has training-phase.

What is this??

Thanks, Edward Cho.

Epoch 0: 100%|█████████████████████████████████████████████████████Epoch 0: 100%|████████████████████████████████████████████████████████████████| 62/62 [00:20<00:00, 2.98it/s, v_num=6]Traceback (most recent call last): File "train.py", line 440, in trainer.fit(model) File "F:\ProgramData\Anaconda3\envs\patchcore\lib\site-packages\pytorch_lightning\trainer\trainer.py", line 696, in fit self._call_and_handle_interrupt( File "F:\ProgramData\Anaconda3\envs\patchcore\lib\site-packages\pytorch_lightning\trainer\trainer.py", line 650, in _call_and_handle_interrupt return trainer_fn(*args, **kwargs) File "F:\ProgramData\Anaconda3\envs\patchcore\lib\site-packages\pytorch_lightning\trainer\trainer.py", line 737, in _fit_impl results = self._run(model, ckpt_path=self.ckpt_path) File "F:\ProgramData\Anaconda3\envs\patchcore\lib\site-packages\pytorch_lightning\trainer\trainer.py", line 1168, in _run results = self._run_stage() File "F:\ProgramData\Anaconda3\envs\patchcore\lib\site-packages\pytorch_lightning\trainer\trainer.py", line 1254, in _run_stage return self._run_train() File "F:\ProgramData\Anaconda3\envs\patchcore\lib\site-packages\pytorch_lightning\trainer\trainer.py", line 1285, in _run_train self.fit_loop.run() File "F:\ProgramData\Anaconda3\envs\patchcore\lib\site-packages\pytorch_lightning\loops\loop.py", line 201, in run self.on_advance_end() File "F:\ProgramData\Anaconda3\envs\patchcore\lib\site-packages\pytorch_lightning\loops\fit_loop.py", line 286, in on_advance_end epoch_end_outputs = self.trainer._call_lightning_module_hook("training_epoch_end", epoch_end_outputs) File "F:\ProgramData\Anaconda3\envs\patchcore\lib\site-packages\pytorch_lightning\trainer\trainer.py", line 1552, in _call_lightning_module_hook output = fn(*args, **kwargs) File "train.py", line 354, in training_epoch_end selected_idx = selector.select_batch(model=self.randomprojector, already_selected=[], N=int(total_embeddings.shape[0]*args.coreset_sampling_ratio)) ValueError: invalid literal for int() with base 10: '0.010.010.010.010.010.010.010.010.010.010.010.010.010.010.010.010.010.010.010.010.010.010.010.010.010.010.010.010.010.010.010.010.010.010.010.010.010.010.010.010.010.010.010.010.010.010.0

I ran the code and got many errors but handled to solve them somehow! here is a problem that I have. I only want to train the model on the bottle dataset and it is already in the dataset address. the original dataset has broken_large, broken_small, contaminated, and good but I do not know how to feed those 4 groups as a train/test dataset.

I also have got an error while training only on good images. I will appreciate it if you could help me

$ python train.py --phase train --dataset_path C:\\projects\\interview\\mvtec --category bottle --project_root_path C:\projects\interview\PatchCore_anomaly_detection\results --coreset_sampling_ratio 0.01 --n_neighbors 9 C:\Users\user\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.10_qbz5n2kfra8p0\LocalCache\local-packages\Python310\site-packages\torchvision\models\detection\anchor_utils.py:63: UserWarning: Failed to initialize NumPy: module compiled against API version 0x10 but this version of numpy is 0xf (Triggered internally at ..\torch\csrc\utils\tensor_numpy.cpp:68.) device: torch.device = torch.device("cpu"), GPU available: False, used: False TPU available: False, using: 0 TPU cores IPU available: False, using: 0 IPUs Using cache found in C:\Users\user/.cache\torch\hub\pytorch_vision_v0.9.0 C:\Users\user\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.10_qbz5n2kfra8p0\LocalCache\local-packages\Python310\site-packages\torchvision\models\_utils.py:208: UserWarning: The parameter 'pretrained' is deprecated since 0.13 and will be removed in 0.15, please use 'weights' instead. warnings.warn( C:\Users\user\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.10_qbz5n2kfra8p0\LocalCache\local-packages\Python310\site-packages\torchvision\models\_utils.py:223: UserWarning: Arguments other than a weight enum orNonefor 'weights' are deprecated since 0.13 and will be removed in 0.15. The current behavior is equivalent to passingweights=Wide_ResNet50_2_Weights.IMAGENET1K_V1. You can also useweights=Wide_ResNet50_2_Weights.DEFAULTto get the most up-to-date weights. warnings.warn(msg) C:\projects\interview\PatchCore_anomaly_detection_2\train.py:259: DeprecationWarning: ANTIALIAS is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.LANCZOS instead. transforms.Resize((args.load_size, args.load_size), Image.ANTIALIAS), C:\Users\user\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.10_qbz5n2kfra8p0\LocalCache\local-packages\Python310\site-packages\torchvision\transforms\transforms.py:332: UserWarning: Argument 'interpolation' of type int is deprecated since 0.13 and will be removed in 0.15. Please use InterpolationMode enum. warnings.warn( C:\Users\user\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.10_qbz5n2kfra8p0\LocalCache\local-packages\Python310\site-packages\pytorch_lightning\trainer\optimizers.py:37: UserWarning:LightningModule.configure_optimizersreturnedNone`, this fit will run with no optimizer rank_zero_warn(

| Name | Type | Params

0 | model | ResNet | 68.9 M 1 | criterion | MSELoss | 0 2 | inv_normalize | Normalize | 0

0 Trainable params 68.9 M Non-trainable params 68.9 M Total params 275.533 Total estimated model params size (MB) Traceback (most recent call last): File "C:\projects\interview\PatchCore_anomaly_detection_2\train.py", line 437, in trainer.fit(model) File "C:\Users\user\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.10_qbz5n2kfra8p0\LocalCache\local-packages\Python310\site-packages\pytorch_lightning\trainer\trainer.py", line 740, in fit self._call_and_handle_interrupt( File "C:\Users\user\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.10_qbz5n2kfra8p0\LocalCache\local-packages\Python310\site-packages\pytorch_lightning\trainer\trainer.py", line 685, in _call_and_handle_interrupt return trainer_fn(*args, **kwargs) File "C:\Users\user\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.10_qbz5n2kfra8p0\LocalCache\local-packages\Python310\site-packages\pytorch_lightning\trainer\trainer.py", line 777, in _fit_impl self._run(model, ckpt_path=ckpt_path) File "C:\Users\user\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.10_qbz5n2kfra8p0\LocalCache\local-packages\Python310\site-packages\pytorch_lightning\trainer\trainer.py", line 1199, in _run self._dispatch() File "C:\Users\user\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.10_qbz5n2kfra8p0\LocalCache\local-packages\Python310\site-packages\pytorch_lightning\trainer\trainer.py", line 1279, in _dispatch self.training_type_plugin.start_training(self) File "C:\Users\user\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.10_qbz5n2kfra8p0\LocalCache\local-packages\Python310\site-packages\pytorch_lightning\plugins\training_type\training_type_plugin.py", line 202, in start_training self._results = trainer.run_stage() File "C:\Users\user\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.10_qbz5n2kfra8p0\LocalCache\local-packages\Python310\site-packages\pytorch_lightning\trainer\trainer.py", line 1289, in run_stage return self._run_train() File "C:\Users\user\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.10_qbz5n2kfra8p0\LocalCache\local-packages\Python310\site-packages\pytorch_lightning\trainer\trainer.py", line 1319, in _run_train self.fit_loop.run() File "C:\Users\user\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.10_qbz5n2kfra8p0\LocalCache\local-packages\Python310\site-packages\pytorch_lightning\loops\base.py", line 140, in run self.on_run_start(*args, **kwargs) File "C:\Users\user\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.10_qbz5n2kfra8p0\LocalCache\local-packages\Python310\site-packages\pytorch_lightning\loops\fit_loop.py", line 197, in on_run_start self.trainer.reset_train_val_dataloaders(self.trainer.lightning_module) File "C:\Users\user\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.10_qbz5n2kfra8p0\LocalCache\local-packages\Python310\site-packages\pytorch_lightning\trainer\data_loading.py", line 595, in reset_train_val_dataloaders self.reset_train_dataloader(model=model) File "C:\Users\user\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.10_qbz5n2kfra8p0\LocalCache\local-packages\Python310\site-packages\pytorch_lightning\trainer\data_loading.py", line 365, in reset_train_dataloader self.train_dataloader = self.request_dataloader(RunningStage.TRAINING, model=model) File "C:\Users\user\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.10_qbz5n2kfra8p0\LocalCache\local-packages\Python310\site-packages\pytorch_lightning\trainer\data_loading.py", line 611, in request_dataloader dataloader = source.dataloader() File "C:\Users\user\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.10_qbz5n2kfra8p0\LocalCache\local-packages\Python310\site-packages\pytorch_lightning\trainer\connectors\data_connector.py", line 296, in dataloader return self.instance.trainer.call_hook(self.name, pl_module=self.instance) File "C:\Users\user\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.10_qbz5n2kfra8p0\LocalCache\local-packages\Python310\site-packages\pytorch_lightning\trainer\trainer.py", line 1501, in call_hook output = model_fx(*args, **kwargs) File "C:\projects\interview\PatchCore_anomaly_detection_2\train.py", line 304, in train_dataloader train_loader = DataLoader(image_datasets, batch_size=args.batch_size, shuffle=True, num_workers=0) #, pin_memory=True) File "C:\Users\user\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.10_qbz5n2kfra8p0\LocalCache\local-packages\Python310\site-packages\torch\utils\data\dataloader.py", line 347, in init sampler = RandomSampler(dataset, generator=generator) # type: ignore[arg-type] File "C:\Users\user\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.10_qbz5n2kfra8p0\LocalCache\local-packages\Python310\site-packages\torch\utils\data\sampler.py", line 107, in init raise ValueError("num_samples should be a positive integer " ValueError: num_samples should be a positive integer value, but got num_samples=0 `

If I understand correctly, the kCenterGreedy.select_batch_ function is merely doing random sampling now. The main reason is that self.already_selected is not updated within the range(N) loop, therefore line 'ind = np.random.choice(np.arange(self.n_obs))' is always executed. To fix it, put self.already_selected = new_batch into the range(N) loop.

Please point it out if I misunderstood anything!

for _ in range(N): if not self.already_selected: # Initialize centers with a randomly selected datapoint ind = np.random.choice(np.arange(self.n_obs)) else: ind = np.argmax(self.min_distances) # New examples should not be in already selected since those points # should have min_distance of zero to a cluster center. assert ind not in already_selected

  self.update_distances([ind], only_new=True, reset_dist=False)
  new_batch.append(ind)
  print('Maximum distance from cluster centers is %0.2f' % max(self.min_distances))

  self.already_selected = new_batch#already_selected

I have tried pip version of faiss for my own experiments, faiss-cpu and faiss-gpu, and I found my results are unstable, after searching on google, https://github.com/facebookresearch/faiss/issues/2273 pip is not a supported way of installing in my user experience, I found that faiss(pip version) is very unstable, and also hard to debug, sometimes my program hangs when running faiss, anomalib provides an alternative way to do nearest neighbor search.