in the indad/model.py line 78

the mask must be convert to int type, otherwise, it will get ValueError: continuous format is not supported

Here is the solution: I changed the mask type from float to int

pixel_labels.extend(mask.flatten().numpy().astype(int))

The initial function is like below

	def evaluate(self, test_ds: VisionDataset) -> Tuple[float, float]:
		"""Calls predict step for each test sample."""
		image_preds = []
		image_labels = []
		pixel_preds = []
		pixel_labels = []

		for sample, mask, label in tqdm(test_ds, **TQDM_PARAMS):
			z_score, fmap = self.predict(sample.unsqueeze(0))
			
			image_preds.append(z_score.numpy())
			image_labels.append(label)
			
			pixel_preds.extend(fmap.flatten().numpy())
			pixel_labels.extend(mask.flatten().numpy())
			
		image_preds = np.stack(image_preds)

		image_rocauc = roc_auc_score(image_labels, image_preds)

		pixel_rocauc = roc_auc_score(pixel_labels, pixel_preds)

		return image_rocauc, pixel_rocauc

Hello and thank for your contribution. Really good job

Trying your code i am wondering if this attribute n_reweight is the neighbourhood size described in the Paper.

Hi there,

I am using the pipeline based on the padim method and backbone_name="wide_resnet50_2". However, I see excessive raise in my memory when I check it with htop command in another terminal. I don't have same experience with spade method.

As picture shows, it consumes up to 121G in the memory. I even tried to reduce the 'd_reduced' parameter, but it is still around 100G. May I ask about your opinion how can I make it better? did U also faced same issue?

Best

I'm trying to convert pytorch to torch script.I want to know whether the detection speed of the converted model has changed. Neither method in the document can be used. https://pytorch.org/tutorials/advanced/cpp_export.html

model = SPADE(k=42)  # , backbone_name="hypernet")
train_dataset = StreamingDataset()
test_dataset = StreamingDataset()
app_custom_train_images =  "D:\\cwge\\Dataset\\bottle\\train\\good"
# train images
for root, dirs, files in os.walk(app_custom_train_images):
    for file in files:
        train_dataset.add_pil_image(Image.open(os.path.join(root, file)))
model.fit(train_dataset)
PATH = "test.pth"
torch.save(model.state_dict(), PATH)
# model.load_state_dict(torch.load(PATH))
#example = torch.rand(1, 3, 224, 224).cuda()
#traced_script_module = torch.jit.trace(model, example)
script_net = torch.jit.script(model)
script_net.save('model_script.pt')
#traced_script_module.save("traced_resnet_model.pt")

My C + + code reported an error when loading the model.

#include <iostream>
#include "torch/script.h"
#include <vector>
int main()
{torch::jit::script::Module module = torch::jit::load("D:\\model_script.pt");}

Unhandled exception at 0x00007FFB76BA4B89 in ConsoleApplication1.exe: Microsoft C++ exception: c10::NotImplementedError at memory location 0x00000095C3EFDD50.

Thanks to your efforts, I've noticed in the code of patchcore, doing k nearest neighbor search, https://github.com/rvorias/ind_knn_ad/blob/5510f0f4bc1fc9688e57d20023985087f90f0f4d/indad/models.py#L286 the value of k is set to 3, can I ask how to determine the value of k? can I set k=1?

Hi there, thanks for sharing your codes publicly. I am trying to follow the custom dataset workflow. My question is about defining my own threshold based on required prec/recall, as you mentioned in here. I am quite newbie and would like to use your repo for my study thesis. My I ask for quick clue where in the code and how can I set a new threshold?

After training with spade method, I test the model on an anomaly test image and got the img anom score: tensor(8.3064), while I obtained even similar value with a good test image.

btw, as you mentioned I defined a custom dataset folder, but do we need to also define a new dataloader? as the already utilized the one in the run.py is for MVTecDataset.
train_ds, test_ds = MVTecDataset(cls).get_dataloaders() My dataset folder structure is like: -custom_dataset --train ---good --test ---good ---bad --ground_truth ---bad

I really appreciate your help and effort.

Hello.

I tested your Patchcore implementation and i need to advice you of a mistake of what the table of results shows I tested your implementation for 1% of coreset with different B values to get the mean as final results

You can see, that the performance is better than what you described in your table.

Moreover, if we use avgpooling (3, 1, 1) instead avgpooling(3, 2, 0) with Interpolation Lanzcos in the training, results are still better. In fact, paper says that increasing the stride decreases the performance

Here is the results of using this new avgpooling

Thank you for sharing your streamlit code.

It was very easy to check my data.

But I made a red pattern even in a normal image.

Because on streamlit color assignment of heatmap is decided by each image. I think it is better to fix the range if necessary.

So I made it. If you don't check the box, the operation will be the same as before.

If you like it, please merge it.

The size of my picture is 1280*1024,I use the command streamlit run streamlit_app.py . The result is very good. But part of my picture is missing. The displayed result is not a complete picture. Can the cropping of the picture be changed? I tried to modify the code, but the result was not good.Can the detection speed be improved?Can I just load the model without training every time?

class SPADE(KNNExtractor):
    def __init__(
            self,
            k: int = 5,
            backbone_name: str = "resnet50",
    ):
        super().__init__(
            backbone_name=backbone_name,
            out_indices=(1, 2, 3),
            pool=True,
        )
        self.k = k
        self.image_size_x = 1280
        self.image_size_y = 1024
        self.z_lib = []
        self.feature_maps = []
        self.threshold_z = None
        self.threshold_fmaps = None
        self.blur = GaussianBlur(4)

    def predict(self, sample):
        feature_maps, z = self(sample)

        distances = torch.linalg.norm(self.z_lib - z, dim=1)
        values, indices = torch.topk(distances.squeeze(), self.k, largest=False)

        z_score = values.mean()

        # Build the feature gallery out of the k nearest neighbours.
        # The authors migh have concatenated all features maps first, then check the minimum norm per pixel.
        # Here, we check for the minimum norm first, then concatenate (sum) in the final layer.
        scaled_s_map = torch.zeros(1, 1, self.image_size_y, self.image_size_x)
        for idx, fmap in enumerate(feature_maps):
            nearest_fmaps = torch.index_select(self.feature_maps[idx], 0, indices)
            # min() because kappa=1 in the paper
            s_map, _ = torch.min(torch.linalg.norm(nearest_fmaps - fmap, dim=1), 0, keepdims=True)
            scaled_s_map += torch.nn.functional.interpolate(
                s_map.unsqueeze(0), size=(self.image_size_y, self.image_size_x), mode='bilinear'
            )

        scaled_s_map = self.blur(scaled_s_map)

        return z_score, scaled_s_map

Hi, great project! I have 8000 images, and I found that the memory increased a lot during training. My computer has 60G RAM but it is still not enough.

model = SPADE(k=42)  # , backbone_name="hypernet")
train_dataset = StreamingDataset()
app_custom_train_images = "D:\\train\\good"
# train images
for root, dirs, files in os.walk(app_custom_train_images):
    for file in files:
        train_dataset.add_pil_image(Image.open(os.path.join(root, file)))
model.fit(train_dataset)
PATH = "test.pth"
torch.save(model.state_dict(), PATH)

pil_img = plt.open("20210115_raw.png")
img_pil_1 = np.array(pil_img) 
tensor_pil = torch.from_numpy(np.transpose(img_pil_1, (2, 0, 1)))
img = tensor_pil.type(torch.float32)  
img = img.unsqueeze(0) 
pil_to_tensor = img 
img_lvl_anom_score, pxl_lvl_anom_score = model.predict(pil_to_tensor)

The test picture does not seem to need to be transformed. Does it support pictures in other formats?

Test results zipper - image_rocauc: 0.94, pixel_rocauc: 0.98 Traceback (most recent call last): File "D:\Code\ind_knn_ad-master\indad\run.py", line 82, in cli_interface() File "D:\ProgramData\Anaconda3\envs\env1\lib\site-packages\click\core.py", line 1137, in call return self.main(*args, **kwargs) File "D:\ProgramData\Anaconda3\envs\env1\lib\site-packages\click\core.py", line 1062, in main rv = self.invoke(ctx) File "D:\ProgramData\Anaconda3\envs\env1\lib\site-packages\click\core.py", line 1404, in invoke return ctx.invoke(self.callback, **ctx.params) File "D:\ProgramData\Anaconda3\envs\env1\lib\site-packages\click\core.py", line 763, in invoke return __callback(*args, **kwargs) File "D:\Code\ind_knn_ad-master\indad\run.py", line 79, in cli_interface write_results(total_results, method) File "D:\Code\ind_knn_ad-master\indad\utils.py", line 65, in write_results with open(f"./results/{name}.yml", "w") as outfile: FileNotFoundError: [Errno 2] No such file or directory: './results/spade_26_07_2021_22_47_37.yml'

Hey the app is currently not responding

by the way do you have a tutorial how can i make the app run on a windows maschine? im new to maschine learning and want to test anomaly detection on pictures of industrial parts.

Hey there :wave: Just wanted to let you know that your app on Streamlit Cloud deployed from this repo has gone over its resource limits. Access to the app is temporarily limited. Visit the app to see more details and possible solutions.

Hey there :wave: Just wanted to let you know that your app on Streamlit Cloud deployed from this repo has gone over its resource limits. Access to the app is temporarily limited. Visit the app to see more details and possible solutions.

Hi! The more image samples are, the greater the 28(Fmap_H)*28(Fmap_W)*num_samples, so it grows because it is projecting. Is there any way to reduce it?

def fit(self, train_dl):
        for sample, _ in tqdm(train_dl, **get_tqdm_params()):
	        feature_maps = self(sample)
        
	        if self.resize is None:
		        largest_fmap_size = feature_maps[0].shape[-2:]
		        self.resize = torch.nn.AdaptiveAvgPool2d(largest_fmap_size)
	        resized_maps = [self.resize(self.average(fmap)) for fmap in feature_maps]
	        patch = torch.cat(resized_maps, 1)
	        patch = patch.reshape(patch.shape[1], -1).T
        
	        self.patch_lib.append(patch)
        
        self.patch_lib = torch.cat(self.patch_lib, 0)
        
        if self.f_coreset < 1:
	        self.coreset_idx = get_coreset_idx_randomp(
		        self.patch_lib,
		        n=int(self.f_coreset * self.patch_lib.shape[0]),
		        eps=self.coreset_eps,
	        )
	        self.patch_lib = self.patch_lib[self.coreset_idx]

How can I model save, model load and model test for one image?

PATH = '/home/agteks/Desktop/ind_knn_ad/model.pt' model = PatchCore(f_coreset= .01, backbone_name= "efficientnet_b0", coreset_eps= .95)

model.load_state_dict(torch.load('/home/agteks/Desktop/ind_knn_ad/model.pt')) model.eval() model = model.to('cuda')

#sample, *_ = test_dataset[st.session_state.test_idx] pil_img = Image.open("/home/agteks/Desktop/ind_knn_ad/datasets/fabric_4/test/broken_large/Cropped_2_276_46_768.png") pil_img = pil_img.resize((224, 224)) pil_to_tensor = transforms.ToTensor()(pil_img).unsqueeze_(0) img_lvl_anom_score, pxl_lvl_anom_score = model.predict(pil_to_tensor) score_range = pxl_lvl_anom_score.min(), pxl_lvl_anom_score.max() show_pred(pil_to_tensor, img_lvl_anom_score, pxl_lvl_anom_score)

-- This code not working. Please help me :(