Visualizing Yolov5's layers using GradCam

Overview

YOLO-V5 GRADCAM

I constantly desired to know to which part of an object the object-detection models pay more attention. So I searched for it, but I didn't find any for Yolov5. Here is my implementation of Grad-cam for YOLO-v5. To load the model I used the yolov5's main codes, and for computing GradCam I used the codes from the gradcam_plus_plus-pytorch repository. Please follow my GitHub account and star the project if this functionality benefits your research or projects.

Installation

pip install -r requirements.txt

Infer

python main.py --model-path yolov5s.pt --img-path images/cat-dog.jpg --output-dir outputs

NOTE: If you don't have any weights and just want to test, don't change the model-path argument. The yolov5s model will be automatically downloaded thanks to the download function from yolov5.

NOTE: For more input arguments, check out the main.py or run the following command:

python main.py -h

Examples

Open In Colab

cat&dog

cat&dog

cat&dog

Note

I checked the code, but I couldn't find an explanation for why the truck's heatmap does not show anything. Please inform me or create a pull request if you find the reason.

TO Do

  1. Add GradCam++
  2. Add ScoreCam
  3. Add the functionality to the deep_utils library

References

  1. https://github.com/1Konny/gradcam_plus_plus-pytorch
  2. https://github.com/ultralytics/yolov5
  3. https://github.com/pooya-mohammadi/deep_utils

Citation

Please cite yolov5-gradcam if it helps your research. You can use the following BibTeX entry:

@misc{deep_utils,
	title = {yolov5-gradcam},
	author = {Mohammadi Kazaj, Pooya},
	howpublished = {\url{github.com/pooya-mohammadi/yolov5-gradcam}},
	year = {2021}
}
Comments
  • No heatmap using a different model

    No heatmap using a different model

    Hi, thank you for your great job!

    I tried to use a custom model, and even though the object is detected, the heatmap is basically empty.

    Do you know a workaround?

    Thanks agin.

    opened by CBarrelet 8
  • get error

    get error "google/protobuf/pyext/descriptor.cc:358: bad argument to internal function"

    Dears

    When I try to operate "python main.py --model-path yolov5l.pt --img-path images/cat-dog.jpg --output-dir outputs" and I get this error

    Traceback (most recent call last): File "main.py", line 6, in from models.yolo_v5_object_detector import YOLOV5TorchObjectDetector File "/content/yolov5-gradcam/models/yolo_v5_object_detector.py", line 2, in from deep_utils.utils.box_utils.boxes import Box File "/usr/local/lib/python3.7/dist-packages/deep_utils/init.py", line 10, in from .vision import * File "/usr/local/lib/python3.7/dist-packages/deep_utils/vision/init.py", line 1, in from .face_detection import * File "/usr/local/lib/python3.7/dist-packages/deep_utils/vision/face_detection/init.py", line 49, in from deep_utils.vision.face_detection.ultralight.tf.ultralight_tf_face_detection import ( File "/usr/local/lib/python3.7/dist-packages/deep_utils/vision/face_detection/ultralight/tf/ultralight_tf_face_detection.py", line 17, in from .utils.rfb_320 import create_rfb_net File "/usr/local/lib/python3.7/dist-packages/deep_utils/vision/face_detection/ultralight/tf/utils/rfb_320.py", line 1, in import tensorflow as tf File "/usr/local/lib/python3.7/dist-packages/tensorflow/init.py", line 37, in from tensorflow.python.tools import module_util as _module_util File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/init.py", line 37, in from tensorflow.python.eager import context File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/eager/context.py", line 29, in from tensorflow.core.framework import function_pb2 File "/usr/local/lib/python3.7/dist-packages/tensorflow/core/framework/function_pb2.py", line 16, in from tensorflow.core.framework import attr_value_pb2 as tensorflow_dot_core_dot_framework_dot_attr__value__pb2 File "/usr/local/lib/python3.7/dist-packages/tensorflow/core/framework/attr_value_pb2.py", line 16, in from tensorflow.core.framework import tensor_pb2 as tensorflow_dot_core_dot_framework_dot_tensor__pb2 File "/usr/local/lib/python3.7/dist-packages/tensorflow/core/framework/tensor_pb2.py", line 16, in from tensorflow.core.framework import resource_handle_pb2 as tensorflow_dot_core_dot_framework_dot_resource__handle__pb2 File "/usr/local/lib/python3.7/dist-packages/tensorflow/core/framework/resource_handle_pb2.py", line 150, in 'module' : 'tensorflow.core.framework.resource_handle_pb2' SystemError: google/protobuf/pyext/descriptor.cc:358: bad argument to internal function

    Could you tell me how to solve this error?

    opened by ChengYao0209 3
  • why register_backward_hook() got the zero gradient

    why register_backward_hook() got the zero gradient

    I took the gradcam for yolov3 and revise a little. I have no idea about why register_backward_hook() couldn't get the gradient. The register_forward_hook() is ok. can you help me, thanks a lot. image

    opened by infochen 3
  • RuntimeError: don't know how to restore data location of torch.HalfStorage (tagged with )

    RuntimeError: don't know how to restore data location of torch.HalfStorage (tagged with )

    D:\anaconda\envs\yolov5\lib\site-packages\deep_utils\vision\face_detection\ultralight_init_.py:3: UserWarning: No module named 'deep_utils.vision.face_detection.ultralight.torch'. If you don't use UltralightTorchFaceDetector ignore this message. UltralightTorchFaceDetector = import_module( D:\anaconda\envs\yolov5\lib\site-packages\deep_utils\vision\face_detection\ultralight_init_.py:6: UserWarning: No module named 'deep_utils.vision.face_detection.ultralight.tf'. If you don't use UltralightTFFaceDetector ignore this message. UltralightTFFaceDetector = import_module( D:\anaconda\envs\yolov5\lib\site-packages\deep_utils\callbacks\tf_keras_init_.py:3: UserWarning: No module named 'tensorflow'. If you don't use LRScalar ignore this message. LRScalar = import_module( [INFO] Loading the model Traceback (most recent call last): File "main.py", line 125, in main(args.img_path) File "main.py", line 64, in main model = YOLOV5TorchObjectDetector(args.model_path, device, img_size=input_size, File "D:\AAAAAAAAA\YOLOV5-4.0\yolov5-gradcam-master\models\yolo_v5_object_detector.py", line 32, in init self.model = attempt_load(model_weight, device=device) File "D:\AAAAAAAAA\YOLOV5-4.0\yolov5-gradcam-master\models\experimental.py", line 80, in attempt_load ckpt = torch.load(attempt_download(w), map_location=device) File "D:\anaconda\envs\yolov5\lib\site-packages\torch\serialization.py", line 594, in load return _load(opened_zipfile, map_location, pickle_module, **pickle_load_args) File "D:\anaconda\envs\yolov5\lib\site-packages\torch\serialization.py", line 853, in _load result = unpickler.load() File "D:\anaconda\envs\yolov5\lib\site-packages\torch\serialization.py", line 845, in persistent_load load_tensor(data_type, size, key, _maybe_decode_ascii(location)) File "D:\anaconda\envs\yolov5\lib\site-packages\torch\serialization.py", line 834, in load_tensor loaded_storages[key] = restore_location(storage, location) File "D:\anaconda\envs\yolov5\lib\site-packages\torch\serialization.py", line 811, in restore_location return default_restore_location(storage, map_location) File "D:\anaconda\envs\yolov5\lib\site-packages\torch\serialization.py", line 178, in default_restore_location raise RuntimeError("don't know how to restore data location of " RuntimeError: don't know how to restore data location of torch.HalfStorage (tagged with )

    opened by YTTT9 2
  • make --img-path parameter accept folder as a path

    make --img-path parameter accept folder as a path

    I added a feature so that --img-path can accept folder as path. So, if it is folder, it will loop through all files and interpret them one by one.

    This is default functionality in YOLO.

    opened by pourmand1376 1
  • How to visualize multi-classes

    How to visualize multi-classes

    Hi, Thanks for your great work and the results look amazing. But I wonder that if it can visualize multi-classes by grad-cam whatever in classification or detection task.

    opened by O2andsAnoxia 1
  • Tensor not  match

    Tensor not match

    hello, I trianed it on DOTA datasets,but get error. RuntimeError: The size of tensor a (80) must match the size of tensor b (1280) at non-singleton dimension 1

    opened by friskzz 0
  • Error when trying with custom model and dataset

    Error when trying with custom model and dataset

    Hi there,

    I'm currently have issues when running the code with the model I trained on yolov5 v6.1, with a custom dataset image Do you know what's happening?

    Thank you so much!!

    opened by bzha5848 0
  • Can't get attribute 'DetectionModel' on <module 'models.yolo' from....>

    Can't get attribute 'DetectionModel' on

    Here's my code...↓

    
    ! pip install -r requirements.txt
    
    import os
    os.chdir("./yolov5-gradcam-master/")
    
    !python main.py --model-path runs/train/hat_person4/weights/best.pt --img-path datasets/hat_person/yolo_dataset/images/train/000490.jpg --output-dir outputs --names hat,person   
    
    

    And then...Something wrong..

    [INFO] Loading the model Traceback (most recent call last): File "main.py", line 124, in main(args.img_path) File "main.py", line 64, in main names=None if args.names is None else args.names.strip().split(",")) File "f:\works\VSCODE\yolov5\yolov5-gradcam-master\models\yolo_v5_object_detector.py", line 32, in __init__ self.model = attempt_load(model_weight, device=device) File "f:\works\VSCODE\yolov5\yolov5-gradcam-master\models\experimental.py", line 80, in attempt_load ckpt = torch.load(attempt_download(w), map_location=device) File "d:\Anaconda3\envs\yolo\lib\site-packages\torch\serialization.py", line 607, in load return _load(opened_zipfile, map_location, pickle_module, **pickle_load_args) File "d:\Anaconda3\envs\yolo\lib\site-packages\torch\serialization.py", line 882, in _load result = unpickler.load() File "d:\Anaconda3\envs\yolo\lib\site-packages\torch\serialization.py", line 875, in find_class return super().find_class(mod_name, name) AttributeError: Can't get attribute 'DetectionModel' on <module 'models.yolo' from 'f:\works\VSCODE\yolov5\yolov5-gradcam-master\models\yolo.py'>

    opened by mjuhbgtfc 7
  • what about yolov5l,yolov5x,yolov5s?

    what about yolov5l,yolov5x,yolov5s?

    hi,i tried to use the yolov5l,yolov5x,yolov5s models,but the results of heatmap were not good,although i tried other targets-layer. please tell me how this work support yolov5l,yolov5x,yolov5s?thanks

    opened by luyuxuan 0
Owner
Pooya Mohammadi Kazaj
Open Source Enthusiast | AI Developer & Researcher
Pooya Mohammadi Kazaj
Compare outputs between layers written in Tensorflow and layers written in Pytorch

Compare outputs of Wasserstein GANs between TensorFlow vs Pytorch This is our testing module for the implementation of improved WGAN in Pytorch Prereq

Hung Nguyen 72 Dec 20, 2022
a reccurrent neural netowrk that when trained on a peice of text and fed a starting prompt will write its on 250 character text using LSTM layers

RNN-Playwrite a reccurrent neural netowrk that when trained on a peice of text and fed a starting prompt will write its on 250 character text using LS

Arno Barton 1 Oct 29, 2021
Bayesian-Torch is a library of neural network layers and utilities extending the core of PyTorch to enable the user to perform stochastic variational inference in Bayesian deep neural networks

Bayesian-Torch is a library of neural network layers and utilities extending the core of PyTorch to enable the user to perform stochastic variational inference in Bayesian deep neural networks. Bayesian-Torch is designed to be flexible and seamless in extending a deterministic deep neural network architecture to corresponding Bayesian form by simply replacing the deterministic layers with Bayesian layers.

Intel Labs 210 Jan 4, 2023
Meta Language-Specific Layers in Multilingual Language Models

Meta Language-Specific Layers in Multilingual Language Models This repo contains the source codes for our paper On Negative Interference in Multilingu

Zirui Wang 20 Feb 13, 2022
Improving Deep Network Debuggability via Sparse Decision Layers

Improving Deep Network Debuggability via Sparse Decision Layers This repository contains the code for our paper: Leveraging Sparse Linear Layers for D

Madry Lab 35 Nov 14, 2022
Unofficial PyTorch implementation of Attention Free Transformer (AFT) layers by Apple Inc.

aft-pytorch Unofficial PyTorch implementation of Attention Free Transformer's layers by Zhai, et al. [abs, pdf] from Apple Inc. Installation You can i

Rishabh Anand 184 Dec 12, 2022
Spectral Tensor Train Parameterization of Deep Learning Layers

Spectral Tensor Train Parameterization of Deep Learning Layers This repository is the official implementation of our AISTATS 2021 paper titled "Spectr

Anton Obukhov 12 Oct 23, 2022
TensorFlow, PyTorch and Numpy layers for generating Orthogonal Polynomials

OrthNet TensorFlow, PyTorch and Numpy layers for generating multi-dimensional Orthogonal Polynomials 1. Installation 2. Usage 3. Polynomials 4. Base C

Chuan 29 May 25, 2022
RepMLP: Re-parameterizing Convolutions into Fully-connected Layers for Image Recognition

RepMLP: Re-parameterizing Convolutions into Fully-connected Layers for Image Recognition (PyTorch) Paper: https://arxiv.org/abs/2105.01883 Citation: @

null 260 Jan 3, 2023
DEMix Layers for Modular Language Modeling

DEMix This repository contains modeling utilities for "DEMix Layers: Disentangling Domains for Modular Language Modeling" (Gururangan et. al, 2021). T

Suchin 43 Nov 11, 2022
Accelerate Neural Net Training by Progressively Freezing Layers

FreezeOut A simple technique to accelerate neural net training by progressively freezing layers. This repository contains code for the extended abstra

Andy Brock 203 Jun 19, 2022
A library to inspect itermediate layers of PyTorch models.

A library to inspect itermediate layers of PyTorch models. Why? It's often the case that we want to inspect intermediate layers of a model without mod

archinet.ai 380 Dec 28, 2022
Simple tools for logging and visualizing, loading and training

TNT TNT is a library providing powerful dataloading, logging and visualization utilities for Python. It is closely integrated with PyTorch and is desi

null 1.5k Jan 2, 2023
Your interactive network visualizing dashboard

Your interactive network visualizing dashboard Documentation: Here What is Jaal Jaal is a python based interactive network visualizing tool built usin

Mohit 177 Jan 4, 2023
A module for solving and visualizing Schrödinger equation.

qmsolve This is an attempt at making a solid, easy to use solver, capable of solving and visualize the Schrödinger equation for multiple particles, an

null 506 Dec 28, 2022
Code for CVPR2021 "Visualizing Adapted Knowledge in Domain Transfer". Visualization for domain adaptation. #explainable-ai

Visualizing Adapted Knowledge in Domain Transfer @inproceedings{hou2021visualizing, title={Visualizing Adapted Knowledge in Domain Transfer}, auth

Yunzhong Hou 80 Dec 25, 2022
Code for visualizing the loss landscape of neural nets

Visualizing the Loss Landscape of Neural Nets This repository contains the PyTorch code for the paper Hao Li, Zheng Xu, Gavin Taylor, Christoph Studer

Tom Goldstein 2.2k Jan 9, 2023
Improving Contrastive Learning by Visualizing Feature Transformation, ICCV 2021 Oral

Improving Contrastive Learning by Visualizing Feature Transformation This project hosts the codes, models and visualization tools for the paper: Impro

Bingchen Zhao 83 Dec 15, 2022