Code for the head detector (HeadHunter) proposed in our CVPR 2021 paper Tracking Pedestrian Heads in Dense Crowd.

Overview

Head Detector

Code for the head detector (HeadHunter) proposed in our CVPR 2021 paper Tracking Pedestrian Heads in Dense Crowd. The head_detection module can be installed using pip in order to be able to plug-and-play with HeadHunter-T.

Requirements

  1. Nvidia Driver >= 418

  2. Cuda 10.0 and compaitible CudNN

  3. Python packages : To install the required python packages; conda env create -f head_detection.yml.

  4. Use the anaconda environment head_detection by activating it, source activate head_detection or conda activate head_detection.

  5. Alternatively pip can be used to install required packages using pip install -r requirements.txt or update your existing environment with the aforementioned yml file.

Training

  1. To train a model, define environment variable NGPU, config file and use the following command

$python -m torch.distributed.launch --nproc_per_node=$NGPU --use_env train.py --cfg_file config/config_chuman.yaml --world_size $NGPU --num_workers 4

  1. Training is currently supported over (a) ScutHead dataset (b) CrowdHuman + ScutHead combined, (c) Our proposed CroHD dataset. This can be mentioned in the config file.

  2. To train the model, config files must be defined. More details about the config files are mentioned in the section below

Evaluation and Testing

  1. Unlike the training, testing and evaluation does not have a config file. Rather, all the parameters are set as argument variable while executing the code. Refer to the respective files, evaluate.py and test.py.
  2. evaluate.py evaluates over the validation/test set using AP, MMR, F1, MODA and MODP metrics.
  3. test.py runs the detector over a "bunch of images" in the testing set for qualitative evaluation.

Config file

A config file is necessary for all training. It's built to ease the number of arg variable passed during each execution. Each sub-sections are as elaborated below.

  1. DATASET

    1. Set the base_path as the parent directory where the dataset is situated at.
    2. Train and Valid are .txt files that contains relative path to respective images from the base_path defined above and their corresponding Ground Truth in (x_min, y_min, x_max, y_max) format. Generation files for the three datasets can be seen inside data directory. For example,
    /path/to/image.png
    x_min_1, y_min_1, x_max_1, y_max_1
    x_min_2, y_min_2, x_max_2, y_max_2
    x_min_3, y_min_3, x_max_3, y_max_3
    .
    .
    .
    
    1. mean_std are RGB means and stdev of the training dataset. If not provided, can be computed prior to the start of the training
  2. TRAINING

    1. Provide pretrained_model and corresponding start_epoch for resuming.
    2. milestones are epoch at which the learning rates are set to 0.1 * lr.
    3. only_backbone option loads just the Resnet backbone and not the head. Not applicable for mobilenet.
  3. NETWORK

    1. The mentioned parameters are as described in experiment section of the paper.
    2. When using median_anchors, the anchors have to be defined in anchors.py.
    3. We experimented with mobilenet, resnet50 and resnet150 as alternative backbones. This experiment was not reported in the paper due to space constraints. We found the accuracy to significantly decrease with mobilenet but resnet50 and resnet150 yielded an almost same performance.
    4. We also briefly experimented with Deformable Convolutions but again didn't see noticable improvements in performance. The code we used are available in this repository.

Note :

This codebase borrows a noteable portion from pytorch-vision owing to the fact some of their modules cannot be "imported" as a package.

Citation :

@InProceedings{Sundararaman_2021_CVPR,
    author    = {Sundararaman, Ramana and De Almeida Braga, Cedric and Marchand, Eric and Pettre, Julien},
    title     = {Tracking Pedestrian Heads in Dense Crowd},
    booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
    month     = {June},
    year      = {2021},
    pages     = {3865-3875}
}
Issues
  • pretrained weights

    pretrained weights

    Can you share some pretrained weights?

    opened by yzfly 15
  • The data set format is incorrect

    The data set format is incorrect

    The data set format is incorrect

    opened by Freezing-hxy 4
  • when publish the dateset?

    when publish the dateset?

    I want to see the datasets,thank you!!

    opened by wong1998 3
  • load state dict error

    load state dict error

    I run test.py. args.pretrained_model is from this link. I meet this error: RuntimeError: Error(s) in loading state_dict for FasterRCNN: Unexpected key(s) in state_dict: "backbone.ssh1.branch1.0.weight", "backbone.ssh1.branch1.1.weight", "backbone.ssh1.branch1.1.bias", "backbone.ssh1.branch1.1.running_mean", "backbone.ssh1.branch1.1.running_var", "backbone.ssh1.branch1.1.num_batches_tracked", "backbone.ssh1.branch2a.0.weight", "backbone.ssh1.branch2a.1.weight", "backbone.ssh1.branch2a.1.bias", "backbone.ssh1.branch2a.1.running_mean", "backbone.ssh1.branch2a.1.running_var", "backbone.ssh1.branch2a.1.num_batches_tracked", "backbone.ssh1.branch2b.0.weight", "backbone.ssh1.branch2b.1.weight", "backbone.ssh1.branch2b.1.bias", "backbone.ssh1.branch2b.1.running_mean", "backbone.ssh1.branch2b.1.running_var", "backbone.ssh1.branch2b.1.num_batches_tracked", "backbone.ssh1.branch2c.0.weight", "backbone.ssh1.branch2c.1.weight", "backbone.ssh1.branch2c.1.bias", "backbone.ssh1.branch2c.1.running_mean", "backbone.ssh1.branch2c.1.running_var", "backbone.ssh1.branch2c.1.num_batches_tracked", "backbone.ssh1.ssh_1.weight", "backbone.ssh1.ssh_1.bias", "backbone.ssh1.ssh_dimred.weight", "backbone.ssh1.ssh_dimred.bias", "backbone.ssh1.ssh_2.weight", "backbone.ssh1.ssh_2.bias", "backbone.ssh1.ssh_3a.weight", "backbone.ssh1.ssh_3a.bias", "backbone.ssh1.ssh_3b.weight", "backbone.ssh1.ssh_3b.bias", "backbone.ssh1.ssh_final.0.weight", "backbone.ssh1.ssh_final.1.weight", "backbone.ssh1.ssh_final.1.bias", "backbone.ssh1.ssh_final.1.running_mean", "backbone.ssh1.ssh_final.1.running_var", "backbone.ssh1.ssh_final.1.num_batches_tracked", "backbone.ssh2.branch1.0.weight", "backbone.ssh2.branch1.1.weight", "backbone.ssh2.branch1.1.bias", "backbone.ssh2.branch1.1.running_mean", "backbone.ssh2.branch1.1.running_var", "backbone.ssh2.branch1.1.num_batches_tracked", "backbone.ssh2.branch2a.0.weight", "backbone.ssh2.branch2a.1.weight", "backbone.ssh2.branch2a.1.bias", "backbone.ssh2.branch2a.1.running_mean", "backbone.ssh2.branch2a.1.running_var", "backbone.ssh2.branch2a.1.num_batches_tracked", "backbone.ssh2.branch2b.0.weight", "backbone.ssh2.branch2b.1.weight", "backbone.ssh2.branch2b.1.bias", "backbone.ssh2.branch2b.1.running_mean", "backbone.ssh2.branch2b.1.running_var", "backbone.ssh2.branch2b.1.num_batches_tracked", "backbone.ssh2.branch2c.0.weight", "backbone.ssh2.branch2c.1.weight", "backbone.ssh2.branch2c.1.bias", "backbone.ssh2.branch2c.1.running_mean", "backbone.ssh2.branch2c.1.running_var", "backbone.ssh2.branch2c.1.num_batches_tracked", "backbone.ssh2.ssh_1.weight", "backbone.ssh2.ssh_1.bias", "backbone.ssh2.ssh_dimred.weight", "backbone.ssh2.ssh_dimred.bias", "backbone.ssh2.ssh_2.weight", "backbone.ssh2.ssh_2.bias", "backbone.ssh2.ssh_3a.weight", "backbone.ssh2.ssh_3a.bias", "backbone.ssh2.ssh_3b.weight", "backbone.ssh2.ssh_3b.bias", "backbone.ssh2.ssh_final.0.weight", "backbone.ssh2.ssh_final.1.weight", "backbone.ssh2.ssh_final.1.bias", "backbone.ssh2.ssh_final.1.running_mean", "backbone.ssh2.ssh_final.1.running_var", "backbone.ssh2.ssh_final.1.num_batches_tracked", "backbone.ssh3.branch1.0.weight", "backbone.ssh3.branch1.1.weight", "backbone.ssh3.branch1.1.bias", "backbone.ssh3.branch1.1.running_mean", "backbone.ssh3.branch1.1.running_var", "backbone.ssh3.branch1.1.num_batches_tracked", "backbone.ssh3.branch2a.0.weight", "backbone.ssh3.branch2a.1.weight", "backbone.ssh3.branch2a.1.bias", "backbone.ssh3.branch2a.1.running_mean", "backbone.ssh3.branch2a.1.running_var", "backbone.ssh3.branch2a.1.num_batches_tracked", "backbone.ssh3.branch2b.0.weight", "backbone.ssh3.branch2b.1.weight", "backbone.ssh3.branch2b.1.bias", "backbone.ssh3.branch2b.1.running_mean", "backbone.ssh3.branch2b.1.running_var", "backbone.ssh3.branch2b.1.num_batches_tracked", "backbone.ssh3.branch2c.0.weight", "backbone.ssh3.branch2c.1.weight", "backbone.ssh3.branch2c.1.bias", "backbone.ssh3.branch2c.1.running_mean", "backbone.ssh3.branch2c.1.running_var", "backbone.ssh3.branch2c.1.num_batches_tracked", "backbone.ssh3.ssh_1.weight", "backbone.ssh3.ssh_1.bias", "backbone.ssh3.ssh_dimred.weight", "backbone.ssh3.ssh_dimred.bias", "backbone.ssh3.ssh_2.weight", "backbone.ssh3.ssh_2.bias", "backbone.ssh3.ssh_3a.weight", "backbone.ssh3.ssh_3a.bias", "backbone.ssh3.ssh_3b.weight", "backbone.ssh3.ssh_3b.bias", "backbone.ssh3.ssh_final.0.weight", "backbone.ssh3.ssh_final.1.weight", "backbone.ssh3.ssh_final.1.bias", "backbone.ssh3.ssh_final.1.running_mean", "backbone.ssh3.ssh_final.1.running_var", "backbone.ssh3.ssh_final.1.num_batches_tracked", "backbone.ssh4.branch1.0.weight", "backbone.ssh4.branch1.1.weight", "backbone.ssh4.branch1.1.bias", "backbone.ssh4.branch1.1.running_mean", "backbone.ssh4.branch1.1.running_var", "backbone.ssh4.branch1.1.num_batches_tracked", "backbone.ssh4.branch2a.0.weight", "backbone.ssh4.branch2a.1.weight", "backbone.ssh4.branch2a.1.bias", "backbone.ssh4.branch2a.1.running_mean", "backbone.ssh4.branch2a.1.running_var", "backbone.ssh4.branch2a.1.num_batches_tracked", "backbone.ssh4.branch2b.0.weight", "backbone.ssh4.branch2b.1.weight", "backbone.ssh4.branch2b.1.bias", "backbone.ssh4.branch2b.1.running_mean", "backbone.ssh4.branch2b.1.running_var", "backbone.ssh4.branch2b.1.num_batches_tracked", "backbone.ssh4.branch2c.0.weight", "backbone.ssh4.branch2c.1.weight", "backbone.ssh4.branch2c.1.bias", "backbone.ssh4.branch2c.1.running_mean", "backbone.ssh4.branch2c.1.running_var", "backbone.ssh4.branch2c.1.num_batches_tracked", "backbone.ssh4.ssh_1.weight", "backbone.ssh4.ssh_1.bias", "backbone.ssh4.ssh_dimred.weight", "backbone.ssh4.ssh_dimred.bias", "backbone.ssh4.ssh_2.weight", "backbone.ssh4.ssh_2.bias", "backbone.ssh4.ssh_3a.weight", "backbone.ssh4.ssh_3a.bias", "backbone.ssh4.ssh_3b.weight", "backbone.ssh4.ssh_3b.bias", "backbone.ssh4.ssh_final.0.weight", "backbone.ssh4.ssh_final.1.weight", "backbone.ssh4.ssh_final.1.bias", "backbone.ssh4.ssh_final.1.running_mean", "backbone.ssh4.ssh_final.1.running_var", "backbone.ssh4.ssh_final.1.num_batches_tracked". what should i do?

    opened by RichardoMrMu 3
  • The env yml file?

    The env yml file?

    Hi Sentient07 I have no idea about the actual env requirements of this project. Could you tell me more details?

    opened by D-Walter 2
  • About GPU memory requirement.

    About GPU memory requirement.

    How much GPU memory is recommanded when training the model using ScutHead dataset? I've found a memory insufficience error when trying to train the model on a idel GPU with 4041MB memory and batch size 1. Besides, there is some minor bugs in the "head_detection/data/create_scuthead.py" file, as args.dset_path is defined but args.dset_dir is used (and the same bugs to the arg.save_path and args.out_path).

    opened by NonameZ-HTY 2
  • how to solve fast_rcnn.py

    how to solve fast_rcnn.py

    File "/home/project/HeadHunter-master/head_detection/models/fast_rcnn.py", line 496, in forward boxes, scores = self.filter_proposals(proposals, objectness, images.image_sizes, num_anchors_per_level) File "/home/project/HeadHunter-master/head_detection/models/fast_rcnn.py", line 458, in filter_proposals keep = keep[:self.post_nms_top_n] TypeError: slice indices must be integers or None or have an index method

    opened by Freezing-hxy 2
  • How to setup the enviroment?

    How to setup the enviroment?

    Hello, the setup instruction is quite confusing, can you provide the pytorch version, required python package or config.yaml file?

    opened by yzfly 1
  • the code of tracking

    the code of tracking

    Hi, Sentient07! I only found the the code of head detection in GitHub, do you have the plane to place the code of head tracking in Github?

    opened by TUT103 1
  • Could you share your CroHD dataset?

    Could you share your CroHD dataset?

    Hi, I am very interested in your great work, I try to reproduce your work from scratch. But I cannot find anywhere to download CroHD dataset. Could you please give me a way to download it?

    opened by yoyokitartora 1
  •  raise ValueError(

    raise ValueError("Anchors should be Tuple[Tuple[int]] because each feature "

    2021-11-24 10-12-33 的屏幕截图 anchors = self.grid_anchors(grid_sizes, strides) File "/home/haojie/下载/ENTER/envs/head_detection2/lib/python3.8/site-packages/torchvision/models/detection/anchor_utils.py", line 103, in grid_anchors raise ValueError("Anchors should be Tuple[Tuple[int]] because each feature " ValueError: Anchors should be Tuple[Tuple[int]] because each feature map could potentially have different sizes and aspect ratios. There needs to be a match between the number of feature maps passed and the number of sizes / aspect ratios specified. Killing subprocess 1055

    opened by haojie0616 0
  • how to set the anchor and choose the benchmark when running the tracker on a CroHD dataset(use_public

    how to set the anchor and choose the benchmark when running the tracker on a CroHD dataset(use_public" is false)

    when I try to run the tracker on a CroHD dataset(training set) and **set the "use_public" option in the 'det_cfg' to 'False',**the ValueError said that "Anchors should be Tuple[Tuple[int]] because each feature map could potentially have different sizes and aspect ratios. " I noticed that in the obj_detect.py,if det_cfg['median_anchor']:the program can choose different benchmark to import different anchors,so which benchmark I should set in the det_cfg when I want to run the tracker on a CroHD dataset?And I noticed that the benchmark should be associated with the anchor.py,too. Another question is,if i set the "use_public" option in the 'det_cfg' to 'True',does it mean that I will use the data in det.txt which the dataset offers and the detector will not be used to detect the pedestrians or output anything. 1635856049(1) 1635856090(1)

    opened by yx98314 4
  • CorHD dataset process program

    CorHD dataset process program

    For the preprocessing .py program of the headhunter data set, you seem to have not uploaded it yet, can you please upload it? Thank you very much

    opened by jihai-cyber 7
Owner
Ramana Subramanyam
Ramana Subramanyam
code for our ICCV 2021 paper "DeepCAD: A Deep Generative Network for Computer-Aided Design Models"

DeepCAD This repository provides source code for our paper: DeepCAD: A Deep Generative Network for Computer-Aided Design Models Rundi Wu, Chang Xiao,

Rundi Wu 44 Nov 22, 2021
Learning Camera Localization via Dense Scene Matching, CVPR2021

This repository contains code of our CVPR 2021 paper - "Learning Camera Localization via Dense Scene Matching" by Shitao Tang, Chengzhou Tang, Rui Hua

tangshitao 49 Nov 13, 2021
Implement 'Single Shot Text Detector with Regional Attention, ICCV 2017 Spotlight'

SSTDNet Implement 'Single Shot Text Detector with Regional Attention, ICCV 2017 Spotlight' using pytorch. This code is work for general object detecti

HotaekHan 87 Nov 16, 2021
Single Shot Text Detector with Regional Attention

Single Shot Text Detector with Regional Attention Introduction SSTD is initially described in our ICCV 2017 spotlight paper. A third-party implementat

Pan He 219 Nov 16, 2021
A tensorflow implementation of EAST text detector

EAST: An Efficient and Accurate Scene Text Detector Introduction This is a tensorflow re-implementation of EAST: An Efficient and Accurate Scene Text

null 2.8k Nov 22, 2021
Implementation of EAST scene text detector in Keras

EAST: An Efficient and Accurate Scene Text Detector This is a Keras implementation of EAST based on a Tensorflow implementation made by argman. The or

Jan Zdenek 199 Nov 16, 2021
This is a pytorch re-implementation of EAST: An Efficient and Accurate Scene Text Detector.

EAST: An Efficient and Accurate Scene Text Detector Description: This version will be updated soon, please pay attention to this work. The motivation

Dejia Song 510 Dec 2, 2021
PyTorch Re-Implementation of EAST: An Efficient and Accurate Scene Text Detector

Description This is a PyTorch Re-Implementation of EAST: An Efficient and Accurate Scene Text Detector. Only RBOX part is implemented. Using dice loss

null 316 Nov 18, 2021
Packaged, Pytorch-based, easy to use, cross-platform version of the CRAFT text detector

CRAFT: Character-Region Awareness For Text detection Packaged, Pytorch-based, easy to use, cross-platform version of the CRAFT text detector | Paper |

null 119 Nov 26, 2021
TextBoxes++: A Single-Shot Oriented Scene Text Detector

TextBoxes++: A Single-Shot Oriented Scene Text Detector Introduction This is an application for scene text detection (TextBoxes++) and recognition (CR

Minghui Liao 907 Nov 21, 2021
TextBoxes: A Fast Text Detector with a Single Deep Neural Network https://github.com/MhLiao/TextBoxes 基于SSD改进的文本检测算法,textBoxes_note记录了之前整理的笔记。

TextBoxes: A Fast Text Detector with a Single Deep Neural Network Introduction This paper presents an end-to-end trainable fast scene text detector, n

zhangjing1 25 Jan 9, 2020
python ocr using tesseract/ with EAST opencv detector

pytextractor python ocr using tesseract/ with EAST opencv text detector Uses the EAST opencv detector defined here with pytesseract to extract text(de

Danny Crasto 35 Nov 21, 2021
Augmenting Anchors by the Detector Itself

Augmenting Anchors by the Detector Itself Introduction It is difficult to determine the scale and aspect ratio of anchors for anchor-based object dete

null 3 Jun 1, 2021
Motion detector, Full body detection, Upper body detection, Cat face detection, Smile detection, Face detection (haar cascade), Silverware detection, Face detection (lbp), and Sending email notifications

Security camera running OpenCV for object and motion detection. The camera will send email with image of any objects it detects. It also runs a server that provides web interface with live stream video.

Peace 10 Jun 30, 2021
Image Detector and Convertor App created using python's Pillow, OpenCV, cvlib, numpy and streamlit packages.

Image Detector and Convertor App created using python's Pillow, OpenCV, cvlib, numpy and streamlit packages.

Siva Prakash 10 Nov 14, 2021
This is the code for our paper DAAIN: Detection of Anomalous and AdversarialInput using Normalizing Flows

Merantix-Labs: DAAIN This is the code for our paper DAAIN: Detection of Anomalous and Adversarial Input using Normalizing Flows which can be found at

Merantix 13 Jun 10, 2021
Code release for our paper, "SimNet: Enabling Robust Unknown Object Manipulation from Pure Synthetic Data via Stereo"

SimNet: Enabling Robust Unknown Object Manipulation from Pure Synthetic Data via Stereo Thomas Kollar, Michael Laskey, Kevin Stone, Brijen Thananjeyan

null 40 Nov 26, 2021
CVPR 2021 Oral paper "LED2-Net: Monocular 360˚ Layout Estimation via Differentiable Depth Rendering" official PyTorch implementation.

LED2-Net This is PyTorch implementation of our CVPR 2021 Oral paper "LED2-Net: Monocular 360˚ Layout Estimation via Differentiable Depth Rendering". Y

Fu-En Wang 54 Oct 28, 2021