The project is an official implementation of our CVPR2019 paper "Deep High-Resolution Representation Learning for Human Pose Estimation"

Overview

Deep High-Resolution Representation Learning for Human Pose Estimation (CVPR 2019)

News

Introduction

This is an official pytorch implementation of Deep High-Resolution Representation Learning for Human Pose Estimation. In this work, we are interested in the human pose estimation problem with a focus on learning reliable high-resolution representations. Most existing methods recover high-resolution representations from low-resolution representations produced by a high-to-low resolution network. Instead, our proposed network maintains high-resolution representations through the whole process. We start from a high-resolution subnetwork as the first stage, gradually add high-to-low resolution subnetworks one by one to form more stages, and connect the mutli-resolution subnetworks in parallel. We conduct repeated multi-scale fusions such that each of the high-to-low resolution representations receives information from other parallel representations over and over, leading to rich high-resolution representations. As a result, the predicted keypoint heatmap is potentially more accurate and spatially more precise. We empirically demonstrate the effectiveness of our network through the superior pose estimation results over two benchmark datasets: the COCO keypoint detection dataset and the MPII Human Pose dataset.

Illustrating the architecture of the proposed HRNet

Main Results

Results on MPII val

Arch Head Shoulder Elbow Wrist Hip Knee Ankle Mean [email protected]
pose_resnet_50 96.4 95.3 89.0 83.2 88.4 84.0 79.6 88.5 34.0
pose_resnet_101 96.9 95.9 89.5 84.4 88.4 84.5 80.7 89.1 34.0
pose_resnet_152 97.0 95.9 90.0 85.0 89.2 85.3 81.3 89.6 35.0
pose_hrnet_w32 97.1 95.9 90.3 86.4 89.1 87.1 83.3 90.3 37.7

Note:

Results on COCO val2017 with detector having human AP of 56.4 on COCO val2017 dataset

Arch Input size #Params GFLOPs AP Ap .5 AP .75 AP (M) AP (L) AR AR .5 AR .75 AR (M) AR (L)
pose_resnet_50 256x192 34.0M 8.9 0.704 0.886 0.783 0.671 0.772 0.763 0.929 0.834 0.721 0.824
pose_resnet_50 384x288 34.0M 20.0 0.722 0.893 0.789 0.681 0.797 0.776 0.932 0.838 0.728 0.846
pose_resnet_101 256x192 53.0M 12.4 0.714 0.893 0.793 0.681 0.781 0.771 0.934 0.840 0.730 0.832
pose_resnet_101 384x288 53.0M 27.9 0.736 0.896 0.803 0.699 0.811 0.791 0.936 0.851 0.745 0.858
pose_resnet_152 256x192 68.6M 15.7 0.720 0.893 0.798 0.687 0.789 0.778 0.934 0.846 0.736 0.839
pose_resnet_152 384x288 68.6M 35.3 0.743 0.896 0.811 0.705 0.816 0.797 0.937 0.858 0.751 0.863
pose_hrnet_w32 256x192 28.5M 7.1 0.744 0.905 0.819 0.708 0.810 0.798 0.942 0.865 0.757 0.858
pose_hrnet_w32 384x288 28.5M 16.0 0.758 0.906 0.825 0.720 0.827 0.809 0.943 0.869 0.767 0.871
pose_hrnet_w48 256x192 63.6M 14.6 0.751 0.906 0.822 0.715 0.818 0.804 0.943 0.867 0.762 0.864
pose_hrnet_w48 384x288 63.6M 32.9 0.763 0.908 0.829 0.723 0.834 0.812 0.942 0.871 0.767 0.876

Note:

Results on COCO test-dev2017 with detector having human AP of 60.9 on COCO test-dev2017 dataset

Arch Input size #Params GFLOPs AP Ap .5 AP .75 AP (M) AP (L) AR AR .5 AR .75 AR (M) AR (L)
pose_resnet_152 384x288 68.6M 35.3 0.737 0.919 0.828 0.713 0.800 0.790 0.952 0.856 0.748 0.849
pose_hrnet_w48 384x288 63.6M 32.9 0.755 0.925 0.833 0.719 0.815 0.805 0.957 0.874 0.763 0.863
pose_hrnet_w48* 384x288 63.6M 32.9 0.770 0.927 0.845 0.734 0.831 0.820 0.960 0.886 0.778 0.877

Note:

Environment

The code is developed using python 3.6 on Ubuntu 16.04. NVIDIA GPUs are needed. The code is developed and tested using 4 NVIDIA P100 GPU cards. Other platforms or GPU cards are not fully tested.

Quick start

Installation

  1. Install pytorch >= v1.0.0 following official instruction. Note that if you use pytorch's version < v1.0.0, you should following the instruction at https://github.com/Microsoft/human-pose-estimation.pytorch to disable cudnn's implementations of BatchNorm layer. We encourage you to use higher pytorch's version(>=v1.0.0)

  2. Clone this repo, and we'll call the directory that you cloned as ${POSE_ROOT}.

  3. Install dependencies:

    pip install -r requirements.txt
    
  4. Make libs:

    cd ${POSE_ROOT}/lib
    make
    
  5. Install COCOAPI:

    # COCOAPI=/path/to/clone/cocoapi
    git clone https://github.com/cocodataset/cocoapi.git $COCOAPI
    cd $COCOAPI/PythonAPI
    # Install into global site-packages
    make install
    # Alternatively, if you do not have permissions or prefer
    # not to install the COCO API into global site-packages
    python3 setup.py install --user
    

    Note that instructions like # COCOAPI=/path/to/install/cocoapi indicate that you should pick a path where you'd like to have the software cloned and then set an environment variable (COCOAPI in this case) accordingly.

  6. Init output(training model output directory) and log(tensorboard log directory) directory:

    mkdir output 
    mkdir log
    

    Your directory tree should look like this:

    ${POSE_ROOT}
    ├── data
    ├── experiments
    ├── lib
    ├── log
    ├── models
    ├── output
    ├── tools 
    ├── README.md
    └── requirements.txt
    
  7. Download pretrained models from our model zoo(GoogleDrive or OneDrive)

    ${POSE_ROOT}
     `-- models
         `-- pytorch
             |-- imagenet
             |   |-- hrnet_w32-36af842e.pth
             |   |-- hrnet_w48-8ef0771d.pth
             |   |-- resnet50-19c8e357.pth
             |   |-- resnet101-5d3b4d8f.pth
             |   `-- resnet152-b121ed2d.pth
             |-- pose_coco
             |   |-- pose_hrnet_w32_256x192.pth
             |   |-- pose_hrnet_w32_384x288.pth
             |   |-- pose_hrnet_w48_256x192.pth
             |   |-- pose_hrnet_w48_384x288.pth
             |   |-- pose_resnet_101_256x192.pth
             |   |-- pose_resnet_101_384x288.pth
             |   |-- pose_resnet_152_256x192.pth
             |   |-- pose_resnet_152_384x288.pth
             |   |-- pose_resnet_50_256x192.pth
             |   `-- pose_resnet_50_384x288.pth
             `-- pose_mpii
                 |-- pose_hrnet_w32_256x256.pth
                 |-- pose_hrnet_w48_256x256.pth
                 |-- pose_resnet_101_256x256.pth
                 |-- pose_resnet_152_256x256.pth
                 `-- pose_resnet_50_256x256.pth
    
    

Data preparation

For MPII data, please download from MPII Human Pose Dataset. The original annotation files are in matlab format. We have converted them into json format, you also need to download them from OneDrive or GoogleDrive. Extract them under {POSE_ROOT}/data, and make them look like this:

${POSE_ROOT}
|-- data
`-- |-- mpii
    `-- |-- annot
        |   |-- gt_valid.mat
        |   |-- test.json
        |   |-- train.json
        |   |-- trainval.json
        |   `-- valid.json
        `-- images
            |-- 000001163.jpg
            |-- 000003072.jpg

For COCO data, please download from COCO download, 2017 Train/Val is needed for COCO keypoints training and validation. We also provide person detection result of COCO val2017 and test-dev2017 to reproduce our multi-person pose estimation results. Please download from OneDrive or GoogleDrive. Download and extract them under {POSE_ROOT}/data, and make them look like this:

${POSE_ROOT}
|-- data
`-- |-- coco
    `-- |-- annotations
        |   |-- person_keypoints_train2017.json
        |   `-- person_keypoints_val2017.json
        |-- person_detection_results
        |   |-- COCO_val2017_detections_AP_H_56_person.json
        |   |-- COCO_test-dev2017_detections_AP_H_609_person.json
        `-- images
            |-- train2017
            |   |-- 000000000009.jpg
            |   |-- 000000000025.jpg
            |   |-- 000000000030.jpg
            |   |-- ... 
            `-- val2017
                |-- 000000000139.jpg
                |-- 000000000285.jpg
                |-- 000000000632.jpg
                |-- ... 

Training and Testing

Testing on MPII dataset using model zoo's models(GoogleDrive or OneDrive)

python tools/test.py \
    --cfg experiments/mpii/hrnet/w32_256x256_adam_lr1e-3.yaml \
    TEST.MODEL_FILE models/pytorch/pose_mpii/pose_hrnet_w32_256x256.pth

Training on MPII dataset

python tools/train.py \
    --cfg experiments/mpii/hrnet/w32_256x256_adam_lr1e-3.yaml

Testing on COCO val2017 dataset using model zoo's models(GoogleDrive or OneDrive)

python tools/test.py \
    --cfg experiments/coco/hrnet/w32_256x192_adam_lr1e-3.yaml \
    TEST.MODEL_FILE models/pytorch/pose_coco/pose_hrnet_w32_256x192.pth \
    TEST.USE_GT_BBOX False

Training on COCO train2017 dataset

python tools/train.py \
    --cfg experiments/coco/hrnet/w32_256x192_adam_lr1e-3.yaml \

Visualization

Visualizing predictions on COCO val

python visualization/plot_coco.py \
    --prediction output/coco/w48_384x288_adam_lr1e-3/results/keypoints_val2017_results_0.json \
    --save-path visualization/results

Other applications

Many other dense prediction tasks, such as segmentation, face alignment and object detection, etc. have been benefited by HRNet. More information can be found at High-Resolution Networks.

Other implementation

mmpose

Citation

If you use our code or models in your research, please cite with:

@inproceedings{sun2019deep,
  title={Deep High-Resolution Representation Learning for Human Pose Estimation},
  author={Sun, Ke and Xiao, Bin and Liu, Dong and Wang, Jingdong},
  booktitle={CVPR},
  year={2019}
}

@inproceedings{xiao2018simple,
    author={Xiao, Bin and Wu, Haiping and Wei, Yichen},
    title={Simple Baselines for Human Pose Estimation and Tracking},
    booktitle = {European Conference on Computer Vision (ECCV)},
    year = {2018}
}

@article{WangSCJDZLMTWLX19,
  title={Deep High-Resolution Representation Learning for Visual Recognition},
  author={Jingdong Wang and Ke Sun and Tianheng Cheng and 
          Borui Jiang and Chaorui Deng and Yang Zhao and Dong Liu and Yadong Mu and 
          Mingkui Tan and Xinggang Wang and Wenyu Liu and Bin Xiao},
  journal   = {TPAMI}
  year={2019}
}
Comments
  • Error when training/testing on COCO dataset

    Error when training/testing on COCO dataset

    My problem is similar to #100 , but I have tried all possible solutions in that issue and still can not solve the problem...

    Now I want to training/testing on COCO dataset, and visualize the model by using tensorboardX. Unfortunately, errors occurred one by one... At first, I met the same problem as this, then I downgrade my tensorboardX version to 1.6, and met the same problem as this, then I tried to degrade other package like torch, torchvision, etc., but it didn't work as well.

    Currently my version is as follows:

    torch 1.0.0
    torchvision 0.2.1
    tensorboard 1.6.0
    tensorboardX 1.6
    tensorflow 1.0.0
    cudatoolkit 9.0
    

    When I want to start training using the following command:

    python tools/train.py --cfg experiments/coco/hrnet/w32_256x192_adam_lr1e-3_coco2014.yaml
    

    It tells me:

    RuntimeError: CUDA out of memory. Tried to allocate 12.00 MiB (GPU 1; 11.93 GiB total capacity; 2.48 GiB already allocated; 2.81 MiB free; 5.46 MiB cached)
    

    It seems that the part to do with tensorboardX is successfully completed, and error occurred when begin training. What I want to know is that: Is this problem really caused by the memory of CUDA?Or it is caused by the old version of pytorch? Any code need to modified?

    And I also add the following codes into test.py, in order to visualize the model by using tensorboardX.

        writer_dict = {
            'writer': SummaryWriter(log_dir=tb_log_dir),
            'train_global_steps': 0,
            'valid_global_steps': 0,
        }
        dump_input = torch.rand(
            (1, 3, cfg.MODEL.IMAGE_SIZE[1], cfg.MODEL.IMAGE_SIZE[0])
        )
        writer_dict['writer'].add_graph(model, (dump_input, ))
    

    When I use the following command to test...

    python tools/test.py --cfg experiments/coco/hrnet/w32_256x192_adam_lr1e-3_coco2014.yaml TEST.MODEL_FILE models/pytorch/pose_coco/pose_hrnet_w32_256x192.pth TEST.USE_GT_BBOX False
    

    The error to do with tensorboardX occurred again!

    /home/fangqingkai/anaconda3/envs/HRNet/lib/python3.6/site-packages/torch/nn/modules/upsampling.py:129: UserWarning: nn.Upsample is deprecated. Use nn.functional.interpolate instead.
      warnings.warn("nn.{} is deprecated. Use nn.functional.interpolate instead.".format(self.name))
    Traceback (most recent call last):
      File "tools/test.py", line 142, in <module>
        main()
      File "tools/test.py", line 110, in main
        writer_dict['writer'].add_graph(model, (dump_input, ))
      File "/home/fangqingkai/anaconda3/envs/HRNet/lib/python3.6/site-packages/tensorboardX/writer.py", line 566, in add_graph
        self.file_writer.add_graph(graph(model, input_to_model, verbose))
      File "/home/fangqingkai/anaconda3/envs/HRNet/lib/python3.6/site-packages/tensorboardX/pytorch_graph.py", line 235, in graph
        _optimize_trace(trace, torch.onnx.utils.OperatorExportTypes.ONNX)
      File "/home/fangqingkai/anaconda3/envs/HRNet/lib/python3.6/site-packages/tensorboardX/pytorch_graph.py", line 175, in _optimize_trace
        trace.set_graph(_optimize_graph(trace.graph(), operator_export_type))
      File "/home/fangqingkai/anaconda3/envs/HRNet/lib/python3.6/site-packages/tensorboardX/pytorch_graph.py", line 201, in _optimize_graph
        torch._C._jit_pass_lower_all_tuples(graph)
    RuntimeError: tuple appears in op that does not forward tuples (VisitNode at /opt/conda/conda-bld/pytorch_1544174967633/work/torch/csrc/jit/passes/lower_tuples.cpp:109)
    frame #0: c10::Error::Error(c10::SourceLocation, std::string const&) + 0x45 (0x7fc59951fcc5 in /home/fangqingkai/anaconda3/envs/HRNet/lib/python3.6/site-packages/torch/lib/libc10.so)
    frame #1: <unknown function> + 0x6d6882 (0x7fc596bf0882 in /home/fangqingkai/anaconda3/envs/HRNet/lib/python3.6/site-packages/torch/lib/libtorch.so.1)
    frame #2: <unknown function> + 0x6d6ad4 (0x7fc596bf0ad4 in /home/fangqingkai/anaconda3/envs/HRNet/lib/python3.6/site-packages/torch/lib/libtorch.so.1)
    frame #3: torch::jit::LowerAllTuples(std::shared_ptr<torch::jit::Graph>&) + 0x13 (0x7fc596bf0be3 in /home/fangqingkai/anaconda3/envs/HRNet/lib/python3.6/site-packages/torch/lib/libtorch.so.1)
    frame #4: <unknown function> + 0x350b24 (0x7fc5c99fbb24 in /home/fangqingkai/anaconda3/envs/HRNet/lib/python3.6/site-packages/torch/lib/libtorch_python.so)
    frame #5: <unknown function> + 0x111e36 (0x7fc5c97bce36 in /home/fangqingkai/anaconda3/envs/HRNet/lib/python3.6/site-packages/torch/lib/libtorch_python.so)
    <omitting python frames>
    frame #35: __libc_start_main + 0xf0 (0x7fc5db560830 in /lib/x86_64-linux-gnu/libc.so.6)
    

    It confused me that why the same code has not caused error in train.py but in test.py...? And anyone can help me solve this problem? Thanks a lot!!!

    opened by Poeroz 18
  • ModuleNotFoundError: No module named 'nms.cpu_nms'

    ModuleNotFoundError: No module named 'nms.cpu_nms'

    [email protected]:~/Desktop/deep-high-resolution-net.pytorch-master$ python tools/test.py \ --cfg experiments/coco/hrnet/w32_256x192_adam_lr1e-3.yaml \ TEST.MODEL_FILE models/pytorch/pose_coco/pose_hrnet_w32_256x192.pth \ TEST.USE_GT_BBOX False Traceback (most recent call last): File "tools/test.py", line 31, in import dataset File "/home/myubuntu/Desktop/deep-high-resolution-net.pytorch-master/tools/../lib/dataset/init.py", line 12, in from .coco import COCODataset as coco File "/home/myubuntu/Desktop/deep-high-resolution-net.pytorch-master/tools/../lib/dataset/coco.py", line 22, in from nms.nms import oks_nms File "/home/myubuntu/Desktop/deep-high-resolution-net.pytorch-master/tools/../lib/nms/nms.py", line 13, in from .cpu_nms import cpu_nms ModuleNotFoundError: No module named 'nms.cpu_nms'

    opened by tianjiahao 11
  • AttributeError: 'NoneType' object has no attribute '__name__'

    AttributeError: 'NoneType' object has no attribute '__name__'

    when i do: python tools/train.py
    --cfg experiments/mpii/hrnet/w32_256x256_adam_lr1e-3.yaml an error occurrs: 图片

    However,when i do: python tools/test.py
    --cfg experiments/mpii/hrnet/w32_256x256_adam_lr1e-3.yaml
    TEST.MODEL_FILE models/pytorch/pose_mpii/pose_hrnet_w32_256x256.pth there is nothing wrong.

    opened by angechen 8
  • Low mAP obtained in testing Simple-baseline model using code in this repo.

    Low mAP obtained in testing Simple-baseline model using code in this repo.

    I test the simple baseline models downloaded from your link using the code in this repo. I just get 65.3 mAP on pose_resnet50. Flip-test, Ground-truth bbox, are used, and input size is 256*192. GPU: 4 k80, batch-size:32. And I test onresnet101, 152, it's still lower than the results in simple-baseline repo.

    opened by qiuzhongwei-USTB 6
  • How Can I Get The Pretrained Weights

    How Can I Get The Pretrained Weights

    I am in china and cannot open OneDrive or GoogleDrive ,please tell me how can i get the pretrained pth file? Can you upload the file to Baidu cloud disk?thanks a lot

    opened by JiangWeiHn 5
  • nvidia-docker is deprecated.

    nvidia-docker is deprecated.

    readme of demo said, we can run container by nvidia-docker, but latest version of docker does not support nvidia-docker command.

    we can use

    docker run --gpus all --rm -it \
        -v $(pwd)/output:/output \
        -v $(pwd)/videos:/videos \
        -v $(pwd)/models:/models \
        -w /pose_root \
        hrnet_demo_inference \
        /bin/bash
    

    instead of nvidia-docker

    opened by sklationd 5
  • I test the model on MPII

    I test the model on MPII

    File "tools/test.py", line 131, in main() File "tools/test.py", line 99, in main model = torch.nn.DataParallel(model, device_ids=cfg.GPUS).cuda()

    lib/python3.6/site-packages/torch/cuda/init.py", line 318, in get_device_properties raise AssertionError("Invalid device id") AssertionError: Invalid device id

    opened by henbucuoshanghai 4
  • How can I evaluate my model on MPII test set?

    How can I evaluate my model on MPII test set?

    How can I test my model on test set and get the output file like evaluatePCKh.m on MPII website? Is there any tools for converting test results to .m file? THX

    opened by xtyDoge 4
  • opencv error

    opencv error

    Thanks for sharing the code. When I tried to run the test commands both failed with cv2.error: OpenCV(3.4.1) /io/opencv/modules/imgproc/src/color.cpp:11115: error: (-215) scn == 3 || scn == 4 in function cvtColor. Am I missing something? Thanks,

    opened by mistycheney 4
  • 请教代码意思。

    请教代码意思。

    https://github.com/leoxiaobin/deep-high-resolution-net.pytorch/blob/ba50a82dce412df97f088c572d86d7977753bf74/lib/dataset/mpii.py#L58

    作者您好,请问下这里避免裁剪到肢体, 是自己自定义轻微调整center 和 scale的值吗? 具体表示的是什么意思呢? 纵坐标和尺度发生变化?

    opened by YuQi9797 3
  • mAP difference in MSCOCO val

    mAP difference in MSCOCO val

    您好!HRNet的工作真的非常棒,非常感谢开源代码! 有几个问题,想请教一下: person_keypoints_val2017.json和COCO_val2017_detections_AP_H_56_person.json有什么区别吗? 如果理解无误的话,使用的都是val2017验证集。那为什么在每次训练epoch结束的模型评价mAP值,要比test.py所得的mAP 低

    这是在train中最后一个训练轮次结束所得mAP

    2020-06-22 19:56:58,671 | Arch | AP | Ap .5 | AP .75 | AP (M) | AP (L) | AR | AR .5 | AR .75 | AR (M) | AR (L) | 2020-06-22 19:56:58,672 |---|---|---|---|---|---|---|---|---|---|---| 2020-06-22 19:56:58,672 | pose_hrnet | 0.744 | 0.926 | 0.816 | 0.722 | 0.783 | 0.774 | 0.935 | 0.838 | 0.746 | 0.817 |

    这是通过val检测final_state.pth的mAP

    2020-06-23 04:13:00,041 | Arch | AP | Ap .5 | AP .75 | AP (M) | AP (L) | AR | AR .5 | AR .75 | AR (M) | AR (L) | 2020-06-23 04:13:00,043 |---|---|---|---|---|---|---|---|---|---|---| 2020-06-23 04:13:00,043 | pose_hrnet | 0.724 | 0.892 | 0.799 | 0.692 | 0.789 | 0.780 | 0.932 | 0.848 | 0.740 | 0.839 |

    两者相差了2个点,我认为这应该与json文件的选取相关。 以及,请问在论文中:Table 1. Comparisons on the COCO validation set,使用的是哪个mAP值。 还望予以赐教,谢谢!

    opened by ZW-XXX 3
  • How to convert mpii mat file to json

    How to convert mpii mat file to json

    When I test on MPII and COCO dataset using model zoo's models, I found COCO dataset will get a json file, but MPII file only get a mat file, does anyone know how to convert mpii mat file to json?

    opened by STRUGGLE1999 0
  • Inferencing queries

    Inferencing queries

    Can the model infer on CPU or only on GPU? Can it infer high resolution image or at max, what resolution of the image can be inferred? Can it perform 3D pose estimation? What is the Inference speed in FPS?

    opened by SoumyajitMukherjee-droid 0
  • RuntimeError: CUDA error: out of memory -- although GPU is empty

    RuntimeError: CUDA error: out of memory -- although GPU is empty

    Only reason I can think of is my cuda version is 11.7, but the latest version of PyTorch available is for cuda 11.6. Could that be the issue?

    image

    Log:

     python tools/test.py \
        --cfg experiments/coco/hrnet/w48_384x288_adam_lr1e-3.yaml \
        TEST.MODEL_FILE models/pytorch/pose_hrnet_w48_384x288.pth \
        TEST.USE_GT_BBOX False
    
    => creating log/coco/pose_hrnet/w48_384x288_adam_lr1e-3_2022-07-19-07-56
    Namespace(cfg='experiments/coco/hrnet/w48_384x288_adam_lr1e-3.yaml', opts=['TEST.MODEL_FILE', 'models/pytorch/pose_hrnet_w48_384x288.pth', 'TEST.USE_GT_BBOX', 'False'], modelDir='', logDir='', dataDir='', prevModelDir='')
    AUTO_RESUME: True
    CUDNN:
      BENCHMARK: True
      DETERMINISTIC: False
      ENABLED: True
    DATASET:
      COLOR_RGB: True
      DATASET: coco
      DATA_FORMAT: jpg
      FLIP: True
      HYBRID_JOINTS_TYPE: 
      NUM_JOINTS_HALF_BODY: 8
      PROB_HALF_BODY: 0.3
      ROOT: data/coco/
      ROT_FACTOR: 45
      SCALE_FACTOR: 0.35
      SELECT_DATA: False
      TEST_SET: val2017
      TRAIN_SET: train2017
    DATA_DIR: 
    DEBUG:
      DEBUG: True
      SAVE_BATCH_IMAGES_GT: True
      SAVE_BATCH_IMAGES_PRED: True
      SAVE_HEATMAPS_GT: True
      SAVE_HEATMAPS_PRED: True
    GPUS: (0, 1, 2, 3)
    LOG_DIR: log
    LOSS:
      TOPK: 8
      USE_DIFFERENT_JOINTS_WEIGHT: False
      USE_OHKM: False
      USE_TARGET_WEIGHT: True
    MODEL:
      EXTRA:
        FINAL_CONV_KERNEL: 1
        PRETRAINED_LAYERS: ['conv1', 'bn1', 'conv2', 'bn2', 'layer1', 'transition1', 'stage2', 'transition2', 'stage3', 'transition3', 'stage4']
        STAGE2:
          BLOCK: BASIC
          FUSE_METHOD: SUM
          NUM_BLOCKS: [4, 4]
          NUM_BRANCHES: 2
          NUM_CHANNELS: [48, 96]
          NUM_MODULES: 1
        STAGE3:
          BLOCK: BASIC
          FUSE_METHOD: SUM
          NUM_BLOCKS: [4, 4, 4]
          NUM_BRANCHES: 3
          NUM_CHANNELS: [48, 96, 192]
          NUM_MODULES: 4
        STAGE4:
          BLOCK: BASIC
          FUSE_METHOD: SUM
          NUM_BLOCKS: [4, 4, 4, 4]
          NUM_BRANCHES: 4
          NUM_CHANNELS: [48, 96, 192, 384]
          NUM_MODULES: 3
      HEATMAP_SIZE: [72, 96]
      IMAGE_SIZE: [288, 384]
      INIT_WEIGHTS: True
      NAME: pose_hrnet
      NUM_JOINTS: 17
      PRETRAINED: models/pytorch/imagenet/hrnet_w48-8ef0771d.pth
      SIGMA: 3
      TAG_PER_JOINT: True
      TARGET_TYPE: gaussian
    OUTPUT_DIR: output
    PIN_MEMORY: True
    PRINT_FREQ: 100
    RANK: 0
    TEST:
      BATCH_SIZE_PER_GPU: 24
      BBOX_THRE: 1.0
      COCO_BBOX_FILE: data/coco/person_detection_results/COCO_val2017_detections_AP_H_56_person.json
      FLIP_TEST: True
      IMAGE_THRE: 0.0
      IN_VIS_THRE: 0.2
      MODEL_FILE: models/pytorch/pose_hrnet_w48_384x288.pth
      NMS_THRE: 1.0
      OKS_THRE: 0.9
      POST_PROCESS: True
      SHIFT_HEATMAP: True
      SOFT_NMS: False
      USE_GT_BBOX: False
    TRAIN:
      BATCH_SIZE_PER_GPU: 24
      BEGIN_EPOCH: 0
      CHECKPOINT: 
      END_EPOCH: 210
      GAMMA1: 0.99
      GAMMA2: 0.0
      LR: 0.001
      LR_FACTOR: 0.1
      LR_STEP: [170, 200]
      MOMENTUM: 0.9
      NESTEROV: False
      OPTIMIZER: adam
      RESUME: False
      SHUFFLE: True
      WD: 0.0001
    WORKERS: 24
    => loading model from models/pytorch/pose_hrnet_w48_384x288.pth
    loading annotations into memory...
    Done (t=0.12s)
    creating index...
    index created!
    => classes: ['__background__', 'person']
    => num_images: 5000
    => Total boxes: 104125
    => Total boxes after fliter low [email protected]: 104125
    => load 104125 samples
    Traceback (most recent call last):
      File "/mnt/e/hi_5/deep-high-resolution-net.pytorch/tools/test.py", line 130, in <module>
        main()
      File "/mnt/e/hi_5/deep-high-resolution-net.pytorch/tools/test.py", line 125, in main
        validate(cfg, valid_loader, valid_dataset, model, criterion,
      File "/mnt/e/hi_5/deep-high-resolution-net.pytorch/tools/../lib/core/function.py", line 118, in validate
        for i, (input, target, target_weight, meta) in enumerate(val_loader):
      File "/home/roc-hci/anaconda3/envs/hi5/lib/python3.10/site-packages/torch/utils/data/dataloader.py", line 652, in __next__
        data = self._next_data()
      File "/home/roc-hci/anaconda3/envs/hi5/lib/python3.10/site-packages/torch/utils/data/dataloader.py", line 1347, in _next_data
        return self._process_data(data)
      File "/home/roc-hci/anaconda3/envs/hi5/lib/python3.10/site-packages/torch/utils/data/dataloader.py", line 1373, in _process_data
        data.reraise()
      File "/home/roc-hci/anaconda3/envs/hi5/lib/python3.10/site-packages/torch/_utils.py", line 461, in reraise
        raise exception
    RuntimeError: Caught RuntimeError in pin memory thread for device 0.
    Original Traceback (most recent call last):
      File "/home/roc-hci/anaconda3/envs/hi5/lib/python3.10/site-packages/torch/utils/data/_utils/pin_memory.py", line 34, in _pin_memory_loop
        data = pin_memory(data, device)
      File "/home/roc-hci/anaconda3/envs/hi5/lib/python3.10/site-packages/torch/utils/data/_utils/pin_memory.py", line 65, in pin_memory
        return type(data)([pin_memory(sample, device) for sample in data])  # type: ignore[call-arg]
      File "/home/roc-hci/anaconda3/envs/hi5/lib/python3.10/site-packages/torch/utils/data/_utils/pin_memory.py", line 65, in <listcomp>
        return type(data)([pin_memory(sample, device) for sample in data])  # type: ignore[call-arg]
      File "/home/roc-hci/anaconda3/envs/hi5/lib/python3.10/site-packages/torch/utils/data/_utils/pin_memory.py", line 50, in pin_memory
        return data.pin_memory(device)
    RuntimeError: CUDA error: out of memory
    CUDA kernel errors might be asynchronously reported at some other API call,so the stacktrace below might be incorrect.
    For debugging consider passing CUDA_LAUNCH_BLOCKING=1.
    
    terminate called without an active exception```
    
    
    ### --------------------------------
    
    
    > nvidia-smi
    ```Tue Jul 19 07:54:49 2022       
    +-----------------------------------------------------------------------------+
    | NVIDIA-SMI 515.57       Driver Version: 516.59       CUDA Version: 11.7     |
    |-------------------------------+----------------------+----------------------+
    | GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
    | Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
    |                               |                      |               MIG M. |
    |===============================+======================+======================|
    |   0  NVIDIA GeForce ...  On   | 00000000:0A:00.0  On |                  N/A |
    |  0%   49C    P8    24W / 370W |    683MiB / 24576MiB |      2%      Default |
    |                               |                      |                  N/A |
    +-------------------------------+----------------------+----------------------+
    |   1  NVIDIA GeForce ...  On   | 00000000:0B:00.0 Off |                  N/A |
    |  0%   39C    P8    14W / 370W |      0MiB / 24576MiB |      0%      Default |
    |                               |                      |                  N/A |
    +-------------------------------+----------------------+----------------------+
                                                                                   
    +-----------------------------------------------------------------------------+
    | Processes:                                                                  |
    |  GPU   GI   CI        PID   Type   Process name                  GPU Memory |
    |        ID   ID                                                   Usage      |
    |=============================================================================|
    |  No running processes found                                                 |
    +-----------------------------------------------------------------------------+```
    opened by Masum06 0
This project is the official implementation of our accepted ICLR 2021 paper BiPointNet: Binary Neural Network for Point Clouds.

BiPointNet: Binary Neural Network for Point Clouds Created by Haotong Qin, Zhongang Cai, Mingyuan Zhang, Yifu Ding, Haiyu Zhao, Shuai Yi, Xianglong Li

Haotong Qin 58 Nov 6, 2022
The project is an official implementation of our paper "3D Human Pose Estimation with Spatial and Temporal Transformers".

3D Human Pose Estimation with Spatial and Temporal Transformers This repo is the official implementation for 3D Human Pose Estimation with Spatial and

Ce Zheng 354 Nov 15, 2022
PyTorch implementation of our Adam-NSCL algorithm from our CVPR2021 (oral) paper "Training Networks in Null Space for Continual Learning"

Adam-NSCL This is a PyTorch implementation of Adam-NSCL algorithm for continual learning from our CVPR2021 (oral) paper: Title: Training Networks in N

Shipeng Wang 33 Nov 3, 2022
This project is the PyTorch implementation of our CVPR 2022 paper:

Requirements and Dependency Install PyTorch with CUDA (for GPU). (Experiments are validated on python 3.8.11 and pytorch 1.7.0) (For visualization if

Lei Huang 22 Nov 5, 2022
Official implementation of our paper "LLA: Loss-aware Label Assignment for Dense Pedestrian Detection" in Pytorch.

LLA: Loss-aware Label Assignment for Dense Pedestrian Detection This project provides an implementation for "LLA: Loss-aware Label Assignment for Dens

null 34 Aug 31, 2022
Official implementation of our CVPR2021 paper "OTA: Optimal Transport Assignment for Object Detection" in Pytorch.

OTA: Optimal Transport Assignment for Object Detection This project provides an implementation for our CVPR2021 paper "OTA: Optimal Transport Assignme

null 214 Nov 17, 2022
This is an official implementation of our CVPR 2021 paper "Bottom-Up Human Pose Estimation Via Disentangled Keypoint Regression" (https://arxiv.org/abs/2104.02300)

Bottom-Up Human Pose Estimation Via Disentangled Keypoint Regression Introduction In this paper, we are interested in the bottom-up paradigm of estima

HRNet 356 Nov 23, 2022
The official pytorch implementation of our paper "Is Space-Time Attention All You Need for Video Understanding?"

TimeSformer This is an official pytorch implementation of Is Space-Time Attention All You Need for Video Understanding?. In this repository, we provid

Facebook Research 1k Nov 22, 2022
Official implementation of GraphMask as presented in our paper Interpreting Graph Neural Networks for NLP With Differentiable Edge Masking.

GraphMask This repository contains an implementation of GraphMask, the interpretability technique for graph neural networks presented in our ICLR 2021

Michael Schlichtkrull 29 Sep 2, 2022
The official implementation of our CVPR 2021 paper - Hybrid Rotation Averaging: A Fast and Robust Rotation Averaging Approach

Graph Optimizer This repo contains the official implementation of our CVPR 2021 paper - Hybrid Rotation Averaging: A Fast and Robust Rotation Averagin

Chenyu 107 Nov 13, 2022
The repository offers the official implementation of our paper in PyTorch.

Cloth Interactive Transformer (CIT) Cloth Interactive Transformer for Virtual Try-On Bin Ren1, Hao Tang1, Fanyang Meng2, Runwei Ding3, Ling Shao4, Phi

Bingoren 48 Oct 1, 2022
This is the official pytorch implementation for our ICCV 2021 paper "TRAR: Routing the Attention Spans in Transformers for Visual Question Answering" on VQA Task

?? ERASOR (RA-L'21 with ICRA Option) Official page of "ERASOR: Egocentric Ratio of Pseudo Occupancy-based Dynamic Object Removal for Static 3D Point C

Hyungtae Lim 211 Nov 10, 2022
Official implementation of our paper "Learning to Bootstrap for Combating Label Noise"

Learning to Bootstrap for Combating Label Noise This repo is the official implementation of our paper "Learning to Bootstrap for Combating Label Noise

null 21 Apr 9, 2022
Convolutional neural network web app trained to track our infant’s sleep schedule using our Google Nest camera.

Machine Learning Sleep Schedule Tracker What is it? Convolutional neural network web app trained to track our infant’s sleep schedule using our Google

g-parki 7 Jul 15, 2022
This is the official code of our paper "Diversity-based Trajectory and Goal Selection with Hindsight Experience Relay" (PRICAI 2021)

Diversity-based Trajectory and Goal Selection with Hindsight Experience Replay This is the official implementation of our paper "Diversity-based Traje

Tianhong Dai 6 Jul 18, 2022
Official code for our ICCV paper: "From Continuity to Editability: Inverting GANs with Consecutive Images"

GANInversion_with_ConsecutiveImgs Official code for our ICCV paper: "From Continuity to Editability: Inverting GANs with Consecutive Images" https://a

QingyangXu 36 Oct 6, 2022
Official PyTorch implemention of our paper "Learning to Rectify for Robust Learning with Noisy Labels".

WarPI The official PyTorch implemention of our paper "Learning to Rectify for Robust Learning with Noisy Labels". Run python main.py --corruption_type

Haoliang Sun 3 Sep 3, 2022
The official codes of our CVPR2022 paper: A Differentiable Two-stage Alignment Scheme for Burst Image Reconstruction with Large Shift

TwoStageAlign The official codes of our CVPR2022 paper: A Differentiable Two-stage Alignment Scheme for Burst Image Reconstruction with Large Shift Pa

Shi Guo 32 Oct 11, 2022