The project is an official implementation of our CVPR2019 paper "Deep High-Resolution Representation Learning for Human Pose Estimation"

Overview

Deep High-Resolution Representation Learning for Human Pose Estimation (CVPR 2019)

News

Introduction

This is an official pytorch implementation of Deep High-Resolution Representation Learning for Human Pose Estimation. In this work, we are interested in the human pose estimation problem with a focus on learning reliable high-resolution representations. Most existing methods recover high-resolution representations from low-resolution representations produced by a high-to-low resolution network. Instead, our proposed network maintains high-resolution representations through the whole process. We start from a high-resolution subnetwork as the first stage, gradually add high-to-low resolution subnetworks one by one to form more stages, and connect the mutli-resolution subnetworks in parallel. We conduct repeated multi-scale fusions such that each of the high-to-low resolution representations receives information from other parallel representations over and over, leading to rich high-resolution representations. As a result, the predicted keypoint heatmap is potentially more accurate and spatially more precise. We empirically demonstrate the effectiveness of our network through the superior pose estimation results over two benchmark datasets: the COCO keypoint detection dataset and the MPII Human Pose dataset.

Illustrating the architecture of the proposed HRNet

Main Results

Results on MPII val

Arch Head Shoulder Elbow Wrist Hip Knee Ankle Mean [email protected]
pose_resnet_50 96.4 95.3 89.0 83.2 88.4 84.0 79.6 88.5 34.0
pose_resnet_101 96.9 95.9 89.5 84.4 88.4 84.5 80.7 89.1 34.0
pose_resnet_152 97.0 95.9 90.0 85.0 89.2 85.3 81.3 89.6 35.0
pose_hrnet_w32 97.1 95.9 90.3 86.4 89.1 87.1 83.3 90.3 37.7

Note:

Results on COCO val2017 with detector having human AP of 56.4 on COCO val2017 dataset

Arch Input size #Params GFLOPs AP Ap .5 AP .75 AP (M) AP (L) AR AR .5 AR .75 AR (M) AR (L)
pose_resnet_50 256x192 34.0M 8.9 0.704 0.886 0.783 0.671 0.772 0.763 0.929 0.834 0.721 0.824
pose_resnet_50 384x288 34.0M 20.0 0.722 0.893 0.789 0.681 0.797 0.776 0.932 0.838 0.728 0.846
pose_resnet_101 256x192 53.0M 12.4 0.714 0.893 0.793 0.681 0.781 0.771 0.934 0.840 0.730 0.832
pose_resnet_101 384x288 53.0M 27.9 0.736 0.896 0.803 0.699 0.811 0.791 0.936 0.851 0.745 0.858
pose_resnet_152 256x192 68.6M 15.7 0.720 0.893 0.798 0.687 0.789 0.778 0.934 0.846 0.736 0.839
pose_resnet_152 384x288 68.6M 35.3 0.743 0.896 0.811 0.705 0.816 0.797 0.937 0.858 0.751 0.863
pose_hrnet_w32 256x192 28.5M 7.1 0.744 0.905 0.819 0.708 0.810 0.798 0.942 0.865 0.757 0.858
pose_hrnet_w32 384x288 28.5M 16.0 0.758 0.906 0.825 0.720 0.827 0.809 0.943 0.869 0.767 0.871
pose_hrnet_w48 256x192 63.6M 14.6 0.751 0.906 0.822 0.715 0.818 0.804 0.943 0.867 0.762 0.864
pose_hrnet_w48 384x288 63.6M 32.9 0.763 0.908 0.829 0.723 0.834 0.812 0.942 0.871 0.767 0.876

Note:

Results on COCO test-dev2017 with detector having human AP of 60.9 on COCO test-dev2017 dataset

Arch Input size #Params GFLOPs AP Ap .5 AP .75 AP (M) AP (L) AR AR .5 AR .75 AR (M) AR (L)
pose_resnet_152 384x288 68.6M 35.3 0.737 0.919 0.828 0.713 0.800 0.790 0.952 0.856 0.748 0.849
pose_hrnet_w48 384x288 63.6M 32.9 0.755 0.925 0.833 0.719 0.815 0.805 0.957 0.874 0.763 0.863
pose_hrnet_w48* 384x288 63.6M 32.9 0.770 0.927 0.845 0.734 0.831 0.820 0.960 0.886 0.778 0.877

Note:

Environment

The code is developed using python 3.6 on Ubuntu 16.04. NVIDIA GPUs are needed. The code is developed and tested using 4 NVIDIA P100 GPU cards. Other platforms or GPU cards are not fully tested.

Quick start

Installation

  1. Install pytorch >= v1.0.0 following official instruction. Note that if you use pytorch's version < v1.0.0, you should following the instruction at https://github.com/Microsoft/human-pose-estimation.pytorch to disable cudnn's implementations of BatchNorm layer. We encourage you to use higher pytorch's version(>=v1.0.0)

  2. Clone this repo, and we'll call the directory that you cloned as ${POSE_ROOT}.

  3. Install dependencies:

    pip install -r requirements.txt
    
  4. Make libs:

    cd ${POSE_ROOT}/lib
    make
    
  5. Install COCOAPI:

    # COCOAPI=/path/to/clone/cocoapi
    git clone https://github.com/cocodataset/cocoapi.git $COCOAPI
    cd $COCOAPI/PythonAPI
    # Install into global site-packages
    make install
    # Alternatively, if you do not have permissions or prefer
    # not to install the COCO API into global site-packages
    python3 setup.py install --user
    

    Note that instructions like # COCOAPI=/path/to/install/cocoapi indicate that you should pick a path where you'd like to have the software cloned and then set an environment variable (COCOAPI in this case) accordingly.

  6. Init output(training model output directory) and log(tensorboard log directory) directory:

    mkdir output 
    mkdir log
    

    Your directory tree should look like this:

    ${POSE_ROOT}
    ├── data
    ├── experiments
    ├── lib
    ├── log
    ├── models
    ├── output
    ├── tools 
    ├── README.md
    └── requirements.txt
    
  7. Download pretrained models from our model zoo(GoogleDrive or OneDrive)

    ${POSE_ROOT}
     `-- models
         `-- pytorch
             |-- imagenet
             |   |-- hrnet_w32-36af842e.pth
             |   |-- hrnet_w48-8ef0771d.pth
             |   |-- resnet50-19c8e357.pth
             |   |-- resnet101-5d3b4d8f.pth
             |   `-- resnet152-b121ed2d.pth
             |-- pose_coco
             |   |-- pose_hrnet_w32_256x192.pth
             |   |-- pose_hrnet_w32_384x288.pth
             |   |-- pose_hrnet_w48_256x192.pth
             |   |-- pose_hrnet_w48_384x288.pth
             |   |-- pose_resnet_101_256x192.pth
             |   |-- pose_resnet_101_384x288.pth
             |   |-- pose_resnet_152_256x192.pth
             |   |-- pose_resnet_152_384x288.pth
             |   |-- pose_resnet_50_256x192.pth
             |   `-- pose_resnet_50_384x288.pth
             `-- pose_mpii
                 |-- pose_hrnet_w32_256x256.pth
                 |-- pose_hrnet_w48_256x256.pth
                 |-- pose_resnet_101_256x256.pth
                 |-- pose_resnet_152_256x256.pth
                 `-- pose_resnet_50_256x256.pth
    
    

Data preparation

For MPII data, please download from MPII Human Pose Dataset. The original annotation files are in matlab format. We have converted them into json format, you also need to download them from OneDrive or GoogleDrive. Extract them under {POSE_ROOT}/data, and make them look like this:

${POSE_ROOT}
|-- data
`-- |-- mpii
    `-- |-- annot
        |   |-- gt_valid.mat
        |   |-- test.json
        |   |-- train.json
        |   |-- trainval.json
        |   `-- valid.json
        `-- images
            |-- 000001163.jpg
            |-- 000003072.jpg

For COCO data, please download from COCO download, 2017 Train/Val is needed for COCO keypoints training and validation. We also provide person detection result of COCO val2017 and test-dev2017 to reproduce our multi-person pose estimation results. Please download from OneDrive or GoogleDrive. Download and extract them under {POSE_ROOT}/data, and make them look like this:

${POSE_ROOT}
|-- data
`-- |-- coco
    `-- |-- annotations
        |   |-- person_keypoints_train2017.json
        |   `-- person_keypoints_val2017.json
        |-- person_detection_results
        |   |-- COCO_val2017_detections_AP_H_56_person.json
        |   |-- COCO_test-dev2017_detections_AP_H_609_person.json
        `-- images
            |-- train2017
            |   |-- 000000000009.jpg
            |   |-- 000000000025.jpg
            |   |-- 000000000030.jpg
            |   |-- ... 
            `-- val2017
                |-- 000000000139.jpg
                |-- 000000000285.jpg
                |-- 000000000632.jpg
                |-- ... 

Training and Testing

Testing on MPII dataset using model zoo's models(GoogleDrive or OneDrive)

python tools/test.py \
    --cfg experiments/mpii/hrnet/w32_256x256_adam_lr1e-3.yaml \
    TEST.MODEL_FILE models/pytorch/pose_mpii/pose_hrnet_w32_256x256.pth

Training on MPII dataset

python tools/train.py \
    --cfg experiments/mpii/hrnet/w32_256x256_adam_lr1e-3.yaml

Testing on COCO val2017 dataset using model zoo's models(GoogleDrive or OneDrive)

python tools/test.py \
    --cfg experiments/coco/hrnet/w32_256x192_adam_lr1e-3.yaml \
    TEST.MODEL_FILE models/pytorch/pose_coco/pose_hrnet_w32_256x192.pth \
    TEST.USE_GT_BBOX False

Training on COCO train2017 dataset

python tools/train.py \
    --cfg experiments/coco/hrnet/w32_256x192_adam_lr1e-3.yaml \

Visualization

Visualizing predictions on COCO val

python visualization/plot_coco.py \
    --prediction output/coco/w48_384x288_adam_lr1e-3/results/keypoints_val2017_results_0.json \
    --save-path visualization/results

Other applications

Many other dense prediction tasks, such as segmentation, face alignment and object detection, etc. have been benefited by HRNet. More information can be found at High-Resolution Networks.

Other implementation

mmpose

Citation

If you use our code or models in your research, please cite with:

@inproceedings{sun2019deep,
  title={Deep High-Resolution Representation Learning for Human Pose Estimation},
  author={Sun, Ke and Xiao, Bin and Liu, Dong and Wang, Jingdong},
  booktitle={CVPR},
  year={2019}
}

@inproceedings{xiao2018simple,
    author={Xiao, Bin and Wu, Haiping and Wei, Yichen},
    title={Simple Baselines for Human Pose Estimation and Tracking},
    booktitle = {European Conference on Computer Vision (ECCV)},
    year = {2018}
}

@article{WangSCJDZLMTWLX19,
  title={Deep High-Resolution Representation Learning for Visual Recognition},
  author={Jingdong Wang and Ke Sun and Tianheng Cheng and 
          Borui Jiang and Chaorui Deng and Yang Zhao and Dong Liu and Yadong Mu and 
          Mingkui Tan and Xinggang Wang and Wenyu Liu and Bin Xiao},
  journal   = {TPAMI}
  year={2019}
}
Comments
  • Error when training/testing on COCO dataset

    Error when training/testing on COCO dataset

    My problem is similar to #100 , but I have tried all possible solutions in that issue and still can not solve the problem...

    Now I want to training/testing on COCO dataset, and visualize the model by using tensorboardX. Unfortunately, errors occurred one by one... At first, I met the same problem as this, then I downgrade my tensorboardX version to 1.6, and met the same problem as this, then I tried to degrade other package like torch, torchvision, etc., but it didn't work as well.

    Currently my version is as follows:

    torch 1.0.0
    torchvision 0.2.1
    tensorboard 1.6.0
    tensorboardX 1.6
    tensorflow 1.0.0
    cudatoolkit 9.0
    

    When I want to start training using the following command:

    python tools/train.py --cfg experiments/coco/hrnet/w32_256x192_adam_lr1e-3_coco2014.yaml
    

    It tells me:

    RuntimeError: CUDA out of memory. Tried to allocate 12.00 MiB (GPU 1; 11.93 GiB total capacity; 2.48 GiB already allocated; 2.81 MiB free; 5.46 MiB cached)
    

    It seems that the part to do with tensorboardX is successfully completed, and error occurred when begin training. What I want to know is that: Is this problem really caused by the memory of CUDA?Or it is caused by the old version of pytorch? Any code need to modified?

    And I also add the following codes into test.py, in order to visualize the model by using tensorboardX.

        writer_dict = {
            'writer': SummaryWriter(log_dir=tb_log_dir),
            'train_global_steps': 0,
            'valid_global_steps': 0,
        }
        dump_input = torch.rand(
            (1, 3, cfg.MODEL.IMAGE_SIZE[1], cfg.MODEL.IMAGE_SIZE[0])
        )
        writer_dict['writer'].add_graph(model, (dump_input, ))
    

    When I use the following command to test...

    python tools/test.py --cfg experiments/coco/hrnet/w32_256x192_adam_lr1e-3_coco2014.yaml TEST.MODEL_FILE models/pytorch/pose_coco/pose_hrnet_w32_256x192.pth TEST.USE_GT_BBOX False
    

    The error to do with tensorboardX occurred again!

    /home/fangqingkai/anaconda3/envs/HRNet/lib/python3.6/site-packages/torch/nn/modules/upsampling.py:129: UserWarning: nn.Upsample is deprecated. Use nn.functional.interpolate instead.
      warnings.warn("nn.{} is deprecated. Use nn.functional.interpolate instead.".format(self.name))
    Traceback (most recent call last):
      File "tools/test.py", line 142, in <module>
        main()
      File "tools/test.py", line 110, in main
        writer_dict['writer'].add_graph(model, (dump_input, ))
      File "/home/fangqingkai/anaconda3/envs/HRNet/lib/python3.6/site-packages/tensorboardX/writer.py", line 566, in add_graph
        self.file_writer.add_graph(graph(model, input_to_model, verbose))
      File "/home/fangqingkai/anaconda3/envs/HRNet/lib/python3.6/site-packages/tensorboardX/pytorch_graph.py", line 235, in graph
        _optimize_trace(trace, torch.onnx.utils.OperatorExportTypes.ONNX)
      File "/home/fangqingkai/anaconda3/envs/HRNet/lib/python3.6/site-packages/tensorboardX/pytorch_graph.py", line 175, in _optimize_trace
        trace.set_graph(_optimize_graph(trace.graph(), operator_export_type))
      File "/home/fangqingkai/anaconda3/envs/HRNet/lib/python3.6/site-packages/tensorboardX/pytorch_graph.py", line 201, in _optimize_graph
        torch._C._jit_pass_lower_all_tuples(graph)
    RuntimeError: tuple appears in op that does not forward tuples (VisitNode at /opt/conda/conda-bld/pytorch_1544174967633/work/torch/csrc/jit/passes/lower_tuples.cpp:109)
    frame #0: c10::Error::Error(c10::SourceLocation, std::string const&) + 0x45 (0x7fc59951fcc5 in /home/fangqingkai/anaconda3/envs/HRNet/lib/python3.6/site-packages/torch/lib/libc10.so)
    frame #1: <unknown function> + 0x6d6882 (0x7fc596bf0882 in /home/fangqingkai/anaconda3/envs/HRNet/lib/python3.6/site-packages/torch/lib/libtorch.so.1)
    frame #2: <unknown function> + 0x6d6ad4 (0x7fc596bf0ad4 in /home/fangqingkai/anaconda3/envs/HRNet/lib/python3.6/site-packages/torch/lib/libtorch.so.1)
    frame #3: torch::jit::LowerAllTuples(std::shared_ptr<torch::jit::Graph>&) + 0x13 (0x7fc596bf0be3 in /home/fangqingkai/anaconda3/envs/HRNet/lib/python3.6/site-packages/torch/lib/libtorch.so.1)
    frame #4: <unknown function> + 0x350b24 (0x7fc5c99fbb24 in /home/fangqingkai/anaconda3/envs/HRNet/lib/python3.6/site-packages/torch/lib/libtorch_python.so)
    frame #5: <unknown function> + 0x111e36 (0x7fc5c97bce36 in /home/fangqingkai/anaconda3/envs/HRNet/lib/python3.6/site-packages/torch/lib/libtorch_python.so)
    <omitting python frames>
    frame #35: __libc_start_main + 0xf0 (0x7fc5db560830 in /lib/x86_64-linux-gnu/libc.so.6)
    

    It confused me that why the same code has not caused error in train.py but in test.py...? And anyone can help me solve this problem? Thanks a lot!!!

    opened by Poeroz 18
  • ModuleNotFoundError: No module named 'nms.cpu_nms'

    ModuleNotFoundError: No module named 'nms.cpu_nms'

    myubuntu@myubuntu-Precision-WorkStation-T5500:~/Desktop/deep-high-resolution-net.pytorch-master$ python tools/test.py \ --cfg experiments/coco/hrnet/w32_256x192_adam_lr1e-3.yaml \ TEST.MODEL_FILE models/pytorch/pose_coco/pose_hrnet_w32_256x192.pth \ TEST.USE_GT_BBOX False Traceback (most recent call last): File "tools/test.py", line 31, in import dataset File "/home/myubuntu/Desktop/deep-high-resolution-net.pytorch-master/tools/../lib/dataset/init.py", line 12, in from .coco import COCODataset as coco File "/home/myubuntu/Desktop/deep-high-resolution-net.pytorch-master/tools/../lib/dataset/coco.py", line 22, in from nms.nms import oks_nms File "/home/myubuntu/Desktop/deep-high-resolution-net.pytorch-master/tools/../lib/nms/nms.py", line 13, in from .cpu_nms import cpu_nms ModuleNotFoundError: No module named 'nms.cpu_nms'

    opened by tianjiahao 11
  • AttributeError: 'NoneType' object has no attribute '__name__'

    AttributeError: 'NoneType' object has no attribute '__name__'

    when i do: python tools/train.py
    --cfg experiments/mpii/hrnet/w32_256x256_adam_lr1e-3.yaml an error occurrs: 图片

    However,when i do: python tools/test.py
    --cfg experiments/mpii/hrnet/w32_256x256_adam_lr1e-3.yaml
    TEST.MODEL_FILE models/pytorch/pose_mpii/pose_hrnet_w32_256x256.pth there is nothing wrong.

    opened by angechen 8
  • Low mAP obtained in testing Simple-baseline model using code in this repo.

    Low mAP obtained in testing Simple-baseline model using code in this repo.

    I test the simple baseline models downloaded from your link using the code in this repo. I just get 65.3 mAP on pose_resnet50. Flip-test, Ground-truth bbox, are used, and input size is 256*192. GPU: 4 k80, batch-size:32. And I test onresnet101, 152, it's still lower than the results in simple-baseline repo.

    opened by qiuzhongwei-USTB 6
  • How Can I Get The Pretrained Weights

    How Can I Get The Pretrained Weights

    I am in china and cannot open OneDrive or GoogleDrive ,please tell me how can i get the pretrained pth file? Can you upload the file to Baidu cloud disk?thanks a lot

    opened by JiangWeiHn 5
  • nvidia-docker is deprecated.

    nvidia-docker is deprecated.

    readme of demo said, we can run container by nvidia-docker, but latest version of docker does not support nvidia-docker command.

    we can use

    docker run --gpus all --rm -it \
        -v $(pwd)/output:/output \
        -v $(pwd)/videos:/videos \
        -v $(pwd)/models:/models \
        -w /pose_root \
        hrnet_demo_inference \
        /bin/bash
    

    instead of nvidia-docker

    opened by sklationd 5
  • I test the model on MPII

    I test the model on MPII

    File "tools/test.py", line 131, in main() File "tools/test.py", line 99, in main model = torch.nn.DataParallel(model, device_ids=cfg.GPUS).cuda()

    lib/python3.6/site-packages/torch/cuda/init.py", line 318, in get_device_properties raise AssertionError("Invalid device id") AssertionError: Invalid device id

    opened by henbucuoshanghai 4
  • How can I evaluate my model on MPII test set?

    How can I evaluate my model on MPII test set?

    How can I test my model on test set and get the output file like evaluatePCKh.m on MPII website? Is there any tools for converting test results to .m file? THX

    opened by xtyDoge 4
  • opencv error

    opencv error

    Thanks for sharing the code. When I tried to run the test commands both failed with cv2.error: OpenCV(3.4.1) /io/opencv/modules/imgproc/src/color.cpp:11115: error: (-215) scn == 3 || scn == 4 in function cvtColor. Am I missing something? Thanks,

    opened by mistycheney 4
  • 请教代码意思。

    请教代码意思。

    https://github.com/leoxiaobin/deep-high-resolution-net.pytorch/blob/ba50a82dce412df97f088c572d86d7977753bf74/lib/dataset/mpii.py#L58

    作者您好,请问下这里避免裁剪到肢体, 是自己自定义轻微调整center 和 scale的值吗? 具体表示的是什么意思呢? 纵坐标和尺度发生变化?

    opened by YuQi9797 3
  • mAP difference in MSCOCO val

    mAP difference in MSCOCO val

    您好!HRNet的工作真的非常棒,非常感谢开源代码! 有几个问题,想请教一下: person_keypoints_val2017.json和COCO_val2017_detections_AP_H_56_person.json有什么区别吗? 如果理解无误的话,使用的都是val2017验证集。那为什么在每次训练epoch结束的模型评价mAP值,要比test.py所得的mAP 低

    这是在train中最后一个训练轮次结束所得mAP

    2020-06-22 19:56:58,671 | Arch | AP | Ap .5 | AP .75 | AP (M) | AP (L) | AR | AR .5 | AR .75 | AR (M) | AR (L) | 2020-06-22 19:56:58,672 |---|---|---|---|---|---|---|---|---|---|---| 2020-06-22 19:56:58,672 | pose_hrnet | 0.744 | 0.926 | 0.816 | 0.722 | 0.783 | 0.774 | 0.935 | 0.838 | 0.746 | 0.817 |

    这是通过val检测final_state.pth的mAP

    2020-06-23 04:13:00,041 | Arch | AP | Ap .5 | AP .75 | AP (M) | AP (L) | AR | AR .5 | AR .75 | AR (M) | AR (L) | 2020-06-23 04:13:00,043 |---|---|---|---|---|---|---|---|---|---|---| 2020-06-23 04:13:00,043 | pose_hrnet | 0.724 | 0.892 | 0.799 | 0.692 | 0.789 | 0.780 | 0.932 | 0.848 | 0.740 | 0.839 |

    两者相差了2个点,我认为这应该与json文件的选取相关。 以及,请问在论文中:Table 1. Comparisons on the COCO validation set,使用的是哪个mAP值。 还望予以赐教,谢谢!

    opened by ZW-XXX 3
  • Training on COCO dataset

    Training on COCO dataset

    Thank you for the amazing work! I am trying to train the model on Coco dataset on two Gpus. As soon the images are loaded my machine goes into an unresponsive stage. I am using python 3.9, pytorch 1.11.0, tensorboardX= 2.1, no_of_workers=12, Batch_Size_Per_GPU = 16. Could you please guide me what could be causing this issue. Thanks

    opened by anas-zafar 0
  • cv2.error: Caught error in DataLoader worker process 0.

    cv2.error: Caught error in DataLoader worker process 0.

    when I run :"python tools/test.py
    --cfg experiments/coco/hrnet/w32_256x192_adam_lr1e-3.yaml
    TEST.MODEL_FILE models/pytorch/pose_coco/pose_hrnet_w32_256x192.pth
    TEST.USE_GT_BBOX False" to test on COCO val2017 dataset using model zoo's models,it occurs the error: => creating output/coco/pose_hrnet/w32_256x192_adam_lr1e-3 => creating log/coco/pose_hrnet/w32_256x192_adam_lr1e-3_2022-12-07-21-01 Namespace(cfg='experiments/coco/hrnet/w32_256x192_adam_lr1e-3.yaml', dataDir='', logDir='', modelDir='', opts=['TEST.MODEL_FILE', 'models/pytorch/pose_coco/pose_hrnet_w32_256x192.pth', 'TEST.USE_GT_BBOX', 'False'], prevModelDir='') AUTO_RESUME: True CUDNN: BENCHMARK: True DETERMINISTIC: False ENABLED: True DATASET: COLOR_RGB: True DATASET: coco DATA_FORMAT: jpg FLIP: True HYBRID_JOINTS_TYPE: NUM_JOINTS_HALF_BODY: 8 PROB_HALF_BODY: 0.3 ROOT: data/coco/ ROT_FACTOR: 45 SCALE_FACTOR: 0.35 SELECT_DATA: False TEST_SET: val2017 TRAIN_SET: train2017 DATA_DIR: DEBUG: DEBUG: True SAVE_BATCH_IMAGES_GT: True SAVE_BATCH_IMAGES_PRED: True SAVE_HEATMAPS_GT: True SAVE_HEATMAPS_PRED: True GPUS: (0, 1, 2, 3) LOG_DIR: log LOSS: TOPK: 8 USE_DIFFERENT_JOINTS_WEIGHT: False USE_OHKM: False USE_TARGET_WEIGHT: True MODEL: EXTRA: FINAL_CONV_KERNEL: 1 PRETRAINED_LAYERS: ['conv1', 'bn1', 'conv2', 'bn2', 'layer1', 'transition1', 'stage2', 'transition2', 'stage3', 'transition3', 'stage4'] STAGE2: BLOCK: BASIC FUSE_METHOD: SUM NUM_BLOCKS: [4, 4] NUM_BRANCHES: 2 NUM_CHANNELS: [32, 64] NUM_MODULES: 1 STAGE3: BLOCK: BASIC FUSE_METHOD: SUM NUM_BLOCKS: [4, 4, 4] NUM_BRANCHES: 3 NUM_CHANNELS: [32, 64, 128] NUM_MODULES: 4 STAGE4: BLOCK: BASIC FUSE_METHOD: SUM NUM_BLOCKS: [4, 4, 4, 4] NUM_BRANCHES: 4 NUM_CHANNELS: [32, 64, 128, 256] NUM_MODULES: 3 HEATMAP_SIZE: [48, 64] IMAGE_SIZE: [192, 256] INIT_WEIGHTS: True NAME: pose_hrnet NUM_JOINTS: 17 PRETRAINED: models/pytorch/imagenet/hrnet_w32-36af842e.pth SIGMA: 2 TAG_PER_JOINT: True TARGET_TYPE: gaussian OUTPUT_DIR: output PIN_MEMORY: True PRINT_FREQ: 100 RANK: 0 TEST: BATCH_SIZE_PER_GPU: 32 BBOX_THRE: 1.0 COCO_BBOX_FILE: data/coco/person_detection_results/COCO_val2017_detections_AP_H_56_person.json FLIP_TEST: True IMAGE_THRE: 0.0 IN_VIS_THRE: 0.2 MODEL_FILE: models/pytorch/pose_coco/pose_hrnet_w32_256x192.pth NMS_THRE: 1.0 OKS_THRE: 0.9 POST_PROCESS: True SHIFT_HEATMAP: True SOFT_NMS: False USE_GT_BBOX: False TRAIN: BATCH_SIZE_PER_GPU: 32 BEGIN_EPOCH: 0 CHECKPOINT: END_EPOCH: 210 GAMMA1: 0.99 GAMMA2: 0.0 LR: 0.001 LR_FACTOR: 0.1 LR_STEP: [170, 200] MOMENTUM: 0.9 NESTEROV: False OPTIMIZER: adam RESUME: False SHUFFLE: True WD: 0.0001 WORKERS: 24 => loading model from models/pytorch/pose_coco/pose_hrnet_w32_256x192.pth loading annotations into memory... Done (t=0.15s) creating index... index created! => classes: ['background', 'person'] => num_images: 5000 => Total boxes: 104125 => Total boxes after fliter low [email protected]: 104125 => load 104125 samples Traceback (most recent call last): File "tools/test.py", line 130, in main() File "tools/test.py", line 125, in main validate(cfg, valid_loader, valid_dataset, model, criterion, File "/home/ycj/deep-high-resolution-net.pytorch-master/tools/../lib/core/function.py", line 118, in validate for i, (input, target, target_weight, meta) in enumerate(val_loader): File "/home/ycj/anaconda3/lib/python3.8/site-packages/torch/utils/data/dataloader.py", line 435, in next data = self._next_data() File "/home/ycj/anaconda3/lib/python3.8/site-packages/torch/utils/data/dataloader.py", line 1085, in _next_data return self._process_data(data) File "/home/ycj/anaconda3/lib/python3.8/site-packages/torch/utils/data/dataloader.py", line 1111, in _process_data data.reraise() File "/home/ycj/anaconda3/lib/python3.8/site-packages/torch/_utils.py", line 428, in reraise raise self.exc_type(msg) cv2.error: Caught error in DataLoader worker process 0. Original Traceback (most recent call last): File "/home/ycj/anaconda3/lib/python3.8/site-packages/torch/utils/data/_utils/worker.py", line 198, in _worker_loop data = fetcher.fetch(index) File "/home/ycj/anaconda3/lib/python3.8/site-packages/torch/utils/data/_utils/fetch.py", line 44, in fetch data = [self.dataset[idx] for idx in possibly_batched_index] File "/home/ycj/anaconda3/lib/python3.8/site-packages/torch/utils/data/_utils/fetch.py", line 44, in data = [self.dataset[idx] for idx in possibly_batched_index] File "/home/ycj/deep-high-resolution-net.pytorch-master/tools/../lib/dataset/JointsDataset.py", line 131, in getitem data_numpy = cv2.cvtColor(data_numpy, cv2.COLOR_BGR2RGB) cv2.error: OpenCV(4.5.1) /tmp/pip-req-build-jr1ur_cf/opencv/modules/imgproc/src/color.cpp:182: error: (-215:Assertion failed) !_src.empty() in function 'cvtColor'

    2022-12-07 21-06-21 的屏幕截图

    can you give some advice? thank you!

    opened by ycj1124 0
  • How to convert mpii mat file to json

    How to convert mpii mat file to json

    When I test on MPII and COCO dataset using model zoo's models, I found COCO dataset will get a json file, but MPII file only get a mat file, does anyone know how to convert mpii mat file to json?

    opened by STRUGGLE1999 0
This project is the official implementation of our accepted ICLR 2021 paper BiPointNet: Binary Neural Network for Point Clouds.

BiPointNet: Binary Neural Network for Point Clouds Created by Haotong Qin, Zhongang Cai, Mingyuan Zhang, Yifu Ding, Haiyu Zhao, Shuai Yi, Xianglong Li

Haotong Qin 59 Dec 17, 2022
The project is an official implementation of our paper "3D Human Pose Estimation with Spatial and Temporal Transformers".

3D Human Pose Estimation with Spatial and Temporal Transformers This repo is the official implementation for 3D Human Pose Estimation with Spatial and

Ce Zheng 363 Dec 28, 2022
PyTorch implementation of our Adam-NSCL algorithm from our CVPR2021 (oral) paper "Training Networks in Null Space for Continual Learning"

Adam-NSCL This is a PyTorch implementation of Adam-NSCL algorithm for continual learning from our CVPR2021 (oral) paper: Title: Training Networks in N

Shipeng Wang 34 Dec 21, 2022
This project is the PyTorch implementation of our CVPR 2022 paper:

Requirements and Dependency Install PyTorch with CUDA (for GPU). (Experiments are validated on python 3.8.11 and pytorch 1.7.0) (For visualization if

Lei Huang 23 Nov 29, 2022
Official implementation of our paper "LLA: Loss-aware Label Assignment for Dense Pedestrian Detection" in Pytorch.

LLA: Loss-aware Label Assignment for Dense Pedestrian Detection This project provides an implementation for "LLA: Loss-aware Label Assignment for Dens

null 35 Dec 6, 2022
Official implementation of our CVPR2021 paper "OTA: Optimal Transport Assignment for Object Detection" in Pytorch.

OTA: Optimal Transport Assignment for Object Detection This project provides an implementation for our CVPR2021 paper "OTA: Optimal Transport Assignme

null 217 Jan 3, 2023
This is an official implementation of our CVPR 2021 paper "Bottom-Up Human Pose Estimation Via Disentangled Keypoint Regression" (https://arxiv.org/abs/2104.02300)

Bottom-Up Human Pose Estimation Via Disentangled Keypoint Regression Introduction In this paper, we are interested in the bottom-up paradigm of estima

HRNet 367 Dec 27, 2022
The official pytorch implementation of our paper "Is Space-Time Attention All You Need for Video Understanding?"

TimeSformer This is an official pytorch implementation of Is Space-Time Attention All You Need for Video Understanding?. In this repository, we provid

Facebook Research 1k Dec 31, 2022
Official implementation of GraphMask as presented in our paper Interpreting Graph Neural Networks for NLP With Differentiable Edge Masking.

GraphMask This repository contains an implementation of GraphMask, the interpretability technique for graph neural networks presented in our ICLR 2021

Michael Schlichtkrull 29 Sep 2, 2022
The official implementation of our CVPR 2021 paper - Hybrid Rotation Averaging: A Fast and Robust Rotation Averaging Approach

Graph Optimizer This repo contains the official implementation of our CVPR 2021 paper - Hybrid Rotation Averaging: A Fast and Robust Rotation Averagin

Chenyu 109 Dec 23, 2022
The repository offers the official implementation of our paper in PyTorch.

Cloth Interactive Transformer (CIT) Cloth Interactive Transformer for Virtual Try-On Bin Ren1, Hao Tang1, Fanyang Meng2, Runwei Ding3, Ling Shao4, Phi

Bingoren 49 Dec 1, 2022
This is the official pytorch implementation for our ICCV 2021 paper "TRAR: Routing the Attention Spans in Transformers for Visual Question Answering" on VQA Task

?? ERASOR (RA-L'21 with ICRA Option) Official page of "ERASOR: Egocentric Ratio of Pseudo Occupancy-based Dynamic Object Removal for Static 3D Point C

Hyungtae Lim 225 Dec 29, 2022
Official implementation of our paper "Learning to Bootstrap for Combating Label Noise"

Learning to Bootstrap for Combating Label Noise This repo is the official implementation of our paper "Learning to Bootstrap for Combating Label Noise

null 21 Apr 9, 2022
Convolutional neural network web app trained to track our infant’s sleep schedule using our Google Nest camera.

Machine Learning Sleep Schedule Tracker What is it? Convolutional neural network web app trained to track our infant’s sleep schedule using our Google

g-parki 7 Jul 15, 2022
This is the official code of our paper "Diversity-based Trajectory and Goal Selection with Hindsight Experience Relay" (PRICAI 2021)

Diversity-based Trajectory and Goal Selection with Hindsight Experience Replay This is the official implementation of our paper "Diversity-based Traje

Tianhong Dai 6 Jul 18, 2022
Official code for our ICCV paper: "From Continuity to Editability: Inverting GANs with Consecutive Images"

GANInversion_with_ConsecutiveImgs Official code for our ICCV paper: "From Continuity to Editability: Inverting GANs with Consecutive Images" https://a

QingyangXu 38 Dec 7, 2022
Official PyTorch implemention of our paper "Learning to Rectify for Robust Learning with Noisy Labels".

WarPI The official PyTorch implemention of our paper "Learning to Rectify for Robust Learning with Noisy Labels". Run python main.py --corruption_type

Haoliang Sun 3 Sep 3, 2022
The official codes of our CVPR2022 paper: A Differentiable Two-stage Alignment Scheme for Burst Image Reconstruction with Large Shift

TwoStageAlign The official codes of our CVPR2022 paper: A Differentiable Two-stage Alignment Scheme for Burst Image Reconstruction with Large Shift Pa

Shi Guo 32 Dec 15, 2022