Realtime_Multi-Person_Pose_Estimation

Hi,

I tried the code but I got different values in torch than in the matlab version. Also, for some images I get:

ZeroDivisionError Traceback (most recent call last) in () 20 21 score_midpts = np.multiply(vec_x, vec[0]) + np.multiply(vec_y, vec[1]) ---> 22 score_with_dist_prior = sum(score_midpts)/len(score_midpts) + min(0.5*oriImg.shape[0]/norm-1, 0) 23 criterion1 = len(np.nonzero(score_midpts > param_['thre2'])[0]) > 0.8 * len(score_midpts) 24 criterion2 = score_with_dist_prior > 0

ZeroDivisionError: float division by zero

Another problem is that you get parts from a different person in the list of parts of a subject. I also think that if the algorithm is unable to find a joint for a subject to put 0 (as in Matlab), because I think that this is the cause of the second problem. Otherwise we are unable to determine automatically to whom the detected parts belong.

I really appreciate your work! Thank you!

I want to train with train2017 and I've tried using coco.annToMask() to get mask but the output are different from yours. How can I get mask for train2017?

Thank you for sharing your code. Apologies if I missed it, but I don't see a license specified. Could you please let us know what license this is shared under, and if it can be used commercially?

Hi, I change the model used in picture_demo.py to your "pose_model_scratch.pth" but it gives me error "RuntimeError: Error(s) in loading state_dict for rtpose_model: Missing key(s) in state_dict: "model0.0.bias", ..........." Did I miss something? The model "pose_model.pth" works well. Thank you.

1. in train_VGG19.py, the validation also backwards the val_loss (actually also "training"), making the ReduceLROnPlateau adjust training lr on part of the "training" data. Should validation only calculates the val_loss but not backwards loss?
2. you say "first 2644 images of val2014 marked by 'isValidation = 1', as our minival dataset." So the openpose selected 1160 images for evaluation is all selected from those 2644 images? otherwise, I think the program has already trained on some evaluation images...

python train_VGG19.py --json_path training/dataset/COCO.json --data_dir training/dataset/COCO/images --mask_dir training/dataset/COCO

Loading dataset... train dataset len: 121522 val dataset len: 4873 Bulding VGG19 load imagenet pretrained model: ./network/weight/ OpenCV Error: Assertion failed (ssize.width > 0 && ssize.height > 0) in resize, file /install_opencv/opencv-3.3.0/modules/imgproc/src/imgwarp.cpp, line 3483 OpenCV Error: Assertion failed (ssize.width > 0 && ssize.height > 0) in resize, file /install_opencv/opencv-3.3.0/modules/imgproc/src/imgwarp.cpp, line 3483 OpenCV Error: Assertion failed (ssize.width > 0 && ssize.height > 0) in resize, file /install_opencv/opencv-3.3.0/modules/imgproc/src/imgwarp.cpp, line 3483 OpenCV Error: Assertion failed (ssize.width > 0 && ssize.height > 0) in resize, file /install_opencv/opencv-3.3.0/modules/imgproc/src/imgwarp.cpp, line 3483 Traceback (most recent call last): File "train_VGG19.py", line 310, in train_loss = train(train_data, model, optimizer, epoch) File "train_VGG19.py", line 157, in train for i, (img, heatmap_target, heat_mask, paf_target, paf_mask) in enumerate(train_loader): File "anaconda3/envs/py35/lib/python3.5/site-packages/torch/utils/data/dataloader.py", line 286, in next return self._process_next_batch(batch) File "anaconda3/envs/py35/lib/python3.5/site-packages/torch/utils/data/dataloader.py", line 307, in _process_next_batch raise batch.exc_type(batch.exc_msg) cv2.error: Traceback (most recent call last): File "anaconda3/envs/py35/lib/python3.5/site-packages/torch/utils/data/dataloader.py", line 57, in _worker_loop samples = collate_fn([dataset[i] for i in batch_indices]) File "anaconda3/envs/py35/lib/python3.5/site-packages/torch/utils/data/dataloader.py", line 57, in samples = collate_fn([dataset[i] for i in batch_indices]) File "pytorch_Realtime_Multi-Person_Pose_Estimation/training/datasets/coco_data/COCO_data_pipeline.py", line 238, in getitem meta_data, img, mask_miss, self.params_transform) File "pytorch_Realtime_Multi-Person_Pose_Estimation/training/datasets/coco_data/ImageAugmentation.py", line 42, in aug_scale fy=scale, interpolation=cv2.INTER_CUBIC) cv2.error: /install_opencv/opencv-3.3.0/modules/imgproc/src/imgwarp.cpp:3483: error: (-215) ssize.width > 0 && ssize.height > 0 in function resize

Christians-iMac:pytorch_Realtime_Multi-Person_Pose_Estimation-master sugarbank$ python picture_demo.py File "picture_demo.py", line 162 print 'time is %.5f'%(toc-tic)

I tried to run python demo/web_demo.py. I am using pytorch 1.3 with python 3.7.4 at Anaconda3 on Ubuntu 18.04 system. It reports:

  File "demo/web_demo.py", line 129
    break
    ^
SyntaxError: 'break' outside loop

I think there maybe something wrong with the loop. Were you meant to include the lines 90~129 inside the looping while True: at line 77?

python evaluation.py using pose_model.pth(via rtpose original link https://www.dropbox.com/s/ae071mfm2qoyc8v/pose_model.pth?dl=0)

Traceback (most recent call last): File "evaluation.py", line 34, in model=model, preprocess='vgg') File "../evaluate/coco_eval.py", line 311, in run_eval return eval_coco(outputs=outputs, dataDir=anno_dir, imgIds=img_ids) File "../evaluate/coco_eval.py", line 50, in eval_coco json.dump(outputs, f)
File "/home/anaconda3/envs/py35/lib/python3.5/json/init.py", line 178, in dump for chunk in iterable: File "/home/anaconda3/envs/py35/lib/python3.5/json/encoder.py", line 427, in _iterencode yield from _iterencode_list(o, _current_indent_level) File "/home/anaconda3/envs/py35/lib/python3.5/json/encoder.py", line 324, in _iterencode_list yield from chunks File "/home/anaconda3/envs/py35/lib/python3.5/json/encoder.py", line 403, in _iterencode_dict yield from chunks File "/home/anaconda3/envs/py35/lib/python3.5/json/encoder.py", line 436, in _iterencode o = _default(o) File "/home/anaconda3/envs/py35/lib/python3.5/json/encoder.py", line 179, in default raise TypeError(repr(o) + " is not JSON serializable") TypeError: 192 is not JSON serializable

What is this error?

The link to COCO.json https://www.dropbox.com/s/0sj2q24hipiiq5t/COCO.json?dl=0 provides an incomplete JSON file (it is abruptly cut off at line 311090)

JSON linter output:

Error: Parse error on line 311090:
...00, 391.000, 1.000],
-----------------------^
Expecting 'STRING', 'NUMBER', 'NULL', 'TRUE', 'FALSE', '{', '[', got 'EOF'

Hi @tensorboy,

Could you upload the converted pytorch model to the website of "www.pan.baidu.com" or some other places?

I cannot visit the web of DropBox on my internet.

Thank you!

YIng Wong

Hi, I try to change model from VGG19 to MobileNetV2. Although i found "rtpose_mobilenetV2.py" in "lib/network",but i'm not sure how to modify "train_VGG19.py" to training. Could you help me please ? Thanks.

Hi author, I am just a curious how this model can achieve real-time inference? The model is so huge and the inference speed is very slow even on 3090.

hello authors,

i have trained a simple model using FCN for face keypoint regression. Can you please tell me how we can get heatmaps of muliple faces in the bottom-up approach. My current model takes input of 96x96x1 image and gives 96x96x15 size heatmaps for 15 keypoints. I trained my model using datset consisting of images with single face. Do i need the datset with multiple faces? and do I need bouding box information or mask information too?

Please give me your advice thank you

In the demo code, update the lines to load the model as below:

model = get_model('vgg19')
wts_dict = torch.load(args.weight) wts_dict_corrected = {k.replace('module.',''): v for k, v in wts_dict.items()} model.load_state_dict(wts_dict_corrected)

Originally posted by @SaraSherinThomas in https://github.com/tensorboy/pytorch_Realtime_Multi-Person_Pose_Estimation/issues/145#issuecomment-954444431