Hi, Thanks for your amazing work. I want to test posenet on a single image(already cropped by Detectron).Could you please share a demo script to visualization?

I download the annotation file about MUPOTS dataset from your link, the 2D coordination is in keypoints_img, and the corresponding x and y of 3D is the same as keypoints_img, the z comes from keypoints_cam-z. But I found the z is misplaced with real depth position in the corresponding image. Are there any extra steps to do?

Hi,

I wanted to apply PoseNet to test my own dataset. However, I found that the result from demo.py and test.py are very different.

In demo.py, each time, we pass a single processed image that only contains one person, this is the code that we used for processing:

    bbox = process_bbox(np.array(bbox_list[n]), original_img_width, original_img_height)
    img, img2bb_trans = generate_patch_image(original_img, bbox, False, 1.0, 0.0, False) 
    img = transform(img).cuda()[None,:,:,:]

Then we feed "img" to the model:

    # forward
    with torch.no_grad():
        pose_3d = model(img) # x,y: pixel, z: root-relative depth (mm)

The result I get in demo.py is very correct for both 2D and 3D estimation.

However, when I use test.py (I define a new class for my dataset like you did for other datasets), the result is very different. After debugging, I found that starting here it is different:

        for itr, input_img in enumerate(tqdm(tester.batch_generator)):
            
            # forward
            coord_out = tester.model(input_img)

Then we pass this "coord_out", which has a shape of (128,21,3) to evaluate. But when I take coord_out[0] and compare with the first person's pose_3d in demo.py (using the same pre-trained model), I found them very different. Then I found that even the img and input_img[0] is different.

Is that because in test.py we gave the whole batch 128 people to the network without cropping and it has a problem? How can I address this?

Thanks!

Hi,

I was trying to add another convolution kernel before the convolution in the posenet backbone. And there is a parameter updated in that kernel function based on the epoch number. It's like a decay factor, when epoch = 0 it is 1 and epoch *= 0.9 every 10 epochs. However I couldn't find a proper way to pass the epoch parameter.

In base.py, I see this: https://github.com/mks0601/3DMPPE_POSENET_RELEASE/blob/3f92ebaef214a0eb1574b7265e836456fbf3508a/common/base.py#L123 This is where we initialize the model, this is under the function._make_model(). I did some changes like:

model = get_pose_net(cfg, True, self.joint_num, epoch)

But in train.py, unlike trainer.set_lr(epoch) which is in the for loop of epoch, we only call the trainer._make_model() one time at the very first beginning outside the for loop: https://github.com/mks0601/3DMPPE_POSENET_RELEASE/blob/3f92ebaef214a0eb1574b7265e836456fbf3508a/main/train.py#L34

So the epoch number cannot be updated properly if I pass it thru the _make_model().

How can I properly pass the epoch number to the posenet model?

Hello, first of all, thank you for your wonderful research, but I found that the download link for the Human36 dataset you provided is broken. Could you please provide a new download address code? thank you very much!!!!!!

hi, it is really a great work!

But I have a question about the output. It is clearly the keypoints coordinates can be got from the volumetric heatmaps, but if I can got the confidence score about the keypoint? I try to got them by take the max value like the 2d heatmap, but it seems not convincing. Could you give me some advice?

Hi, when doing the data augment, you normalized the depth to -1~1 by diving bbox_3d_shape[0] as the "joint_img[i, 2] /= (cfg.bbox_3d_shape[0]/2.) # expect depth lies in -bbox_3d_shape[0]/2 ~ bbox_3d_shape[0]/2 -> -1.0 ~ 1.0", does it mean the depths in camera space are in the range of [-1000, 1000]? And why?

Then you did 'joint_img[:, 2] = joint_img[:, 2] * cfg.depth_dim', why did you do this?

Hi, first of all, thank you for your great work. There were different results in two different environments.

1. windows10, cuda9, torch=1.1, I can get a better result.
2. ubuntu16.04 cuda10.1 torch1.4, I can get another result. Why do you think the same code gets inconsistent results in two environments?

thanks!

Thank your for great project. I have some question for help.

1. About PoseNet , your paper cite Integral human pose regression. What has been modify in this base?
2. In paper As the results in the table are based on the same PoseNet, we can conclude that AUCrel, which is an evaluation of the root-relative 3D human pose estimation highly depends on the accuracy of the PoseNe from 8.3 Ablation study--Effect of the PoseNet. However, Why this paper didn't focus on PoseNet.

Hi, thanks for your great work! I have already got "bbox_root_pose_human36m_output.json" file by run "python test.py --gpu 0 --test_epoch 24". We could get positions(x,y,z) of body joints by parsing "bbox_root_human36m_output.json" and "bbox_root_pose_human36m_output.json". But how to show these infomation in a picture? I am not clear what the parameters (kpt_3d, kpt_3d_vis, kps_lines) under "./common/utils/vis.py" exactly mean. def vis_3d_skeleton(kpt_3d, kpt_3d_vis, kps_lines, filename=None):

what's more, how to get test result files (preds_2d_kpt_$DB_NAME.mat, preds_3d_kpt_$DB_NAME.mat)? Looking forward to your reply.

https://github.com/mks0601/3DMPPE_POSENET_RELEASE/blob/3f92ebaef214a0eb1574b7265e836456fbf3508a/demo/demo.py#L116 에서는 visible를 모두 1로 표시하였는데, 사물 및 기타 환경에 의해 keypoints가 가려져 보이지 않을 경우 해당 keypoints를 확인하고 싶습니다.

예를 들면 아래와 같은 keypoints에서 R_Knee_x와 R_Ankle, L_Knee, L_Ankle가 보이지 않을 경우 visible 형태로 얻고 싶은데 방법이 있을까요? keypoints = [ (Head_top_x, Head_top_y), (Thorax_x, Thorax_y), (R_Shoulder_x, R_Shoulder_y), (R_Elbow_x, R_Elbow_y), (R_Wrist_x, R_Wrist_y), (L_Shoulder_x, L_Shoulder_y), (L_Elbow_x, L_Elbow_y), (L_Wrist_x, L_Wrist_y), (R_Hip_x, R_Hip_y), (R_Knee_x, R_Knee_y), (R_Ankle_x, R_Ankle_y), (L_Hip_x, L_Hip_y), (L_Knee_x, L_Knee_y), (L_Ankle_x, L_Ankle_y), (Pelvis_x, Pelvis_y), (Spine_x, Spine_y), (Head_x, Head_y), (R_Hand_x, R_Hand_y), (L_Hand_x, L_Hand_y), (R_Toe_x, R_Toe_y), (L_Toe_x, L_Toe_y) ] visible = [ True, True, True, True, True, True, True, True, False, False, True, False, False, True, True, True, True, True, True, True, True ]

Thank you for your great work. I love your project. I would like to ask two questions:

1. What is your method of two-dimensional pose estimation?
2. My goal is to do multi-person pose estimation based on video. Is it possible to combine your ROOTNET and VideoPose3D with TCN to introduce time information? Thank you very much again. Looking forward to your reply.

Hi, thank you for your great work, I want to know what dataset you trained on. There are 21 key points. I hope you can reply me when you see it. Thank you!!!

Thank you for your great job, I like it very much. I want to know how to get a custom image bbox_list? I know deep_list depond on RootNet. Please reply me, because i really want to study your great project. Thank you so much.

Hi Thank you for your great work, I am a beginner, I have some questions to ask: 1, this project can run on my Windows10 computer? 2. What if the test runs my own video file? Thank you very much.

After I downloaded the 2.6m dataset, there is no distinction between manual labeling and machine labeling pictures, both are in the train folder, how do I know which one is manually labeled? At the same time, I want to delete all the pictures and json information that have two-handed interaction or appear with both hands, and only keep the pictures and json information with one hand, what should I do?