Hi author,

Thanks for your such excellent work. I did some training and tests based on your paper and codes and the results are good. I am now curious about the inference in real time. My intention is to estimate the 3D coordinates while playing back a video. According to your strategy and demo code, estimation against a center frame need the 2D poses before and after it, which means the 3D pose of a certain frame cannot be achieved until the 2D poses after it are calculated. I have no idea how to handle such case. I don't think it's a good idea just to pad some dummy data such as zeros. Appreciate any suggestions from you. Thanks.

Hi author,

Looks like it's a top-down model which needs an extra detector. A naive question is, for single pose 3d estimation, is it possible to just use the raw image/frame as the model input so as to omit the detector and speed up the inference time? Wait for your feedback, thanks.

How do you guys handle in the wild inference since you only release models that have been trained on cpn_ft_detections that are already in line with the h36m skeleton?

Excuse me, I can not reproduce your results.For 81 frames, I can only achieve 44.83mm MPJPE, and for 351 frame, just 43.17mm.Would you please tell me the reason or show me the way to revisit results? It would be better if you could share the training log.Thanks a lot!!!

I used the script provided by the P-STMO model(https://github.com/paTRICK-swk/P-STMO) to train and test the MHFormer model on the 3dhp dataset. The average value of PCK and AUC indexes obtained was 20 and mpjpe was 300, but the results were very unreliable.

python demo/vis.py --video sample_video.mp4

Generating 2D pose... 100%|█████████████████████████████████████████| 291/291 [00:19<00:00, 14.91it/s] Generating 2D pose successful! ['checkpoint/pretrained/351/model_4294.pth']

Generating 3D pose... 0%| | 0/291 [00:00<?, ?it/s] Traceback (most recent call last): File "demo/vis.py", line 279, in get_pose3D(video_path, output_dir) File "demo/vis.py", line 225, in get_pose3D show3Dpose( post_out, ax) File "demo/vis.py", line 72, in show3Dpose ax.set_aspect('equal') File "anaconda3/envs/lili/lib/python3.8/site-packages/mpl_toolkits/mplot3d/axes3d.py", line 323, in set_aspect raise NotImplementedError( NotImplementedError: Axes3D currently only supports the aspect argument 'auto'. You passed in 'equal'.

非常棒的工作！ 我正在拼命学习你们的研究并且尝试将其应用于实际领域。

我想问的是，

1. output文件夹下的'input_2D'文件夹内是否已经是3D 人体姿势估计的结果（也就是你们论文中所描述的主要工作生成的结果）？因为我想将其用在优化虚拟人物动作中。
2. 目前使用的是略显“古老”的YOLOv3和体积很大的HRNet来工作，是否有可能将其换成新的YOLOv7和Lite-HRNet，以加快识别姿态的速度，并最终实现实时识别和多目标同时识别？

如果可行的话，能否给我讲一下修改方法？例如如何生成“ YOLOv7.weights”文件，以及能否直接在github仓库中下载Lite-HRNet来替换使用？

请原谅我作为一个cv领域的初学者可能提问过于小白，感谢您的耐心

Great Great Job! I'm trying my best to study your research and try to apply it in real world.

What I want to ask is,

1. Is the 'input_2D' folder that in the ‘output’ folder the result of 3D human pose estimation(that is, the main work in your paper)? Because I want to use it in optimizing avatar motivation.
2. Currently working with a slightly "old" YOLOv3 and a large HRNet, is it possible to replace it with the new YOLOv7 and Lite-HRNet to speed up the recognition of poses and finally achieve real-time recognition and multi-tasking? target at the same time?

If possible, can you tell me how to modify it? For example, how to generate the "YOLOv7.weights" file, and can I download Lite-HRNet directly in the github repository to replace it?

Please forgive me as a beginner in cv area for asking such primary questions, thank you for your patience.

hi，I finished reading your paper and I have a question, it seems that the Human3.6M and MPI-INF-3DHP dataset are used for evaluation, so does it mean that the training dataset is marked by yourself? I learned from other papers(e.g. blazepalm) that the widely used method is to use a 3d software to generate keypoints-marked dataset, then use this initial model to iteratively update. 您好，我阅读了您的文章，十分厉害！但是我有一个问题想请教你们，就是我看文章里面说的那两个数据集是用来评估的，我的理解是测试集，所以训练集是您在谷歌drive上的npz文件？是您自己标注的吗？因为我目前从其他文章了解到的方法是先用3d软件通过获取准确关节点的坐标生成数据集和初始模型，然后不停的迭代（标注其他数据集）。

Thank you for the amazing work!

Traceback (most recent call last):
  File "C:/Users/92336/Desktop/humanposeestimation/MHFormer-main/demo/vis.py", line 278, in <module>
    get_pose2D(video_path, output_dir)
  File "C:/Users/92336/Desktop/humanposeestimation/MHFormer-main/demo/vis.py", line 94, in get_pose2D
    keypoints, scores = hrnet_pose(video_path, det_dim=416, num_peroson=1, gen_output=True)
  File "C:\Users\92336\Desktop\humanposeestimation\MHFormer-main\demo\lib\hrnet\gen_kpts.py", line 118, in gen_video_kpts
    bboxs = bboxs_pre
UnboundLocalError: local variable 'bboxs_pre' referenced before assignment

I am having this issue while trying to run inference. Any fixes for this? @Vegetebird

Hey there! Amazing work on this new paper. Can you all specify a license for your work? I'm the founder of NatML and I would like to bring this model to NatML Hub.

Also, do you accept pull requests? If you all provide an open-source license for your work, I would like to add a link to the NatML implementation of your model in the README. A lot of augmented reality developers in Unity will find this work very useful. Thank you!

Hi author, I'm running vis.py, and I want to get three-dimensional coordinate information x,y,z. I see there are post_out and output_3D in the code, may I ask which coordinate is it? What's the difference between these two parameters?

Hey @Vegetebird! 👋

This pull request makes it possible to run your model inside a Docker environment, which makes it easier for other people to run it. We're using an open source tool called Cog to make this process easier. To make the model work in Cog, a few edits have been made (for example, parsing args from command line using argparser wouldn't work so we manually set the default args in set_default_options() whenever use_cog= True).

This also means we can make a web demo where other people can try out your model! View it here: https://replicate.com/vegetebird/human-pose-estimation

We've added some examples to the web demo; please click the black "Claim this model" button andclaim your page here so you own it/edit it.

In case you're wondering who I am, I'm from Replicate, where we're trying to make machine learning reproducible. We got frustrated that we couldn't run all the really interesting ML work being done. So, we're going round implementing models we like. 😊