Real-Time and Accurate Full-Body Multi-Person Pose Estimation&Tracking System

你好作者，我这几天正在尝试在coco数据集上训练SPPE，第一阶段按照你们的步骤python train.py --dataset coco --expID exp1 --nClasses 17 --LR 1e-4 没有加入dpg训练，50个epoch之后acc大概只有55%左右，我看您在另一个issue中的解答是 等acc达到90以后再加入dpg操作，所以我尝试着在第一阶段的基础上python train.py --dataset coco --expID exp1 --nClasses 17 --LR 1e-5 --loadmodel '第一阶段的模型model_49.pkl' 将lr调为1e-5 然后load第一阶段的模型继续训练想进一步提高acc，但好像20个epoch之后acc也只有56% 提升并不明显。 想请问一下在这种情况下，我是继续衰减 lr且不加入dpg训练，还是直接加入dpg进行训练呢？望作者还能够指点一下，十分感谢！

Running on python3.6 got this error:

  File "video_demo.py", line 41, in <module>
    data_loader = VideoLoader(videofile, batchSize=args.detbatch).start()
  File "/~/AlphaPose/dataloader.py", line 214, in start
    p.start()
  File "/usr/lib/python3.6/multiprocessing/process.py", line 105, in start
    self._popen = self._Popen(self)
  File "/usr/lib/python3.6/multiprocessing/context.py", line 223, in _Popen
    return _default_context.get_context().Process._Popen(process_obj)
  File "/usr/lib/python3.6/multiprocessing/context.py", line 291, in _Popen
    return Popen(process_obj)
  File "/usr/lib/python3.6/multiprocessing/popen_forkserver.py", line 35, in __init__
    super().__init__(process_obj)
  File "/usr/lib/python3.6/multiprocessing/popen_fork.py", line 19, in __init__
    self._launch(process_obj)
  File "/usr/lib/python3.6/multiprocessing/popen_forkserver.py", line 47, in _launch
    reduction.dump(process_obj, buf)
  File "/usr/lib/python3.6/multiprocessing/reduction.py", line 60, in dump
    ForkingPickler(file, protocol).dump(obj)
TypeError: can't pickle cv2.VideoCapture objects

Running code doesnt work. I get the following error:

(venv) juggernaut@xmen9:/hdd/AlphaPose$ python demo.py --indir examples/demo/
Loading YOLO model..
torch_shm_manager: error while loading shared libraries: libcudart.so.10.0: cannot open shared object file: No such file or directory
Traceback (most recent call last):
  File "/usr/lib/python3.6/multiprocessing/queues.py", line 234, in _feed
    obj = _ForkingPickler.dumps(obj)
  File "/usr/lib/python3.6/multiprocessing/reduction.py", line 51, in dumps
    cls(buf, protocol).dump(obj)
  File "/hdd/kps_pipeline/venv/lib/python3.6/site-packages/torch/multiprocessing/reductions.py", line 314, in reduce_storage
    metadata = storage._share_filename_()
RuntimeError: error executing torch_shm_manager at "/hdd/kps_pipeline/venv/lib/python3.6/site-packages/torch/bin/torch_shm_manager" at /pytorch/torch/lib/libshm/core.cpp:99
torch_shm_manager: error while loading shared libraries: libcudart.so.10.0: cannot open shared object file: No such file or directory
Traceback (most recent call last):
  File "/usr/lib/python3.6/multiprocessing/queues.py", line 234, in _feed
    obj = _ForkingPickler.dumps(obj)
  File "/usr/lib/python3.6/multiprocessing/reduction.py", line 51, in dumps
    cls(buf, protocol).dump(obj)
  File "/hdd/kps_pipeline/venv/lib/python3.6/site-packages/torch/multiprocessing/reductions.py", line 314, in reduce_storage
    metadata = storage._share_filename_()
RuntimeError: error executing torch_shm_manager at "/hdd/kps_pipeline/venv/lib/python3.6/site-packages/torch/bin/torch_shm_manager" at /pytorch/torch/lib/libshm/core.cpp:99
torch_shm_manager: error while loading shared libraries: libcudart.so.10.0: cannot open shared object file: No such file or directory
torch_shm_manager: error while loading shared libraries: libcudart.so.10.0: cannot open shared object file: No such file or directory
Traceback (most recent call last):
  File "demo.py", line 50, in <module>
    det_loader = DetectionLoader(data_loader, batchSize=args.detbatch).start()
  File "/hdd/AlphaPose/dataloader.py", line 309, in start
    p.start()
  File "/usr/lib/python3.6/multiprocessing/process.py", line 105, in start
    self._popen = self._Popen(self)
  File "/usr/lib/python3.6/multiprocessing/context.py", line 223, in _Popen
    return _default_context.get_context().Process._Popen(process_obj)
  File "/usr/lib/python3.6/multiprocessing/context.py", line 291, in _Popen
    return Popen(process_obj)
  File "/usr/lib/python3.6/multiprocessing/popen_forkserver.py", line 35, in __init__
    super().__init__(process_obj)
  File "/usr/lib/python3.6/multiprocessing/popen_fork.py", line 19, in __init__
    self._launch(process_obj)
  File "/usr/lib/python3.6/multiprocessing/popen_forkserver.py", line 47, in _launch
    reduction.dump(process_obj, buf)
  File "/usr/lib/python3.6/multiprocessing/reduction.py", line 60, in dump
    ForkingPickler(file, protocol).dump(obj)
  File "/hdd/kps_pipeline/venv/lib/python3.6/site-packages/torch/multiprocessing/reductions.py", line 314, in reduce_storage
    metadata = storage._share_filename_()
RuntimeError: error executing torch_shm_manager at "/hdd/kps_pipeline/venv/lib/python3.6/site-packages/torch/bin/torch_shm_manager" at /pytorch/torch/lib/libshm/core.cpp:99

Although, when i add flag --sp it works fine.

Python 3.6
CUDA 9.0
CUDNN 7
torch 1.2.0    
torchfile 0.1.0    
torchvision 0.4.0 

Hello,

I'm very impressed with your project AlphaPose!

I really would like to use AlphaPose to track pose in my project, but it seems PoseFlow is not implemented on pytorch branch now. Are you planning to implement PoseFlow on pytorch branch?

If not, is it possible to run PoseFlow and get tracked pose with following steps?

• Get alpha-pose-results-{...}.json by executing video_demo.py from pytorch branch
• Extract images from each frame of video for the deepmatching
• Execute deepmatching and get result file ({...}.txt)
• Edit PoseFlow/tracker.py to enable it to read images that were extracted from my video and deepmatching result
• Execute PoseFlow/tracker.py and get result

Hi,i'm working on Windows 10,python 3.6.7 and get error as below when runing .scripts\demo_inference.py with nessary parameters.

Traceback (most recent call last):
  File "D:/WorkProgram/PoseCheck/code/AlphaPose/scripts/demo_inference.py", line 13, in <module>
    from alphapose.models import builder
  File "D:\WorkProgram\PoseCheck\code\AlphaPose\alphapose\__init__.py", line 1, in <module>
    from .datasets import *  # noqa: F401,F403
  File "D:\WorkProgram\PoseCheck\code\AlphaPose\alphapose\datasets\__init__.py", line 1, in <module>
    from .coco_det import Mscoco_det
  File "D:\WorkProgram\PoseCheck\code\AlphaPose\alphapose\datasets\coco_det.py", line 17, in <module>
    from alphapose.models.builder import DATASET
  File "D:\WorkProgram\PoseCheck\code\AlphaPose\alphapose\models\__init__.py", line 1, in <module>
    from .fastpose import FastPose
  File "D:\WorkProgram\PoseCheck\code\AlphaPose\alphapose\models\fastpose.py", line 10, in <module>
    from .layers.SE_Resnet import SEResnet
  File "D:\WorkProgram\PoseCheck\code\AlphaPose\alphapose\models\layers\SE_Resnet.py", line 9, in <module>
    from .dcn import DeformConv, ModulatedDeformConv
  File "D:\WorkProgram\PoseCheck\code\AlphaPose\alphapose\models\layers\dcn\__init__.py", line 1, in <module>
    from .deform_conv import (DeformConv, DeformConvPack, ModulatedDeformConv,
  File "D:\WorkProgram\PoseCheck\code\AlphaPose\alphapose\models\layers\dcn\deform_conv.py", line 9, in <module>
    from . import deform_conv_cuda
ImportError: cannot import name 'deform_conv_cuda'

Hi，according to your paper, the method of Pose Flow NMS is different from the conventional NMS method using frame-by-frame Pose-NMS. However, the code in your github means that the NMS process depends on frame-by-frame Pose-NMS. The code about the Pose flow building puzzled me also, could you help me .

您好，大神，根据您之前的指导初步完成了在COCO数据上的训练，当用模型进行检测的时候却出现了权重维度大小不匹配的情况，折腾很久没有解决，求指点。。。谢谢 Loading pose model from ./models/sppe/model_49.pkl Traceback (most recent call last): File "demo.py", line 56, in pose_model = InferenNet_fast(4 * 1 + 1, pose_dataset) File "/home/zhong/AlphaPose/SPPE/src/main_fast_inference.py", line 61, in init model.load_state_dict(torch.load('./models/sppe/model_49.pkl')) File "/usr/local/lib/python3.5/dist-packages/torch/nn/modules/module.py", line 719, in load_state_dict self.class.name, "\n\t".join(error_msgs))) RuntimeError: Error(s) in loading state_dict for FastPose: size mismatch for conv_out.bias: copying a param of torch.Size([33]) from checkpoint, where the shape is torch.Size([17]) in current model. size mismatch for conv_out.weight: copying a param of torch.Size([33, 128, 3, 3]) from checkpoint, where the shape is torch.Size([17, 128, 3, 3]) in current model.

Hello, thanks for a fantastic project - it was the most accurate & easiest of all posetracking projects that I could get to run! I used the Colab notebook provided https://github.com/tugstugi/dl-colab-notebooks/blob/master/notebooks/AlphaPose.ipynb and it worked pretty much out of the box (I ran into issues with a local GPU though). However, this notebook installs from the pytorch-subbranch of the repository, which is older. I tried to install the master, but failed. In particular, I would like to obtain tracking results about person identity so that I can track multiple persons. However, the options --tracking or --pose_track are not working (it complains that the module 'detector' is missing - that seems to be only present in master). Does anyone have suggestions for how to get the master branch to work on colab? I tried to using the instructions, but didn't succeed (the old pytorch branch had a requirements-file -- the setup/build isn't working as smoothly). Thank you very much R

您好！ 我的电脑是windows10, cuda9.2, pytorch0.4.1, python3.6 请问要到哪里查看检测后的图片呢？ 我按照您的说明运行程序后，example/res文件夹里有一个.json文件和一个vis文件夹，但是vis文件夹是空的。。。 我运行的代码如下 hp@pc MINGW64 /d/AlphaPose-pytorch/AlphaPose-pytorch
$ python demo.py --indir examples/demo --outdir examples/res --save_img --sp --p osebatch 10 --vis Loading YOLO model..
D:\Anaconda3\lib\site-packages\torch\nn\modules\upsampling.py:122: UserWarning: nn.Upsampling is deprecated. Use nn.functional.interpolate instead.
warnings.warn("nn.Upsampling is deprecated. Use nn.functional.interpolate inst ead.") Loading pose model from ./models/sppe/duc_se.pth 100%|▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒| 2/2 [00:02<00:00, 1.28s/it] ===========================> Finish Model Running. ===========================> Rendering remaining images in the queue... ===========================> If this step takes too long, you can enable the --v is_fast flag to use fast rendering (real-time).

此外，运行时会出现如上所示的 warnings.warn("nn.Upsampling is deprecated. Use nn.functional.interpolate inst ead.，请问这是因为什么呢？会影响结果吗？怎么能使它不报这个warning呢？ 最后，如果要检测视频，视频需要什么格式呢？mp4吗？ 特别需要您的回答QAQ感谢~！！

Hi, could you tell me the detailed name of the data used in PoseFlow/deepmatching.py, I donn't know which the data you used that satisfied the condition of "len(a.split("/")) == 9". When I debug the code, the value of vidnames is empty.

Is there a requirement for the CUDA version? ImportError happens when I run the demo：

$ ./scripts/inference.sh configs/coco/resnet/256x192_res50_lr1e-3_1x.yaml pretrained_models/simple_res50_256x192.pth ./data/1.mp4
++ CONFIG=configs/coco/resnet/256x192_res50_lr1e-3_1x.yaml
++ CKPT=pretrained_models/simple_res50_256x192.pth
++ VIDEO=./data/1.mp4
++ OUTDIR=./examples/res
++ python scripts/demo_inference.py --cfg configs/coco/resnet/256x192_res50_lr1e-3_1x.yaml --checkpoint pretrained_models/simple_res50_256x192.pth --video ./data/1.mp4 --outdir ./examples/res --detector yolo --save_img --save_video
Traceback (most recent call last):
  File "scripts/demo_inference.py", line 13, in <module>
    from alphapose.models import builder
  File "/home/tianxin/AlphaPose/alphapose/models/__init__.py", line 1, in <module>
    from .fastpose import FastPose
  File "/home/tianxin/AlphaPose/alphapose/models/fastpose.py", line 10, in <module>
    from .layers.SE_Resnet import SEResnet
  File "/home/tianxin/AlphaPose/alphapose/models/layers/SE_Resnet.py", line 9, in <module>
    from .dcn import DeformConv, ModulatedDeformConv
  File "/home/tianxin/AlphaPose/alphapose/models/layers/dcn/__init__.py", line 1, in <module>
    from .deform_conv import (DeformConv, DeformConvPack, ModulatedDeformConv,
  File "/home/tianxin/AlphaPose/alphapose/models/layers/dcn/deform_conv.py", line 9, in <module>
    from . import deform_conv_cuda
ImportError: /home/tianxin/AlphaPose/alphapose/models/layers/dcn/deform_conv_cuda.cpython-36m-x86_64-linux-gnu.so: undefined symbol: _ZN2at7getTypeERKNS_6TensorE

Current environment： pytorch==1.2.0 torchvision==0.4.0 cudatoolkit=9.2

how can i solve this error?

this code error: !scripts/validate.sh --cfg /content/drive/MyDrive/AlphaPose/AlphaPose/configs/coco/resnet/256x192_res50_lr1e-3_1x.yaml --checkpoint /content/drive/MyDrive/AlphaPose/AlphaPose/pretrained_models/fast_res50_256x192.pth

scripts/validate.sh: Permission denied

! scripts /content/drive/MyDrive/AlphaPose/AlphaPose/scripts/train.sh configs /content/drive/MyDrive/AlphaPose/AlphaPose/configs/halpe_26/resnet/256x192_res50_lr1e-3_1x.yaml

/bin/bash: scripts: command not found

I want to generate JSON files with MPII keypoint order, not COCO.

So I ran the following command. I changed --format coco to --format MPII.

python scripts/demo_inference.py \
--cfg configs/coco/resnet/256x192_res50_lr1e-3_1x-simple.yaml \
--checkpoint pretrained_models/simple_res50_256x192.pth \
--video example.mp4 \
--outdir examples/res \
--detbatch 1 \
--posebatch 30 \
--detector tracker \
--showbox \
--format MPII \
--save_video

However, a portion of the output data shows that the number of elements in keypoints is 51, which is keypoints in COCO.

{
        "image_id": "0.jpg",
        "category_id": 1,
        "keypoints": [
            1470.4735107421875,
            167.3895721435547,
            0.9206457138061523,
            1473.953857421875,
            163.9093017578125,
            0.9169428944587708,
            1466.9932861328125,
            163.9093017578125,
            0.9339861273765564,
            1480.9144287109375,
            165.64944458007812,
            0.8938273787498474,
            1463.512939453125,
            165.64944458007812,
            0.9008600115776062,
            1487.8748779296875,
            184.79098510742188,
            0.9028074145317078,
            1461.7728271484375,
            186.5311279296875,
            0.8846182823181152,
            1482.654541015625,
            209.15293884277344,
            0.8575469255447388,
            1458.2926025390625,
            209.15293884277344,
            0.887753427028656,
            1468.7333984375,
            230.0346221923828,
            0.881260097026825,
            1458.2926025390625,
            230.0346221923828,
            0.8903049826622009,
            1482.654541015625,
            236.9951934814453,
            0.8013991713523865,
            1465.2530517578125,
            236.9951934814453,
            0.8115606904029846,
            1480.9144287109375,
            273.53814697265625,
            0.8705073595046997,
            1465.2530517578125,
            273.53814697265625,
            0.8841249942779541,
            1484.3946533203125,
            308.3409423828125,
            0.8712695240974426,
            1465.2530517578125,
            306.6007995605469,
            0.9167873859405518
        ],
        "score": 3.051330089569092,
        "box": [
            1437.9459228515625,
            145.28985595703125,
            70.275634765625,
            178.19027709960938
        ],
        "idx": 2.0
    },

How can I output the JSON file for the MPII case?

The google colab sample program is presented in ReadMe.md, but it does not work properly.

Please refer to this isuue #1087 I set up for more details.

So, I put the properly modified program on ReadMe.md and made a pull request. Please check it out.

URL from : https://colab.research.google.com/drive/1c7xb_7U61HmeJp55xjXs24hf1GUtHmPs?usp=sharing to : https://colab.research.google.com/drive/1_3Wxi4H3QGVC28snL3rHIoeMAwI2otMR?usp=sharing

Running this command from a conda environment: python setup.py build develop --user

/home/julian/miniconda3/lib/python3.9/site-packages/torch/cuda/__init__.py:88: UserWarning: CUDA initialization: Unexpected error from cudaGetDeviceCount(). Did you run some cuda functions before calling NumCudaDevices() that might have already set an error? Error 804: forward compatibility was attempted on non supported HW (Triggered internally at ../c10/cuda/CUDAFunctions.cpp:109.)
  return torch._C._cuda_getDeviceCount() > 0
No CUDA runtime is found, using CUDA_HOME='/usr/local/cuda'
/home/julian/miniconda3/lib/python3.9/site-packages/setuptools/installer.py:27: SetuptoolsDeprecationWarning: setuptools.installer is deprecated. Requirements should be satisfied by a PEP 517 installer.
  warnings.warn(
running build
running build_py
copying alphapose/version.py -> build/lib.linux-x86_64-3.9/alphapose
running build_ext
/home/julian/miniconda3/lib/python3.9/site-packages/torch/utils/cpp_extension.py:387: UserWarning: The detected CUDA version (11.8) has a minor version mismatch with the version that was used to compile PyTorch (11.7). Most likely this shouldn't be a problem.
  warnings.warn(CUDA_MISMATCH_WARN.format(cuda_str_version, torch.version.cuda))
/home/julian/miniconda3/lib/python3.9/site-packages/torch/utils/cpp_extension.py:397: UserWarning: There are no g++ version bounds defined for CUDA version 11.8
  warnings.warn(f'There are no {compiler_name} version bounds defined for CUDA version {cuda_str_version}')
building 'detector.nms.soft_nms_cpu' extension
Emitting ninja build file /home/julian/Documents/Alphapose attempt/AlphaPose/build/temp.linux-x86_64-3.9/build.ninja...
Compiling objects...
Allowing ninja to set a default number of workers... (overridable by setting the environment variable MAX_JOBS=N)
ninja: no work to do.
g++ -pthread -B /home/julian/miniconda3/compiler_compat -shared -Wl,-rpath,/home/julian/miniconda3/lib -Wl,-rpath-link,/home/julian/miniconda3/lib -L/home/julian/miniconda3/lib -L/home/julian/miniconda3/lib -Wl,-rpath,/home/julian/miniconda3/lib -Wl,-rpath-link,/home/julian/miniconda3/lib -L/home/julian/miniconda3/lib "/home/julian/Documents/Alphapose attempt/AlphaPose/build/temp.linux-x86_64-3.9/detector/nms/src/soft_nms_cpu.o" -o build/lib.linux-x86_64-3.9/detector/nms/soft_nms_cpu.cpython-39-x86_64-linux-gnu.so
/home/julian/miniconda3/compiler_compat/ld: /lib/x86_64-linux-gnu/libm.so.6: unknown type [0x13] section `.relr.dyn'
/home/julian/miniconda3/compiler_compat/ld: skipping incompatible /lib/x86_64-linux-gnu/libm.so.6 when searching for /lib/x86_64-linux-gnu/libm.so.6
/home/julian/miniconda3/compiler_compat/ld: cannot find /lib/x86_64-linux-gnu/libm.so.6
/home/julian/miniconda3/compiler_compat/ld: /lib/x86_64-linux-gnu/libm.so.6: unknown type [0x13] section `.relr.dyn'
/home/julian/miniconda3/compiler_compat/ld: skipping incompatible /lib/x86_64-linux-gnu/libm.so.6 when searching for /lib/x86_64-linux-gnu/libm.so.6
/home/julian/miniconda3/compiler_compat/ld: /lib/x86_64-linux-gnu/libmvec.so.1: unknown type [0x13] section `.relr.dyn'
/home/julian/miniconda3/compiler_compat/ld: skipping incompatible /lib/x86_64-linux-gnu/libmvec.so.1 when searching for /lib/x86_64-linux-gnu/libmvec.so.1
/home/julian/miniconda3/compiler_compat/ld: cannot find /lib/x86_64-linux-gnu/libmvec.so.1
/home/julian/miniconda3/compiler_compat/ld: /lib/x86_64-linux-gnu/libmvec.so.1: unknown type [0x13] section `.relr.dyn'
/home/julian/miniconda3/compiler_compat/ld: skipping incompatible /lib/x86_64-linux-gnu/libmvec.so.1 when searching for /lib/x86_64-linux-gnu/libmvec.so.1
collect2: error: ld returned 1 exit status
error: command '/usr/bin/g++' failed with exit code 1

Do you know what could cause this? Thanks in advance

Bumps pillow from 5.3.0 to 9.3.0.

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.