Hi Author,

I have some questions about the settings in config.py a. the default output size is 256x256 and there's an assertion in input size check: assert input_size[1] in [256] does it mean other input size cannot be applied? such as 320x320 etc.

b. the default output shape is devided by 8: output_shape = (input_shape[0]//8, input_shape[1]//8) can it be changed? e.g, divided by 4 instead. Meanwhile, the default depth_dim is set to 32. Is there any requirement on depth and output_shape? Since I notice there are some reshape operations in soft_max, I am a little confused. Based on the reshape operations, it looks that depth_dim and output_shape need to be the same value.

Wait for your reply, big thanks.

Hi, Really appreciate your efforts on this repo. I'm following your work to generate a model for my own "toy" project. But I can't reach the same metrics as you did. The loss_coord keeps fluctuating around 0.4-0.5 even though the learning rate decayed to 1e-5. This is my settings:

## input, output
input_shape = (256, 256) 
output_shape = (input_shape[0]//8, input_shape[1]//8)
width_multiplier = 1.0
depth_dim = 32
bbox_3d_shape = (2000, 2000, 2000) # depth, height, width
pixel_mean = (0.485, 0.456, 0.406)
pixel_std = (0.229, 0.224, 0.225)

## training config
embedding_size = 2048
lr_dec_epoch = [17, 21]
end_epoch = 25
lr = 1e-3
lr_dec_factor = 10
batch_size = 24

Thanks

Thanks for your great work! But i'm confused about the output of the network, your paper said the output is 64641152 which means (output_resoutput_res(joint_num*64)), what dose this 64 mean? And what it stands for?

Hi Author,

Thanks for your excellent work. The default setting are defined in config.py as follows:

trainset_3d = ['MuCo'] trainset_2d = ['MSCOCO']

testset = 'MuPoTS'

Based on your paper, it looks to have 2 different training/test configs, one is Human3.6M+MPII, the other is Muco+COCO. So which setting is better or they are just two alternatives and any one is applicable? Thanks for your time.

Dear SangbumChoi,

In repository it is said that there is pretrained model of your work. but I can not find it (I'm new to github community). Could you please guide me in the right direction where to find it or if it is not there could you share it with me. I would like to test it out and see how it works.

I managed to find ONNX version of your work in: https://github.com/PINTO0309/PINTO_model_zoo/blob/main/156_MobileHumanPose/download_mobile_human_pose_working_well.sh but I'm not sure if it works as your model and I would like to run it using pytorch. :)

Thank you for your work!

I thought that only 3d joint coordinate x,y,z are needed to train model for 3d dateset, But when I analyzed model.py, there are target_vis, target_have_depth for loss function.

loss_coord = torch.abs(coord - target_coord) * target_vis
            loss_coord = (loss_coord[:,:,0] + loss_coord[:,:,1] + loss_coord[:,:,2] * target_have_depth)/3.
            return loss_coord

Can you explain what they are?

When trying each of the backbones I am getting the following error (using LPRES as example, but same with all networks):

============================================================
LPRES BackBone Generated
============================================================
Traceback (most recent call last):
  File "main/train.py", line 84, in <module>
    main()
  File "main/train.py", line 39, in main
    trainer._make_model()
  File "main/../common/base.py", line 125, in _make_model
    model = get_pose_net(self.backbone, True, self.joint_num)
  File "main/model.py", line 83, in get_pose_net
    model.backbone.init_weight()
  File "/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1130, in __getattr__
    raise AttributeError("'{}' object has no attribute '{}'".format(
AttributeError: 'LpNetResConcat' object has no attribute 'init_weight'

When looking at the commit history the problem arrived at this commit bdbc6c14a58d857cab7b654fd02a356ae4b9c6c7

Hi SangbumChoi,

I was trying to understand your code a little bit and I got a confusion a little bit (I'm new to this so I might be wrong). I was checking your residual inverted block configuration. according your paper: you modified first 4 layers. here is what i noticed:

1st. At first bottleneck you increase the channel output to 64. Looking at the MobilenetV2 structure. shouldn't it go down to 24 and then go up?

2th. bottleneck layers 4,5 has swapped stride. is this a typo or new modification since your paper came out?

thank you for your good work!

ERROR:EngineBuilder:In node 143 (importResize): UNSUPPORTED_NODE: Assertion failed: scales.is_weights() && "Resize scales must be an initializer!"

hi, thanks for sharing the great job. If using custom data, what pre-process should I do? For example, normalizing the 3D coordinate data into [2000,2000,2000], normalizing the 2D data into [256, 256]?

Hello, I'm trying to train my own dataset using this model. I'm trying to make this to COCO format. However, this dataset has problem because it only includes [x, y, z] and no bounding box. The dataset has 1person per image with [x, y, z(depth)] coordinates. Can I use it? Or should I make bounding box per image?

Hi

Thank you for providing the training , eval and data preparation scripts. I followed the readme and set up the data and all the scripts in correct locations as indicated by t h file structure in readme. I kept the exact same configuration as the config.py and ran the training script for the same number of epochs. The train datase was human36 and mpii and tes set was human36 . protocol followed was the default in the code which is 2.

when i used the saved checkpoints to run the inference i get : Protocol 2 error (MPJPE) >> tot: 67.92 Directions: 60.23 Discussion: 72.39 Eating: 60.08 Greeting: 62.74 Phoning: 66.70 Posing: 59.33 Purchases: 61.72 Sitting: 81.54 SittingDown: 90.77 Smoking: 67.13 Photo: 80.52 Waiting: 63.39 Walking: 51.68 WalkDog: 71.55 WalkTogether: 58.84

which is obviously pretty high than what is expected. Just as a note i was getting an error while using broadcast in model.py module 'torch.nn.parallel' has no attribute 'comm' So I changed the original script from

accu_x = accu_x * torch.nn.parallel.comm.broadcast(torch.arange(1,cfg.output_shape[1]+1).type(torch.cuda.FloatTensor), devices=[accu_x.device.index])[0]
accu_y = accu_y * torch.nn.parallel.comm.broadcast(torch.arange(1,cfg.output_shape[0]+1).type(torch.cuda.FloatTensor), devices=[accu_y.device.index])[0]
accu_z = accu_z * torch.nn.parallel.comm.broadcast(torch.arange(1,cfg.depth_dim+1).type(torch.cuda.FloatTensor), devices=[accu_z.device.index])[0]

TO:

accu_x = accu_x * torch.arange(1, cfg.output_shape[1] + 1).type(torch.cuda.FloatTensor)
accu_y = accu_y * torch.arange(1, cfg.output_shape[0] + 1).type(torch.cuda.FloatTensor)
accu_z = accu_z * torch.arange(1, cfg.depth_dim + 1).type(torch.cuda.FloatTensor)

I didnot change anything else including batch size. And again same config file. If you can please help me root cause the issue it will be very helpful. Thanks

First, thanks for your great work.

I trained the model using script 'python train.py --gpu 0-1 --backbone LPSKI' with Human3.6M and MPII datasets. The protocol is 1 and train epoch was 25.

And I tested the model with test.py and the result is like below : Protocol 1 error (PA MPJPE) >> tot: 42.72 Directions: 37.63 Discussion: 39.01 Eating: 45.51 Greeting: 43.06 Phoning: 41.33 Posing: 41.10 Purchases: 35.78 Sitting: 43.50 SittingDown: 57.36 Smoking: 47.08 Photo: 51.04 Waiting: 38.32 Walking: 30.94 WalkDog: 46.21 WalkTogether: 38.39

I found the average MPJPE of Protocol1 on paper is 35.2 which is different from my result. Did I miss something to get the right result?? Like other settings in config.py...

Also, my train time was 16 hours with RTX2080 and the train time on paper is 3 days with 2 RTX titans. So I also wonder what makes time difference between my result and the paper.