Hi,

Great work! I'm wondering if it is possible to reconstruct smpl model parameters when the human keypoint aren't the same format as the default one. For example, I'm using a in-house dataset obtained keypoint coordinates from Kinect V2, which output 25 keypoints coordinates in 3D (https://lisajamhoury.medium.com/understanding-kinect-v2-joints-and-coordinate-system-4f4b90b9df16). How do I specify the correspondence from my keypoint to the mesh? Does this code include any similar functions?

Appreciate it!

Hi,

Thank you for making this repository. Can you please guide me how I can convert c3d file to smpl file using this repository. I am unable to find any relevant tutorial.

Thank you

hi, would you mind to provide me the file pose_body_prior.pkl for the smal model ? the results I generate now are almost correct except the body pose(limb pose), when the model_type is smal without a pose_body_prior.pkl for it.

as the figures show below, the shape and the orientation are alright, but the limb poses are not as expected as I think, the limb poses keep same although they have been changed in fact in the left image.

the code can work normally when the model type is smplx. and now I try to fit the smpl params instead of the smplx following the smplx moshpp process. at first I donwloaded the smpl basic models, and put them under support_files/smpl directory, then change the surface_model.typefrom smplx to smpl, and addedopt_weights.smpl config which is a copy of smplx as below.

opt_weights:
  smpl:
    stagei_wt_poseH: 3.0
    stagei_wt_poseF: 3.
    stagei_wt_expr: 34.
    stagei_wt_pose: 3.
    stagei_wt_poseB: 3.
    stagei_wt_init_finger_left: 400.0
    stagei_wt_init_finger_right: 400.0
    stagei_wt_init_finger: 400.0
    stagei_wt_betas: 10.
    stagei_wt_init: 300
    stagei_wt_data: 75.
    stagei_wt_surf: 10000.
    stagei_wt_annealing: [ 1., .5, .25, .125 ]
    stageii_wt_data: 400
    stageii_wt_velo: 2.5
    stageii_wt_dmpl: 1.0
    stageii_wt_expr: 1.0
    stageii_wt_poseB: 1.6
    stageii_wt_poseH: 1.0
    stageii_wt_poseF: 1.0
    stageii_wt_annealing: 2.5

the error when I run the mosh job(task)

2022-07-07 08:11:49.227 | INFO     | soma.tools.parallel_tools:run_parallel_jobs:54 - #Job(s) submitted: 83
2022-07-07 08:11:49.227 | INFO     | soma.tools.parallel_tools:run_parallel_jobs:67 - Will run the jobs in random order.
soma_subject1 -- custom_low_head2_yz_swap -- mosh_head:__init__:95 -- mocap_fname: /data/hxh/project/soma/SOMA_FOLDER_TEMPLATE//training_experiments/V48_02_SOMA/OC_05_G_03_real_000_synt_100/evaluations/soma_labeled_mocap_tracklet/SOMA_unlabeled_mpc/soma_subject1/custom_low_head2_yz_swap.pkl
soma_subject1 -- custom_low_head2_yz_swap -- mosh_head:__init__:97 -- stagei_fname: /data/hxh/project/soma/SOMA_FOLDER_TEMPLATE//training_experiments/V48_02_SOMA/OC_05_G_03_real_000_synt_100/evaluations/mosh_results_tracklet/SOMA_unlabeled_mpc/soma_subject1/male_stagei.pkl
soma_subject1 -- custom_low_head2_yz_swap -- mosh_head:__init__:98 -- stageii_fname: /data/hxh/project/soma/SOMA_FOLDER_TEMPLATE//training_experiments/V48_02_SOMA/OC_05_G_03_real_000_synt_100/evaluations/mosh_results_tracklet/SOMA_unlabeled_mpc/soma_subject1/custom_low_head2_yz_swap_stageii.pkl
soma_subject1 -- custom_low_head2_yz_swap -- mosh_head:__init__:103 -- surface_model: type: smpl; gender: male; fname:/data/hxh/project/soma/SOMA_FOLDER_TEMPLATE/support_files/smpl/male/model.pkl
soma_subject1 -- custom_low_head2_yz_swap -- mosh_head:__init__:107 -- optimize_fingers: False, optimize_face: False, optimize_toes: False, optimize_betas: True, optimize_dynamics: False
soma_subject1 -- custom_low_head2_yz_swap -- mosh_head:prepare_stagei_frames:157 -- Selecting 12 frames using method manual on frames with 100% least_avail_markers
soma_subject1 -- custom_low_head2_yz_swap -- mosh_head:prepare_stagei_frames:197 -- Using stagei_fnames for stage-i: ['/data/hxh/project/soma/SOMA_FOLDER_TEMPLATE/training_experiments/V48_02_SOMA/OC_05_G_03_real_000_synt_100/evaluations/soma_labeled_mocap_tracklet/SOMA_unlabeled_mpc/soma_subject1/run_002.pkl_001091'
 '/data/hxh/project/soma/SOMA_FOLDER_TEMPLATE/training_experiments/V48_02_SOMA/OC_05_G_03_real_000_synt_100/evaluations/soma_labeled_mocap_tracklet/SOMA_unlabeled_mpc/soma_subject1/jump_001.pkl_000137'
 '/data/hxh/project/soma/SOMA_FOLDER_TEMPLATE/training_experiments/V48_02_SOMA/OC_05_G_03_real_000_synt_100/evaluations/soma_labeled_mocap_tracklet/SOMA_unlabeled_mpc/soma_subject1/run_001.pkl_001366'
 '/data/hxh/project/soma/SOMA_FOLDER_TEMPLATE/training_experiments/V48_02_SOMA/OC_05_G_03_real_000_synt_100/evaluations/soma_labeled_mocap_tracklet/SOMA_unlabeled_mpc/soma_subject1/jump_001.pkl_000509'
 '/data/hxh/project/soma/SOMA_FOLDER_TEMPLATE/training_experiments/V48_02_SOMA/OC_05_G_03_real_000_synt_100/evaluations/soma_labeled_mocap_tracklet/SOMA_unlabeled_mpc/soma_subject1/throw_001.pkl_000596'
 '/data/hxh/project/soma/SOMA_FOLDER_TEMPLATE/training_experiments/V48_02_SOMA/OC_05_G_03_real_000_synt_100/evaluations/soma_labeled_mocap_tracklet/SOMA_unlabeled_mpc/soma_subject1/dance_003.pkl_001488'
 '/data/hxh/project/soma/SOMA_FOLDER_TEMPLATE/training_experiments/V48_02_SOMA/OC_05_G_03_real_000_synt_100/evaluations/soma_labeled_mocap_tracklet/SOMA_unlabeled_mpc/soma_subject1/jump_001.pkl_000588'
 '/data/hxh/project/soma/SOMA_FOLDER_TEMPLATE/training_experiments/V48_02_SOMA/OC_05_G_03_real_000_synt_100/evaluations/soma_labeled_mocap_tracklet/SOMA_unlabeled_mpc/soma_subject1/squat_002.pkl_001134'
 '/data/hxh/project/soma/SOMA_FOLDER_TEMPLATE/training_experiments/V48_02_SOMA/OC_05_G_03_real_000_synt_100/evaluations/soma_labeled_mocap_tracklet/SOMA_unlabeled_mpc/soma_subject1/jump_002.pkl_000471'
 '/data/hxh/project/soma/SOMA_FOLDER_TEMPLATE/training_experiments/V48_02_SOMA/OC_05_G_03_real_000_synt_100/evaluations/soma_labeled_mocap_tracklet/SOMA_unlabeled_mpc/soma_subject1/run_001.pkl_000032'
 '/data/hxh/project/soma/SOMA_FOLDER_TEMPLATE/training_experiments/V48_02_SOMA/OC_05_G_03_real_000_synt_100/evaluations/soma_labeled_mocap_tracklet/SOMA_unlabeled_mpc/soma_subject1/dance_001.pkl_001042'
 '/data/hxh/project/soma/SOMA_FOLDER_TEMPLATE/training_experiments/V48_02_SOMA/OC_05_G_03_real_000_synt_100/evaluations/soma_labeled_mocap_tracklet/SOMA_unlabeled_mpc/soma_subject1/dance_001.pkl_000289']
soma_subject1 -- custom_low_head2_yz_swap -- mosh_head:mosh_stagei:241 -- Attempting mosh stagei to create /data/hxh/project/soma/SOMA_FOLDER_TEMPLATE//training_experiments/V48_02_SOMA/OC_05_G_03_real_000_synt_100/evaluations/mosh_results_tracklet/SOMA_unlabeled_mpc/soma_subject1/male_stagei.pkl
soma_subject1 -- custom_low_head2_yz_swap -- chmosh:mosh_stagei:120 -- using marker_layout_fname: /data/hxh/project/soma/SOMA_FOLDER_TEMPLATE//training_experiments/V48_02_SOMA/OC_05_G_03_real_000_synt_100/evaluations/mosh_results_tracklet/SOMA_unlabeled_mpc/SOMA_unlabeled_mpc_smpl.json
soma_subject1 -- custom_low_head2_yz_swap -- bodymodel_loader:load_moshpp_models:93 -- Loading model: /data/hxh/project/soma/SOMA_FOLDER_TEMPLATE/support_files/smpl/male/model.pkl
soma_subject1 -- custom_low_head2_yz_swap -- chmosh:mosh_stagei:172 -- can_model.betas.shape: (300,)
soma_subject1 -- custom_low_head2_yz_swap -- chmosh:mosh_stagei:173 -- opt_models[0].betas.shape: (300,)
soma_subject1 -- custom_low_head2_yz_swap -- chmosh:mosh_stagei:192 -- Estimating for #latent markers: 53
soma_subject1 -- custom_low_head2_yz_swap -- chmosh:mosh_stagei:229 -- Number of available markers in each stagei selected frames: (F00, 53), (F01, 53), (F02, 53), (F03, 53), (F04, 53), (F05, 53), (F06, 53), (F07, 53), (F08, 53), (F09, 53), (F10, 53), (F11, 53)
soma_subject1 -- custom_low_head2_yz_swap -- chmosh:mosh_stagei:237 -- Rigidly aligning the body to the markers
soma_subject1 -- custom_low_head2_yz_swap -- chmosh:mosh_stagei:262 -- MoSh stagei weights before annealing:
stagei_wt_poseH: 3.0
stagei_wt_poseF: 3.0
stagei_wt_expr: 34.0
stagei_wt_pose: 3.0
stagei_wt_poseB: 3.0
stagei_wt_init_finger_left: 400.0
stagei_wt_init_finger_right: 400.0
stagei_wt_init_finger: 400.0
stagei_wt_betas: 10.0
stagei_wt_init: 300
stagei_wt_data: 75.0
stagei_wt_surf: 10000.0
stagei_wt_annealing: [1.0, 0.5, 0.25, 0.125]
soma_subject1 -- custom_low_head2_yz_swap -- chmosh:mosh_stagei:266 -- head_marker_corr_fname is provided and is being loaded: /data/hxh/project/soma/SOMA_FOLDER_TEMPLATE/support_files/ssm_head_marker_corr.npz
soma_subject1 -- custom_low_head2_yz_swap -- chmosh:mosh_stagei:274 -- Successfully took into account the correlation of the head markers
soma_subject1 -- custom_low_head2_yz_swap -- chmosh:mosh_stagei:280 -- Beginning mosh stagei with opt_settings.weights_type: smpl
soma_subject1 -- custom_low_head2_yz_swap -- chmosh:mosh_stagei:354 -- Step 1/4 : Opt. wt_anneal_factor = 1.00, wt_data = 1.00, wt_poseB = 65.09
soma_subject1 -- custom_low_head2_yz_swap -- chmosh:mosh_stagei:356 -- stagei_wt_init for different marker types body = 300.00:
Traceback (most recent call last):
  File "test_soma.py", line 115, in <module>
    'randomly_run_jobs': True,
  File "/data/hxh/project/soma/src/soma/tools/run_soma_multiple.py", line 283, in run_soma_on_multiple_settings
    run_parallel_jobs(func=run_moshpp_once, jobs=mosh_jobs, parallel_cfg=moshpp_parallel_cfg)
  File "/data/hxh/project/soma/src/soma/tools/parallel_tools.py", line 79, in run_parallel_jobs
    func(job)
  File "/root/anaconda3/envs/soma/lib/python3.7/site-packages/moshpp-3.0-py3.7.egg/moshpp/mosh_head.py", line 598, in run_moshpp_once
    mp.mosh_stagei(mosh_stagei)
  File "/root/anaconda3/envs/soma/lib/python3.7/site-packages/moshpp-3.0-py3.7.egg/moshpp/mosh_head.py", line 245, in mosh_stagei
    v_template_fname=self.cfg.moshpp.v_template_fname)
  File "/root/anaconda3/envs/soma/lib/python3.7/site-packages/moshpp-3.0-py3.7.egg/moshpp/chmosh.py", line 421, in mosh_stagei
    [f'{k} = {np.sum(opt_objs[k].r ** 2):2.2e}' for k in sorted(opt_objs)])))
  File "/root/anaconda3/envs/soma/lib/python3.7/site-packages/moshpp-3.0-py3.7.egg/moshpp/chmosh.py", line 421, in <listcomp>
    [f'{k} = {np.sum(opt_objs[k].r ** 2):2.2e}' for k in sorted(opt_objs)])))
  File "/root/anaconda3/envs/soma/lib/python3.7/site-packages/chumpy/ch.py", line 596, in r
    self._cache['r'] = np.asarray(np.atleast_1d(self.compute_r()), dtype=np.float64, order='C')
  File "/root/anaconda3/envs/soma/lib/python3.7/site-packages/chumpy/ch_ops.py", line 708, in compute_r
    return self.a.r * self.b.r
  File "/root/anaconda3/envs/soma/lib/python3.7/site-packages/chumpy/ch.py", line 596, in r
    self._cache['r'] = np.asarray(np.atleast_1d(self.compute_r()), dtype=np.float64, order='C')
  File "/root/anaconda3/envs/soma/lib/python3.7/site-packages/chumpy/reordering.py", line 376, in compute_r
    return np.concatenate([t.r for t in self.our_terms], axis=self.axis)
  File "/root/anaconda3/envs/soma/lib/python3.7/site-packages/chumpy/reordering.py", line 376, in <listcomp>
    return np.concatenate([t.r for t in self.our_terms], axis=self.axis)
  File "/root/anaconda3/envs/soma/lib/python3.7/site-packages/chumpy/ch.py", line 594, in r
    self._call_on_changed()
  File "/root/anaconda3/envs/soma/lib/python3.7/site-packages/chumpy/ch.py", line 589, in _call_on_changed
    self.on_changed(self._dirty_vars)
  File "/root/anaconda3/envs/soma/lib/python3.7/site-packages/moshpp-3.0-py3.7.egg/moshpp/prior/gmm_prior_ch.py", line 62, in on_changed
    for logl, w in zip(self.loglikelihoods, self.weights)])
  File "/root/anaconda3/envs/soma/lib/python3.7/site-packages/moshpp-3.0-py3.7.egg/moshpp/prior/gmm_prior_ch.py", line 62, in <listcomp>
    for logl, w in zip(self.loglikelihoods, self.weights)])
  File "/root/anaconda3/envs/soma/lib/python3.7/site-packages/chumpy/ch.py", line 596, in r
    self._cache['r'] = np.asarray(np.atleast_1d(self.compute_r()), dtype=np.float64, order='C')
  File "/root/anaconda3/envs/soma/lib/python3.7/site-packages/chumpy/ch_ops.py", line 319, in compute_r
    return np.sum(self.x.r, axis=self.axis)
  File "/root/anaconda3/envs/soma/lib/python3.7/site-packages/chumpy/ch.py", line 596, in r
    self._cache['r'] = np.asarray(np.atleast_1d(self.compute_r()), dtype=np.float64, order='C')
  File "/root/anaconda3/envs/soma/lib/python3.7/site-packages/chumpy/ch_ops.py", line 584, in compute_r
    return self.safe_power(self.x.r, self.pow.r)
  File "/root/anaconda3/envs/soma/lib/python3.7/site-packages/chumpy/ch.py", line 596, in r
    self._cache['r'] = np.asarray(np.atleast_1d(self.compute_r()), dtype=np.float64, order='C')
  File "/root/anaconda3/envs/soma/lib/python3.7/site-packages/chumpy/ch_ops.py", line 708, in compute_r
    return self.a.r * self.b.r
  File "/root/anaconda3/envs/soma/lib/python3.7/site-packages/chumpy/ch.py", line 596, in r
    self._cache['r'] = np.asarray(np.atleast_1d(self.compute_r()), dtype=np.float64, order='C')
  File "/root/anaconda3/envs/soma/lib/python3.7/site-packages/chumpy/ch_ops.py", line 731, in compute_r
    return self.a.r.dot(self.b.r)
  File "/root/anaconda3/envs/soma/lib/python3.7/site-packages/chumpy/ch.py", line 596, in r
    self._cache['r'] = np.asarray(np.atleast_1d(self.compute_r()), dtype=np.float64, order='C')
  File "/root/anaconda3/envs/soma/lib/python3.7/site-packages/chumpy/ch_ops.py", line 566, in compute_r
    return self.a.r - self.b.r
ValueError: operands could not be broadcast together with shapes (69,) (63,)

Hi, could you give me some advice about how to get the smal parameters with Mosh++ if I have some 3d animation models bought from unity store. (ie:https://assetstore.unity.com/packages/3d/characters/animals/mammals/fully-animated-cats-185493#description)

currently, I manually labeled 53 marker locations which are defined in person smpl-x marker-layout, and the mesh result with mosh++ looks like a person with four cilmbs down on ground, the pose is what we thought it would be except the shape is a person, because of the smpl-x model. we need a smal model rather than a smpl-x for the animals.

besides replacing the smpl-x model with smal, what should I do if I want to fitting the smal params to animal mocap data manual picked from the unity 3d model?

below is a mesh results of cat under smpl-x param model, a person-like cat.