Hi,

I successfully ran the example.py and got gt.obj and est.obj. But I didn't see where I can visualize them.

After searching I found that I can visualize it use open3d, but when I load the .obj file it throw error:

import copy
import open3d as o3d

print("Testing mesh in open3d ...")
mesh = o3d.io.read_triangle_mesh("est.obj")
print(mesh)

It gives: Testing mesh in open3d ... Read geometry::TriangleMesh failed: unknown file extension.

Any suggestions? Thanks!

I've been trying to get this IK solver to work with the FreiHand data but so far haven't been unable to. I wanted to see if there was something that I am doing wrong. Is there any sort of preprocessing needed to keypoints before being fed into the solver?

I have managed to get it working no problem when using the keypoints provided by the mesh

_, keypoints = mesh.set_params(pose_pca=pose_pca, pose_glb=pose_glb, shape=shape)

and using the solver I get a really good result. But when using the XYZ keypoints from the FreiHand dataset I get the following

I have done the following things. First I made sure that I had the joints in the correct order following the MANO convention rather than MPII that FreiHand is in. So

class MANOHandJoints:
    n_joints = 21

    labels = [
        'W',  # 0
        'I0', 'I1', 'I2',  # 3
        'M0', 'M1', 'M2',  # 6
        'L0', 'L1', 'L2',  # 9
        'R0', 'R1', 'R2',  # 12
        'T0', 'T1', 'T2',  # 15
        'I3', 'M3', 'L3', 'R3', 'T3'  # 20, tips are manually added (not in MANO)
    ]

instead of this

class MPIIHandJoints:
    n_joints = 21

    labels = [
        'W',  # 0
        'T0', 'T1', 'T2', 'T3',  # 4
        'I0', 'I1', 'I2', 'I3',  # 8
        'M0', 'M1', 'M2', 'M3',  # 12
        'R0', 'R1', 'R2', 'R3',  # 16
        'L0', 'L1', 'L2', 'L3',  # 20
    ]

Secondly I made sure to scale the coordinates to be in millimeters rather than meters. Do I also need to mean center and normalize similar to how it's done in IKNet? I tried this as well but still was not getting good results. Any help would be greatly appreciated!

Hello,CalciferZh.When I input detect 3D points in IK that result is strange.

points = np.asarray([[-0.05845946, -0.03575248, -0.01891485, -0.00118977,  0.02288574, -0.02119502,
                    -0.00542915,  0.01052371,  0.02606209, -0.01342221,  0.01144132,  0.02687377,
                    0.04031715, -0.01040107,  0.01338105 , 0.03538704,  0.05632934, -0.00652865,
                    0.01324268,  0.03186151,  0.05112882],\
                    [ 0.02216181, -0.00476968, -0.02193613, -0.02797434, -0.04152796, -0.02755484,
                    -0.03023094, -0.03193476, -0.03731135, -0.01063601, -0.01484661, -0.01565503,
                    -0.01747507 , 0.01185602,  0.0031302,  -0.00127476, -0.00702907,  0.02705859,
                    0.02097698 , 0.0164664,   0.00978883],\
                    [ 0.86504179,  0.8595106,   0.84762412,  0.83012652,  0.80829155,  0.80431879,
                    0.78179353,  0.78447574,  0.80182135,  0.7954936,   0.78770196,  0.79872048,
                    0.81998879,  0.80215442,  0.79682922 , 0.80361056,  0.8165772,   0.81301314,
                    0.8120501,   0.81251454,  0.81184596]])

Hi @CalciferZh In spmlx , the shape of keypoint is (24,3), as follow:

In https://github.com/CalciferZh/Minimal-IK/blob/72bfcbb768355fcc6706ca8cbdf50c3da25f9f63/example.py#L47 the shape of keypoint is (29,3)

Could padding (5,3) to solve this gap?

SMPL parameters : betas(10 x 1) + pose(24 x 3) -------> joint coordinates (and optional other keypoints)

your project: joint coordinates (and optional other keypoints) -----> betas + pose

   I assume that the betas are fixed , only solve pose paramters。As it's only a "minimal" solution, is there any other solution?

您好，minimal-IK是一个非常有启发的项目

function 关于update函数我有个问题，我注意到优化的时候只有keypoints被用到，而verts没有被用到，同时要计算verts涉及到几个778size的运算。 请问如果为了速度，只输出keypoints，是否有可能略过verts的部分相关计算

谢谢！

Hi! After going through the code, I ain't able to understand the following:

If I want to find the pose_abs of the estimated hand using the MANO hand model, what should be the format in which I am required to input the hand key points? By that what I mean is, with respect to which coordinate system should I input the hand key points?

Thanks in advance!

Hi, when I run example.py. I got errors as follows:

Traceback (most recent call last): File "/home/haomeng/PycharmProjects/Minimal-IK/example.py", line 20, in <module> mesh = KinematicModel(config.MANO_MODEL_PATH, MANOArmature, scale=1000) File "/home/haomeng/PycharmProjects/Minimal-IK/models.py", line 23, in __init__ params = pickle.load(f) UnicodeDecodeError: 'ascii' codec can't decode byte 0xd5 in position 2: ordinal not in range(128)

It seems that code need to run in python2.X version but another dependices called open3d couldn't run in python2.X.

Could you export your virtual environment and provide the .yaml file. Thanks!

For SMPL-H following https://github.com/vchoutas/smplx/blob/master/smplx/joint_names.py

for SMPL following line 18 to 42? for SMPL-H following line 18 to 72? for MANO following line 43 to 72?

Hi,

Thanks for the great work. I try to run the example for smpl but there are key errors. In the code we load these parameters:

` self.J_regressor = params['J_regressor']

  self.skinning_weights = params['skinning_weights']

  self.mesh_pose_basis = params['mesh_pose_basis'] # pose blend shape
  self.mesh_shape_basis = params['mesh_shape_basis']
  self.mesh_template = params['mesh_template']

  self.faces =  params['faces']

  self.parents = params['parents']`

But in the basicModel_f_lbs_10_207_0_v1.0.0.pkl model I only have these parameters: dict_keys(['J_regressor_prior', 'f', 'J_regressor', 'kintree_table', 'J', 'weights_prior', 'weights', 'vert_sym_idxs', 'posedirs', 'pose_training_info', 'bs_style', 'v_template', 'shapedirs', 'bs_type'])

Loos like only J_regressor is matching. Could you please give me some pointers about it?

https://github.com/CalciferZh/Minimal-IK/blob/0c189028f9c2b5f03a78c8209f47a523a87bda17/solver.py#L53

How about accuracy of this method? Do you compare it with other methods?

Hi! I'm very new to this. I have coco keypoints of a person (18 keypoint format) for multiple frames. I would like to convert my keypoints to the SMPL format so that I can render some animations on Mixamo. Is this at all possible? And if so, how can I do this? My keypoint file is of the format [frames,joints,3] -> 300,18,3 (3 refers to positions along the x,y,z axis)

Hi, when I run smpl, the fit is poor when using the keypoints output from my own program as input. I think this may be due to the large rotation of the root node.

Again, we note that this approach cannot handle large global rotations (R0) due to the high non-convexity. For example, when the subject keeps the T pose but faces backwards.

I rotated the keypoint to roughly the same pose as T pose, but it does not solve the problem, do you have any comments?