I am using BMX055 sensor as input. It's a 9-dof sensor that includes Accelerometer, Gyroscope, Magnetometer.

IMU data, each sensor

Accelerometer

   [ xAccl, yAccl, zAccl]        [3 x 1]

Gyroscope

   [ xGyro, yGyro, zGyro]        [3 x 1]

Magnetometer

   [ xMag, yMag, zMag]        [3 x 1]

Model input data, each sensor

Accelerations

   [ 1.0883e+00,  2.7423e+00,  1.7926e+00]        [3 x 1]

Rotations

   [[ 0.2445,  0.1298,  0.9609],       [3 x 3]
   [-0.9482,  0.2391,  0.2090],
   [-0.2026, -0.9623,  0.1815]]

How to convert IMU data to model input data? Thankyou!

In your paper you mention you synthesise global virtual "sensor" orientations. I'm confused how I might go about synthesising sensor relative orientations from the amass dataset (so that I can use this model with imus that only provide sensor relative measurements). Do you guys have any insights?

Thanks! Vimal

Hi Xinyu-Yi,

Thank you for sharing the code. Maybe I missed it, but I couldn't figure out where in the code you actually do the Calibration stated in section A 1. in the paper. For example the Raw DIP-IMU data is uncalibrated, but you just adapt it for NaN values and the npass it to the network after normalization. But nevertheless I obtain the same results as in the paper. Why do you not do the calibration in here?

Also would it be possible to share the training routines, so that we can reproduce your results from scratch? The latter might be also in your interest as I will list your method as a comparison in a coming paper, and in best case I plug and play your method with my Input to have the fairest comparison.

Thank you in advance

Hello Xinyu, I have a question about the dimension of Pose-S3 output. As mentioned in the paper, The output of Pose-S3 is R6(j-1), and j is 24. So the output dimension is 138 But the in the code, net.py, The pose_s3 output dimension is joint_set.n_reduced * 6, which is 90. I have no idea about it. Can you help me to figure out why the dimension is it?

So, when I want to calculate the loss of Pose-S3. I need to use the func:_reduced_glb_6d_to_full_local_mat. to get the full joint rotation matrix. And use func:rotation_matrix_to_r6d, to get the full joint rotation in 6d, right?

Hi!

I'm trying to reproduce paper results using lower-cost IMU.

Since I am not tall enough (1m63) and have not edited the leg_length information, the estimate is not accurate. I tried and tested a lot to get the first results: Video Link

IMU: I'm supported six Hi229 9-dof from HiPNUC (datasheet) Frequency: 50Hz (record 100Hz step=2)

I recorded the data(data_align, data_T_Pose) and modified live_hi229.py based live_demo.py:

• Put imu (x = Left, y = Up, z = Forward)
  • record data_align in 10s length: 500 frame
  • data_align[50:300]: align
• Stand straight in T-pose
  • record data_T_Pose in 60s length: 2999 frame
  • data_T_Pose[50:300]: T_Pose
• Demo
  • data_T_Pose[500:2980]: action

The acceleration read from Noitom is global. But in my data: the acceleration is local, and the quaternion is local. I don't know how to convert local to global acceleration, the local accelerometer is passed to the model. This could be the reason the body estimate is not accurate.

Can you help me revise the code to fit my data? (live_hi229.py) If you are too busy, read through the code once and give me some advice. Thanks a lot!

Hi Xinyu,

Thank you and your colleagues for this great work. Really impressed by the results.

We are working on a project where part of your project might be very helpful. Specifically, how do you select the subset of AMASS and generate the synthetic data (~1200 min out of total 60 hours)? Do you need to downweight some common motions (like walking) and upweight some rare motions during training?

I am also wondering the details about how you place the virtual IMU sensors on the SMPL model -- is it same as DIP? Would it be possible for you to send me (yifengj at stanford.edu) the training / data generation code? ("Dirty" and uncommented version is totally fine.)

Thanks a lot! Yifeng

Dear the author of the article Transpose

I am a student at the University of Danang in Vietnam.

I want to use the results of the paper "Real-time 3D Human Translation and Pose Estimation with Six Inertial Sensors" into VRChat.

Please allow me to conduct research for academic purposes.

Thank you very much!

Link: https://twitter.com/VRChat/status/1493314642071986182 https://twitter.com/Tn23008624/status/1494618444821839877

你好，请问在对totalcapture处理过程中 代码如下 pos[:, :, 0].neg_() #对Tensor中的每个元素取负 pos[:, :, 2].neg_() trans.append(pos[:, 2] - pos[:1, 2]) # N, 3 #相当与每帧的脊柱坐标减去第一帧的脊柱坐标 请问为什么要进行neg操作，此外左右脚数据好像没有使用到 十分感谢

您好，我在尝试使用其他传感器运行TransPose代码时，遇到一些问题，想请教一下。

我基于live_demo.py，将TCP接收诺亦腾数据部分改为接收自己的传感器数据，按照左手臂、右手臂、左腿、右腿、头、胯顺序，分别存入self._quat_buffer和self._acc_buffer中。其中，四元数是按照（w, x, y, z）顺序存放，加速度数据是经过处理去掉重力部分的数据。 后续代码基本没变，最后使用提供的U3D插件显示结果。 运行时，按照指示先将传感器按照x左y上z前摆放，再穿戴好做T-POSE（佩戴时传感器朝向和文章中的不一样）。

一开始显示的姿态异常，例如向前举手，模型手会绕一圈再举起来，我尝试将_read()中，加速度乘以-9.8的操作去掉后，姿态显示好了很多，但模型始终无法跳跃（地上行走ok），猜测是诺亦腾的加速度数据与我的传感器加速度数据不一致，或者我是否漏了一些处理操作，因此有以下几个问题：

1、是否可以提供一小段诺亦腾数据供参考数据的格式？（例如加速度数据是否含有重力部分） 2、_read()中为什么加速度要乘以-9.8？ 3、TransPoseNet初始化中gravity_velocity=-0.018是如何计算得到的？ 4、config.py中提供的acc_scale = 30和vel_scale = 3是如何计算得到的？ 5、传感器佩戴方式是否有限制？如果有，方便提供一下传感器在佩戴时的轴向情况吗？

非常感谢！

Very nice work! I have a question about the smpl vertex index. In the smpl model, we need to manually select some vertices and place virtual sensors on these vertices. Each of these vertices corresponds to an index. I would like to ask you how you get these indices? Is it by calculation or do you get them through unity?

Hi! Congrats on the amazing paper and thanks so much for providing your source code! I was wondering if you guys used variable input sequences while training and if you could provide code for handling variable input sequences?

Thanks!

大佬你好，感谢你的工作。 我在使用其他项目计算的SMPLX数据（右手坐标系，X-right，Y-up，Z-forward）截取了前24个节点坐标和trans后导入你创建的unity项目中实现可视化的时候遇到了一些问题，人体动作很扭曲。 想请教你的是，你的代码计算后传输到unity的数据是左手坐标系还是右手坐标系？x,y,z分别都是哪些方向？ MotionController.cs中的坐标转换看得有点懵。请大佬指教

I'm trying to make train.py with your given code and datasets. I made synthetic amass datasets and checked it worked well in evaluate.py.

Most of train.py was written, but there is difficulty in calibrating amass data.

In the paper, t-pose calibration was used, and to do so, ori_raw and acc_raw values of t-pose are required. How can I get data? Are they already in synthetic amass datasets?

In calibration process of the live_demo.py, I can check that smpl2imu and device2bone are created by directly collecting t-pose data. I'm trying to use a code similar to this, but I don't know how to get ori_raw and acc_raw.

Hi, I'm trying to reproduce test result using DIP-IMU and TotalCapture datasets.

When run preprocess.py and evaluate.py, It worked well in DIP-IMU, but not in TotalCapture.

Is anyone who reproduce test result with TotalCapture dataset?

Should I change code or some data?

作者在论文中写到输入的数据是 校准后的6个IMU的加速度 accelerations 6 * 3， 还有方向矩阵 rotations 633 我自己买的九轴传感器其中一个数据是 三轴加速度 + 三轴角速度 +三轴磁场 我理解的是 accelerations 是由三轴加速度校准得来的 但我并不清楚rotations是 九轴IMU的哪些数据得来的 作者能解答一下rotations是怎么得来的吗

Hi, I've been using the XSens IMU sensors to collect data and run pose prediction using the TransPose algorithm. The sensor placement and configuration follow that of the DIP dataset. I use both single sensor calibration (x=Left, y=Up, z=Forward) and T-pose calibration, like what is done in live_demo.py. However, what I've found, is that upper limb predictions are pretty accurate, but the lower limb predictions are not correct (for example, the Unity avatar is supposed to raise the leg but it is just standing normally). Any possible suggestions, or someone with similar problems?