Hey, I have been trying out combining the SURREACT dataset with the STAR model and can not seem to get satisfying results. From what I can see this is very similar to the problem described here: https://github.com/gulvarol/surreal/issues/10#issuecomment-447336600

However, I implemented the code regarding the rotation and still the generated meshes seems to have the wrong orientation. Tested this with a number of different examples but the rotation always seems to be off (without me being able to notice any similarity in the way they are oriented). Included the used code below, would appreciate any kind of help on this matter.

import math from star.pytorch.star import STAR import numpy as np import torch import open3d import scipy.io import transforms3d

def rotateBody(RzBody, pelvisRotVec):

pelvisRotVec = pelvisRotVec.cpu().detach().numpy()
angle = np.linalg.norm(pelvisRotVec)
Rpelvis = transforms3d.axangles.axangle2mat(pelvisRotVec / angle, angle, True)
globRotMat = np.dot(RzBody, Rpelvis)
R90 = transforms3d.euler.euler2mat(np.pi / 2, 0, 0)
globRotAx, globRotAngle = transforms3d.axangles.mat2axangle(np.dot(R90,globRotMat))
globRotVec = globRotAx * globRotAngle
return torch.cuda.FloatTensor(globRotVec)

num_betas=10 batch_size=1 m = STAR(gender='male',num_betas=num_betas)

mat = scipy.io.loadmat('/SURREACT_DATA/a37_d1_p019_c1_color.avi_v225_r00_info.mat')

betas = torch.cuda.FloatTensor(mat['shape']) trans = torch.cuda.FloatTensor(np.zeros((batch_size, 3)))

max_frame = torch.cuda.FloatTensor(mat['joints3D']).shape[2]

for x in range(0, max_frame+1, 2):

pose = torch.cuda.FloatTensor(mat['pose'][:, :, x])
betas = torch.cuda.FloatTensor(mat['shape'])

zrot = mat['zrot_euler'][0][0]
zrot = math.radians(zrot)
RzBody = np.array(((math.cos(zrot), -math.sin(zrot), 0),
                   (math.sin(zrot), math.cos(zrot), 0),
                   (0, 0, 1)))
pose[0, 0:3] = rotateBody(RzBody, pose[0, 0:3])

model = m.forward(pose, betas, trans)
shaped = model.v_shaped[-1, :, :]

# 3d visualization with open3d
mesh = open3d.geometry.TriangleMesh()
vertices_list = np.array(model[0].tolist())
faces_list = np.array(model.f.tolist()).astype(np.int32)
mesh.vertices = open3d.utility.Vector3dVector(vertices_list)
mesh.triangles = open3d.utility.Vector3iVector(faces_list)
open3d.visualization.draw_geometries([mesh])

Hi! I found the camera height is strange that positive height gives upward view while negetive height looks down. It is quite different from the paper Fig. A.4. Could you help to figure out this problem？Thanks! For example surreact_uestc_hmmr/train/a1_d1_p001_c1_color.avi/a1_d1_p001_c1_color.avi_v135_r00.mp4 with height 2.9454

surreact_uestc_hmmr/train/a10_d1_p003_c1_color.avi/a10_d1_p003_c1_color.avi_v225_r00.mp4 with height -0.96642