Hi, thanks for your nice work!

How can I get a pre-trained model(0811_in_256_4.model and point.model) of pointnet?

Could you share the pre-trained model of pointnet?

Also, could you share the pre-trained model of cvae, RouteNet and PoseNet?

Thanks!

Hi Jiashun, thanks for your nice work! Could you please illustrate more about the intuition behind this for loop? https://github.com/jiashunwang/Long-term-Motion-in-3D-Scenes/blob/5673d342093ff92e677da2d90e0dd8b7dd766c5b/utils_optimize.py#L312

Thanks for your interesting work. When I attempt to use the code in generate_sub_data.ipynb, it seems human_body_prior has changed its function, can you tell me which commit of human_body_prior do you use for the generate_sub_data.ipynb?

I have noticed that you set the middle position manually in demo.ipynb. So, I am wondering that if I want to synthesize a long-term sequence with the start position and end position as input, I need to manually set the middle position by myself? According to my understanding, the sub-goal network proposed in your paper only synthesizes the body with a given position, it doesn't generate a middle position in a long-term sequence synthesis, am I right? I am looking forward to your reply.

Thanks for your instruction for the demo. But the demo can only generate an SMPLX model for each frame, can you publish the code which generates the video visualization of your method on your project page?

Hi

I have a question about evaluation. I implemented the evaluation code for the evaluation split of the PROX dataset. In Table 1 of the paper(Ours w/o opt), translation, orientation, and pose errors are reported as 6.91, 9,71, and 41.17, respectively. The results of the code I implemented are 59.02, 60.23 and 1459.95 respectively. What's wrong with my implementation?

import os
import random
import torch
import numpy as np

from route_data import ROUTEDATAEVAL
from route_data import ROUTEDATA
from route import ROUTENET
from pose_after_route import POSEAFTERROUTE
from utils import GeometryTransformer
from utils import AverageMeter
from progress.bar import Bar

def l1_error(prediction, target):
    return torch.abs(prediction - target).sum(dim=1).mean().item()


SEED_VALUE = 0
print(f'Seed value for the experiment {SEED_VALUE}')
os.environ['PYTHONHASHSEED'] = str(SEED_VALUE)
random.seed(SEED_VALUE)
torch.manual_seed(SEED_VALUE)
np.random.seed(SEED_VALUE)


batch_size = 16

dataset = ROUTEDATAEVAL()
dataloader = torch.utils.data.DataLoader(dataset, batch_size=batch_size, shuffle=False)


model_route = ROUTENET(input_dim=9,hid_dim=64)
model_route = model_route.cuda()

print('use pretrained routenet')
model_route.load_state_dict(torch.load('saved_model/route.model'))
model_route.eval()

model_pose = POSEAFTERROUTE(input_dim=65-9,hid_dim=256)
model_pose = model_pose.cuda()

print('use pretrained posenet')
model_pose.load_state_dict(torch.load('saved_model/pose_after_route.model'))
model_pose.eval()

error_translation_ = AverageMeter()
error_orientation_ = AverageMeter()
error_pose_ = AverageMeter()

num_iter = len(dataloader)
bar = Bar('==>', max=num_iter)

for j, data in enumerate(dataloader, 0):

    input_list, middle_list, frame_name, scene_name, sdf, scene_points, cam_extrinsic, s_grid_min, s_grid_max = data

    input_list = input_list[:, [0, -1], :]
    body = middle_list[:, 0:1, 6:16].cuda()
    input_list = torch.cat([input_list[:, :, :6], input_list[:, :, 16:]], dim=2)
    middle_list = torch.cat([middle_list[:, :, :6], middle_list[:, :, 16:]], dim=2)

    scene_points = scene_points.cuda()

    input_list = input_list.view(-1, 62)
    six_d_input_list = GeometryTransformer.convert_to_6D_rot(input_list)
    six_d_input_list = six_d_input_list.view(-1, 2, 65)
    x = six_d_input_list.cuda()
    x1 = six_d_input_list[:, :, :9].cuda()

    middle_list = middle_list.view(-1, 62)
    six_d_middle_list = GeometryTransformer.convert_to_6D_rot(middle_list)
    six_d_middle_list = six_d_middle_list.view(-1, 60, 65)  # 60: 2s 30fps

    y = six_d_middle_list[:, :, :9].cuda()

    out_route = model_route(x1, scene_points.transpose(1, 2))

    pred_trans = out_route[: , :, :3]
    pred_6d = out_route[:, :, 3:]

    gt_trans = y[:, :, :3]
    gt_6d = y[:, :, 3:]

    route_prediction = out_route.detach().view(x1.shape[0],-1)

    out_pose = model_pose(x[:,:,9:],scene_points.transpose(1,2),route_prediction)

    y = six_d_middle_list[:, :, 9:].cuda()

    pred_body_pose = out_pose[:, :, :32]
    gt_body_pose = y[:, :, :32]

    error_translation = l1_error(pred_trans.reshape(-1,3).detach(), gt_trans.reshape(-1,3)) * 100.
    error_orientation = l1_error(pred_6d.reshape(-1,6).detach(), gt_6d.reshape(-1,6)) * 100.
    error_pose = l1_error(pred_body_pose.reshape(-1,32).detach(), gt_body_pose.reshape(-1,32).detach()) * 100.


    error_translation_.update(error_translation, pred_trans.shape[0] * pred_trans.shape[1])
    error_orientation_.update(error_orientation, pred_trans.shape[0] * pred_trans.shape[1])
    error_pose_.update(error_pose, pred_trans.shape[0] * pred_trans.shape[1])



    summary_string = ' EVAL :[{0}/{1}]'.format(j, num_iter)
    summary_string += ' | ET {error_translation.avg:.4f} | EO {error_orientation.avg:.4f} | EP {error_pose.avg:.4f}'.format(error_translation=error_translation_, error_orientation=error_orientation_, error_pose=error_pose_)

    Bar.suffix = summary_string
    bar.next()

bar.finish()

Thanks, Seonghyun