@dazinovic How about the result on real-world datasets. I collect the datasets with my own RGBD camera and estimate the pose with colmap. But the results is a mass. Any advices?

I'm a little confused about the translation in config. For instance, in configs/scene0050_00.txt there is a config named translation. Each config contains a different translation, what is the effect of the translation?

Hi, @dazinovic Thx for sharing your great work! I'm confused about the meaning of sdf used in your code. I hope you can help me figure it out. https://github.com/dazinovic/neural-rgbd-surface-reconstruction/blob/01782741271b5ace1083bfc4aadd787c97b31836/losses.py#L31-L38 As in this function, it seems that sdf is a ratio between the actual distance and the truncation value. While, when it turns to this render function, https://github.com/dazinovic/neural-rgbd-surface-reconstruction/blob/01782741271b5ace1083bfc4aadd787c97b31836/optimize.py#L172-L183 it seems that sdf represents a distance in normalized space. So, if we use the second meaning which is sdf is a distance in normalized space, should the sdf loss function change to this?

def get_sdf_loss(z_vals, target_d, predicted_sdf, truncation, loss_type):

    front_mask, sdf_mask, fs_weight, sdf_weight = get_masks(z_vals, target_d, truncation)

    fs_loss = compute_loss(predicted_sdf * front_mask, truncation * tf.ones_like(predicted_sdf) * front_mask, loss_type) * fs_weight
    sdf_loss = compute_loss((z_vals + predicted_sdf) * sdf_mask, target_d * sdf_mask, loss_type) * sdf_weight

    return fs_loss, sdf_loss

Thanks for your wonderful work! I have got the reconed mesh as:

So I wonder: Are the wired blocks around the scene proposed? If so, how can I alleviate their effects in the evaluation period? I have tried code from retrieval-fuse. While I get the metric with: Iou: 0.5294117647058824,chamferL1:0.21610536071424724, normals_correctness:0.7708863087762083, f-score1:0.8835909549382578, f-score-1.5：0.9605475835329947

looking forward to your kind reply!

Original NeRF can simulate a virtual camera for extract the vertices color from the NeRF, I did the same operation as https://github.com/bmild/nerf/issues/44 , but get a bad results, any ideas to extract color mesh from the trained model?

Hi @dazinovic , Thanks for your great work! I'm trying to train a ScanNet Scene using your code. Could you tell me how you transform ScanNet poses? I investigate the synthesis dataset you provide, I guess trainval_poses are: [ 1 0 0 0 0 0 -1 0 0 1 0 0 0 0 0 1] @ inv(T0) @ Tx where T0 is the first pose of poses(poses.txt) and Tx is the rest. But they are not equal after I verify the "complete kitchen" scene.

Thanks.

Hello, author.

Thank you for completing and providing this great work.

I ran the code and got perfect results on the data set you provided, so I want to try to make my own data set and see how the code works. Could you describe in detail how to create own datasets, such as how to use BundleFusion you mentioned in readme.md.

Seems like the link : http://kaldir.vc.in.tum.de/neural_rgbd/neural_rgbd_data.zip is broken.

Could you please upload the data to some place else (maybe on gdrive?)

Hi, Dejan! Thanks for opensourcing this work :muscle:! I cannot get how to transform poses from custom format to the format in this work. May be I did not get properly OpenGL coordinate system you mentioned, but it looks like the usual yUp coordinate system. So I expect that y axis from camera pose is "up" direction. However the poses for breakfast_room (and other provided scenes) are oriented differently: I have rotated MeshLab axes to match OpenGL coordinate system and draw camera poses (each 10th), and y axis from camera pose (in green) is aligned with z axis from coordinate system and vice versa: I have checked #2, #5, #9, #16, #21 and still in stuck. Can you please clarify why camera axes are not aligned with the corresponding OpenGL coordinate axes?

Hi, @dazinovic , thx for the great work. I've investaged [#2 ] & [#4 ] , and finally found the corret operation for aligning point clouds from each frame in the given dataset, for instance the breakfast_room data. I've got complete sence points using both trainval_pose.txt & pose.txt, although there exist a rigid transformation between them, key idea is the transform the ponit cloud recovered from each frame using: T0 @ trainval_pose or pose @ T1 @ point_cloud, with T0 = [ 1., 0., 0., 0.], [ 0., 0., 1., 0.], [ 0., -1., 0., 0.], [ 0., 0., 0., 1.]

T1 = array([[ 1., 0., 0., 0.], [ 0., -1., 0., 0.], [ 0., 0., -1., 0.], [ 0., 0., 0., 1.]])

But I'm still confused about the meaning of given T0 & T1。I guess that T1 is used for converting from OpenCV coo to OpenGL coo, T0 for converting to Blender, in which the datasets were created. Do I understand it right? Since that you use the trainval_pose in the code, are the transformations, namely T0 & T1 here, implicitly included in your code? And did the trainval_pose came directly from BundleFusion? If I want to use a Kinect to create a custom data, what should I do with these transformations?

hey! I just found that in the latest paper, there don't exist any quantitative results of Neural RGB-D surface recon on SCANNET dataset. Can you share the results so that I won't need to re-eval. Many thanks for your wonderful work!

Hi @dazinovic , Thanks for your great work! Based on the pose conversion you provided #2 , T0 @ scannet_pose @ T1, I want to make sure that the coordinate system of official scannet pose is that: positive x-axis is right, positive y-axis is out and positive z-axis is down. Am I right or not?

Thanks.

hi, many thanks for your great works! I tried to run the transform_mesh.py and cull_mesh.py scripts on the breakfast_room data on remote server. But I get a error with the cull_mesh script. Do you have any advice how I can resolve this error?

/home/dl/kaiduo.zhang/LumiNet/cullmesh/neural_rgbd_data/ breakfast_room Traceback (most recent call last): File "/home/dl/kaiduo.zhang/LumiNet/cullmesh/frustum_culling/cull_mesh.py", line 172, in <module> cull_mesh(mesh_path, save_path, pose_file, training_poses, intrinsics_path, scene_bounds=scene_bounds) File "/home/dl/kaiduo.zhang/LumiNet/cullmesh/frustum_culling/cull_mesh.py", line 81, in cull_mesh proc1 = subprocess.Popen(['scontrol', 'show', 'job', os.environ['SLURM_JOBID'], '-d'], stdout=subprocess.PIPE) File "/home/dl/kaiduo.zhang/anaconda3/envs/cull/lib/python3.9/os.py", line 679, in __getitem__ raise KeyError(key) from None KeyError: 'SLURM_JOBID'

Hi, Thanks for sharing you great work! I found that neural-rgbd performs well in estimating the camera poses, so I wonder if you can share the estimated camera poses on 10 synthetic scenes. I would like to use the estimated camera poses of neural-rgbd as an initialization. Thanks for your time!

Hi, firstly thanks so much for this amazing work.

Now I finished all the preparation steps and started running "optimize.py".

1. After traininng I will get a reconstruction mesh which is exactly as same as the "meshes.zip"?
2. If not, which step should i do to get the same result from the zip file?

Thanks for your reply.

Hi, I am trying to get this repo working. It seems like the environment.yml file is outdated. I get a bunch of dependencies conflicts. ANy ideas what the right .yml file should be?

Hi:

thank you for sharing this amazing work!

Is it possible to share the training/evaluation script of the compared methods (like routed fusion) on the scannet dataset?

thank you!