Hello! Thank you for sharing this wonderful work I wonder why the author used alpha * (0.5 + 0.5 * sdf.sign() * torch.expm1(-sdf.abs() / beta)) instead of psi = torch.where(sdf > 0, exp, 1 - exp) with exp = 0.5 * torch.expm1(-sdf.abs() / beta).

Hi, thank you for your decent work. I try to follow your work recently and I meet some problems which I wish to get answers from this issue.

1. First question: In function load_K_Rt_from_P at line 48 in rend_util.py: https://github.com/lioryariv/volsdf/blob/a974c883eb70af666d8b4374e771d76930c806f3/code/utils/rend_util.py#L48-L50 This code really makes me confused and I'm not able to give an explanation to it. I read the following code at line 78 in rend_util.py: https://github.com/lioryariv/volsdf/blob/a974c883eb70af666d8b4374e771d76930c806f3/code/utils/rend_util.py#L73-L78 It seems that you use pose as a cameraToWorld matrix. I did an experiment in advance, the following code is from stackoverflow:

k = np.array([[631,   0, 384],
              [  0, 631, 288],
              [  0,   0,   1]])
r = np.array([[-0.30164902,  0.68282439, -0.66540117],
              [-0.63417301,  0.37743435,  0.67480953],
              [ 0.71192167,  0.6255351 ,  0.3191761 ]])
t = np.array([ 3.75082481, -1.18089565,  1.06138781])

C = np.eye(4)
C[:3, :3] = k @ r
C[:3, 3] = k @ r @ t

out = cv2.decomposeProjectionMatrix(C[:3, :])

If I convert r and t into a homogeneous coordinate, then I take the R@T, which is the worldToCamera matrix. I will get:

>>> T=np.eye(4)
>>> T[:3,3]=t
>>> R=np.eye(4)
>>> R[:3,:3]=r
>>> R@T
array([[-0.30164902,  0.68282439, -0.66540117, -2.64402567],
       [-0.63417301,  0.37743435,  0.67480953, -2.10814783],
       [ 0.71192167,  0.6255351 ,  0.3191761 ,  2.27037141],
       [ 0.        ,  0.        ,  0.        ,  1.        ]])

Then if I take the inverse of R@T, which I think is the cameraToWorld matrix. I will get:

>>> np.linalg.inv((R@T))
array([[-0.30164902, -0.63417301,  0.71192166, -3.75082481],
       [ 0.6828244 ,  0.37743435,  0.6255351 ,  1.18089565],
       [-0.66540117,  0.67480953,  0.3191761 , -1.06138781],
       [ 0.        ,  0.        ,  0.        ,  1.        ]])

This result seems that, to get the cameraToWorld matrix, we should concatenate the R^(-1) and -T, instead of R^(-1) and T referred in line 31 in rend_util.py: https://github.com/lioryariv/volsdf/blob/a974c883eb70af666d8b4374e771d76930c806f3/code/utils/rend_util.py#L48-L50 I don't know why it takes R^(-1) and T here.

2. Second question: In function lift in line 96 in rend_util.py: https://github.com/lioryariv/volsdf/blob/a974c883eb70af666d8b4374e771d76930c806f3/code/utils/rend_util.py#L96-L109 I don't know why the x_lift takes y and fy into consideration. It seems that sk should be 0, but I test it in runtime and I get:

intrinsics
tensor([[[ 2.8923e+03, -2.1742e-04,  8.2320e+02,  0.0000e+00],
         [ 0.0000e+00,  2.8832e+03,  6.1907e+02,  0.0000e+00],
         [ 0.0000e+00,  0.0000e+00,  1.0000e+00,  0.0000e+00],
         [ 0.0000e+00,  0.0000e+00,  0.0000e+00,  1.0000e+00]]],
       device='cuda:0')

It seems that sk is not 0. So the transformation becomes:

$$ \begin{bmatrix} x'\\y'\\z \end{bmatrix}= \begin{bmatrix} f_x&sk&c_x&0\
0&f_y&c_y&0\
0&0&1&0 \end{bmatrix} \begin{bmatrix} x\_lift\\y\_lift\\z\\1 \end{bmatrix} $$

Here [x,y,z,1] is the point in the camera coordinates. I find that:

$$ x'=f_x \cdot x\_lift + sk \cdot y\_lift + c_x \cdot z $$

The actual result of x_lift is:

$$ x\_lift = \cfrac{x'-c_x \cdot z}{f_x} - sk \cdot y\_lift $$

But in rend_list.py, x_lift is like to be:

$$ x\_lift = \cfrac{(x'-c_x)\cdot z}{f_x} - sk \cdot y\_lift $$

So when z=1, the code is correct. Would it be better if it is simply changed to be:

x_lift = (x / z - cx.unsqueeze(-1) + cy.unsqueeze(-1)*sk.unsqueeze(-1)/fy.unsqueeze(-1) - sk.unsqueeze(-1)*y/fy.unsqueeze(-1)) / fx.unsqueeze(-1) * z

(/ z is added to the x)

The first question means more to me than the second question. Would you please explain the logic of pose matrix to me.

Hope this issue would help other people as well.

Dear @lioryariv

Thank you so much for your work. I have not found how to evaluate VolSDF on BlendedMVS dataset, i.e computing the chamfer distance. The BlendedMVS repo doesn't seem to provide such script.

Would you mind pointing me to the right place? Thanks!

Hi, thank you for your decent work. I try to follow your work recently and I meet some problems which I wish to get answers from this issue.

1. First question: In function load_K_Rt_from_P at line 48 in rend_util.py: https://github.com/lioryariv/volsdf/blob/a974c883eb70af666d8b4374e771d76930c806f3/code/utils/rend_util.py#L48-L50 This code really makes me confused and I'm not able to give an explanation to it. I read the following code at line 78 in rend_util.py: https://github.com/lioryariv/volsdf/blob/a974c883eb70af666d8b4374e771d76930c806f3/code/utils/rend_util.py#L73-L78 It seems that you use pose as a cameraToWorld matrix. I did an experiment in advance, the following code is from stackoverflow:

k = np.array([[631,   0, 384],
              [  0, 631, 288],
              [  0,   0,   1]])
r = np.array([[-0.30164902,  0.68282439, -0.66540117],
              [-0.63417301,  0.37743435,  0.67480953],
              [ 0.71192167,  0.6255351 ,  0.3191761 ]])
t = np.array([ 3.75082481, -1.18089565,  1.06138781])

C = np.eye(4)
C[:3, :3] = k @ r
C[:3, 3] = k @ r @ t

out = cv2.decomposeProjectionMatrix(C[:3, :])

If I convert r and t into a homogeneous coordinate, then I take the R@T, which is the worldToCamera matrix. I will get:

>>> T=np.eye(4)
>>> T[:3,3]=t
>>> R=np.eye(4)
>>> R[:3,:3]=r
>>> R@T
array([[-0.30164902,  0.68282439, -0.66540117, -2.64402567],
       [-0.63417301,  0.37743435,  0.67480953, -2.10814783],
       [ 0.71192167,  0.6255351 ,  0.3191761 ,  2.27037141],
       [ 0.        ,  0.        ,  0.        ,  1.        ]])

Then if I take the inverse of R@T, which I think is the cameraToWorld matrix. I will get:

>>> np.linalg.inv((R@T))
array([[-0.30164902, -0.63417301,  0.71192166, -3.75082481],
       [ 0.6828244 ,  0.37743435,  0.6255351 ,  1.18089565],
       [-0.66540117,  0.67480953,  0.3191761 , -1.06138781],
       [ 0.        ,  0.        ,  0.        ,  1.        ]])

This result seems that, to get the cameraToWorld matrix, we should concatenate the R^(-1) and -T, instead of R^(-1) and T referred in line 31 in rend_util.py: https://github.com/lioryariv/volsdf/blob/a974c883eb70af666d8b4374e771d76930c806f3/code/utils/rend_util.py#L48-L50 I don't know why it takes R^(-1) and T here.

2. Second question: In function lift in line 96 in rend_util.py: https://github.com/lioryariv/volsdf/blob/a974c883eb70af666d8b4374e771d76930c806f3/code/utils/rend_util.py#L96-L109 I don't know why the x_lift takes y and fy into consideration. It seems that sk should be 0, but I test it in runtime and I get:

intrinsics
tensor([[[ 2.8923e+03, -2.1742e-04,  8.2320e+02,  0.0000e+00],
         [ 0.0000e+00,  2.8832e+03,  6.1907e+02,  0.0000e+00],
         [ 0.0000e+00,  0.0000e+00,  1.0000e+00,  0.0000e+00],
         [ 0.0000e+00,  0.0000e+00,  0.0000e+00,  1.0000e+00]]],
       device='cuda:0')

It seems that sk is not 0. So the transformation becomes:

$$ \begin{bmatrix} x'\\y'\\z \end{bmatrix}= \begin{bmatrix} f_x&sk&c_x&0\
0&f_y&c_y&0\
0&0&1&0 \end{bmatrix} \begin{bmatrix} x\_lift\\y\_lift\\z\\1 \end{bmatrix} $$

Here [x,y,z,1] is the point in the camera coordinates. I find that:

$$ x'=f_x \cdot x\_lift + sk \cdot y\_lift + c_x \cdot z $$

The actual result of x_lift is:

$$ x\_lift = \cfrac{x'-c_x \cdot z}{f_x} - sk \cdot y\_lift $$

But in rend_list.py, x_lift is like to be:

$$ x\_lift = \cfrac{(x'-c_x)\cdot z}{f_x} - sk \cdot y\_lift $$

So when z=1, the code is correct. Would it be better if it is simply changed to be:

x_lift = (x / z - cx.unsqueeze(-1) + cy.unsqueeze(-1)*sk.unsqueeze(-1)/fy.unsqueeze(-1) - sk.unsqueeze(-1)*y/fy.unsqueeze(-1)) / fx.unsqueeze(-1) * z

(/ z is added to the x)

The first question means more to me than the second question. Would you please explain the logic of pose matrix to me.

Hope this issue would help other people as well.

I try my best to express my question as clear as possible. If there's something unclear or wrong with me, please inform of me.

After running your source code, I found these issues:

• Your source code does not handle the cases with lower resolutions. e.g. for DTU dataset, if you lower the resolution to something like [ 300, 400 ], the code will crash and the default resolution [ 1200, 1600 ] is hard-coded in the source code.

• The first issue crashes the code also in the psnr computation due to the same reason

• The renderings during evaluation generate blank squares. Here bellow you can see the renderings from scan 114 of the DTU dataset:

• It seems the your model is not training at all! Because the loss and the psnr value dose not change from the beginning and remains at something like 0.1 and 12 respectively.

• Your source code will not run without lowering the config. You will immediately get OOM error.

• Your code generates messages like face_normals incorrect shape, ignoring! and face_normals all zero, ignoring! all the times. I still don't know what they mean, or if they are the cause of the issues above.

Some of the issues I mentioned above have been already found by others in other issues. I did not see any response from the authors, so I am writing these here. I am looking forward to the authors response.

dear authors: very nice work! but when i run the program, i met some "problems". after loading the dataset, i found that the program will display "face_normals incorrect shape, ignoring!", however, i didn't see where print the sentence in code. so i want to know if i miss something, what is going on when i see "face_normals incorrect shape...". regards

Hello! Thank you for sharing this wonderful work Vol-sdf is very fantastic work at least for me. I have a minor question about code in preprocess.py, though.

I wonder why below multiplication is needed rather than only inverse K. If you answer it, it would be really appreciated. Thanks a lot!

=========================================

  def get_center_point(num_cams,cameras):

    ....
    v = np.linalg.inv(K) @ np.array([800, 600, 1]) #why is it needed?
    v = v / np.linalg.norm(v)

    ....  
  return soll,camera_centers

=======================================

Hello! In eq (6), it shows tua_t = density * transperancy(or transmittance) but in the code, tua_t(or weights) = alpha * transmittance where alpha = 1.0 - torch.exp(-dists * density).

Hi! Thanks for the amazing work!

I've got a question and hope you could give me a hint. To my understand, the "camera normalization" step (as stated in Supplementary material B.1) is only used for placing the camera inside a sphere in the world coordinate. In the .conf file there is a parameter scene_bounding_sphere, and according to the class ImplicitNetwork, if scene_bounding_sphere is set to 0, the step "Clamping the SDF with the scene bounding sphere" would be ignored.

My question is I'm not sure if I could skip this step. In my experiment setting, the camera is placed on the surfce of a sphere (rather than at the center of the sphere). If I do not use normalize_cameras.py to undergo the data convention, and set the parameter scene_bounding_sphere to the radius of the sphere, I'm not sure whether it would effect the opacity approximation error bound.

Could you illustrate more about the function of the "camera normalization" step? Thank you very much! 😆