Neural-Pull: Learning Signed Distance Functions from Point Clouds by Learning to Pull Space onto Surfaces(ICML 2021)

Hi， Recently, I have read the pytorch code of your work,but I canot fully understand how to reconstrcut surface. According to code,It doesnot use any information in GT_mesh and even the query_point is random generated. How to make sure that our generated mesh can correspond to GT_mesh one by one? Could you please answer my question?Thank you so much !

Dear baorui,

Thansk for sharing your great work! From my understanding, for the surface reconstruction experiments, we need one network for each shape. And the Tab.1 and Tab 2 show the metric in ABC, FAMOUS dataset and ShapeNet. I want to make sure whether you just use the test part of these dataset ( ABC, FAMOUS dataset and ShapeNet) for the evaluation?

looking for your reply!

Best, Runsong

Thanks very much.. I have made as you say for the first 2 command lines, and as result, I have 3 environments, the first is base for anaconda, the second is env for the first line of command, the last one is called TensorFlow for the second command line. Now I'm facing another problem in the third command line which is 'python setup.py build_ext --inplace', I have implemented it in the env. of the first command, is that right or I'm in the wrong direction? the error that I have is like the photo is attached below, could you help me, please? thanks for your efforts. Heyam

Originally posted by @HeyamM in https://github.com/mabaorui/NeuralPull/issues/2#issuecomment-887088549

Dear Baorui,

First of all, I show my appreciation to you for your Neural-Pull code. But I found that under the dataset released by Points2Surf such as FAMOUS no-noise dataset, the produced chamfer-distance results of your method are inconsistent with Table 1 in the main paper. I used the evaluation code released by Points2Surf.

| Chamfer | Points2Surf(Ori Paper) | Points2Surf(My result) | Neural-Pull(Your Paper) | Neural-Pull (My result) | | :-------------: | :----------------: | :---------------: | :----------------: | :--------------: | | FAMOUS no-noise | 1.41 | 1.46 | 0.22 | 3.41 |

I have no idea what happened. Could you help me? Thank you very much!

P.S. I only found the chamfer-distance formulation defined by occupancy_networks in your code, and I haven't seen the chamfer-distance formulation of Points2Surf. Based on my understanding, the chamfer distance defined by occupancy_networks and Points2Surf is different (ErlerPhilipp/points2surf#20). I guess that it may need to utilize the chamfer-distance formulation given by Points2Surf to evaluate rec results under the dataset released by Points2Surf for fairness, just like POCO doing.

So do you think the statistics in your paper are a little misleading?

Best

https://github.com/mabaorui/NeuralPull/blob/c093a52308a9b74446d24cc6c1b0fee5ee5bb7bb/NeuralPull.py#L92-L139

Full log:

 conda env create -f NeuralPull.yaml                                                                           

Collecting package metadata (repodata.json): done
Solving environment: failed

ResolvePackageNotFound:
  - python==2.7.12=1
  - tk==8.5.18=0
  - readline==6.2=2
  - qt==5.6.2=5
  - sqlite==3.13.0=0
  - fontconfig==2.12.1=3
  - pillow==4.2.1=py27_0
  - cairo==1.14.8=0
  - matplotlib==2.0.2=np113py27_0
  - freetype==2.5.5=2
  - icu==54.1=0
  - libiconv==1.14=0
  - cudatoolkit==8.0=3

Hi @mabaorui, I have got some cycles to test the code :)

I am going around this issue by removing the version constraints on these packages but it would be great to have an official YAML that works because now i won't know if any subsequent issue come from a bad package version or something else,

Best regards, Thibault

Hi,

Thanks a lot for the code release, I greatly enjoyed your paper. I would like to experiment with surface reconstruction on my own pointclouds and ran into a few issues.

As per my understanding from your pape, NeuralPull does not condition the surface reconstruction on anything :

We do not leverage any condition c in Fig. 1 and overfit the neural network
to the shape by minimizing the loss in Eq. 2

However, the surface reconstruction script seems to be conditioning the reconstruction on a code, and learning surface reconstruction jointly?

Ideally, to test NeuralPull, it would be great if the repo included a script that takes a pointcloud as input, fits neuralPull and returns a mesh.

I am sorry if I misunderstood the code and thanks a lot for your help,

Thibault

Hi Baorui,

I have some questions regarding this experiment presented in Section 4.2, hope you can help me with this.

1. Is the L1 CD metric reported in Table 7 multiplied by a constant factor of 0.5, as shown here?
2. Is there any (additional) normalization applied on the point clouds obtained from the Occupancy Networks ShapeNet dataset for training & evaluation respectively? If not, does that mean the point clouds are bounded within a unit cube centered at 0 with a padding of 0.05 on all sides?
3. Is it true that only the first view for every object in the dataset is used for evaluation, as done in Occupancy Networks?

Thanks!

Hi, I was wondering how to extract one possible sdf array so I can render it and use it for data base search?

Siren made a good demo. https://www.shadertoy.com/view/wtVyWK I wanted to do the same to NeuralPull.

Hi @mabaorui,

Congrats again and sorry for the repeated queries. I am still trying to figure out a few aspects of the method and am thus opening various discussions, I hope that is ok.

I have been experimenting for surface reconstruction using the Pytorch version of the code. As you know : bearprin/neuralpull-pytorch#2

I observe that the results are better with a threshold of 0.005 than 0. Some large chunks of the shapes are sometimes not reconstructed with a threshold of 0.

Does this mean that the fitted SDF has converged to an unsigned distance function?

Thanks a lot, Thibault

Hi @mabaorui,

Congrats again and sorry for the repeated queries. I am still trying to figure out a few aspects of the method and am thus opening various discussions, I hope that is ok.

I have been experimenting for surface reconstruction using the Pytorch version of the code. As you know : https://github.com/bearprin/neuralpull-pytorch/issues/2

I observe that the results are not very stable aven after a long time of overfitting a single shape. I can pick a good reconstruction by selecting a step where the fitted mesh is close to the input pointcloud with an ad-hoc metric (like L1 or L2 or Hausdorff), but the next step might be a poor reconstruction.

I wonder if this instability is a bug or a normal behaviour of the method?

Thanks a lot, Thibault

Hi @mabaorui,

Congrats again and sorry for the repeated queries. I am still trying to figure out a few aspects of the method and am thus opening various discussions, I hope that is ok.

I wonder if you use higher-order gradients to backprop in equation 1 of the main paper. I noticed that the pytorch version does not use higher order gradients. And I have hardly ever used Tensorflow in my life, so it is hard for me to understand the official code.

Beside the answer, I wonder what are the pros and cons of using higher-order gradients in this equation, and how you made this design decision,

Thanks a lot, Thibault

Hi @mabaorui, thanks for your code. I'm interested in the single-view reconstruction part and I want to train it on new data. But before that I want to reproduce it.

I noticed that the image features are pre-extracted by a pre-trained model from SoftRas as you mentioned in the issue #4. But they released three models with silhouettes supervision, with shading supervision and with color supervision. Which one was used here? Or could you provide the image feature data here if possible? Thank you again.