Hi! Thank you very much for your excellent work!

I use the provided script python run.py --config experiments/train_celeba.yml --gpu 0 --num_workers 4 to train a model for the synface dataset.

And then I use python run.py --config experiments/test_celeba.yml --gpu 0 --num_workers 4 to test the model.

Finally, I got 0.0092±0.002 SIDE and 17.77±1.92 MAD, which is not good as in the paper(0.793 ±0.140 and 16.51 ±1.56 MAD in Table 2).

May I have a problem with my operation?

Thank you!

Hello, thank you very much for being open source. I have a problem that I only see about face and cat in your code, but your paper has about car. So I want to ask you how to train cars? Thank you very much! @elliottwu

Hi, wu!

I‘m sorry to disturb you again. Inspired from your repo, I just replace the neural renderer with pytorch3d. Specially, I choose MeshRenderer in pytorch3d.

I first eliminated the inconsistency between NMR and PyTorch3D coordinate systems by overriding PerspectiveCameras class in pytorch3d. However, I found that the noise for shape is so big that the network can not learn a reasonable face shape. Note that I have trained the model with 100 epochs. The figure below is a comparison between NMR and PyTorch3D.

NMR:

PyTorch3D:

RasterizationSettings is showed as below:

     # hard rasterization
     raster_settings = RasterizationSettings(
            image_size=self.image_size,
            blur_radius=0.0,
            faces_per_pixel=1,
            bin_size=0
        )

I noticed that you mentioned one can add a smoothing loss to the depth map for alleviating the noisy depth problem in #9. So can you provide the specific form of such a smoothing loss? I really don't have a clue about this.

Thank you very much!

Hi! Thank you very much for such a great work!

I'm trying to train a network in the size of 512×512. However, the time for a single forward propagation (batch size = 8) is too long (~60.12s). Furthermore, I find that the time is mainly spent on rendering the reconstructed depth map (~59.98s). I want to know what causes the rendering to be so slow.

Again thank you for your work and code, looking forward to your reply :>.

Hello, Thank you for sharing your amazing work. We are students trying to replicate this on the car dataset. Unfortunately, it seems like there are some permission issues to download the data.

curl -o syncar.zip "https://www.robots.ox.ac.uk/~vgg/research/unsup3d/data/syncar.zip"

does not work because of a 403 error ( Permission denied ). Would it be possible to grant access for the same as the others seems to be working fine ?

Hi, when I tried to test unsup3d on Google Colab, I had some problems, such as:

/usr/local/bin/python: Error while finding module specification for 'demo.demo' (AttributeError: module 'demo' has no attribute 'path')

so, how to write a Colab notebook can successfully test unsup3d? just like: https://blog.csdn.net/yrwang_xd/article/details/103150691

Thanks!

Hi, @elliottwu , Sorry to bother you again, but I have 2 questions about the setting of image size:

1. Will increasing the input image_size improve the reconstruction effect? Since I have another dataset trained on the unsup3d model, but I didn't get satisfactory recon results, so I wonder if increasing the input image_size will heal the problem;

2. I have tried increasing the image_size of the input image, set image_size in data_loader as 128 (2 times as original image_size=64), but I encountered the following error:

RuntimeError: The size of tensor a (128) must match the size of tensor b (4224) at non-singleton dimension 0

After checking, I found the two tensors are canon_normal and canon_light_d.view(-1,1,1,3) in the forward process, a element-wise multiplication will be operated on them, but they are unequal on the first dimension respectively:

torch.Size([128, 128, 128, 3])
torch.Size([4224, 1, 1, 3])

So I wonder if you have encountered this kind of error, and how you solved it. Thank you very much, and looking forward to your response.

Congratulations! I am so interested in your project. When i was running the .sh file to download the pretrained models, i always get the network problem. It is so nice of you if you could send your all pretrained models to my e-mail box. Best wishes to you! Have a nice day! Here is my address : [email protected].

Hi, thx for your work, very impressive!

I got a query about the output form of the mode. To my knowledge, the output (reconstruction) and the canonical view are all 2D images, but with depth value which could be used to reconstruct 3D volumes. Is that right? Or canonical view (reconstruction image) is a 3D volume already?

Again thank you for your work and code, looking forward to your reply :>.

Congratulation to be this best paper! The 3d reconstruction results is really impressive with only single view image used! I'm new to 3d cv task, when I run the demo.py, I'm confused with the "view_after" and "yaw_rotations". I think since the yaw_rotations has rotate the view, why do we need the view_after here

Hi Elliott,

Really nice work!

Do you plan to release the car dataset you use in the paper for further research? Also, how did you choose the light parameters when rendering the images? Thank you very much.

hello, I am trying to predict the depth directly from a face image, however the final depth is always predicted for different camera pose. How can I retain the camera pose and face angle, that I have in my initial input image? Thank you

I used celeba dataset and webface dataset, and then used your code and settings for training. After training the first epoch, the results are as follows: Render image： Source image： But at the second epoch, the rendered image becomes extremely poor. It seems that a large part of the face has been filtered out

Can this algorithm generate 3d model with different num of Vertices and faces? and how to config? As you know, 3d Face models will be used in different usage scenarios. Thanks

Hi, Thanks for the paper. I appreciate the awesome work. We are students trying to re-train the Unsup3D model on front view car images. However, the results of the training were not good. We trained with 100 epochs and the validation loss in final epoch is 0.70303. We are definitely sure that we are missing something.

Can you please guide/advise on tweaking any parameters to construct good 3D model for front-view car images ?

Thanks

@elliottwu I'm using a Windows environment, RTX 3080 when using the install neural_renderer_pytorch , an error is displayed: error: command 'C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v9.2\bin\nvcc.exe' failed with exit status 2 ---------------------------------------- ERROR: Command errored out with exit status 1: 'D:\Anaconda3\envs\unsup\python.exe' -u -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'C:\Users\ADMINI~1\AppData\Local\Temp\pip-install-s57_gk9g\neural-renderer-pytorch_baa3a 7b911c743b6a25d2f949292e84f\setup.py'"'"'; file='"'"'C:\Users\ADMINI~1\AppData\Local\Temp\pip-install-s57_gk9g\neural-renderer-pytorch_baa3a7b911c743b6a25d2f949292e84f\setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(_file _) if os.path.exists(file) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, file, '"'"'exec'"'"'))' install --record 'C:\Users\ADMINI~1\Ap pData\Local\Temp\pip-record-zg4vslmq\install-record.txt' --single-version-externally-managed --compile --install-headers 'D:\Anaconda3\envs\unsup\Include\neural-renderer-pytorch' Check the logs for full command output.

Is the version of the graphics driver too high？