For example here

https://github.com/krrish94/nerf-pytorch/blob/a14357da6cada433d28bf11a45c7bcaace76c06e/train_nerf.py#L117-L123

The hidden_size in the config is never applied... which means it will always be set to the default value 128 no matter what you set in the config file... I suspect that there are still many places where the config is not fully applied, which means your implementation might differ a lot from the original repo.

I was first astonished at the speed you mentioned in the readme, but now it seems that your model is not the same as the original repo... yours is tinier (hidden_size=128 in contrast to original 256), so no wonder your speed is faster!

Could you please carefully check again, or mention in the readme that you use a smaller model? Otherwise it is misleading.

The code is thoroughly tested (to the best of my abilities) to match the original implementation (and be much faster)! In particular, I have ensured that

Every individual module exactly (numerically) matches that of the TensorFlow implementation. This Colab notebook has all the tests, matching op for op (but is very scratchy to look at)! Training works as expected (for Lego and LLFF scenes).

Hey,

Thank you for the awesome work on this implementation.

I have been playing around with it quite a bit. I however can't get any results close to that of Table 4, found in the supplementary material of the paper. For instance, they report a PSNR of ~30 on the Lego scene of their synthetic dataset.

In my experiments, either using the pre-trained model of yours or my own trained from scratch, I get a PSNR of only ~20.

Hi, Whenever I try to train I am getting 'Killed' as an output. This is the same even when trying to cache rays from the output. Can I have some suggestions for resolving this problem? Thanks in advance.

I followed the instructions in the README. After installing, I ran the command

python train_nerf.py --config config/lego.yml

and got the following error

Traceback (most recent call last):
  File "train_nerf.py", line 404, in <module>
    main()
  File "train_nerf.py", line 59, in main
    testskip=cfg.dataset.testskip,
  File "D:\dev\nerf\nerf\load_blender.py", line 44, in load_blender_data
    with open(os.path.join(basedir, f"transforms_{s}.json"), "r") as fp:
FileNotFoundError: [Errno 2] No such file or directory: 'cache/nerf_synthetic/lego\\transforms_train.json'

This is on Windows with Python 3.7.

First, great work and thanks for making the effort to port this work to pytorch. In the readme you mention a requirements.txt which is used for installing dependencies, but it looks like it is missing in the repository.

What is the point of this line of code? dists = dists * ray_directions[..., None, :].norm(p=2, dim=-1) on line 24 in volume_rendering_utils.py ? The paper only mentions the dists but not multiplying it with the ray directions.

I don't know whether you are using the NERF author's released blender data. I have not find the 'transforms_train.json' in that data. Could you help with this?

Hello and thank you for great work!

I’m not sure requrements.txt is fully up to date, so please accept created with care environment.yml. To create an environment simply cd nerf-pytorch and run conda env create. To activate an environment you run conda activate nerf and you’re good to run training! There is no need to go to the torchsearchsorted repo and follow the readme, it’s installed automatically with proper GPU support (cudatoolkit-dev package contains nvcc and sets CUDA_HOME). If you’d like I also can update the README.

Hi, Lately I have been working on nerf and found your repo really helpful. Is there a way to convert the model to Open Neural Network Exchange (.onnx) format?

Hi, nice work with NeRF, but I have a question: I try to run the example on https://colab.research.google.com/drive/1L6QExI2lw5xhJ-MLlIwpbgf7rxW7fcz3, but even with 92.450 iter the result is close to 0.17 to Val Loss and 7.6 to PSNR without images for coarse and fine. So, why did this happen? or did I lose something? Adjunct an image with this.

Another thing I did was uncommenting ReplicateNeRFModel, because another error appeared with this;

Only with this action did the example run

model_coarse = ReplicateNeRFModel(
    hidden_size=128,
    num_encoding_fn_xyz=num_encoding_fn_xyz,
    num_encoding_fn_dir=num_encoding_fn_dir,
    include_input_xyz=include_input_xyz,
    include_input_dir=include_input_dir
)
# model_coarse = VeryTinyNeRFModel()
model_coarse.to(device)

# Initialize a fine-resolution model, if specified.
model_fine = ReplicateNeRFModel(
    hidden_size=128,
    num_encoding_fn_xyz=num_encoding_fn_xyz,
    num_encoding_fn_dir=num_encoding_fn_dir,
    include_input_xyz=include_input_xyz,
    include_input_dir=include_input_dir
)
# model_fine = VeryTinyNeRFModel()
model_fine.to(device)

Thx

It looks like when evaluating the loss in the last cell of the colab, the code uses loss = torch.nn.functional.mse_loss(rgb_predicted, target_img)

But believe this should actually be loss = torch.nn.functional.mse_loss(rgb_predicted, testimg)

Since we use testpose for rgb_predicted we should compare to testimg not target_img, which is just a random training sample.

thanks for your great work! I have question about getting ray_directions. can you give me some sepcific introduction? ` ii, jj = meshgrid_xy( torch.arange( width, dtype=tform_cam2world.dtype, device=tform_cam2world.device ).to(tform_cam2world), torch.arange( height, dtype=tform_cam2world.dtype, device=tform_cam2world.device ), )

directions = torch.stack(
    [
        (ii - width * 0.5) / focal_length,
        -(jj - height * 0.5) / focal_length,
        -torch.ones_like(ii),
    ],
    dim=-1,
)
ray_directions = torch.sum(
    directions[..., None, :] * tform_cam2world[:3, :3], dim=-1
)`

https://github.com/krrish94/nerf-pytorch/blob/a14357da6cada433d28bf11a45c7bcaace76c06e/cache_dataset.py#L70

The caching code does loop num-variations times to sample rays from the source images, but it saves the current ray batch into the same file from within the loop, overwriting the file each time. So in the end the cache of each source file only contains the last variation.