Thanks a lot for your great work!

I noticed the resolution (fixed_resolution) for tiny MLPs defined in cfgs/paper/distill/$SEQ.yaml is not the same across scenes: for lego, [9,16,10], for hotdog: [16,16, 6].

So, I wonder how did you determine the resolution for each scene.

Here are my naive approaches: a. just optimize for each scene using grid search b. define the size for a voxel in each scene, then derive the resolution based on global_{max, min} and voxel size

• e.g. ceil ((global_max - global_min) / voxel size)

I would appreciate if you have any suggestions.

Hi! First of all, I would like to thank you for your incredible work speeding up NeRF.

I'm trying to train a new model, but I'm struggling with the configuration. I've read the paper but I can't seem to understand what 'resolution' on cfg files stands for, so I'm not sure which values I should have there for my model. If you could guide me a little with this I would be immensely grateful.

Thank you.

Hi Christian @creiser !

Thank you a lot for the great work and the open-sourced code! It is very helpful in the NeRF community!

I noticed that the number of layers and hidden dims seems to be fixed in the CUDA implementation, regardless of the number of layers and hidden dim we set in the .yaml file.

If I understand it correctly:

• this part fixed the hidden dims to be 32
• this part fixed the number of layers to be 5.

I was wondering:

• The reason why you fixed them here (for less pre-allocated memory?)
• Do you have any plan to extend it to support an arbitrary number of layers and hidden dim?

Thank you!

Really great work!

I have a question about current LPIPS implementation. From the lpips document, the input RGB images should be in scale [-1,+1]. But it seems that the current implementation feed RGB in [0, 1] to lpips.

I have also traced the baseline NSVF's lpips evaluation and they scale the RGB from [0, 1] to [-1, 1] (see their code).

I would appreciate clarification about this. Thanks!

Hi! Firstly, I would like to thank you for work on KiloNeRF and the benefits it provides over basic NeRFs.

I added another output layer to the base NeRF and now the local_distill.py is always killed during its execution. My system reports that the RAM is running out of memory. Did you experience anything similar and know how to fix that? Is maybe the network that has to be distilled too big?

Hello, first of all thanks for your very interesting work.

I'm trying to train a very simple custom scene, but the PSNR remains low even after many iterations on the Vanilla NeRF. The Lego data set has a mean PSNR of 16 on the test set after only 500 iterations, while mine has a mean PSNR of 12 after 10k iterations. I generated my dataset with BlenderProc using the same poses of the Lego dataset, and I checked that the intrinsics, bbox and poses are taken correctly.

These are two images of the train set:

And these are the results after 10k iterations on the test set:

And if I try to render images of the train set the results are slightly better, but still not acceptable.

I can see during the training that the Loss is decreasing and the PSNR is increasing but not as fast as it did with the Lego data set, and they are oscillating a lot.

Do you have any suggestion to fix this behavior? Thanks a lot!

Hi, thanks for opening the source code!

I'm just curious about the acceleration speed of the cuda implementation compared to pytorch.bmm operation, if the input to each MLPs are equal. The test() part code in multi_module.py cannot run successfully due to some flags, and I have no idea how to measure the speed of the cuda implementation against bmm. Could you please give me some guidance? Thanks!

Hi! I met this CUDA error while running render_to_screen.sh: CUDA error at /home/chris/anti/cuda/render_to_screen.cpp:113 code=999(cudaErrorUnknown) "cudaGraphicsGLRegisterBuffer(&cuda_pbo_resource, pbo, cudaGraphicsMapFlagsWriteDiscard)" render_to_screen.sh: line 3: 28062 Segmentation fault (core dumped) python run_nerf.py cfgs/paper/finetune/$DATASET.yaml -rcfg cfgs/render/render_to_screen.yaml

I'm running kilonerf on Ubuntu18.04, CUDA11.1, GPU is A6000. Could you please help me with this? Thank you very much!

Here's the output:

(kilonerf) nesc525@nesc525:~/drivers/5/kilonerf$ bash render_to_screen.sh
auto log path: logs/paper/finetune/Synthetic_NeRF_Lego
{'checkpoint_interval': 50000, 'chunk_size': 40000, 'distilled_cfg_path': 'cfgs/paper/distill/Synthetic_NeRF_Lego.yaml', 'distilled_checkpoint_path': 'logs/paper/distill/Synthetic_NeRF_Lego/checkpoint.pth', 'initial_learning_rate': 0.001, 'iterations': 1000000, 'l2_regularization_lambda': 1e-06, 'learing_rate_decay_rate': 500, 'no_batching': True, 'num_rays_per_batch': 8192, 'num_samples_per_ray': 384, 'occupancy_cfg_path': 'cfgs/paper/pretrain_occupancy/Synthetic_NeRF_Lego.yaml', 'occupancy_log_path': 'logs/paper/pretrain_occupancy/Synthetic_NeRF_Lego/occupancy.pth', 'perturb': 1.0, 'precrop_fraction': 0.5, 'precrop_iterations': 0, 'raw_noise_std': 0.0, 'render_only': False, 'no_color_sigmoid': False, 'render_test': True, 'render_factor': 0, 'testskip': 8, 'deepvoxels_shape': 'greek', 'blender_white_background': True, 'blender_half_res': False, 'llff_factor': 8, 'llff_no_ndc': False, 'llff_lindisp': False, 'llff_spherify': False, 'llff_hold': False, 'print_interval': 100, 'render_testset_interval': 10000, 'render_video_interval': 100000000, 'network_chunk_size': 65536, 'rng_seed': 0, 'use_same_initialization_for_all_networks': False, 'use_initialization_fix': False, 'num_importance_samples_per_ray': 0, 'model_type': 'multi_network', 'random_direction_probability': -1, 'von_mises_kappa': -1, 'view_dependent_dropout_probability': -1}
Using GPU: RTX A6000
/home/nesc525/drivers/5/kilonerf/utils.py:254: VisibleDeprecationWarning: Creating an ndarray from ragged nested sequences (which is a list-or-tuple of lists-or-tuples-or ndarrays with different lengths or shapes) is deprecated. If you meant to do this, you must specify 'dtype=object' when creating the ndarray.
  return np.array([[float(w) for w in line.strip().split()] for line in open(path)]).astype(np.float32)
Loaded a NSVF-style dataset (138, 800, 800, 4) (138, 4, 4) (0,) data/nsvf/Synthetic_NeRF/Lego
(100,) (13,) (25,)
Converting alpha to white.
global_domain_min: [-0.67 -1.2  -0.37], global_domain_max: [0.67 1.2  1.03], near: 2.0, far: 6.0, background_color: tensor([1., 1., 1.])
Loading logs/paper/finetune/Synthetic_NeRF_Lego/checkpoint_1000000.pth
Loading occupancy grid from logs/paper/pretrain_occupancy/Synthetic_NeRF_Lego/occupancy.pth
CUDA error at /home/chris/anti/cuda/render_to_screen.cpp:113 code=999(cudaErrorUnknown) "cudaGraphicsGLRegisterBuffer(&cuda_pbo_resource, pbo, cudaGraphicsMapFlagsWriteDiscard)" 
render_to_screen.sh: line 3: 28062 Segmentation fault      (core dumped) python run_nerf.py cfgs/paper/finetune/$DATASET.yaml -rcfg cfgs/render/render_to_screen.yaml

Thanks a lot for your great work.

When debugging distillation, I found that sometimes alpha from student NN takes the negative value (but very close to 0). This comes from using different activation for density in teacher NN and student NN:

• teacher NN: Relu (https://github.com/creiser/kilonerf/blob/master/local_distill.py#L335)
• student NN: leakey Relu (https://github.com/creiser/kilonerf/blob/master/local_distill.py#L114)

I guess the difference may be minor, but is there any reason for using leaky Relu for student NN?

Thank you for your contribution.

I want to train the KiloNeRF for LLFF dataset.

How do I set the best configuration for the best training in LLFF dataset? Such as follows: Resolution, ,Global_domain_min, Global_domain_max, fixed_resolution