Hello! Sorry if this is the wrong place to post this question. In mipnerf, during inference, LLFF scenes have their near and far distances set as 0 and 1 with the use of NDC. However in mipnerf360, during training, without the usage of NDC, I assume near and far distances are used directly from the COLMAP calculations. If so, how do you set these values during inference? Has it got something to do with the contract(.) operator? Or perhaps I am approaching this wrongly?

Hello. First of all, thank you for sharing this great work.

I would like to kindly ask you to elaborate more on how did you derive the formulas for IPE.

Why do you concatenate it this way?

How did you get the formula for the y_var?

I cannot quiet get it from the article. Thank you :)

Positional encoding (PE) or IPE is required to allow NeRF/MipNeRF learn high frequencies. I understand that volume density and color as a function of 3d point can have discontinuities, especially at object boundaries. However, for a given 3d position, the color as a function of viewing direction is a smooth function, right? I can't imagine a scenario where it may have high frequencies. So, why do we need PE for viewing direction?

@jonbarron, I've pondered over this question for quite some time now, but didn't understand. Any intuition will be helpful.

The radii computation code is different for non-ndc and ndc spaces. In particular, without ndc, radii computation uses only dx derived from directions, but when ndc is enabled, (dx+dy)/2 is used, which is derived from ndc_origins. Can you please shed some light on why it is done so?

Hi, when I am trying to reproduce the result, I installed on cuda10.1 py3.6 tensorflow2.3.1, keras2.7.0

Then I met the issue when I import tensorflow, and from keras import optimizers,

"another metric with the same name already exists".

I would like to know what's the version you use during your experiments?

Thanks in advance!

hi,thanks for your exciting work.I have one question. In the code,the initialization values of directions are (x,y,1),so i think the values of origins should be (x,y,0).So,the values of directions are Xdirections=R*Xc+t,the values of origins are Xdirections-R[0:3,2].

Hi,

While sampling points along rays, the code uses rays.directions for the direction vectors instead of rays.viewdirs. https://github.com/google/mipnerf/blob/84c969e0a623edd183b75693aed72a7e7c22902d/internal/models.py#L70-L81

Original NeRF uses normalized direction vectors for the sampling points. Can you clarify if we need to replace rays.directions with rays.viewdirs?

I use the following command to train on multiscale datasets, but get "killed" output.

bash ./scripts/train_multiblender.sh

I have generated multiscale datesets, and changed the correct path in ./scripts/train_multiblender.sh. It works well on ./scripts/train_blender.sh with original datasets.

My computer has 16G MEM and 4G SWAP and I'd like to know the minimum requirements.

Thanks.

After the sigmoid activation I noticed that you are doing this

rgb = rgb * (1 + 2 * self.rgb_padding) - self.rgb_padding

where rgb_padding = 0.001 by default.

Is this intentional or did you mean to move the range to (rgb_padding, 1 - rgb_padding)?

If we estimate the poses of the lego dataset using colmap, will it work? I tried doing the same, and tried training it using train_llff.sh but it showed inconsistent results.

Hi! Thanks for sharing the great work!

I was just going through the paper and might find a tiny typo. So I am posting an issue in case you did not already notice. In the 1st section of supplemental material, I guess it should be (sinθ)^2 here instead of sinθ.

AS you said: in https://github.com/google/mipnerf/blob/84c969e0a623edd183b75693aed72a7e7c22902d/internal/datasets.py#L193

BUT you use directions (not-norm) rather than viewdirs （unit-norm）: https://github.com/google/mipnerf/blob/84c969e0a623edd183b75693aed72a7e7c22902d/internal/datasets.py#L187-L195 This is different,right? https://github.com/google/mipnerf/blob/84c969e0a623edd183b75693aed72a7e7c22902d/internal/datasets.py#L194-L200 If you use directions (not-norm) ,dx (or radii) will be the same for different pixel rays in the image . If you use viewdirs （unit-norm）, dx (or radii) will be smaller for pixel rays away from image center, and bigger for pixel rays around image center.

`

Distance from each unit-norm direction vector to its x-axis neighbor.

dx = [
    np.sqrt(np.sum((v[:-1, :, :] - v[1:, :, :])**2, -1)) for v in directions
]

dx = [np.concatenate([v, v[-2:-1, :]], 0) for v in dx]

# Cut the distance in half, and then round it out so that it's
# halfway between inscribed by / circumscribed about the pixel.

radii = [v[..., None] * 2 / np.sqrt(12) for v in dx]`

https://github.com/google/mipnerf/blob/84c969e0a623edd183b75693aed72a7e7c22902d/internal/mip.py#L183

Hello author! I recently tried to reproduce the mip-nerf by myself, and found a doubt about the IPE part.

In NeRF it is coded in the order of [(sinx,cosx),...], while Mip-NeRF seems to put the sinx-related ones together, followed by the cosx-related ones,[(sinx , ...),(cosx , ...)], as the following equation shows:

So I'm curious, have you tried coding in the same order as in NeRF, or does the current coding layout work better? Thank you so much! I just noticed this while writing the code, so I wanted to ask for some advice

Hi,

I ran the command

bash scripts/train_blender.sh

and the terminal indicated the following error:

scripts/train_blender.sh: line 31: 11426 Segmentation fault (core dumped) python -m train --data_dir=$DATA_DIR --train_dir=$TRAIN_DIR --gin_file=configs/blender.gin --logtostderr

Could you tell me how to address it? Thanks

hi,thanks for your exciting work.I have two question about avg matrics.

first，I'm confused about the meaning of avg matric，will there be more advantages？

second，I find that when you compute MSE from PSNR，the implementation ofthe code in is different from that in the paper. so I'm a bit confunsed. Can you help me ?