Hi, thank you for the inspiring work and your open source code!When I run the following script, I get an ValueError report the at step II. Joint Optimization in Training, Validation, and Testing:
I. Shape Pre-Training and II. Joint Optimization (training and validation)
'''
scene='hotdog_2163'
gpus='2'
model='nerfactor'
overwrite='True'
proj_root='/lyy/nerfactor'
repo_dir="$proj_root/nerfactor"
viewer_prefix='' # or just use ''
I. Shape Pre-Training
data_root="$proj_root/data/selected/$scene"
if [[ "$scene" == scan* ]]; then
# DTU scenes
imh='256'
else
imh='512'
fi
if [[ "$scene" == pinecone || "$scene" == vasedeck || "$scene" == scan* ]]; then
# Real scenes: NeRF & DTU
near='0.1'; far='2'
else
near='2'; far='6'
fi
if [[ "$scene" == pinecone || "$scene" == vasedeck || "$scene" == scan* ]]; then
# Real scenes: NeRF & DTU
use_nerf_alpha='True'
else
use_nerf_alpha='False'
fi
surf_root="$proj_root/output/surf/$scene"
shape_outdir="$proj_root/output/train/${scene}_shape"
REPO_DIR="$repo_dir" "$repo_dir/nerfactor/trainvali_run.sh" "$gpus" --config='shape.ini' --config_override="data_root=$data_root,imh=$imh,near=$near,far=$far,use_nerf_alpha=$use_nerf_alpha,data_nerf_root=$surf_root,outroot=$shape_outdir,viewer_prefix=$viewer_prefix,overwrite=$overwrite"
II. Joint Optimization (training and validation)
shape_ckpt="$shape_outdir/lr1e-2/checkpoints/ckpt-2"
brdf_ckpt="$proj_root/output/train/merl/lr1e-2/checkpoints/ckpt-50"
if [[ "$scene" == pinecone || "$scene" == vasedeck || "$scene" == scan* ]]; then
# Real scenes: NeRF & DTU
xyz_jitter_std=0.001
else
xyz_jitter_std=0.01
fi
test_envmap_dir="$proj_root/data/envmaps/for-render_h16/test"
shape_mode='finetune'
outroot="$proj_root/output/train/${scene}_$model"
REPO_DIR="$repo_dir" "$repo_dir/nerfactor/trainvali_run.sh" "$gpus" --config="$model.ini" --config_override="data_root=$data_root,imh=$imh,near=$near,far=$far,use_nerf_alpha=$use_nerf_alpha,data_nerf_root=$surf_root,shape_model_ckpt=$shape_ckpt,brdf_model_ckpt=$brdf_ckpt,xyz_jitter_std=$xyz_jitter_std,test_envmap_dir=$test_envmap_dir,shape_mode=$shape_mode,outroot=$outroot,viewer_prefix=$viewer_prefix,overwrite=$overwrite"
III. Simultaneous Relighting and View Synthesis (testing)
ckpt="$outroot/lr5e-3/checkpoints/ckpt-10"
if [[ "$scene" == pinecone || "$scene" == vasedeck || "$scene" == scan* ]]; then
# Real scenes: NeRF & DTU
color_correct_albedo='false'
else
color_correct_albedo='true'
fi
REPO_DIR="$repo_dir" "$repo_dir/nerfactor/test_run.sh" "$gpus" --ckpt="$ckpt" --color_correct_albedo="$color_correct_albedo"
'''
[trainvali] For results, see:
/lyy/nerfactor/output/train/hotdog_2163_nerfactor/lr5e-3
[datasets/nerf_shape] Number of 'train' views: 100
[datasets/nerf_shape] Number of 'vali' views: 8
[models/base] Trainable layers registered:
['net_normal_mlp_layer0', 'net_normal_mlp_layer1', 'net_normal_mlp_layer2', 'net_normal_mlp_layer3', 'net_normal_out_layer0', 'net_lvis_mlp_layer0', 'net_lvis_mlp_layer1', 'net_lvis_mlp_layer2', 'net_lvis_mlp_layer3', 'net_lvis_out_layer0']
[models/base] Trainable layers registered:
['net_brdf_mlp_layer0', 'net_brdf_mlp_layer1', 'net_brdf_mlp_layer2', 'net_brdf_mlp_layer3', 'net_brdf_out_layer0']
Traceback (most recent call last):
File "/lyy/nerfactor/nerfactor/nerfactor/trainvali.py", line 341, in
app.run(main)
File "/home/ly/anaconda3/envs/nerfactor/lib/python3.6/site-packages/absl/app.py", line 308, in run
_run_main(main, args)
File "/home/ly/anaconda3/envs/nerfactor/lib/python3.6/site-packages/absl/app.py", line 254, in _run_main
sys.exit(main(argv))
File "/lyy/nerfactor/nerfactor/nerfactor/trainvali.py", line 106, in main
model = Model(config, debug=FLAGS.debug)
File "/lyy/nerfactor/nerfactor/nerfactor/models/nerfactor.py", line 68, in init
ioutil.restore_model(self.brdf_model, brdf_ckpt)
File "/lyy/nerfactor/nerfactor/nerfactor/util/io.py", line 48, in restore_model
ckpt.restore(ckpt_path).expect_partial()
File "/home/ly/anaconda3/envs/nerfactor/lib/python3.6/site-packages/tensorflow/python/training/tracking/util.py", line 2009, in restore
status = self._saver.restore(save_path=save_path)
File "/home/ly/anaconda3/envs/nerfactor/lib/python3.6/site-packages/tensorflow/python/training/tracking/util.py", line 1304, in restore
checkpoint=checkpoint, proto_id=0).restore(self._graph_view.root)
File "/home/ly/anaconda3/envs/nerfactor/lib/python3.6/site-packages/tensorflow/python/training/tracking/base.py", line 209, in restore
restore_ops = trackable._restore_from_checkpoint_position(self) # pylint: disable=protected-access
File "/home/ly/anaconda3/envs/nerfactor/lib/python3.6/site-packages/tensorflow/python/training/tracking/base.py", line 907, in _restore_from_checkpoint_position
tensor_saveables, python_saveables))
File "/home/ly/anaconda3/envs/nerfactor/lib/python3.6/site-packages/tensorflow/python/training/tracking/util.py", line 289, in restore_saveables
validated_saveables).restore(self.save_path_tensor)
File "/home/ly/anaconda3/envs/nerfactor/lib/python3.6/site-packages/tensorflow/python/training/saving/functional_saver.py", line 281, in restore
restore_ops.update(saver.restore(file_prefix))
File "/home/ly/anaconda3/envs/nerfactor/lib/python3.6/site-packages/tensorflow/python/training/saving/functional_saver.py", line 103, in restore
restored_tensors, restored_shapes=None)
File "/home/ly/anaconda3/envs/nerfactor/lib/python3.6/site-packages/tensorflow/python/distribute/values.py", line 647, in restore
for v in self._mirrored_variable.values))
File "/home/ly/anaconda3/envs/nerfactor/lib/python3.6/site-packages/tensorflow/python/distribute/values.py", line 647, in
for v in self._mirrored_variable.values))
File "/home/ly/anaconda3/envs/nerfactor/lib/python3.6/site-packages/tensorflow/python/distribute/values.py", line 392, in _assign_on_device
return variable.assign(tensor)
File "/home/ly/anaconda3/envs/nerfactor/lib/python3.6/site-packages/tensorflow/python/ops/resource_variable_ops.py", line 846, in assign
self._shape.assert_is_compatible_with(value_tensor.shape)
File "/home/ly/anaconda3/envs/nerfactor/lib/python3.6/site-packages/tensorflow/python/framework/tensor_shape.py", line 1117, in assert_is_compatible_with
raise ValueError("Shapes %s and %s are incompatible" % (self, other))
ValueError: Shapes (0, 3) and (100, 3) are incompatible
The shape checkpoints are generated by step I. Shape Pre-Training and the BRDF checkpoints are downloaded from your page.
Does it mean i need to pre-train brdf model by myself?
Very much looking forward to your help!
2.8k Feb 12, 2021
33 Jan 4, 2023
195 Dec 29, 2022
2 Dec 14, 2022
1 Oct 27, 2021
3 Dec 27, 2022
6 Nov 22, 2022
4 Nov 8, 2022
116 Jan 3, 2023
22 Jun 14, 2022
68 Dec 14, 2022
161 Dec 20, 2022
296 Dec 6, 2022
167 Jul 6, 2022
9 Jun 10, 2022
172 Dec 22, 2022
892 Dec 28, 2022
13 Dec 23, 2022
71 Dec 30, 2022