I met some errors when I had a try to run this code. Please help me!

I'm running Ubuntu 18.04.5 LTS Python 3.7 Pytorch 1.9 Cuda 11.1

Depth stats 0.19367019493628088 315.65333461743836 61.20548491109312
Done with imgs2poses
Depths prior training begins !
Found cache /home/xxx/code/NerfingMVS/models/depth_priors/mc.pth
/home/xxx/miniconda2/envs/py37/lib/python3.7/site-packages/torchvision/transforms/transforms.py:281: UserWarning: Argument interpolation should be of type InterpolationMode instead of int. Please, use InterpolationMode enum.
  "Argument interpolation should be of type InterpolationMode instead of int. "
point number: 3922949
Traceback (most recent call last):
  File "run.py", line 19, in <module>
    depth_priors.train(args)
  File "/home/xxx/code/NerfingMVS/src/depth_priors.py", line 70, in train
    args.depth_H, args.depth_W)
  File "/home/xxx/code/NerfingMVS/utils/io_utils.py", line 27, in load_colmap
    ply_masks = read_ply_mask(ply_path)
  File "/home/xxx/code/NerfingMVS/utils/colmap_utils.py", line 52, in read_ply_mask
    load_point_vis(ply_vis_path, masks)
  File "/home/xxx/code/NerfingMVS/utils/colmap_utils.py", line 36, in load_point_vis
    idx, u, v = struct.unpack('<III', f.read(4 * 3))
struct.error: unpack requires a buffer of 12 bytes

Thank you!

sorry to bother.but I wanted to try to train my own data.I download the data from the web:https://vision.middlebury.edu/mview/data/ and I found you mentioned in "Run on Your Own Data!" says factor parameter controls the resolution of output depth maps.for example ,the resolution of the data I download is 640*480,so what number should the "factor" be in my configs?by the way ,could you tell me what does the other parameters as follows exactly mean? N_rand = 1024 N_samples = 64 use_viewdirs = True raw_noise_std = 1e0 thank you very much~

when I follow the instructions and do the "sh run.sh scene0000_01" ,it was successful at first but when it comes to the :"Depths prior training begins !",it happens to occur the error as follow: Depths prior training begins ! Dowloading https://storage.googleapis.com/mannequinchallenge-data/checkpoints/best_depth_Ours_Bilinear_inc_3_net_G.pth to /home/ycj/NerfingMVS/models/depth_priors/mc.pth ... 100% [....................................................] 21619796 / 21619796point number: 4696662 Traceback (most recent call last): File "run.py", line 19, in depth_priors.train(args) File "/home/ycj/NerfingMVS/src/depth_priors.py", line 70, in train args.depth_H, args.depth_W) File "/home/ycj/NerfingMVS/utils/io_utils.py", line 27, in load_colmap ply_masks = read_ply_mask(ply_path) File "/home/ycj/NerfingMVS/utils/colmap_utils.py", line 49, in read_ply_mask load_point_vis(ply_vis_path, masks) File "/home/ycj/NerfingMVS/utils/colmap_utils.py", line 35, in load_point_vis masks[idx][v, u] = 1 IndexError: list index out of range

could you tell me the reason?thx

Hi,

I am trying to train NerfingMVS with my own data but I don't understand what is the "depth mask" that is used (folder: raw_data/scene*/depth/). How do I obtain this mask for my own data and is it also essential?

This is not the GT depth map used to train monocular depth network, right? The depth map obtained from MVS of colmap is used as GT, atleast thats what I understood from reading the paper.

Hi, I am trying to reproduce your work. However, I face the issue about install col map. I try to install from official web, conda and https://github.com/B1ueber2y/colmap.git. They all fail to solve the problem. Would you mind to give me some suggestion?

Hi! Thank you for your work! It was said in the paper, that it's possible to render novel views by using adapted depth priors from the nearest view. However, nothing was said about how to deal with missed regions, in case we adapt these predicted priors (I guess, here we need to warp them). I also didn't manage to find any code doing that. Could you please clarify this point?

I downloaded the results you provided and found the results of NeRF do not match the ones shown in the paper. The NeRF results in the paper seem much worse. Could you please expain what is going on here? Thanks!

Hi, I found you used COLMAP's MVS pipline in your code to gennerate fused Point Cloud and then get the depth map. But you said you use COLMAP's SfM in your paper (the sparse depth in Fig.2 is also from the MVS instead of SfM). The sparse depth from SfM is actually very sparse. If you use COLMAP's MVS to reconsrtuct the point cloud, the depth from point cloud is already accurate (though a little sparse). So it confused me a lot. Why do you use nerf to generate depth? The Tab. 3 shows the depth from DepthNet is very accurate, the Nerf helps a little. I guess if you use a deeper DepthNet and a better loss you can get a better depth.

Hi, Thanks for your great work! I think you can replace the 'torchsearchsorted' module with 'torch.searchsorted()'. When I use NVIDIA GeForce RTX 2080 (same with you) to intall 'torchsearchsorted', I met some 'unsupported gpu architecture'. Maybe you can share how to install 'torchsearchsorted' on 2080 gpu with me. This is a nice work. Thanks!

In the paper: tn = D(1 − max(e, αl)), tf = D(1 + min(e, αh)) In the code:

near = (depth_priors * (1 - torch.clamp(depth_confidences, min=near_bound, max=far_bound))).unsqueeze(1)
far = (depth_priors * (1 + torch.clamp(depth_confidences, min=near_bound, max=far_bound))).unsqueeze(1))

Why do you clamp the confidence in the code?

I'm wondering if there's a option or script in your code to generate the 3D mesh of the trained scene? There's such option in the nerf-pytorch, but your code seems to be based on a older version of nerf-pytorch. And it will be great if you can also explain how to generate images from a specific view, like from an unseen direction. Many thanks

Hi, i have tried to training your model on own dataset (59 images in total, outdoor scenarios). The trained depth priors seem proper: However, the NeRF cannot be trained at all. Its loss is not decreased and the PSNR is nearly unchanged. And finally, the trained NeRF just renders totally black images for both depth and rgb: The experimental setups are the same as the ones in the default configuration. Could you give some advices on this problem? Thanks at advance :)

Hi, thanks for your remarkable work! After training a model, I want to visualize the voxel in the 3D space. Could you please give me some advice or related visualization codes? Many thanks.

Hi, Thank you for your work. Everything seems to work well except if I use my own data. This error happens:

Traceback (most recent call last): File "run.py", line 19, in depth_priors.train(args) File "/mnt/work/Dev/Sources/mesh_refinement/NerfingMVS-main/src/depth_priors.py", line 86, in train loss = compute_depth_loss(depth_pred, depth_gt, mask_gt) File "/mnt/work/Dev/Sources/mesh_refinement/NerfingMVS-main/utils/depth_priors_utils.py", line 6, in compute_depth_loss log_pred = torch.log(pred[mask]) IndexError: too many indices for tensor of dimension 1

not sure what's happening. Do you have any idea?

thank you

Hi, In NerfingMVS, inappropriate dependency versioning constraints can cause risks.

Below are the dependencies and version constraints that the project is using

lpips==0.1.3
imageio
opencv-python
scikit-image==0.15.0
tqdm
wget
configargparse
tensorboard
h5py

The version constraint == will introduce the risk of dependency conflicts because the scope of dependencies is too strict. The version constraint No Upper Bound and * will introduce the risk of the missing API Error because the latest version of the dependencies may remove some APIs.

After further analysis, in this project, The version constraint of dependency imageio can be changed to >=1.1-linux32,<=2.19.3. The version constraint of dependency tqdm can be changed to >=4.36.0,<=4.64.0. The version constraint of dependency configargparse can be changed to >=0.9.3,<=1.5.3. The version constraint of dependency h5py can be changed to >=2.5.0,<=3.7.0.

The above modification suggestions can reduce the dependency conflicts as much as possible, and introduce the latest version as much as possible without calling Error in the projects.

The invocation of the current project includes all the following methods.

imageio.imwrite
imageio.imread

tqdm.tqdm.write
tqdm.tqdm
tqdm.trange

configargparse.ArgumentParser

h5py.File.close
h5py.File

bistochastize
targets.cuda
data_term.item
numpy.genfromtxt
self.blurs.append
hourglass.HourglassModel
self.slice
self.GradientLoss
numpy.ndarray.max
pix_coords.permute.to
mask.sum
numpy.linalg.norm
Image
torch.cat
rgb.cpu.numpy
pred_confidence.data.unsqueeze
loss_list.torch.stack.mean
img_grad_y.abs
numpy.isfinite
scipy.sparse.diags
file.write
poses.np.moveaxis.astype
i.input_depth.data.cpu.numpy
self.S.dot
k.ret.torch.isinf.any
torch.clamp
super.__init__
i.d_gt.cpu
get_valid_idx
torch.Tensor.expand
h5py.File.create_dataset
torch.rand
sc_inv_full.item
i.prediction_d.data.cpu.numpy
torch.nn.Identity
i.pred_d.cpu.numpy
poses.torch.from_numpy.to.clone
list
numpy.array
torch.Tensor
torch.ones_like
eo.embed
i.d_gt.cpu.numpy
self.list.append
torch.pow
lpips_list.append
masks.torch.from_numpy.to
argparse.ArgumentParser
rays_o.torch.reshape.float.clone
numpy.argsort
self.netG.parameters
img2mse
read_cameras_text
render
i.full_flow.data.cpu.numpy
zip
time.time
torch.nn.DataParallel.load_state_dict
self.register_buffer
models.depth_priors.mannequin_challenge_model.MannequinChallengeModel.train
subprocess.check_output
torch.nn.init.xavier_normal_
minify
self.blur
input.view
models.depth_priors.mannequin_challenge_model.MannequinChallengeModel
h5py.File
skimage.io.imsave
lpips_metric.cpu
load_depths
k.all_ret.append
ret.append
rays_d.torch.reshape.float.clone
bds.np.moveaxis.astype.max
read_images_text
rgb_evaluation
rays_rgb.torch.Tensor.to.astype
network.cpu
rays_rgb.torch.Tensor.to
numpy.eye
disp.cpu
sigma_spatial.Iy.astype
prediction_d.data.cpu
render_path_spiral
torch.optim.lr_scheduler.StepLR
cv2.imread
img_grad_x.abs.mean
invK.torch.unsqueeze.repeat
lpips_list.np.concatenate.mean
torch.cumprod
net_.parameters
numpy.flatnonzero
numpy.mean
visualize_depth
X.copy
numpy.array.reshape
rays_o.torch.reshape.float
torch.autograd.set_detect_anomaly
Channels4
self.seq
Z.reshape.astype
numpy.expand_dims
self.Channels3.super.__init__
trans.append
f.read
scipy.sparse.csr_matrix.dot
torch.median
input_.cuda
numpy.array.append
compute_depth_loss.backward
numpy.matmul
numpy.arange
p_fn
vars.items
tqdm.tqdm.write
models.depth_priors.mannequin_challenge_model.MannequinChallengeModel.forward
coord.reshape
load_rgbs_np
self.new_model
i.self.views_linears
i.full_flow.data.cpu
input_images.data.cpu
torch.nn.init.kaiming_normal_
x.squeeze.squeeze
torch.nn.Sequential
i.gt_mask.cpu.numpy
depths.cpu.numpy
prediction_d.data.cpu.torch.exp.unsqueeze
self.loss_joint_var.backward
targets.unsqueeze
f.strip.replace
numpy.argmin
c.t.bs_params.grid.BilateralSolver.solve.reshape
numpy.percentile
models.depth_priors.mannequin_challenge_model.MannequinChallengeModel.eval
self.input.cuda
ssim_list.append
numpy.sqrt
render_rays
numpy.searchsorted
numpy.dstack
depth_grad_yx.abs.mean
img.dim
numpy.tensordot
im.qvec2rotmat
compute_errors
depth_model.model.netG.state_dict
i.input_imgs.cpu.numpy
numpy.load.transpose
numpy.argmax
torch.FloatTensor.unsqueeze
self.backward_G
image_name.replace.split
optimizer_depth.zero_grad
input.size
img_lap.mean.exp.squeeze
inception
depths.torch.from_numpy.to
numpy.shape
main
optimizer_depth.step
float
mse2psnr
self.model.netG.forward
run_network
torchvision.transforms.ToTensor
lpips.LPIPS
batch.depth_model.forward.cpu
numpy.load.astype
torch.sigmoid.cpu
torch.nn.BatchNorm2d
self.create_embedding_fn
f.endswith
predicts.transpose.torch.from_numpy.type
numpy.sum
bds.np.moveaxis.astype
grid.splat
self.rgb_linear
grad_term.item
torch.nn.Sigmoid
numpy.column_stack
torch.matmul.unsqueeze
im.astype
any
W.reshape.astype
perm.append
numpy.linalg.eigh
depth_confidences.torch.reshape.float
numpy.logical_not
tqdm.trange
torch.transpose
numpy.concatenate.append
poses.astype.sum
masks.append
self.criterion_joint
set
src.depth_priors.train
metadata.unsqueeze.cuda
torch.nn.functional.relu
self.writer.add_image
torch.nn.Linear
mannequin_challenge.options.train_options.TrainOptions
LaplacianLayer
i.input_imgs.cpu
numpy.newaxis.pts_arr.transpose
numpy.stack.split
depth.view
self.HourglassModel.super.__init__
struct.unpack
w.reshape.reshape
self.write_summary
numpy.max
self.convs.append
numpy.maximum.mean
self.grid.slice
torch.log
self.laplacian_func
numpy.moveaxis
self.Data_Loss
network.cpu.state_dict
cal_depth_confidences
int
vec_from_R
torch.abs
ndc_rays
numpy.linalg.inv.transpose
skimage.measure.compare_ssim
depth_priors.torch.reshape.float
torch.utils.tensorboard.SummaryWriter
self.criterion_joint.compute_rmse_error
src.initialize.main
torch.optim.Adam.step
torch.stack.cpu
skimage.transform.resize
torch.FloatTensor.gts.transpose.torch.from_numpy.type.cuda
self.model.switch_to_train
depth_grad_yx.abs
image_name.split
gts.astype.max
open
sigma_chroma.im_yuv.astype
valid_z.max
self.LaplacianLayer.super.__init__
W.reshape
embed_fns.append
torch.nn.init.normal_
disp.cpu.numpy
poses.torch.Tensor.to
mse2psnr.item
self.Dn.dot
i.mvs_depth.data.cpu.numpy
solve_image_ldl3
img_name.split
numpy.unique
read_model
torch.inverse
sigma_luma.im_yuv.astype
fid.seek
self.opt.gpu_ids.split
self.Channels4.super.__init__
i.i.cam_points.repeat
pred_confidence.data.unsqueeze.repeat
configargparse.ArgumentParser
Project3D
viewdirs.torch.reshape.float.expand
torch.nn.init.orthogonal_
multiprocessing.Pool
temp.cpu.numpy
self.estimate_depth
cv2.imwrite
images.reshape.cuda
scipy.sparse.linalg.cg
numpy.load.reshape
self.targets.cuda
format
fid.readline
i.targets.split
poses_avg
raw2outputs
torch.stack
torch.unsqueeze
imgs.np.moveaxis.astype.split
exit
EPSILON.mask.data.pred_d.data.mask.data.gt_d.data.torch.median.item
i.pred_confidence.data.cpu
depth_grad_y2.abs
torch.norm
functools.partial
A.diagonal
self.load_network
os.listdir
i_train.len.np.arange.repeat
load_gt_depths
fid.readline.split
i.t.t
src.run_nerf.train
depth_model.model.netG.load_state_dict
i.human_mask.data.cpu.numpy
self.Data_Human_Loss
torch.nn.Sequential.children
meshgrid.np.stack.astype
torch.FloatTensor
self.criterion_joint.compute_si_rmse
os.path.exists
img_lap.mean.exp.view
numpy.random.shuffle
self.uncertainty_layer
fid.read
self.criterion_joint.get_loss_var
self.netG.train
self.optimizer_G.zero_grad
numpy.reshape
rays_d.torch.reshape.float
numpy.logical_or
img_grad_y.abs.mean
input.unsqueeze.unsqueeze
self.model.netG.parameters
init_weights
numpy.cross
predicts.gts.mean.mean.mean
h5py.File.close
zipfile.ZipFile
A_i.A_i.np.transpose.mean
b_i.mean
torch.nn.functional.conv2d.abs
self.netG.eval
Channels1
depth.cpu
bds.np.moveaxis.astype.min
numpy.uint8
torch.randn
compute_depth_loss
img_lap.mean.exp.mean
self.model.netG.state_dict
numpy.linspace
sc_inv_human.item
torch.sum.item
mannequin_challenge.options.train_options.TrainOptions.initialize
numpy.random.rand
_load_data
targets.cuda.autograd.Variable.unsqueeze
gts.astype.transpose
gts.astype.astype
networks.get_scheduler
load_point_vis
self.save_network
network_query_fn
cal_depth_confidences.astype
BilateralSolver
batch.depth_model.forward.cpu.numpy
get_rays_np
multiprocessing.Pool.close
self.splat
torch.autograd.Variable
image_name.replace.replace
prediction_log_d.squeeze
H.focal.W.focal.torch.FloatTensor.to
cv2.resize.astype
img_lap.mean.exp
os.path.join
sc_inv_env.item
recenter_poses
numpy.isnan
self.HourglassVariant.super.__init__
torch.matmul
z_vals.expand.expand
weight_y.depth_grad_y.abs.weight_x.depth_grad_x.abs.mean
read_images_binary.items
pil_loader
torch.randperm
SuppressedStdout
os.path.isfile
self.parser.parse_args
torch.max
i.i.T.repeat
render_path
torch.ones
f.w.h.np.array.reshape
numpy.zeros
images.astype.astype
depth_confidences.append
utils.colmap_read_model.read_cameras_binary
torch.abs.sum
self.scheduler.step
self.feature_linear
local_path.rstrip.rstrip
torchvision.utils.make_grid
tran_dis.sum.sum
line.strip.strip
self.model.prediction_d.squeeze
self.netG.forward
torch.isinf
torch.load
network.cuda
torch.meshgrid
self.grid.splat
wget.download
torch.split
torch.FloatTensor.view.cuda
numpy.ndarray.min
gt_depths_valid.append
os.getcwd
saved_imgs.astype.astype
self.alpha_linear
netchunk.fn.batchify
os.remove
Z.reshape
networks.print_network
k.ret.torch.isnan.any
torch.nn.functional.grid_sample
depth_grad_y2.abs.mean
utils.colmap_read_model.read_points3d_binary
predicts.astype.astype
sh.np.array.reshape
dict
torch.sum
pix_coords.permute.permute
torch.matmul.view
utils.colmap_read_model.read_images_binary
self.inception.super.__init__
numpy.abs
Channels3
torch.autograd.Variable.size
angles.append
i.input_depth.data.cpu
depths.append
self.pred_confidence.squeeze
images.torch.Tensor.to
super
read_images_binary
sorted
f.readlines
print
numpy.sin
numpy.save
depth_grad_x.abs
numpy.transpose
min_line_dist
pred_confidence.squeeze.squeeze
ValueError
Project3D_depth
configargparse.ArgumentParser.add_argument
gradient
mkdir
numpy.log
qvec2rotmat
angle_dis.sum.sum
numpy.linalg.inv
render_poses.np.array.astype
load_rgbs.clone
get_embedder
isinstance
torch.device
poses_tensor.shape.bottom.repeat.to.repeat
sys.stdout.close
bs_params.grid.BilateralSolver.solve
errors.np.array.mean.tolist
self.initialize
numpy.arcsin
im.tvec.reshape
numpy.log10
numpy.meshgrid
depth_grad_y.abs
options.config_parser.parse_args
util.util.mkdirs
numpy.median
self.forward
i.pred_confidence.data.cpu.numpy
numpy.dstack.reshape
self.criterion_joint.compute_l1_rel_error
metadata.unsqueeze
f.writelines
batchify_rays
base_options.BaseOptions.initialize
param.numel
numpy.tile
i.imgs_down.astype
self.JointLoss.super.__init__
torch.from_numpy.view
torch.sqrt
rot.transpose
numpy.square
i.image_list.split
_minify
spherify_poses
prediction_d.data.cpu.torch.exp.unsqueeze.repeat
prediction_d.data.cpu.torch.exp.unsqueeze.repeat.numpy
read_points3D_text
i.mvs_depth.data.cpu
i.input_confidence.data.cpu.numpy
array.np.transpose.squeeze
self.pred_layer
numpy.fromfile
vars
torch.nn.AvgPool2d
input.dim
join
self._compute_factorization
load_llff.load_llff_data
render_poses.np.array.astype.append
i.pred_d.cpu
depth_confidence.mean.cpu.numpy
torch.isnan
self.grid.blur
torch.no_grad
depth_confidence.mean.cpu
Channels2
torch.mean
get_model_from_url
NeRF
numpy.empty_like
f.extractall
torch.nn.DataParallel.to
FixedMcModel
self.optimizer_G.step
torch.FloatTensor.view
create_nerf
poses_arr.reshape.transpose
numpy.ones_like
imgs.np.moveaxis.astype
numpy.stack
sys.path.append
K.torch.unsqueeze.repeat
cv2.applyColorMap
ssim_list.np.array.mean
args.config.open.read
numpy.maximum
networks.JointLoss
numpy.load
sm_term.item
i.prediction_d.data.cpu
sc_inv_intra.item
imread
pred_depths_valid.append
predicts.gts.mean.mean
numpy.ones
self.parser.add_argument
self.laplacian_func.mean
depth_grad_xy.abs
rgbs.append
valid_z.mean
predicts.gts.mean
imageio.imwrite
numpy.clip
errors.np.array.mean
torch.optim.Adam.load_state_dict
downsample
line.strip.split
BackprojectDepth
total_loss.item
max
read_array
i.pred_d.data.cpu
input.unsqueeze
viewdirs.torch.reshape.float
depth_grad_xy.abs.mean
numpy.linalg.lstsq
torch.optim.Adam.zero_grad
images.reshape.reshape
pix_coords_ref.i_train.depths.unsqueeze.F.grid_sample.squeeze
BilateralGrid
fn
torch.nn.UpsamplingBilinear2d
i.poses.ravel
numpy.squeeze
self.S.T.dot
loss.backward
torch.nn.init.constant_
numpy.cos
predicts.astype.transpose
filt
os.chdir
numpy.broadcast_to
p34_to_44
self.compute_image_aware_1st_smoothness_cost
depth_evaluation
far_bound.near_bound.depth_confidences.torch.clamp.depth_priors.unsqueeze
utils.pose_utils.gen_poses
i.self.pts_linears
render_kwargs_test.update
compute_depth_loss.item
numpy.ceil
loss.item
rgb2yuv
self.LaplacianSmoothnessLoss
array.reshape.reshape
Embedder
pred_d.unsqueeze
torch.reshape
sys.exit
i.poses.copy
CameraModel
torch.save
map
PIL.Image.open
embed_fn
torch.abs.topk
hasattr
torch.cuda.is_available
classname.find
src.filter.main
collections.namedtuple
str
read_points3d_binary
self._hash_coords
colmap_depths.shape.np.ones.astype
x.np.clip.astype
depth_confidences.torch.from_numpy.to
torch.FloatTensor.predicts.transpose.torch.from_numpy.type.cuda
args.use_viewdirs.input_ch_views.skips.output_ch.input_ch.args.netwidth.args.netdepth.NeRF.to
self.opt.gpu_ids.append
len
i.pred_d.data.cpu.numpy
read_next_bytes.decode
depth_grad_x2.abs.mean
os.makedirs
Camera
numpy.random.seed
torch.optim.lr_scheduler.ReduceLROnPlateau
read_next_bytes
errors.append
load_img_list
tuple
pix_coords.permute.view
mse.np.log10.mean
f.strip
mask_0.size
tqdm.tqdm
torch.linspace
poses.torch.from_numpy.to
sigma_spatial.Ix.astype
optimizer_depth.state_dict
torch.set_default_tensor_type
batchify
self.prediction_d.squeeze
self.model.switch_to_eval
act_fn
depth.cpu.numpy
torch.utils.tensorboard.SummaryWriter.add_scalar
utils.colmap_read_model.read_cameras_binary.keys
load_rgbs
poses.astype.mean
i_train.depths.unsqueeze
to_tensor
to8b
numpy.sqrt.mean
torch.mul
depth_model
NotImplementedError
lpips_metric
loss_list.append
parser.parser.parse_args
save_poses
models.depth_priors.mannequin_challenge_model.MannequinChallengeModel.parameters
torch.sigmoid
read_ply_mask
create_depth_model
depth_grad_x2.abs
log_pred.log_target.sum
torchvision.transforms.Resize
get_rays
options.config_parser
poses_tensor.shape.bottom.repeat.to
self.NeRF.super.__init__
poses.astype.astype
self.output_linear
render_kwargs_train.update
calculate_coords
align_scales
args.use_viewdirs.input_ch_views.skips.output_ch.input_ch.args.netwidth.args.netdepth.NeRF.to.load_state_dict
numpy.concatenate
normalize
conv
imageio.imread
i.gt_mask.cpu
line.strip
f_list.readlines
self.list
numpy.dot
grid.blur
torch.multiprocessing.set_start_method
load_colmap_data
c2w.expand
center.poses.mean
torch.stack.append
self.targets.cuda.autograd.Variable.unsqueeze
torch.optim.lr_scheduler.LambdaLR
enumerate
planar_filter
i.human_mask.data.cpu
raw2alpha
i.input_confidence.data.cpu
range
embeddirs_fn
torch.exp.squeeze
torch.nn.DataParallel
img.convert
scipy.sparse.csr_matrix
torch.nn.functional.conv2d
args.use_viewdirs.input_ch_views.skips.output_ch.input_ch.args.netwidth.args.netdepth.NeRF.to.parameters
viewmatrix
render_kwargs_train.state_dict
os.path.dirname
img_grad_x.abs
torch.nn.Conv2d
train
src.evaluation.main
Point3D
j.t.t
multiprocessing.Pool.join
time.time.time
self.writer.add_scalar
cv2.resize
read_cameras_binary
cal_neighbor_idx
opt_file.write
getattr
torch.from_numpy
torch.nn.ModuleList
i_test.len.np.arange.repeat
torch.optim.Adam.state_dict
torch.nn.DataParallel.apply
gt_rgbs.pred_rgbs.np.abs.mean
self.Channels1.super.__init__
valid_z.min
torch.optim.Adam
self.model.prediction_d.reshape
gts.transpose.torch.from_numpy.type
depth_priors.torch.from_numpy.to
resize
pred_d.unsqueeze.repeat
sc_inv_inter.item
numpy.stack.append
self.Channels2.super.__init__
torch.nn.ReLU
depth_confidence.mean.cpu.numpy.mean
load_colmap
multiprocessing.Pool.map_async
targets.unsqueeze.repeat
torch.exp

@developer Could please help me check this issue? May I pull a request to fix it? Thank you very much.

Why would this problem happen? I used your colmap version(without git clone or checkout): cmake & make. Then it's wring during the process of making

[ 50%] Building CXX object src/CMakeFiles/colmap.dir/mvs/meshing.cc.o [ 50%] Building CXX object src/CMakeFiles/colmap.dir/optim/bundle_adjustment.cc.o [ 50%] Building CXX object src/CMakeFiles/colmap.dir/ui/undistortion_widget.cc.o /home/lichao/Documents/NerfingMVS/colmap/src/estimators/pose.cc: In function ‘bool colmap::RefineAbsolutePose(const colmap::AbsolutePoseRefinementOptions&, const std::vector&, const std::vector<Eigen::Matrix<double, 2, 1>, std::allocator<Eigen::Matrix<double, 2, 1> > >&, const std::vector<Eigen::Matrix<double, 3, 1> >&, Eigen::Vector4d*, Eigen::Vector3d*, colmap::Camera*)’: /home/lichao/Documents/NerfingMVS/colmap/src/estimators/pose.cc:242:20: error: ‘QuaternionParameterization’ in namespace ‘ceres’ does not name a type 242 | new ceres::QuaternionParameterization; | ^~~~~~~~~~~~~~~~~~~~~~~~~~ /home/lichao/Documents/NerfingMVS/colmap/src/estimators/pose.cc:243:13: error: ‘class ceres::Problem’ has no member named ‘SetParameterization’; did you mean ‘SetParameterLowerBound’? 243 | problem.SetParameterization(qvec->data(), quaternion_parameterization); | ^~~~~~~~~~~~~~~~~~~ | SetParameterLowerBound /home/lichao/Documents/NerfingMVS/colmap/src/estimators/pose.cc:276:16: error: ‘SubsetParameterization’ is not a member of ‘ceres’; did you mean ‘LocalParameterization’? 276 | ceres::SubsetParameterization* camera_params_parameterization = | ^~~~~~~~~~~~~~~~~~~~~~ | LocalParameterization /home/lichao/Documents/NerfingMVS/colmap/src/estimators/pose.cc:276:40: error: ‘camera_params_parameterization’ was not declared in this scope 276 | ceres::SubsetParameterization* camera_params_parameterization = | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ /home/lichao/Documents/NerfingMVS/colmap/src/estimators/pose.cc:277:17: error: expected type-specifier 277 | new ceres::SubsetParameterization( | ^~~~~ /home/lichao/Documents/NerfingMVS/colmap/src/estimators/pose.cc:279:17: error: ‘class ceres::Problem’ has no member named ‘SetParameterization’; did you mean ‘SetParameterLowerBound’? 279 | problem.SetParameterization(camera->ParamsData(), | ^~~~~~~~~~~~~~~~~~~ | SetParameterLowerBound /home/lichao/Documents/NerfingMVS/colmap/src/estimators/pose.cc: In function ‘bool colmap::RefineRelativePose(const ceres::Solver::Options&, const std::vector<Eigen::Matrix<double, 2, 1>, std::allocator<Eigen::Matrix<double, 2, 1> > >&, const std::vector<Eigen::Matrix<double, 2, 1>, std::allocator<Eigen::Matrix<double, 2, 1> > >&, Eigen::Vector4d*, Eigen::Vector3d*)’: /home/lichao/Documents/NerfingMVS/colmap/src/estimators/pose.cc:333:18: error: ‘QuaternionParameterization’ in namespace ‘ceres’ does not name a type 333 | new ceres::QuaternionParameterization; | ^~~~~~~~~~~~~~~~~~~~~~~~~~ /home/lichao/Documents/NerfingMVS/colmap/src/estimators/pose.cc:334:11: error: ‘class ceres::Problem’ has no member named ‘SetParameterization’; did you mean ‘SetParameterLowerBound’? 334 | problem.SetParameterization(qvec->data(), quaternion_parameterization); | ^~~~~~~~~~~~~~~~~~~ | SetParameterLowerBound /home/lichao/Documents/NerfingMVS/colmap/src/estimators/pose.cc:336:10: error: ‘HomogeneousVectorParameterization’ is not a member of ‘ceres’ 336 | ceres::HomogeneousVectorParameterization* homogeneous_parameterization = | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ /home/lichao/Documents/NerfingMVS/colmap/src/estimators/pose.cc:336:45: error: ‘homogeneous_parameterization’ was not declared in this scope; did you mean ‘quaternion_parameterization’? 336 | ceres::HomogeneousVectorParameterization* homogeneous_parameterization = | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~ | quaternion_parameterization /home/lichao/Documents/NerfingMVS/colmap/src/estimators/pose.cc:337:11: error: expected type-specifier 337 | new ceres::HomogeneousVectorParameterization(3); | ^~~~~ /home/lichao/Documents/NerfingMVS/colmap/src/estimators/pose.cc:338:11: error: ‘class ceres::Problem’ has no member named ‘SetParameterization’; did you mean ‘SetParameterLowerBound’? 338 | problem.SetParameterization(tvec->data(), homogeneous_parameterization); | ^~~~~~~~~~~~~~~~~~~ | SetParameterLowerBound /home/lichao/Documents/NerfingMVS/colmap/src/optim/bundle_adjustment.cc: In member function ‘void colmap::BundleAdjuster::AddImageToProblem(colmap::image_t, colmap::Reconstruction*, ceres::LossFunction*)’: /home/lichao/Documents/NerfingMVS/colmap/src/optim/bundle_adjustment.cc:420:22: error: ‘QuaternionParameterization’ in namespace ‘ceres’ does not name a type 420 | new ceres::QuaternionParameterization; | ^~~~~~~~~~~~~~~~~~~~~~~~~~ /home/lichao/Documents/NerfingMVS/colmap/src/optim/bundle_adjustment.cc:421:17: error: ‘class ceres::Problem’ has no member named ‘SetParameterization’; did you mean ‘SetParameterLowerBound’? 421 | problem_->SetParameterization(qvec_data, quaternion_parameterization); | ^~~~~~~~~~~~~~~~~~~ | SetParameterLowerBound /home/lichao/Documents/NerfingMVS/colmap/src/optim/bundle_adjustment.cc:425:16: error: ‘SubsetParameterization’ is not a member of ‘ceres’; did you mean ‘LocalParameterization’? 425 | ceres::SubsetParameterization* tvec_parameterization = | ^~~~~~~~~~~~~~~~~~~~~~ | LocalParameterization /home/lichao/Documents/NerfingMVS/colmap/src/optim/bundle_adjustment.cc:425:40: error: ‘tvec_parameterization’ was not declared in this scope; did you mean ‘quaternion_parameterization’? 425 | ceres::SubsetParameterization* tvec_parameterization = | ^~~~~~~~~~~~~~~~~~~~~ | quaternion_parameterization /home/lichao/Documents/NerfingMVS/colmap/src/optim/bundle_adjustment.cc:426:17: error: expected type-specifier 426 | new ceres::SubsetParameterization(3, constant_tvec_idxs); | ^~~~~ /home/lichao/Documents/NerfingMVS/colmap/src/optim/bundle_adjustment.cc:427:19: error: ‘class ceres::Problem’ has no member named ‘SetParameterization’; did you mean ‘SetParameterLowerBound’? 427 | problem_->SetParameterization(tvec_data, tvec_parameterization); | ^~~~~~~~~~~~~~~~~~~ | SetParameterLowerBound /home/lichao/Documents/NerfingMVS/colmap/src/optim/bundle_adjustment.cc:423:33: warning: unused variable ‘constant_tvec_idxs’ [-Wunused-variable] 423 | const std::vector& constant_tvec_idxs = | ^~~~~~~~~~~~~~~~~~ /home/lichao/Documents/NerfingMVS/colmap/src/optim/bundle_adjustment.cc: In member function ‘void colmap::BundleAdjuster::ParameterizeCameras(colmap::Reconstruction*)’: /home/lichao/Documents/NerfingMVS/colmap/src/optim/bundle_adjustment.cc:514:16: error: ‘SubsetParameterization’ is not a member of ‘ceres’; did you mean ‘LocalParameterization’? 514 | ceres::SubsetParameterization* camera_params_parameterization = | ^~~~~~~~~~~~~~~~~~~~~~ | LocalParameterization /home/lichao/Documents/NerfingMVS/colmap/src/optim/bundle_adjustment.cc:514:40: error: ‘camera_params_parameterization’ was not declared in this scope 514 | ceres::SubsetParameterization* camera_params_parameterization = | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ /home/lichao/Documents/NerfingMVS/colmap/src/optim/bundle_adjustment.cc:515:17: error: expected type-specifier 515 | new ceres::SubsetParameterization( | ^~~~~ /home/lichao/Documents/NerfingMVS/colmap/src/optim/bundle_adjustment.cc:517:19: error: ‘class ceres::Problem’ has no member named ‘SetParameterization’; did you mean ‘SetParameterLowerBound’? 517 | problem_->SetParameterization(camera.ParamsData(), | ^~~~~~~~~~~~~~~~~~~ | SetParameterLowerBound /home/lichao/Documents/NerfingMVS/colmap/src/optim/bundle_adjustment.cc: In member function ‘void colmap::RigBundleAdjuster::AddImageToProblem(colmap::image_t, colmap::Reconstruction*, std::vectorcolmap::CameraRig, ceres::LossFunction)’: /home/lichao/Documents/NerfingMVS/colmap/src/optim/bundle_adjustment.cc:1061:14: error: ‘SubsetParameterization’ is not a member of ‘ceres’; did you mean ‘LocalParameterization’? 1061 | ceres::SubsetParameterization* tvec_parameterization = | ^~~~~~~~~~~~~~~~~~~~~~ | LocalParameterization /home/lichao/Documents/NerfingMVS/colmap/src/optim/bundle_adjustment.cc:1061:38: error: ‘tvec_parameterization’ was not declared in this scope 1061 | ceres::SubsetParameterization* tvec_parameterization = | ^~~~~~~~~~~~~~~~~~~~~ /home/lichao/Documents/NerfingMVS/colmap/src/optim/bundle_adjustment.cc:1062:15: error: expected type-specifier 1062 | new ceres::SubsetParameterization(3, constant_tvec_idxs); | ^~~~~ /home/lichao/Documents/NerfingMVS/colmap/src/optim/bundle_adjustment.cc:1063:17: error: ‘class ceres::Problem’ has no member named ‘SetParameterization’; did you mean ‘SetParameterLowerBound’? 1063 | problem_->SetParameterization(tvec_data, tvec_parameterization); | ^~~~~~~~~~~~~~~~~~~ | SetParameterLowerBound /home/lichao/Documents/NerfingMVS/colmap/src/optim/bundle_adjustment.cc:1059:31: warning: unused variable ‘constant_tvec_idxs’ [-Wunused-variable] 1059 | const std::vector& constant_tvec_idxs = | ^~~~~~~~~~~~~~~~~~ /home/lichao/Documents/NerfingMVS/colmap/src/optim/bundle_adjustment.cc: In member function ‘void colmap::RigBundleAdjuster::ParameterizeCameraRigs(colmap::Reconstruction*)’: /home/lichao/Documents/NerfingMVS/colmap/src/optim/bundle_adjustment.cc:1147:20: error: ‘QuaternionParameterization’ in namespace ‘ceres’ does not name a type 1147 | new ceres::QuaternionParameterization; | ^~~~~~~~~~~~~~~~~~~~~~~~~~ /home/lichao/Documents/NerfingMVS/colmap/src/optim/bundle_adjustment.cc:1148:15: error: ‘class ceres::Problem’ has no member named ‘SetParameterization’; did you mean ‘SetParameterLowerBound’? 1148 | problem_->SetParameterization(qvec_data, quaternion_parameterization); | ^~~~~~~~~~~~~~~~~~~ | SetParameterLowerBound make[2]: *** [src/CMakeFiles/colmap.dir/build.make:661: src/CMakeFiles/colmap.dir/estimators/pose.cc.o] Error 1 make[2]: *** Waiting for unfinished jobs.... make[2]: *** [src/CMakeFiles/colmap.dir/build.make:882: src/CMakeFiles/colmap.dir/optim/bundle_adjustment.cc.o] Error 1 make[1]: *** [CMakeFiles/Makefile2:858: src/CMakeFiles/colmap.dir/all] Error 2 make: *** [Makefile:149: all] Error 2

I hope to receive your reply! THX

Hi sorry to bother you i want to integrate other keypoint detector and extractor in another matching method, but i dont have any experience with development in ubuntu, could you give me some suggestions with me many thanks!!!