Hit the subscribe button on the right of this issue if you wish to be notified of the code release. Please do not reply to this issue to not spam other subscribers. Please do not contact us to ask for early-access to the code.

[ 43%] Built target pixsfm
Scanning dependencies of target pypixsfm
[ 50%] Building CXX object pixsfm/CMakeFiles/pypixsfm.dir/base/bindings.cc.o
[ 87%] Building CXX object pixsfm/CMakeFiles/pypixsfm.dir/features/bindings.cc.o
[ 87%] Building CXX object pixsfm/CMakeFiles/pypixsfm.dir/util/bindings.cc.o
[ 90%] Building CXX object pixsfm/CMakeFiles/pypixsfm.dir/localization/bindings.cc.o
[ 90%] Building CXX object pixsfm/CMakeFiles/pypixsfm.dir/keypoint_adjustment/bindings.cc.o
[ 90%] Building CXX object pixsfm/CMakeFiles/pypixsfm.dir/residuals/bindings.cc.o
[ 90%] Building CXX object pixsfm/CMakeFiles/pypixsfm.dir/bundle_adjustment/bindings.cc.o
[ 90%] Building CXX object pixsfm/CMakeFiles/pypixsfm.dir/pyceres/cost_functions.cc.o
[ 90%] Building CXX object pixsfm/CMakeFiles/pypixsfm.dir/pyceres/pyceres.cc.o
[ 90%] Building CXX object pixsfm/CMakeFiles/pypixsfm.dir/pyceres/solver.cc.o
[ 90%] Building CXX object pixsfm/CMakeFiles/pypixsfm.dir/pyceres/parameterization.cc.o
[ 90%] Building CXX object pixsfm/CMakeFiles/pypixsfm.dir/pyceres/callbacks.cc.o
[ 90%] Building CXX object pixsfm/CMakeFiles/pypixsfm.dir/pyceres/loss_functions.cc.o
[ 90%] Building CXX object pixsfm/CMakeFiles/pypixsfm.dir/pyceres/types.cc.o
[ 90%] Building CXX object pixsfm/CMakeFiles/pypixsfm.dir/pyceres/problem.cc.o
[ 93%] Linking CXX static library libpypixsfm.a
[ 93%] Built target pypixsfm
Scanning dependencies of target _pixsfm
[ 96%] Building CXX object pixsfm/CMakeFiles/_pixsfm.dir/_pixsfm/bindings.cc.o
[100%] Linking CXX shared module ../../../pixsfm/_pixsfm.cpython-38-x86_64-linux-gnu.so
/usr/bin/ld: /usr/local/lib/libceres.a(covariance_impl.cc.o): in function `ceres::internal::CovarianceImpl::GetCovarianceBlockInTangentOrAmbientSpace(double const*, double const*, bool, double*) const':
covariance_impl.cc:(.text+0xa81b): undefined reference to `void Eigen::internal::call_dense_assignment_loop<Eigen::Matrix<double, -1, -1, 1, -1, -1>, Eigen::Matrix<double, -1, -1, 1, -1, -1>, Eigen::internal::assign_op<double, double> >(Eigen::Matrix<double, -1, -1, 1, -1, -1>&, Eigen::Matrix<double, -1, -1, 1, -1, -1> const&, Eigen::internal::assign_op<double, double> const&)'
collect2: error: ld returned 1 exit status
make[2]: *** [pixsfm/CMakeFiles/_pixsfm.dir/build.make:133: ../../pixsfm/_pixsfm.cpython-38-x86_64-linux-gnu.so] Error 1
make[1]: *** [CMakeFiles/Makefile2:403: pixsfm/CMakeFiles/_pixsfm.dir/all] Error 2
make: *** [Makefile:130: all] Error 2
/usr/local/lib/python3.8/dist-packages/setuptools/command/easy_install.py:156: EasyInstallDeprecationWarning: easy_install command is deprecated. Use build and pip and other standards-based tools.
  warnings.warn(
/usr/local/lib/python3.8/dist-packages/setuptools/command/install.py:34: SetuptoolsDeprecationWarning: setup.py install is deprecated. Use build and pip and other standards-based tools.
  warnings.warn(
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/home/prajwal.chidananda/pixel-perfect-sfm/setup.py", line 112, in <module>
    setup(
  File "/usr/local/lib/python3.8/dist-packages/setuptools/__init__.py", line 153, in setup
    return distutils.core.setup(**attrs)
  File "/usr/lib/python3.8/distutils/core.py", line 148, in setup
    dist.run_commands()
  File "/usr/lib/python3.8/distutils/dist.py", line 966, in run_commands
    self.run_command(cmd)
  File "/usr/lib/python3.8/distutils/dist.py", line 985, in run_command
    cmd_obj.run()
  File "/usr/local/lib/python3.8/dist-packages/setuptools/command/develop.py", line 34, in run
    self.install_for_development()
  File "/usr/local/lib/python3.8/dist-packages/setuptools/command/develop.py", line 114, in install_for_development
    self.run_command('build_ext')
  File "/usr/lib/python3.8/distutils/cmd.py", line 313, in run_command
    self.distribution.run_command(command)
  File "/usr/lib/python3.8/distutils/dist.py", line 985, in run_command
    cmd_obj.run()
  File "/home/prajwal.chidananda/pixel-perfect-sfm/setup.py", line 55, in run
    self.build_extension(ext)
  File "/home/prajwal.chidananda/pixel-perfect-sfm/setup.py", line 98, in build_extension
    subprocess.check_call(['cmake', '--build', '.'] + build_args,
  File "/usr/lib/python3.8/subprocess.py", line 364, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['cmake', '--build', '.', '--config', 'Release', '--', '-j']' returned non-zero exit status 2.
----------------------------------------

ERROR: Command errored out with exit status 1: /usr/bin/python3 -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/home/prajwal.chidananda/pixel-perfect-sfm/setup.py'"'"'; file='"'"'/home/prajwal.chidananda/pixel-perfect-sfm/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(file);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, file, '"'"'exec'"'"'))' develop --no-deps Check the logs for full command output.

Building CXX object pixsfm/CMakeFiles/pypixsfm.dir/keypoint_adjustment/bindings.cc.o In file included from /work/pixel-perfect-sfm/pixsfm/keypoint_adjustment/src/featuremetric_keypoint_optimizer.h:8:0, from /work/pixel-perfect-sfm/pixsfm/keypoint_adjustment/bindings.cc:8: /work/pixel-perfect-sfm/pixsfm/base/src/parallel_optimizer.h:90:6: error: prototype for ‘std::unordered_map<long unsigned int, decltype (static_cast<Optimizer*>(nullptr)->.RunSubset<Ns ...>(((pixsfm::ParallelOptimizer<Optimizer, idx_t>)this)->pixsfm::ParallelOptimizer<Optimizer, idx_t>::dummy, pixsfm::ParallelOptimizer::RunParallel::parameters ...))> pixsfm::ParallelOptimizer<Optimizer, idx_t>::RunParallel(std::vector, Param& ...)’ does not match any in class ‘pixsfm::ParallelOptimizer<Optimizer, idx_t>’ auto ParallelOptimizer<Optimizer, idx_t>::RunParallel( ^ /work/pixel-perfect-sfm/pixsfm/base/src/parallel_optimizer.h:78:8: error: candidate is: template<class Optimizer, class idx_t> template<int ...Ns, class ... Param> std::unordered_map<long unsigned int, decltype (static_cast<Optimizer>(nullptr)->.RunSubset<Ns ...>(((pixsfm::ParallelOptimizer<Optimizer, idx_t>*)this)->pixsfm::ParallelOptimizer<Optimizer, idx_t>::dummy, pixsfm::ParallelOptimizer::RunParallel::parameters ...))> pixsfm::ParallelOptimizer<Optimizer, idx_t>::RunParallel(std::vector, Param& ...) auto RunParallel(std::vector problem_labels, Param&... parameters) ^ pixsfm/CMakeFiles/pypixsfm.dir/build.make:134: recipe for target 'pixsfm/CMakeFiles/pypixsfm.dir/keypoint_adjustment/bindings.cc.o' failed

gcc -v gcc version 4.8.5 (Ubuntu 4.8.5-4ubuntu8~16.04.1)

First of all, thanks for the great work. In the paper and project page the authors mention scaling PixSfM to thousands of images. I'm however running into issues doing the same with datasets ranging in the 1600-3000 image range. Two of the datasets in question can be found at: https://meganerf.cmusatyalab.org/#data

My code looks roughly like:

def main(hparams: Namespace) -> None:
    output_path = Path(hparams.output_path)
    output_path.mkdir(exist_ok=True, parents=True)

    feature_conf = extract_features.confs['superpoint_max']
    matcher_conf = match_features.confs['superglue']

    images_path = Path(hparams.images_path)

    images = sorted(images_path.iterdir())
    references = [str(images[i].relative_to(images_path)) for i in range(len(images))]
    print(len(references), 'mapping images')

    features_path = output_path / 'features.h5'
    sfm_pairs_path = output_path / 'pairs-sfm.txt'
    matches_path = output_path / 'matches.h5'

    extract_features.main(feature_conf, images_path, image_list=references, feature_path=features_path)
    pairs_from_gps.main(sfm_pairs_path, images_path, references, 50) # Custom matcher that uses the GPS metadata to match only to the 50 closest pairs
    match_features.main(matcher_conf, sfm_pairs_path, features=features_path, matches=matches_path)

    sfm = PixSfM(conf={"dense_features": {"use_cache": True}})

    ref_dir = output_path / 'ref'
    refined, sfm_outputs = sfm.reconstruction(ref_dir, images_path, sfm_pairs_path, features_path, matches_path,
                                              image_list=references)

And in particular the keypoint adjustment phase seems to be taking a very long time. Some examples so far:

1678 mapping images                                                                                                                                                                                                                                
[2022/02/04 18:28:11 pixsfm.features.models.s2dnet INFO] Loading S2DNet checkpoint at /home/ubuntu/pixel-perfect-sfm/pixsfm/features/models/checkpoints/s2dnet_weights.pth.                                                                        
[2022/02/04 18:28:25 pixsfm INFO] Loaded dense extractor with configuration:                                                                                                                                                                       
{'cache_format': 'chunked',                                                                                                                                                                                                                        
 'device': 'auto',                                                                                                                                                                                                                                 
 'dtype': 'half',                                                                                                                                                                                                                                  
 'fast_image_load': False,                                                                                                                                                                                                                         
 'l2_normalize': True,                                                                                                                                                                                                                             
 'load_cache_on_init': False,                                                                                                                                                                                                                      
 'max_edge': 1600,                                                                                                                                                                                                                                 
 'model': {'name': 's2dnet', 'num_layers': 1, 'checkpointing': None, 'output_dim': 128, 'pretrained': 's2dnet', 'remove_pooling_layers': False, 'combine': False},                                                                                 
 'overwrite_cache': False,                                                                                                                                                                                                                         
 'patch_size': 16,                                                                                                                                                                                                                                 
 'pyr_scales': [1.0],                                                                                                                                                                                                                              
 'resize': 'LANCZOS',                                                                                                                                                                                                                              
 'sparse': True,                                                                                                                                                                                                                                   
 'use_cache': True}                                                                                                                                                                                                                                [2022/02/04 18:29:52 pixsfm INFO] Building matching graph...                                                                                                                                                                                       
[2022/02/04 18:31:00 pixsfm INFO] Extracting dense features...                                                                                                                                                                                                                                                                                                                                                                                                                                      
100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1678/1678 [1:05:39<00:00,  2.35s/it]|
[2022/02/04 19:36:40 pixsfm INFO] Loading featuremaps from H5 File.                                                                                                                                                                                
100%[████████████████████] 1678/1678 [00:51, 32.4514it/s]                                                                                                                                                                                          
[2022/02/04 19:37:32 pixsfm INFO] Computing tracks...                                                                                                                                                                                              
[2022/02/04 19:37:32 pixsfm INFO] # graph nodes: 6001799                                                                                                                                                                                           
[2022/02/04 19:37:38 pixsfm INFO] # graph edges: 87565132                                                                                                                                                                                          
[2022/02/04 19:39:26 pixsfm INFO] # tracks: 655998                                                                                                                                                                                                 
[2022/02/04 19:39:31 pixsfm INFO] Start feature-metric keypoint adjustment.                                                                                                                                                                        
[2022/02/04 19:39:35 pixsfm WARNING] 13887 / 115040 problems have more than 50 keypoints.                                                                                                                                                          
         Maximum keypoints in a problem: 202                                                                                                                                                                                                       
 74%[███████████████     ] 4484836/6001799 [171:56:55, 7.24511it/s]

Another example seems to be getting slower over time:

[2022/02/05 12:35:10 pixsfm INFO] Loading featuremaps from H5 File.
1940 mapping images
[2022/02/05 11:47:44 pixsfm.features.models.s2dnet INFO] Loading S2DNet checkpoint at /data/hturki/pixel-perfect-sfm/pixsfm/features/models/checkpoints/s2dnet_weights.pth.
[2022/02/05 11:47:44 pixsfm.features.models.s2dnet INFO] Downloading S2DNet weights.
[2022/02/05 11:47:51 pixsfm INFO] Loaded dense extractor with configuration:
{'cache_format': 'chunked',
 'device': 'auto',
 'dtype': 'half',
 'fast_image_load': False,
 'l2_normalize': True,
 'load_cache_on_init': False,
 'max_edge': 1600,
 'model': {'name': 's2dnet', 'num_layers': 1, 'checkpointing': None, 'output_dim': 128, 'pretrained': 's2dnet', 'remove_pooling_layers': False, 'combine': False},
 'overwrite_cache': False,
 'patch_size': 16,
 'pyr_scales': [1.0],
 'resize': 'LANCZOS',
 'sparse': True,
 'use_cache': True}
[2022/02/05 11:48:51 pixsfm INFO] Building matching graph...
[2022/02/05 11:50:23 pixsfm INFO] Extracting dense features...
100%[████████████████████] 1938/1938 [00:10, 183.748it/s]
[2022/02/05 12:35:21 pixsfm INFO] Computing tracks...
[2022/02/05 12:35:21 pixsfm INFO] # graph nodes: 6696955
[2022/02/05 12:35:28 pixsfm INFO] # graph edges: 84325871
[2022/02/05 12:38:06 pixsfm INFO] # tracks: 761986
[2022/02/05 12:38:13 pixsfm INFO] Start feature-metric keypoint adjustment.
[2022/02/05 12:38:19 pixsfm WARNING] 16060 / 125998 problems have more than 50 keypoints.
         Maximum keypoints in a problem: 265
  0%[                    ]   59491/6696955 [01:18:26, 12.6392it/s]
 10%[██                  ]  729223/6696955 [23:18:52, 8.68822it/s]
 10%[██                  ]  729290/6696955 [23:19:05, 8.68761it/s]
 25%[█████               ] 1718493/6696955 [70:43:00, 6.75028it/s]
 27%[██████              ] 1830843/6696955 [76:13:28, 6.67195it/s]
 41%[████████            ] 2765893/6696955 [121:42:58, 6.31225it/s]
 49%[██████████          ] 3347643/6696955 [150:00:06, 6.19926it/s]

And same for a third example

3019 mapping images
[2022/02/06 12:12:32 pixsfm.features.models.s2dnet INFO] Loading S2DNet checkpoint at /home/cloudlet/hturki/pixel-perfect-sfm/pixsfm/features/models/checkpoints/s2dnet_weights.pth.
[2022/02/06 12:12:36 pixsfm INFO] Loaded dense extractor with configuration:
{'cache_format': 'chunked',
 'device': 'auto',
 'dtype': 'half',
 'fast_image_load': False,
 'l2_normalize': True,
 'load_cache_on_init': False,
 'max_edge': 1600,
 'model': {'name': 's2dnet', 'num_layers': 1, 'checkpointing': None, 'output_dim': 128, 'pretrained': 's2dnet', 'remove_pooling_layers': False, 'combine': False},
 'overwrite_cache': False,
 'patch_size': 16,
 'pyr_scales': [1.0],
 'resize': 'LANCZOS',
 'sparse': True,
 'use_cache': True}
[2022/02/06 12:14:03 pixsfm INFO] Building matching graph...
[2022/02/06 12:16:14 pixsfm INFO] Extracting dense features...
  5%|██████████▏                                                                                                                                                                                              | 153/  5%|██████████▎                                                                                                                                                                                              | 154/  5%|██████████▎                                              100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 3019/3019 [1:08:49<00:00,  1.37s/it]
[2022/02/06 13:25:03 pixsfm INFO] Loading featuremaps from H5 File.
100%[████████████████████] 3019/3019 [00:17, 172.998it/s]
[2022/02/06 13:25:21 pixsfm INFO] Computing tracks...
[2022/02/06 13:25:21 pixsfm INFO] # graph nodes: 11090098
[2022/02/06 13:25:27 pixsfm INFO] # graph edges: 122388899
[2022/02/06 13:34:55 pixsfm INFO] # tracks: 834854
[2022/02/06 13:35:05 pixsfm INFO] Start feature-metric keypoint adjustment.
[2022/02/06 13:35:13 pixsfm WARNING] 41539 / 141697 problems have more than 50 keypoints.
         Maximum keypoints in a problem: 1022
  0%[                    ]    28755/11090098 [12:26, 38.5155it/s]
  0%[                    ]    42692/11090098 [29:23, 24.2052it/s]
  0%[                    ]    62858/11090098 [50:21, 20.8059it/s]
 16%[███                 ]  1884721/11090098 [23:21:01, 22.4207it/s]
 37%[████████            ]  4142028/11090098 [78:01:34, 14.7458it/s]
 45%[█████████           ]  5016151/11090098 [114:52:32, 12.1294it/s]
 46%[█████████           ]  5208388/11090098 [125:06:01, 11.5649it/s]

The specs for the machines across each of these runs varies a bit, but they're pretty powerful machines (>100GB of ram, dozens of cores, SSD mounts)

I've also tried running PixSfM on an even larger example (matching only against the 20 closest neighbors instead of 50), where keypoint adjustment throws an exception. I've trying reducing the output dim and patch size to no avail:

5871 mapping images
[2022/02/11 08:29:37 hloc INFO] Extracting local features with configuration:
{'model': {'max_keypoints': 4096, 'name': 'superpoint', 'nms_radius': 3},
 'output': 'feats-superpoint-n4096-rmax1600',
 'preprocessing': {'grayscale': True, 'resize_force': True, 'resize_max': 1600}}
Loaded SuperPoint model
100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 5871/5871 [16:57<00:00,  5.77it/s]
[2022/02/11 08:46:38 hloc INFO] Finished exporting features.
[2022/02/11 08:46:41 hloc INFO] Obtaining pairwise distances between 5871 images...
[2022/02/11 08:46:43 hloc INFO] Found 117230 pairs.
[2022/02/11 08:46:43 hloc INFO] Matching local features with configuration:
{'model': {'name': 'superglue',
           'sinkhorn_iterations': 50,
           'weights': 'outdoor'},
 'output': 'matches-superglue'}
Loaded SuperGlue model ("outdoor" weights)
100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 117230/117230 [6:16:43<00:00,  5.19it/s]
[2022/02/11 15:03:29 hloc INFO] Finished exporting matches.
[2022/02/11 15:03:31 pixsfm.features.models.s2dnet INFO] Loading S2DNet checkpoint at /compute/autobot-1-1/hturki/pixel-perfect-sfm/pixsfm/features/models/checkpoints/s2dnet_weights.pth.
[2022/02/11 15:03:31 pixsfm INFO] Loaded dense extractor with configuration:
{'cache_format': 'chunked',
 'device': 'auto',
 'dtype': 'half',
 'fast_image_load': False,
 'l2_normalize': True,
 'load_cache_on_init': False,
 'max_edge': 1600,
 'model': {'name': 's2dnet', 'num_layers': 1, 'checkpointing': None, 'output_dim': 64, 'pretrained': 's2dnet', 'remove_pooling_layers': False, 'combine': False, 'patch_size': 8},
 'overwrite_cache': False,
 'patch_size': 8,
 'pyr_scales': [1.0],
 'resize': 'LANCZOS',
 'sparse': True,
 'use_cache': True} # Have also tried keeping everything in memory
[2022/02/11 15:04:33 pixsfm INFO] Building matching graph...
[2022/02/11 15:07:55 pixsfm INFO] Extracting dense features...
100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 5871/5871 [45:08<00:00,  2.17it/s]
[2022/02/11 15:53:04 pixsfm INFO] Computing tracks...
[2022/02/11 15:53:04 pixsfm INFO] # graph nodes: 21615817
[2022/02/11 15:53:12 pixsfm INFO] # graph edges: 106782076
[2022/02/11 15:55:08 pixsfm INFO] # tracks: 2955327
[2022/02/11 15:55:28 pixsfm INFO] Start feature-metric keypoint adjustment.
[2022/02/11 15:55:52 pixsfm WARNING] 206052 / 906513 problems have more than 20 keypoints.
         Maximum keypoints in a problem: 487
  0%[                    ]        0/21615817 [00:00, -nanit/s][2022/02/11 15:56:13 pixsfm ERROR] Child process threw exception in problem 0.

Any thoughts on what I might be doing wrong, or is a week+ runtime to be expected no matter what? I've gotten suboptimal results on these (drone imagery-related) datasets using even commercial photogrammetry software, so excited to see if I can get better results with your featuremetric approach!

Each time I run a script that calls the matcher, it takes about 20s to load SuperGlue model ("outdoor" weights). Is there a way to better reuse cache between consecutive runs? Or is it that big in memory?

For instance loading S2DNet checkpoint at /dependencies/pixel-perfect-sfm/pixsfm/features/models/checkpoints/s2dnet_weights.pth. is instantaneous

Many thanks

I'm trying to refine a Pointcloud extracted using ArCore on my phone, and I get a Segmentation fault error at the end of the triangulation part of PixSfM.

As an input, I provide:

• A folder of images
• A binary colmap model (cameras.bin, images.bin, points3D.bin): The pointcloud is noisy and used only for the pairs_from_covisibility method (In images.bin, point3D_ids make sense but I simply put dummy zeros xys for the 2d features since features should be recomputed by hloc) but the intrinsics/extrinsics estimate from arcore are quite good and should be used as priors

and I want to refine the poses and the pointcloud.

My pipeline is the following:

1. hloc.extract_features
2. hloc.pairs_from_covisibility
3. hloc.match_features
4. PixSfM.triangulation

I use the low memory configuration file, because it crashes otherwise.

Here's my code:

from pathlib import Path
from hloc import extract_features, match_features, pairs_from_covisibility
from pixsfm.refine_hloc import PixSfM

images = Path('data/images')
input_model = Path('data/input_model')
outputs = Path('data/results')

sfm_pairs = outputs / 'pairs-sfm.txt'
features = outputs / 'features.h5'
matches = outputs / 'matches.h5'
sfm_dir = outputs / "sfm"

feature_conf = extract_features.confs['superpoint_aachen']
matcher_conf = match_features.confs['superglue']

references = [str(p.relative_to(images))for p in images.iterdir()]

extract_features.main(feature_conf, images,
                      image_list=references,
                      feature_path=features)
pairs_from_covisibility.main(input_model,
                             sfm_pairs,
                             num_matched=5)
match_features.main(matcher_conf,
                    sfm_pairs,
                    features=features,
                    matches=matches)

refiner = PixSfM(conf="/dependencies/pixel-perfect-sfm/pixsfm/configs/low_memory.yaml")
model, _ = refiner.triangulation(output_dir=sfm_dir,
                                 reference_model_path=input_model,
                                 image_dir=images,
                                 pairs_path=sfm_pairs,
                                 features_path=features,
                                 matches_path=matches
                                 )

and here's the output I got:

[2022/01/27 09:56:05 hloc INFO] Extracting local features with configuration:
{'model': {'max_keypoints': 4096, 'name': 'superpoint', 'nms_radius': 3},
 'output': 'feats-superpoint-n4096-r1024',
 'preprocessing': {'grayscale': True, 'resize_max': 1024}}
[2022/01/27 09:56:05 hloc INFO] Skipping the extraction.
[2022/01/27 09:56:05 hloc INFO] Reading the COLMAP model...
[2022/01/27 09:56:05 hloc INFO] Extracting image pairs from covisibility info...
  0%|                                                                                                                                                                                   | 0/283 [00:00<?, ?it/s][2022/01/27 09:56:05 hloc INFO] Image 28 does not have any covisibility.
[2022/01/27 09:56:05 hloc INFO] Image 29 does not have any covisibility.
[2022/01/27 09:56:05 hloc INFO] Image 30 does not have any covisibility.
[2022/01/27 09:56:05 hloc INFO] Image 57 does not have any covisibility.
[2022/01/27 09:56:05 hloc INFO] Image 74 does not have any covisibility.
[2022/01/27 09:56:05 hloc INFO] Image 75 does not have any covisibility.
[2022/01/27 09:56:05 hloc INFO] Image 131 does not have any covisibility.
[2022/01/27 09:56:05 hloc INFO] Image 191 does not have any covisibility.
[2022/01/27 09:56:05 hloc INFO] Image 192 does not have any covisibility.
[2022/01/27 09:56:05 hloc INFO] Image 202 does not have any covisibility.
[2022/01/27 09:56:05 hloc INFO] Image 203 does not have any covisibility.
[2022/01/27 09:56:05 hloc INFO] Image 211 does not have any covisibility.
[2022/01/27 09:56:05 hloc INFO] Image 223 does not have any covisibility.
[2022/01/27 09:56:05 hloc INFO] Image 224 does not have any covisibility.
[2022/01/27 09:56:05 hloc INFO] Image 243 does not have any covisibility.
[2022/01/27 09:56:05 hloc INFO] Image 244 does not have any covisibility.
[2022/01/27 09:56:05 hloc INFO] Image 245 does not have any covisibility.
[2022/01/27 09:56:05 hloc INFO] Image 246 does not have any covisibility.
[2022/01/27 09:56:05 hloc INFO] Image 247 does not have any covisibility.
[2022/01/27 09:56:05 hloc INFO] Image 248 does not have any covisibility.
[2022/01/27 09:56:05 hloc INFO] Image 249 does not have any covisibility.
[2022/01/27 09:56:05 hloc INFO] Image 264 does not have any covisibility.
[2022/01/27 09:56:05 hloc INFO] Image 265 does not have any covisibility.
[2022/01/27 09:56:05 hloc INFO] Image 266 does not have any covisibility.
[2022/01/27 09:56:05 hloc INFO] Image 269 does not have any covisibility.
[2022/01/27 09:56:05 hloc INFO] Image 270 does not have any covisibility.
[2022/01/27 09:56:05 hloc INFO] Image 271 does not have any covisibility.
100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 283/283 [00:00<00:00, 4739.06it/s]
[2022/01/27 09:56:05 hloc INFO] Found 911 pairs.
[2022/01/27 09:56:05 hloc INFO] Matching local features with configuration:
{'model': {'name': 'superglue',
           'sinkhorn_iterations': 50,
           'weights': 'outdoor'},
 'output': 'matches-superglue'}
Loaded SuperGlue model ("outdoor" weights)
100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 911/911 [00:31<00:00, 28.93it/s]
[2022/01/27 09:56:39 hloc INFO] Finished exporting matches.
[2022/01/27 09:56:40 pixsfm.features.models.s2dnet INFO] Loading S2DNet checkpoint at /dependencies/pixel-perfect-sfm/pixsfm/features/models/checkpoints/s2dnet_weights.pth.
[2022/01/27 09:56:40 pixsfm INFO] Loaded dense extractor with configuration:
{'cache_format': 'chunked',
 'device': 'auto',
 'dtype': 'half',
 'fast_image_load': False,
 'l2_normalize': True,
 'load_cache_on_init': False,
 'max_edge': 1600,
 'model': {'name': 's2dnet', 'num_layers': 1, 'checkpointing': None, 'output_dim': 128, 'pretrained': 's2dnet', 'remove_pooling_layers': False, 'combine': False},
 'overwrite_cache': True,
 'patch_size': 8,
 'pyr_scales': [1.0],
 'resize': 'LANCZOS',
 'sparse': True,
 'use_cache': True}
[2022/01/27 09:56:41 pixsfm INFO] Building matching graph...
[2022/01/27 09:56:41 pixsfm INFO] Extracting dense features...
100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 256/256 [00:12<00:00, 20.47it/s]
[2022/01/27 09:56:54 pixsfm INFO] Loading featuremaps from H5 File.
100%[████████████████████] 256/256 [00:00, 3605.63it/s]
[2022/01/27 09:56:54 pixsfm INFO] Computing tracks...
[2022/01/27 09:56:54 pixsfm INFO] # graph nodes: 109498
[2022/01/27 09:56:54 pixsfm INFO] # graph edges: 249077
[2022/01/27 09:56:54 pixsfm INFO] # tracks: 34883
[2022/01/27 09:56:54 pixsfm INFO] Start feature-metric keypoint adjustment.
[2022/01/27 09:56:54 pixsfm INFO] Start topological-reference keypoint adjustment.
100%[████████████████████] 109498/109498 [00:11, 9302.35it/s]
[2022/01/27 09:57:06 pixsfm INFO] KA Time: 11.7715s, cost change: 0.0232999 --> 0.0206287
[2022/01/27 09:57:06 hloc INFO] Importing features into the database...
100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 283/283 [00:00<00:00, 4536.62it/s]
[2022/01/27 09:57:06 hloc INFO] Importing matches into the database...
100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 911/911 [00:00<00:00, 8098.12it/s]
[2022/01/27 09:57:06 hloc INFO] Performing geometric verification of the matches...
[2022/01/27 09:57:07 hloc INFO] Running 3D triangulation...
[2022/01/27 09:57:10 hloc INFO] Finished the triangulation with statistics:
Reconstruction:
        num_reg_images = 283
        num_cameras = 283
        num_points3D = 1662
        num_observations = 3408
        mean_track_length = 2.05054
        mean_observations_per_image = 12.0424
        mean_reprojection_error = 1.84696
[2022/01/27 09:57:10 pixsfm INFO] Extracting references and costmaps.
Segmentation fault (core dumped)

Did I miss something or is there an actual bug? Might it be related to the low_memory conf? Is is still ok since I got the log "Finished the triangulation" ?

Many thanks !

Installation went smoothly for me until the final pixSfM package install via pip. Compilation reached 40%, then returned this error from the glog source packaged with pixSfM:

/pixel-perfect-sfm/pixsfm/util/src/glog.cc:86:48: error: invalid conversion from ‘void (*)(const char*, int)’ to ‘void (*)(const char*, size_t)’ {aka ‘void (*)(const char*, long unsigned int)’} [-fpermissive] 86 | []() { google::InstallFailureWriter(&PyBindLogStack); })

The full error context is reprinted below:

[ 40%] Building CXX object pixsfm/CMakeFiles/pixsfm.dir/bundle_adjustment/src/bundle_adjustment_options.cc.o
/home/kevin/pixel-perfect-sfm/pixsfm/util/src/glog.cc: In lambda function:
/home/kevin/pixel-perfect-sfm/pixsfm/util/src/glog.cc:86:48: error: invalid conversion from ‘void (*)(const char*, int)’ to ‘void (*)(const char*, size_t)’ {aka ‘void (*)(const char*, long unsigned int)’} [-fpermissive]
86 |            []() { google::InstallFailureWriter(&PyBindLogStack); })
|                                                ^~~~~~~~~~~~~~~
|                                                |
|                                                void (*)(const char*, int)
In file included from /usr/local/include/colmap/util/logging.h:37,
from /home/kevin/pixel-perfect-sfm/pixsfm/util/src/glog.cc:1:
/usr/local/include/glog/logging.h:1972:12: note:   initializing argument 1of ‘void google::InstallFailureWriter(void (*)(const char*, size_t))’
1972 |     void (*writer)(const char* data, size_t size));
|     ~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
make[2]: *** [pixsfm/CMakeFiles/pixsfm.dir/build.make:132: pixsfm/CMakeFiles/pixsfm.dir/util/src/glog.cc.o] Error 1
make[2]: *** Waiting for unfinished jobs....
make[1]: *** [CMakeFiles/Makefile2:368: pixsfm/CMakeFiles/pixsfm.dir/all] Error 2
make: *** [Makefile:136: all] Error 2
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/home/kevin/pixel-perfect-sfm/setup.py", line 112, in <module>
setup(
File "/home/kevin/anaconda3/lib/python3.9/site-packages/setuptools/__init__.py", line 153, in setup
return distutils.core.setup(**attrs)
File "/home/kevin/anaconda3/lib/python3.9/distutils/core.py", line 148, in setup
dist.run_commands()
File "/home/kevin/anaconda3/lib/python3.9/distutils/dist.py", line 966, in run_commands
self.run_command(cmd)
File "/home/kevin/anaconda3/lib/python3.9/distutils/dist.py", line 985, in run_command
cmd_obj.run()
File "/home/kevin/anaconda3/lib/python3.9/site-packages/setuptools/command/develop.py", line 34, in run
self.install_for_development()
File "/home/kevin/anaconda3/lib/python3.9/site-packages/setuptools/command/develop.py", line 114, in install_for_development
self.run_command('build_ext')
File "/home/kevin/anaconda3/lib/python3.9/distutils/cmd.py", line 313, in run_command
self.distribution.run_command(command)
File "/home/kevin/anaconda3/lib/python3.9/distutils/dist.py", line 985, in run_command
cmd_obj.run()
File "/home/kevin/pixel-perfect-sfm/setup.py", line 55, in run
self.build_extension(ext)
File "/home/kevin/pixel-perfect-sfm/setup.py", line 98, in build_extension
subprocess.check_call(['cmake', '--build', '.'] + build_args,
File "/home/kevin/anaconda3/lib/python3.9/subprocess.py", line 373, in check_call
raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['cmake', '--build', '.', '--config', 'Release', '--', '-j']' returned non-zero exit status 2.
----------------------------------------
ERROR: Command errored out with exit status 1: /home/kevin/anaconda3/bin/python -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'/home/kevin/pixel-perfect-sfm/setup.py'"'"'; __file__='"'"'/home/kevin/pixel-perfect-sfm/setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(__file__) if os.path.exists(__file__) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' develop --no-deps Check the logs for full command output.

I'm facing an issue while importing pixsfm . Following is the error-

>>> from pixsfm.util.visualize import init_image, plot_points2D Traceback (most recent call last): File "<stdin>", line 1, in <module> File "/home/ubuntu/pixelperf/pixel-perfect-sfm/pixsfm/__init__.py", line 18, in <module> from . import ( # noqa F403 File "/home/ubuntu/pixelperf/pixel-perfect-sfm/pixsfm/base/__init__.py", line 1, in <module> from .._pixsfm._base import * # noqa F403 ModuleNotFoundError: No module named 'pixsfm._pixsfm._base'

I'm trying to refine keypoint from database，and I get core dump during Start feature-metric keypoint adjustment.

The database is built on superpoint and superglue. When the descriptor does not exist in the database, the program will core dump. If the score is empty, this situation will be handle in the Graph::RegisterMatches, what is the cause of this problem？

void Graph::RegisterMatches(std::string imname1, std::string imname2,
                            size_t* matches,       // Nx2
                            double* similarities,  // Nx1
                            size_t n_matches) {
  for (size_t match_idx = 0; match_idx < n_matches; ++match_idx) {
    colmap::point2D_t feature_idx1 = matches[2 * match_idx];
    colmap::point2D_t feature_idx2 = matches[2 * match_idx + 1];
    **double similarity = similarities ? similarities[match_idx] : 1.0;**

    FeatureNode* node1 = FindOrCreateNode(imname1, feature_idx1);
    FeatureNode* node2 = FindOrCreateNode(imname2, feature_idx2);

    AddEdge(node1, node2, similarity);
  }
}

When I import the descriptor of superpoint and read it accoording to float32, it still core dump in feature-metric keypoint adjustment. But I use the default uint8 to read the descriptor and the program works fine. I don`t think this is normal, the superpoint descriptor should not be of uint8. What is the cause of this problem？

my code

conf = {
        "dense_features": {"use_cache": True,
        "max_edge": 1024},
}
refiner = PixSfM(conf=conf)
keypoints, _, _ = refiner.refine_keypoints_from_db(
        refined_database,
        database,
        image_dir,
        )

Thanks in advance!

my environment： Ubuntu 16.04 Python 3.8

Hi, Thanks for making this code available, and especially for the pycolmap improvements-- those have been needed for a very long time and the effort will have a big impact.

Have you evaluated this method on Tanks and Temples (or considered doing so) ? The object-centric scenes are a perhaps bit different-- there is perhaps more range of depth and background keypoints have less support (fewer images). If not, do you see more relevance with ETH3D?

Hi, thanks for your fantastic work. But I got some problems when loading .db file generated by colmap. Here is the problem trace:

Traceback (most recent call last): File "/home/tsk/anaconda3/envs/pixel/lib/python3.7/runpy.py", line 193, in _run_module_as_main "main", mod_spec) File "/home/tsk/anaconda3/envs/pixel/lib/python3.7/runpy.py", line 85, in _run_code exec(code, run_globals) File "/home/tsk/workspace/pixel-perfect-sfm/pixsfm/refine_colmap.py", line 209, in args.database_path, args.image_dir, cache_path=args.cache_path) File "/home/tsk/workspace/pixel-perfect-sfm/pixsfm/refine_colmap.py", line 108, in refine_keypoints_from_db pairs, matches, scores = read_matches_from_db(database_path) File "/home/tsk/workspace/pixel-perfect-sfm/pixsfm/util/colmap.py", line 38, in read_matches_from_db desc[image_id] = blob_to_array(data, np.uint32, (-1, c)) File "/home/tsk/workspace/pixel-perfect-sfm/pixsfm/util/database.py", line 134, in blob_to_array return np.fromstring(blob, dtype=dtype).reshape(*shape) ValueError: cannot reshape array of size 277536 into shape (128)

This seems to be the dimention of descriptor. However, I build the latest version of colmap and pycolmap from source. The .db file is generated from the colmap. Moreover, I noticed that the output dimention of descriptor is always 128.

Thanks in advance for your reply! Shengkun

pip install -e . Obtaining file:///home/goodix/qxg/groundtruth/pixsfm/pixel-perfect-sfm Preparing metadata (setup.py) ... done Installing collected packages: pixsfm Running setup.py develop for pixsfm error: subprocess-exited-with-error

× python setup.py develop did not run successfully.
│ exit code: 1
╰─> [124 lines of output]
    running develop
    running egg_info
    writing pixsfm.egg-info/PKG-INFO
    writing dependency_links to pixsfm.egg-info/dependency_links.txt
    writing top-level names to pixsfm.egg-info/top_level.txt
    reading manifest file 'pixsfm.egg-info/SOURCES.txt'
    adding license file 'LICENSE'
    writing manifest file 'pixsfm.egg-info/SOURCES.txt'
    running build_ext
    -- A library with BLAS API found.
    -- Found AMD headers in: /usr/include/suitesparse
    -- Found AMD library: /usr/lib/x86_64-linux-gnu/libamd.so
    -- Found CAMD headers in: /usr/include/suitesparse
    -- Found CAMD library: /usr/lib/x86_64-linux-gnu/libcamd.so
    -- Found CCOLAMD headers in: /usr/include/suitesparse
    -- Found CCOLAMD library: /usr/lib/x86_64-linux-gnu/libccolamd.so
    -- Found CHOLMOD headers in: /usr/include/suitesparse
    -- Found CHOLMOD library: /usr/lib/x86_64-linux-gnu/libcholmod.so
    -- Found COLAMD headers in: /usr/include/suitesparse
    -- Found COLAMD library: /usr/lib/x86_64-linux-gnu/libcolamd.so
    -- Found SPQR headers in: /usr/include/suitesparse
    -- Found SPQR library: /usr/lib/x86_64-linux-gnu/libspqr.so
    -- Found Config headers in: /usr/include/suitesparse
    -- Found Config library: /usr/lib/x86_64-linux-gnu/libsuitesparseconfig.so
    -- Did not find Intel TBB library, assuming SuiteSparseQR was not compiled with TBB.
    -- Adding librt to SuiteSparse_config libraries (required on Linux & Unix [not OSX] if SuiteSparse is compiled with timing).
    -- Found required Ceres dependency: Eigen version 3.3.7 in /usr/local/share/eigen3/cmake
    -- Found required Ceres dependency: glog
    -- Found required Ceres dependency: gflags
    -- Found Ceres version: 2.1.0 installed in: /usr/local with components: [EigenSparse, SparseLinearAlgebraLibrary, LAPACK, SuiteSparse, CXSparse, SchurSpecializations, Multithreading]
    -- Boost version: 1.65.1
    -- Found the following Boost libraries:
    --   program_options
    --   filesystem
    --   system
    --   unit_test_framework
    -- Found Eigen
    --   Includes : /usr/local/include/eigen3
    -- Found FreeImage
    --   Includes : /usr/include
    --   Libraries : /usr/lib/x86_64-linux-gnu/libfreeimage.so
    -- Found Glog
    --   Includes : /usr/include
    --   Libraries : /usr/lib/x86_64-linux-gnu/libglog.so
    -- Found Glew
    --   Includes : /usr/include
    --   Libraries : /usr/lib/x86_64-linux-gnu/libGLEW.so
    Compiling with AVX2 support.
    -- HDF5: Using hdf5 compiler wrapper to determine C configuration
    -- Boost version: 1.65.1
    -- Found the following Boost libraries:
    --   system
    --   serialization
    CMake Warning at third-party/HighFive/CMakeLists.txt:84 (message):
      Unit tests have been DISABLED.
    
    
    -- pybind11 v2.10.2
    -- Configuring done
    CMake Warning at third-party/pybind11/tools/pybind11Tools.cmake:177 (add_library):
      Cannot generate a safe runtime search path for target _pixsfm because files
      in some directories may conflict with libraries in implicit directories:
    
        runtime library [libz.so.1] in /usr/lib/x86_64-linux-gnu may be hidden by files in:
          /home/goodix/anaconda3/envs/gt/lib
    
      Some of these libraries may not be found correctly.
    Call Stack (most recent call first):
      cmake/CMakeHelper.cmake:136 (pybind11_add_module)
      pixsfm/CMakeLists.txt:22 (PIXSFM_ADD_PYMODULE)
    
    
    -- Generating done
    -- Build files have been written to: /home/goodix/qxg/groundtruth/pixsfm/pixel-perfect-sfm/build/temp.linux-x86_64-cpython-38
    [ 56%] Built target pixsfm
    [ 91%] Built target pypixsfm
    [ 95%] Linking CXX shared module ../../../pixsfm/_pixsfm.cpython-38-x86_64-linux-gnu.so
    /usr/bin/ld: /usr/local/share/colmap/../../lib/colmap/libpba.a(pba_generated_ProgramCU.cu.o): relocation R_X86_64_PC32 against symbol `_ZN3pba20jte_point_vec_kernelILi2ELi2EEEviiPf' can not be used when making a shared object; recompile with -fPIC
    /usr/bin/ld: 最后的链结失败: 错误的值
    collect2: error: ld returned 1 exit status
    pixsfm/CMakeFiles/_pixsfm.dir/build.make:156: recipe for target '../../pixsfm/_pixsfm.cpython-38-x86_64-linux-gnu.so' failed
    make[2]: *** [../../pixsfm/_pixsfm.cpython-38-x86_64-linux-gnu.so] Error 1
    CMakeFiles/Makefile2:308: recipe for target 'pixsfm/CMakeFiles/_pixsfm.dir/all' failed
    make[1]: *** [pixsfm/CMakeFiles/_pixsfm.dir/all] Error 2
    Makefile:129: recipe for target 'all' failed
    make: *** [all] Error 2
    /home/goodix/anaconda3/envs/gt/lib/python3.8/site-packages/setuptools/command/easy_install.py:144: EasyInstallDeprecationWarning: easy_install command is deprecated. Use build and pip and other standards-based tools.
      warnings.warn(
    /home/goodix/anaconda3/envs/gt/lib/python3.8/site-packages/setuptools/command/install.py:34: SetuptoolsDeprecationWarning: setup.py install is deprecated. Use build and pip and other standards-based tools.
      warnings.warn(
    Traceback (most recent call last):
      File "<string>", line 2, in <module>
      File "<pip-setuptools-caller>", line 34, in <module>
      File "/home/goodix/qxg/groundtruth/pixsfm/pixel-perfect-sfm/setup.py", line 112, in <module>
        setup(
      File "/home/goodix/anaconda3/envs/gt/lib/python3.8/site-packages/setuptools/__init__.py", line 87, in setup
        return distutils.core.setup(**attrs)
      File "/home/goodix/anaconda3/envs/gt/lib/python3.8/site-packages/setuptools/_distutils/core.py", line 185, in setup
        return run_commands(dist)
      File "/home/goodix/anaconda3/envs/gt/lib/python3.8/site-packages/setuptools/_distutils/core.py", line 201, in run_commands
        dist.run_commands()
      File "/home/goodix/anaconda3/envs/gt/lib/python3.8/site-packages/setuptools/_distutils/dist.py", line 968, in run_commands
        self.run_command(cmd)
      File "/home/goodix/anaconda3/envs/gt/lib/python3.8/site-packages/setuptools/dist.py", line 1217, in run_command
        super().run_command(command)
      File "/home/goodix/anaconda3/envs/gt/lib/python3.8/site-packages/setuptools/_distutils/dist.py", line 987, in run_command
        cmd_obj.run()
      File "/home/goodix/anaconda3/envs/gt/lib/python3.8/site-packages/setuptools/command/develop.py", line 34, in run
        self.install_for_development()
      File "/home/goodix/anaconda3/envs/gt/lib/python3.8/site-packages/setuptools/command/develop.py", line 114, in install_for_development
        self.run_command('build_ext')
      File "/home/goodix/anaconda3/envs/gt/lib/python3.8/site-packages/setuptools/_distutils/cmd.py", line 319, in run_command
        self.distribution.run_command(command)
      File "/home/goodix/anaconda3/envs/gt/lib/python3.8/site-packages/setuptools/dist.py", line 1217, in run_command
        super().run_command(command)
      File "/home/goodix/anaconda3/envs/gt/lib/python3.8/site-packages/setuptools/_distutils/dist.py", line 987, in run_command
        cmd_obj.run()
      File "/home/goodix/qxg/groundtruth/pixsfm/pixel-perfect-sfm/setup.py", line 55, in run
        self.build_extension(ext)
      File "/home/goodix/qxg/groundtruth/pixsfm/pixel-perfect-sfm/setup.py", line 98, in build_extension
        subprocess.check_call(['cmake', '--build', '.'] + build_args,
      File "/home/goodix/anaconda3/envs/gt/lib/python3.8/subprocess.py", line 364, in check_call
        raise CalledProcessError(retcode, cmd)
    subprocess.CalledProcessError: Command '['cmake', '--build', '.', '--config', 'Release', '--', '-j']' returned non-zero exit status 2.
    [end of output]

note: This error originates from a subprocess, and is likely not a problem with pip.

error: subprocess-exited-with-error

× python setup.py develop did not run successfully. │ exit code: 1 ╰─> [124 lines of output] running develop running egg_info writing pixsfm.egg-info/PKG-INFO writing dependency_links to pixsfm.egg-info/dependency_links.txt writing top-level names to pixsfm.egg-info/top_level.txt reading manifest file 'pixsfm.egg-info/SOURCES.txt' adding license file 'LICENSE' writing manifest file 'pixsfm.egg-info/SOURCES.txt' running build_ext -- A library with BLAS API found. -- Found AMD headers in: /usr/include/suitesparse -- Found AMD library: /usr/lib/x86_64-linux-gnu/libamd.so -- Found CAMD headers in: /usr/include/suitesparse -- Found CAMD library: /usr/lib/x86_64-linux-gnu/libcamd.so -- Found CCOLAMD headers in: /usr/include/suitesparse -- Found CCOLAMD library: /usr/lib/x86_64-linux-gnu/libccolamd.so -- Found CHOLMOD headers in: /usr/include/suitesparse -- Found CHOLMOD library: /usr/lib/x86_64-linux-gnu/libcholmod.so -- Found COLAMD headers in: /usr/include/suitesparse -- Found COLAMD library: /usr/lib/x86_64-linux-gnu/libcolamd.so -- Found SPQR headers in: /usr/include/suitesparse -- Found SPQR library: /usr/lib/x86_64-linux-gnu/libspqr.so -- Found Config headers in: /usr/include/suitesparse -- Found Config library: /usr/lib/x86_64-linux-gnu/libsuitesparseconfig.so -- Did not find Intel TBB library, assuming SuiteSparseQR was not compiled with TBB. -- Adding librt to SuiteSparse_config libraries (required on Linux & Unix [not OSX] if SuiteSparse is compiled with timing). -- Found required Ceres dependency: Eigen version 3.3.7 in /usr/local/share/eigen3/cmake -- Found required Ceres dependency: glog -- Found required Ceres dependency: gflags -- Found Ceres version: 2.1.0 installed in: /usr/local with components: [EigenSparse, SparseLinearAlgebraLibrary, LAPACK, SuiteSparse, CXSparse, SchurSpecializations, Multithreading] -- Boost version: 1.65.1 -- Found the following Boost libraries: -- program_options -- filesystem -- system -- unit_test_framework -- Found Eigen -- Includes : /usr/local/include/eigen3 -- Found FreeImage -- Includes : /usr/include -- Libraries : /usr/lib/x86_64-linux-gnu/libfreeimage.so -- Found Glog -- Includes : /usr/include -- Libraries : /usr/lib/x86_64-linux-gnu/libglog.so -- Found Glew -- Includes : /usr/include -- Libraries : /usr/lib/x86_64-linux-gnu/libGLEW.so Compiling with AVX2 support. -- HDF5: Using hdf5 compiler wrapper to determine C configuration -- Boost version: 1.65.1 -- Found the following Boost libraries: -- system -- serialization CMake Warning at third-party/HighFive/CMakeLists.txt:84 (message): Unit tests have been DISABLED.

-- pybind11 v2.10.2
-- Configuring done
CMake Warning at third-party/pybind11/tools/pybind11Tools.cmake:177 (add_library):
  Cannot generate a safe runtime search path for target _pixsfm because files
  in some directories may conflict with libraries in implicit directories:

    runtime library [libz.so.1] in /usr/lib/x86_64-linux-gnu may be hidden by files in:
      /home/goodix/anaconda3/envs/gt/lib

  Some of these libraries may not be found correctly.
Call Stack (most recent call first):
  cmake/CMakeHelper.cmake:136 (pybind11_add_module)
  pixsfm/CMakeLists.txt:22 (PIXSFM_ADD_PYMODULE)


-- Generating done
-- Build files have been written to: /home/goodix/qxg/groundtruth/pixsfm/pixel-perfect-sfm/build/temp.linux-x86_64-cpython-38
[ 56%] Built target pixsfm
[ 91%] Built target pypixsfm
[ 95%] Linking CXX shared module ../../../pixsfm/_pixsfm.cpython-38-x86_64-linux-gnu.so
/usr/bin/ld: /usr/local/share/colmap/../../lib/colmap/libpba.a(pba_generated_ProgramCU.cu.o): relocation R_X86_64_PC32 against symbol `_ZN3pba20jte_point_vec_kernelILi2ELi2EEEviiPf' can not be used when making a shared object; recompile with -fPIC
/usr/bin/ld: 最后的链结失败: 错误的值
collect2: error: ld returned 1 exit status
pixsfm/CMakeFiles/_pixsfm.dir/build.make:156: recipe for target '../../pixsfm/_pixsfm.cpython-38-x86_64-linux-gnu.so' failed
make[2]: *** [../../pixsfm/_pixsfm.cpython-38-x86_64-linux-gnu.so] Error 1
CMakeFiles/Makefile2:308: recipe for target 'pixsfm/CMakeFiles/_pixsfm.dir/all' failed
make[1]: *** [pixsfm/CMakeFiles/_pixsfm.dir/all] Error 2
Makefile:129: recipe for target 'all' failed
make: *** [all] Error 2
/home/goodix/anaconda3/envs/gt/lib/python3.8/site-packages/setuptools/command/easy_install.py:144: EasyInstallDeprecationWarning: easy_install command is deprecated. Use build and pip and other standards-based tools.
  warnings.warn(
/home/goodix/anaconda3/envs/gt/lib/python3.8/site-packages/setuptools/command/install.py:34: SetuptoolsDeprecationWarning: setup.py install is deprecated. Use build and pip and other standards-based tools.
  warnings.warn(
Traceback (most recent call last):
  File "<string>", line 2, in <module>
  File "<pip-setuptools-caller>", line 34, in <module>
  File "/home/goodix/qxg/groundtruth/pixsfm/pixel-perfect-sfm/setup.py", line 112, in <module>
    setup(
  File "/home/goodix/anaconda3/envs/gt/lib/python3.8/site-packages/setuptools/__init__.py", line 87, in setup
    return distutils.core.setup(**attrs)
  File "/home/goodix/anaconda3/envs/gt/lib/python3.8/site-packages/setuptools/_distutils/core.py", line 185, in setup
    return run_commands(dist)
  File "/home/goodix/anaconda3/envs/gt/lib/python3.8/site-packages/setuptools/_distutils/core.py", line 201, in run_commands
    dist.run_commands()
  File "/home/goodix/anaconda3/envs/gt/lib/python3.8/site-packages/setuptools/_distutils/dist.py", line 968, in run_commands
    self.run_command(cmd)
  File "/home/goodix/anaconda3/envs/gt/lib/python3.8/site-packages/setuptools/dist.py", line 1217, in run_command
    super().run_command(command)
  File "/home/goodix/anaconda3/envs/gt/lib/python3.8/site-packages/setuptools/_distutils/dist.py", line 987, in run_command
    cmd_obj.run()
  File "/home/goodix/anaconda3/envs/gt/lib/python3.8/site-packages/setuptools/command/develop.py", line 34, in run
    self.install_for_development()
  File "/home/goodix/anaconda3/envs/gt/lib/python3.8/site-packages/setuptools/command/develop.py", line 114, in install_for_development
    self.run_command('build_ext')
  File "/home/goodix/anaconda3/envs/gt/lib/python3.8/site-packages/setuptools/_distutils/cmd.py", line 319, in run_command
    self.distribution.run_command(command)
  File "/home/goodix/anaconda3/envs/gt/lib/python3.8/site-packages/setuptools/dist.py", line 1217, in run_command
    super().run_command(command)
  File "/home/goodix/anaconda3/envs/gt/lib/python3.8/site-packages/setuptools/_distutils/dist.py", line 987, in run_command
    cmd_obj.run()
  File "/home/goodix/qxg/groundtruth/pixsfm/pixel-perfect-sfm/setup.py", line 55, in run
    self.build_extension(ext)
  File "/home/goodix/qxg/groundtruth/pixsfm/pixel-perfect-sfm/setup.py", line 98, in build_extension
    subprocess.check_call(['cmake', '--build', '.'] + build_args,
  File "/home/goodix/anaconda3/envs/gt/lib/python3.8/subprocess.py", line 364, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['cmake', '--build', '.', '--config', 'Release', '--', '-j']' returned non-zero exit status 2.
[end of output]

note: This error originates from a subprocess, and is likely not a problem with pip.

when I run pip install -e . it happens with this error, can you tell me how to fix it? Thanks

Since low_memory.yaml cannot improve the accuracy of the camera poses a lot for my dataset(1764 images in a café), I am trying to use featuremetric strategy for large scale images. However, even if I provide enough cpu for it, the KA cost is -NAN for this strategy. Have you ever test featuremetric strategy for large scale? Is this strategy suitable for such situation? Here is my log:

1764 mapping images
[2022/12/15 11:18:55 hloc INFO] Extracting local features with configuration:
{'model': {'name': 'dog'},
 'output': 'feats-sift',
 'preprocessing': {'grayscale': True, 'resize_max': 1600}}
[2022/12/15 11:18:58 hloc INFO] Skipping the extraction.
[2022/12/15 11:18:59 hloc INFO] Found 1554966 pairs.
[2022/12/15 11:18:59 hloc INFO] Matching local features with configuration:
{'model': {'do_mutual_check': True,
           'name': 'nearest_neighbor',
           'ratio_threshold': 0.8},
 'output': 'matches-NN-mutual-ratio.8'}
[2022/12/15 11:21:18 hloc INFO] Skipping the matching.
[2022/12/15 11:21:22 pixsfm.features.models.s2dnet INFO] Loading S2DNet checkpoint at /data2/x00586938/pixel-perfect-sfm/pixsfm/features/models/checkpoints/s2dnet_weights.pth.
[2022/12/15 11:21:22 pixsfm INFO] Loaded dense extractor with configuration:
{'cache_format': 'chunked',
 'device': 'auto',
 'dtype': 'half',
 'fast_image_load': False,
 'l2_normalize': True,
 'load_cache_on_init': False,
 'max_edge': 1600,
 'model': {'name': 's2dnet'},
 'overwrite_cache': False,
 'patch_size': 16,
 'pyr_scales': [1.0],
 'resize': 'LANCZOS',
 'sparse': True,
 'use_cache': False}
[2022/12/15 11:48:05 pixsfm INFO] Building matching graph...
[2022/12/15 11:49:07 pixsfm INFO] Extracting dense features...
100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1744/1744 [15:26<00:00,  1.88it/s]
[2022/12/15 12:04:37 pixsfm INFO] Computing tracks...
[2022/12/15 12:04:37 pixsfm INFO] # graph nodes: 4093610
[2022/12/15 12:04:39 pixsfm INFO] # graph edges: 26582231
[2022/12/15 12:10:52 pixsfm INFO] # tracks: 195419
[2022/12/15 12:10:57 pixsfm INFO] Start feature-metric keypoint adjustment.
[2022/12/15 12:11:02 pixsfm WARNING] 15746 / 33210 problems have more than 50 keypoints.
         Maximum keypoints in a problem: 684
100%[████████████████████] 4093610/4093610 [19:13, 3549.15it/s]
    Residuals : -1963876342
   Parameters : 7742986
   Iterations : 100
         Time : 98505.3 [s]
 Initial cost : -nan
   Final cost : -nan
  Termination : Convergence

[2022/12/15 12:30:20 pixsfm INFO] KA Time: 1153.4s, cost change: -nan --> -nan
[2022/12/15 12:30:29 hloc INFO] Importing features into the database...
100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1764/1764 [00:01<00:00, 948.60it/s]
[2022/12/15 12:30:31 hloc INFO] Importing matches into the database...
100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1554966/1554966 [41:34<00:00, 623.45it/s]
[2022/12/15 13:12:09 hloc INFO] Performing geometric verification of the matches...
100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1763/1763 [4:58:01<00:00, 10.14s/it]
[2022/12/15 18:10:12 hloc INFO] mean/med/min/max valid matches 3.93/0.00/0.00/100.00%.
[2022/12/15 18:10:14 hloc INFO] Running 3D triangulation...
[2022/12/15 18:18:01 hloc INFO] Finished the triangulation with statistics:
Reconstruction:
        num_reg_images = 1764
        num_cameras = 1
        num_points3D = 281459
        num_observations = 2148514
        mean_track_length = 7.63349
        mean_observations_per_image = 1217.98
        mean_reprojection_error = 1.02626
[2022/12/15 18:18:03 pixsfm INFO] Extracting references.
100%[████████████████████] 281459/281459 [00:10, 27933.6it/s]
[2022/12/15 18:18:14 pixsfm INFO] Reference Extraction Time: 10.0746s
[2022/12/15 18:18:14 pixsfm INFO] Start feature-reference bundle adjustment.
  0%[                    ]   0/101 [00:00, -nanit/s]Aborted (core dumped)

Here is my config:

dense_features:
    sparse : true
    dtype : "half"
    use_cache: true
    overwrite_cache: true
    load_cache_on_init: false
    patch_size: 8
    cache_format: "chunked"
interpolation:
    nodes: [[0.0,0.0]]
    mode: "BICUBIC"
mapping:
  interpolation: ${interpolation}
  KA:
    strategy: featuremetric
    apply: true
    interpolation: ${..interpolation}
    level_indices: null
    max_kps_per_problem: 50
    optimizer:
      loss:
        name: cauchy
        params:
        - 0.25
      solver:
        function_tolerance: 0.0
        gradient_tolerance: 0.0
        parameter_tolerance: 1.0e-05
        minimizer_progress_to_stdout: false
        max_num_iterations: 100
        max_linear_solver_iterations: 200
        max_num_consecutive_invalid_steps: 10
        max_consecutive_nonmonotonic_steps: 10
        use_inner_iterations: false
        use_nonmonotonic_steps: true
        update_state_every_iteration: false
        num_threads: 1
      print_summary: true
      bound: 4.0
      num_threads: -1
    split_in_subproblems: true
  BA:
    apply: true
    interpolation: ${..interpolation}
    level_indices: null
    max_tracks_per_problem: 10
    num_threads: -1
    optimizer:
      loss:
        name: cauchy
        params:
        - 0.25
      solver:
        function_tolerance: 0.0
        gradient_tolerance: 0.0
        parameter_tolerance: 0.0
        minimizer_progress_to_stdout: false
        max_num_iterations: 100
        max_linear_solver_iterations: 200
        max_num_consecutive_invalid_steps: 10
        max_consecutive_nonmonotonic_steps: 10
        use_inner_iterations: true
        use_nonmonotonic_steps: true
        update_state_every_iteration: false
        num_threads: -1
      print_summary: true
      refine_focal_length: true
      refine_principal_point: true
      refine_extra_params: true
      refine_extrinsics: true
    references:
      loss:
        name: cauchy
        params:
        - 0.25
      iters: 100
      keep_observations: false
      compute_offsets3D: false
      num_threads: -1
    repeats: 1
    strategy: feature_reference
localization:
  interpolation: ${interpolation}
  target_reference: nearest
  unique_inliers: min_error
  references:
    loss:
      name: cauchy
      params:
      - 0.25
    iters: 100
    keep_observations: true
    compute_offsets3D: false
    num_threads: -1
  max_tracks_per_problem: 50
  QKA:
    apply: true
    feature_inlier_thresh: -1
    interpolation: ${..interpolation}
    level_indices: null
    overwrite_features_sparse: null
    stacked_correspondences: False
    optimizer:
      loss:
        name: trivial
        params: []
      solver:
        function_tolerance: 0.0
        gradient_tolerance: 0.0
        parameter_tolerance: 1.0e-05
        minimizer_progress_to_stdout: false
        max_num_iterations: 100
        max_linear_solver_iterations: 200
        max_num_consecutive_invalid_steps: 10
        max_consecutive_nonmonotonic_steps: 10
        use_inner_iterations: false
        use_nonmonotonic_steps: true
        update_state_every_iteration: false
        num_threads: -1
      print_summary: false
      bound: 4.0
  PnP:
    estimation:
      ransac:
        max_error: 12
    refinement: {}
  QBA:
    apply: true
    interpolation: ${..interpolation}
    level_indices: null
    optimizer:
      loss:
        name: cauchy
        params:
        - 0.25
      solver:
        function_tolerance: 0.0
        gradient_tolerance: 0.0
        parameter_tolerance: 0.0
        minimizer_progress_to_stdout: false
        max_num_iterations: 100
        max_linear_solver_iterations: 200
        max_num_consecutive_invalid_steps: 10
        max_consecutive_nonmonotonic_steps: 10
        use_inner_iterations: false
        use_nonmonotonic_steps: false
        update_state_every_iteration: false
        num_threads: -1
      print_summary: false
      refine_focal_length: false
      refine_principal_point: false
      refine_extra_params: false

Thanks for your awesome work!

Hi, I get the following error when building:

    -- Generating done
    -- Build files have been written to: /home/larsh/development/pixel-perfect-sfm/build/temp.linux-x86_64-cpython-39
    [ 56%] Built target pixsfm
    [ 60%] Building CXX object pixsfm/CMakeFiles/pypixsfm.dir/bundle_adjustment/bindings.cc.o
    In file included from /home/larsh/development/pixel-perfect-sfm/pixsfm/bundle_adjustment/src/costmap_extractor.h:18,
                     from /home/larsh/development/pixel-perfect-sfm/pixsfm/bundle_adjustment/bindings.cc:14:
    /home/larsh/development/pixel-perfect-sfm/pixsfm/bundle_adjustment/src/reference_extractor.h:331:32: error: redefinition of default argument for ‘int N_NODES’
      331 | template <int N_NODES = Eigen::Dynamic>
          |                                ^~~~~~~
    /home/larsh/development/pixel-perfect-sfm/pixsfm/bundle_adjustment/src/reference_extractor.h:52:32: note: original definition appeared here
       52 | template <int N_NODES = Eigen::Dynamic>
          |                                ^~~~~~~
    make[2]: *** [pixsfm/CMakeFiles/pypixsfm.dir/build.make:132: pixsfm/CMakeFiles/pypixsfm.dir/bundle_adjustment/bindings.cc.o] Error 1
    make[1]: *** [CMakeFiles/Makefile2:376: pixsfm/CMakeFiles/pypixsfm.dir/all] Error 2
    make: *** [Makefile:136: all] Error 2
    Traceback (most recent call last):
      File "<string>", line 2, in <module>
      File "<pip-setuptools-caller>", line 34, in <module>
      File "/home/larsh/development/pixel-perfect-sfm/setup.py", line 112, in <module>
        setup(
      File "/home/larsh/micromamba/envs/pp_sfm/lib/python3.9/site-packages/setuptools/__init__.py", line 87, in setup
        return distutils.core.setup(**attrs)
      File "/home/larsh/micromamba/envs/pp_sfm/lib/python3.9/site-packages/setuptools/_distutils/core.py", line 185, in setup
        return run_commands(dist)
      File "/home/larsh/micromamba/envs/pp_sfm/lib/python3.9/site-packages/setuptools/_distutils/core.py", line 201, in run_commands
        dist.run_commands()
      File "/home/larsh/micromamba/envs/pp_sfm/lib/python3.9/site-packages/setuptools/_distutils/dist.py", line 968, in run_commands
        self.run_command(cmd)
      File "/home/larsh/micromamba/envs/pp_sfm/lib/python3.9/site-packages/setuptools/dist.py", line 1217, in run_command
        super().run_command(command)
      File "/home/larsh/micromamba/envs/pp_sfm/lib/python3.9/site-packages/setuptools/_distutils/dist.py", line 987, in run_command
        cmd_obj.run()
      File "/home/larsh/micromamba/envs/pp_sfm/lib/python3.9/site-packages/setuptools/command/develop.py", line 34, in run
        self.install_for_development()
      File "/home/larsh/micromamba/envs/pp_sfm/lib/python3.9/site-packages/setuptools/command/develop.py", line 114, in install_for_development
        self.run_command('build_ext')
      File "/home/larsh/micromamba/envs/pp_sfm/lib/python3.9/site-packages/setuptools/_distutils/cmd.py", line 319, in run_command
        self.distribution.run_command(command)
      File "/home/larsh/micromamba/envs/pp_sfm/lib/python3.9/site-packages/setuptools/dist.py", line 1217, in run_command
        super().run_command(command)
      File "/home/larsh/micromamba/envs/pp_sfm/lib/python3.9/site-packages/setuptools/_distutils/dist.py", line 987, in run_command
        cmd_obj.run()
      File "/home/larsh/development/pixel-perfect-sfm/setup.py", line 55, in run
        self.build_extension(ext)
      File "/home/larsh/development/pixel-perfect-sfm/setup.py", line 98, in build_extension
        subprocess.check_call(['cmake', '--build', '.'] + build_args,
      File "/home/larsh/micromamba/envs/pp_sfm/lib/python3.9/subprocess.py", line 373, in check_call
        raise CalledProcessError(retcode, cmd)
    subprocess.CalledProcessError: Command '['cmake', '--build', '.', '--config', 'Release', '--', '-j']' returned non-zero exit status 2.
    [end of output]

gcc -v gcc version 11.3.0

I've tried demo with my own datasets. there are 239 images in datasets. everything works fine until "reconstruction ". My kernel always die in this process. what happened? how to solve this?

Hi.

I get the following error when testing the demo.ipynb on another small dataset:

IndexError                                Traceback (most recent call last)
/tmp/ipykernel_5075/3412010250.py in <cell line: 1>()
----> 1 raw.align_points(refined, max_error=0.005, min_inlier_ratio=0.9, min_overlap=3)

IndexError: _Map_base::at

Here's the summary of the reconstruction:

Raw Reconstruction:
	num_reg_images = 25
	num_cameras = 1
	num_points3D = 1262
	num_observations = 4097
	mean_track_length = 3.24643
	mean_observations_per_image = 163.88
	mean_reprojection_error = 1.35567
Refined Reconstruction:
	num_reg_images = 24
	num_cameras = 1
	num_points3D = 1259
	num_observations = 4241
	mean_track_length = 3.36855
	mean_observations_per_image = 176.708
	mean_reprojection_error = 1.292

It worked OK on the demo dataset but not on my dataset. What could be the problem here?