[CoRL 21'] TANDEM: Tracking and Dense Mapping in Real-time using Deep Multi-view Stereo

Related tags

Deep Learning slam 3d-reconstruction visual-odometry multiview-stereo
Overview

TANDEM: Tracking and Dense Mapping
in Real-time using Deep Multi-view Stereo

Lukas Koestler1*    Nan Yang1,2*,†    Niclas Zeller2,3    Daniel Cremers1,2

*equal contribution    †corresponding author

1Technical University of Munich    2Artisense
3Karlsruhe University of Applied Sciences

Conference on Robot Learning (CoRL) 2021, London, UK

3DV 2021 Best Demo Award

arXiv | Video | OpenReview | Project Page

Code and Data

  • 📣 CVA-MVSNet released! Please check cva_mvsnet/.
  • 📣 Replica training data released! Please check replica/.
  • C++ code realse before Christmas. Thank you for your patience!

Abstract

In this paper, we present TANDEM a real-time monocular tracking and dense mapping framework. For pose estimation, TANDEM performs photometric bundle adjustment based on a sliding window of keyframes. To increase the robustness, we propose a novel tracking front-end that performs dense direct image alignment using depth maps rendered from a global model that is built incrementally from dense depth predictions. To predict the dense depth maps, we propose Cascade View-Aggregation MVSNet (CVA-MVSNet) that utilizes the entire active keyframe window by hierarchically constructing 3D cost volumes with adaptive view aggregation to balance the different stereo baselines between the keyframes. Finally, the predicted depth maps are fused into a consistent global map represented as a truncated signed distance function (TSDF) voxel grid. Our experimental results show that TANDEM outperforms other state-of-the-art traditional and learning-based monocular visual odometry (VO) methods in terms of camera tracking. Moreover, TANDEM shows state-of-the-art real-time 3D reconstruction performance.

Poster

Comments
  • No such file or directory

    No such file or directory

    When I make the project, I get

    fatal error: cub/block/block_reduce.cuh: No such file or directory
 #include <cub/block/block_reduce.cuh>

    but I find this .cuh file in my computer. I don't know how to do next, does anyone have the same question? My environment: Ubuntu18.04, CUDA10.2

    opened by Xie-HIT 6
  • Use preset=gui didn't get mesh

    Use preset=gui didn't get mesh

    Hi, I can successfully generate the mesh obj on the Euroc dataset. My problem is when I set preset=gui I can get some frames showing on a window but didn't see the mesh in the same window. Could you let me know the culprit? Thanks. image

    opened by mathmax12 6
  • Cudatoolkit couldn't find

    Cudatoolkit couldn't find

    Hi,

    I installed CUDAToolkit 11.1.0 as per the nvidia official website. When I ran nvcc -V, the terminal displayed the text in the first

    image. I believe It means the installation is complete. However, when I ran 'cmake ..', an error happened(the second image).

    Can anyone help me solve this problem?

    Thank you! image

    image

    opened by Bobkk-k 5
  • make error related to Sophus

    make error related to Sophus

    Hi, I meet some problem related to sophus while building the project, the log is : [ 81%] Building CXX object CMakeFiles/tandem.dir/src/dvo-core/src/dense_tracking_config.cpp.o [ 83%] Building CXX object CMakeFiles/tandem.dir/src/dvo-core/src/util/histogram.cpp.o [ 84%] Building CXX object CMakeFiles/tandem.dir/src/dvo-core/src/dense_tracking_impl.cpp.o [ 86%] Building CXX object CMakeFiles/tandem.dir/src/IOWrapper/OpenCV/ImageDisplay_OpenCV.cpp.o [ 88%] Building CXX object CMakeFiles/tandem.dir/src/IOWrapper/Pangolin/KeyFrameDisplay.cpp.o [ 89%] Building CXX object CMakeFiles/tandem.dir/src/dvo-core/src/util/id_generator.cpp.o [ 91%] Building CXX object CMakeFiles/tandem.dir/src/IOWrapper/Pangolin/DrFrameDisplay.cpp.o [ 93%] Building CXX object CMakeFiles/tandem.dir/src/IOWrapper/OpenCV/ImageRW_OpenCV.cpp.o [ 94%] Building CXX object CMakeFiles/tandem.dir/src/IOWrapper/Pangolin/PangolinDSOViewer.cpp.o In file included from /home/hutiancheng/project/tandem/tandem/src/util/settings.cpp:27:0: /usr/local/include/boost/bind.hpp:41:1: note: #pragma message: The practice of declaring the Bind placeholders (1, 2, ...) in the global namespace is deprecated. Please use <boost/bind/bind.hpp> + using namespace boost::placeholders, or define BOOST_BIND_GLOBAL_PLACEHOLDERS to retain the current behavior. ) ^ [ 94%] Built target dr_mvsnet_test /home/hutiancheng/project/tandem/tandem/src/FullSystem/HessianBlocks.cpp: In member function ‘void dso::FrameHessian::setStateZero(const Vec10&)’: /home/hutiancheng/project/tandem/tandem/src/FullSystem/HessianBlocks.cpp:84:23: error: ‘template<class Scalar, int Options> class Sophus::SE3’ used without template parameters SE3 EepsP = Sophus::SE3::exp(eps); ^~~ /home/hutiancheng/project/tandem/tandem/src/FullSystem/HessianBlocks.cpp:85:23: error: ‘template<class Scalar, int Options> class Sophus::SE3’ used without template parameters SE3 EepsM = Sophus::SE3::exp(-eps); ^~~ /home/hutiancheng/project/tandem/tandem/src/IOWrapper/Pangolin/PangolinDSOViewer.cpp: In member function ‘void dso::IOWrap::PangolinDSOViewer::run()’: /home/hutiancheng/project/tandem/tandem/src/IOWrapper/Pangolin/PangolinDSOViewer.cpp:280:27: error: ‘glDrawColoredVertices’ is not a member of ‘pangolin’ pangolin::glDrawColoredVertices( ^~~~~~~~~~~~~~~~~~~~~ /home/hutiancheng/project/tandem/tandem/src/IOWrapper/Pangolin/PangolinDSOViewer.cpp:280:27: note: suggested alternative: ‘glDrawColouredCube’ pangolin::glDrawColoredVertices( ^~~~~~~~~~~~~~~~~~~~~ glDrawColouredCube /home/hutiancheng/project/tandem/tandem/src/FullSystem/FullSystem.cpp: In member function ‘dso::Vec4 dso::FullSystem::trackNewCoarse(dso::FrameHessian*)’: /home/hutiancheng/project/tandem/tandem/src/FullSystem/FullSystem.cpp:452:46: error: ‘Quaterniond’ is not a member of ‘Sophus’ SE3(Sophus::Quaterniond(1, rotDelta, 0, 0), ^~~~~~~~~~~ /home/hutiancheng/project/tandem/tandem/src/FullSystem/FullSystem.cpp:452:46: note: suggested alternative: In file included from /usr/local/include/eigen3/Eigen/Geometry:42:0, from /usr/local/include/eigen3/Eigen/Dense:6, from /usr/local/include/sophus/rotation_matrix.hpp:7, from /usr/local/include/sophus/so3.hpp:7, from /usr/local/include/sophus/rxso3.hpp:7, from /usr/local/include/sophus/sim3.hpp:7, from /home/hutiancheng/project/tandem/tandem/src/util/NumType.h:28, from /home/hutiancheng/project/tandem/tandem/src/FullSystem/FullSystem.h:29, from /home/hutiancheng/project/tandem/tandem/src/FullSystem/FullSystem.cpp:32: /usr/local/include/eigen3/Eigen/src/Geometry/Quaternion.h:310:28: note: ‘Eigen::Quaterniond’ typedef Quaternion Quaterniond; ^~~~~~~~~~~ /home/hutiancheng/project/tandem/tandem/src/FullSystem/FullSystem.cpp:455:46: error: ‘Quaterniond’ is not a member of ‘Sophus’ SE3(Sophus::Quaterniond(1, 0, rotDelta, 0), ^~~~~~~~~~~ /home/hutiancheng/project/tandem/tandem/src/FullSystem/FullSystem.cpp:455:46: note: suggested alternative: In file included from /usr/local/include/eigen3/Eigen/Geometry:42:0, from /usr/local/include/eigen3/Eigen/Dense:6, from /usr/local/include/sophus/rotation_matrix.hpp:7, from /usr/local/include/sophus/so3.hpp:7, from /usr/local/include/sophus/rxso3.hpp:7, from /usr/local/include/sophus/sim3.hpp:7, from /home/hutiancheng/project/tandem/tandem/src/util/NumType.h:28, from /home/hutiancheng/project/tandem/tandem/src/FullSystem/FullSystem.h:29, from /home/hutiancheng/project/tandem/tandem/src/FullSystem/FullSystem.cpp:32: /usr/local/include/eigen3/Eigen/src/Geometry/Quaternion.h:310:28: note: ‘Eigen::Quaterniond’ typedef Quaternion Quaterniond; ^~~~~~~~~~~ /home/hutiancheng/project/tandem/tandem/src/FullSystem/FullSystem.cpp:458:46: error: ‘Quaterniond’ is not a member of ‘Sophus’ SE3(Sophus::Quaterniond(1, 0, 0, rotDelta), ^~~~~~~~~~~ /home/hutiancheng/project/tandem/tandem/src/FullSystem/FullSystem.cpp:458:46: note: suggested alternative: In file included from /usr/local/include/eigen3/Eigen/Geometry:42:0, from /usr/local/include/eigen3/Eigen/Dense:6, from /usr/local/include/sophus/rotation_matrix.hpp:7, from /usr/local/include/sophus/so3.hpp:7, from /usr/local/include/sophus/rxso3.hpp:7, from /usr/local/include/sophus/sim3.hpp:7, from /home/hutiancheng/project/tandem/tandem/src/util/NumType.h:28, from /home/hutiancheng/project/tandem/tandem/src/FullSystem/FullSystem.h:29, from /home/hutiancheng/project/tandem/tandem/src/FullSystem/FullSystem.cpp:32: /usr/local/include/eigen3/Eigen/src/Geometry/Quaternion.h:310:28: note: ‘Eigen::Quaterniond’ typedef Quaternion Quaterniond; ^~~~~~~~~~~ /home/hutiancheng/project/tandem/tandem/src/FullSystem/FullSystem.cpp:461:46: error: ‘Quaterniond’ is not a member of ‘Sophus’ SE3(Sophus::Quaterniond(1, -rotDelta, 0, 0), ^~~~~~~~~~~ /home/hutiancheng/project/tandem/tandem/src/FullSystem/FullSystem.cpp:461:46: note: suggested alternative: In file included from /usr/local/include/eigen3/Eigen/Geometry:42:0, from /usr/local/include/eigen3/Eigen/Dense:6, from /usr/local/include/sophus/rotation_matrix.hpp:7, from /usr/local/include/sophus/so3.hpp:7, from /usr/local/include/sophus/rxso3.hpp:7, from /usr/local/include/sophus/sim3.hpp:7, from /home/hutiancheng/project/tandem/tandem/src/util/NumType.h:28, from /home/hutiancheng/project/tandem/tandem/src/FullSystem/FullSystem.h:29, from /home/hutiancheng/project/tandem/tandem/src/FullSystem/FullSystem.cpp:32: /usr/local/include/eigen3/Eigen/src/Geometry/Quaternion.h:310:28: note: ‘Eigen::Quaterniond’ typedef Quaternion Quaterniond; ^~~~~~~~~~~ /home/hutiancheng/project/tandem/tandem/src/FullSystem/FullSystem.cpp:464:46: error: ‘Quaterniond’ is not a member of ‘Sophus’ SE3(Sophus::Quaterniond(1, 0, -rotDelta, 0), ^~~~~~~~~~~ /home/hutiancheng/project/tandem/tandem/src/FullSystem/FullSystem.cpp:464:46: note: suggested alternative: In file included from /usr/local/include/eigen3/Eigen/Geometry:42:0, from /usr/local/include/eigen3/Eigen/Dense:6, from /usr/local/include/sophus/rotation_matrix.hpp:7, from /usr/local/include/sophus/so3.hpp:7, from /usr/local/include/sophus/rxso3.hpp:7, from /usr/local/include/sophus/sim3.hpp:7, from /home/hutiancheng/project/tandem/tandem/src/util/NumType.h:28, from /home/hutiancheng/project/tandem/tandem/src/FullSystem/FullSystem.h:29, from /home/hutiancheng/project/tandem/tandem/src/FullSystem/FullSystem.cpp:32: /usr/local/include/eigen3/Eigen/src/Geometry/Quaternion.h:310:28: note: ‘Eigen::Quaterniond’ typedef Quaternion Quaterniond; ^~~~~~~~~~~ /home/hutiancheng/project/tandem/tandem/src/FullSystem/FullSystem.cpp:467:46: error: ‘Quaterniond’ is not a member of ‘Sophus’ SE3(Sophus::Quaterniond(1, 0, 0, -rotDelta), ^~~~~~~~~~~ /home/hutiancheng/project/tandem/tandem/src/FullSystem/FullSystem.cpp:467:46: note: suggested alternative: In file included from /usr/local/include/eigen3/Eigen/Geometry:42:0, from /usr/local/include/eigen3/Eigen/Dense:6, from /usr/local/include/sophus/rotation_matrix.hpp:7, from /usr/local/include/sophus/so3.hpp:7, from /usr/local/include/sophus/rxso3.hpp:7, from /usr/local/include/sophus/sim3.hpp:7, from /home/hutiancheng/project/tandem/tandem/src/util/NumType.h:28, from /home/hutiancheng/project/tandem/tandem/src/FullSystem/FullSystem.h:29, from /home/hutiancheng/project/tandem/tandem/src/FullSystem/FullSystem.cpp:32: /usr/local/include/eigen3/Eigen/src/Geometry/Quaternion.h:310:28: note: ‘Eigen::Quaterniond’ typedef Quaternion Quaterniond; ^~~~~~~~~~~ /home/hutiancheng/project/tandem/tandem/src/FullSystem/FullSystem.cpp:470:46: error: ‘Quaterniond’ is not a member of ‘Sophus’ SE3(Sophus::Quaterniond(1, rotDelta, rotDelta, 0), ^~~~~~~~~~~ /home/hutiancheng/project/tandem/tandem/src/FullSystem/FullSystem.cpp:470:46: note: suggested alternative: In file included from /usr/local/include/eigen3/Eigen/Geometry:42:0, from /usr/local/include/eigen3/Eigen/Dense:6, from /usr/local/include/sophus/rotation_matrix.hpp:7, from /usr/local/include/sophus/so3.hpp:7, from /usr/local/include/sophus/rxso3.hpp:7, from /usr/local/include/sophus/sim3.hpp:7, from /home/hutiancheng/project/tandem/tandem/src/util/NumType.h:28, from /home/hutiancheng/project/tandem/tandem/src/FullSystem/FullSystem.h:29, from /home/hutiancheng/project/tandem/tandem/src/FullSystem/FullSystem.cpp:32: /usr/local/include/eigen3/Eigen/src/Geometry/Quaternion.h:310:28: note: ‘Eigen::Quaterniond’ typedef Quaternion Quaterniond; ^~~~~~~~~~~ /home/hutiancheng/project/tandem/tandem/src/FullSystem/FullSystem.cpp:473:46: error: ‘Quaterniond’ is not a member of ‘Sophus’ SE3(Sophus::Quaterniond(1, 0, rotDelta, rotDelta), ^~~~~~~~~~~ /home/hutiancheng/project/tandem/tandem/src/FullSystem/FullSystem.cpp:473:46: note: suggested alternative: In file included from /usr/local/include/eigen3/Eigen/Geometry:42:0, from /usr/local/include/eigen3/Eigen/Dense:6, from /usr/local/include/sophus/rotation_matrix.hpp:7, from /usr/local/include/sophus/so3.hpp:7, from /usr/local/include/sophus/rxso3.hpp:7, from /usr/local/include/sophus/sim3.hpp:7, from /home/hutiancheng/project/tandem/tandem/src/util/NumType.h:28, from /home/hutiancheng/project/tandem/tandem/src/FullSystem/FullSystem.h:29, from /home/hutiancheng/project/tandem/tandem/src/FullSystem/FullSystem.cpp:32: /usr/local/include/eigen3/Eigen/src/Geometry/Quaternion.h:310:28: note: ‘Eigen::Quaterniond’ typedef Quaternion Quaterniond; ^~~~~~~~~~~ /home/hutiancheng/project/tandem/tandem/src/FullSystem/FullSystem.cpp:476:46: error: ‘Quaterniond’ is not a member of ‘Sophus’ SE3(Sophus::Quaterniond(1, rotDelta, 0, rotDelta), ^~~~~~~~~~~ /home/hutiancheng/project/tandem/tandem/src/FullSystem/FullSystem.cpp:476:46: note: suggested alternative: In file included from /usr/local/include/eigen3/Eigen/Geometry:42:0, from /usr/local/include/eigen3/Eigen/Dense:6, from /usr/local/include/sophus/rotation_matrix.hpp:7, from /usr/local/include/sophus/so3.hpp:7, from /usr/local/include/sophus/rxso3.hpp:7, from /usr/local/include/sophus/sim3.hpp:7, from /home/hutiancheng/project/tandem/tandem/src/util/NumType.h:28, from /home/hutiancheng/project/tandem/tandem/src/FullSystem/FullSystem.h:29, from /home/hutiancheng/project/tandem/tandem/src/FullSystem/FullSystem.cpp:32: /usr/local/include/eigen3/Eigen/src/Geometry/Quaternion.h:310:28: note: ‘Eigen::Quaterniond’ typedef Quaternion Quaterniond; ^~~~~~~~~~~ /home/hutiancheng/project/tandem/tandem/src/FullSystem/FullSystem.cpp:479:46: error: ‘Quaterniond’ is not a member of ‘Sophus’ SE3(Sophus::Quaterniond(1, -rotDelta, rotDelta, 0), ^~~~~~~~~~~ /home/hutiancheng/project/tandem/tandem/src/FullSystem/FullSystem.cpp:479:46: note: suggested alternative: In file included from /usr/local/include/eigen3/Eigen/Geometry:42:0, from /usr/local/include/eigen3/Eigen/Dense:6, from /usr/local/include/sophus/rotation_matrix.hpp:7, from /usr/local/include/sophus/so3.hpp:7, from /usr/local/include/sophus/rxso3.hpp:7, from /usr/local/include/sophus/sim3.hpp:7, from /home/hutiancheng/project/tandem/tandem/src/util/NumType.h:28, from /home/hutiancheng/project/tandem/tandem/src/FullSystem/FullSystem.h:29, from /home/hutiancheng/project/tandem/tandem/src/FullSystem/FullSystem.cpp:32: /usr/local/include/eigen3/Eigen/src/Geometry/Quaternion.h:310:28: note: ‘Eigen::Quaterniond’ typedef Quaternion Quaterniond; ^~~~~~~~~~~ /home/hutiancheng/project/tandem/tandem/src/FullSystem/FullSystem.cpp:482:46: error: ‘Quaterniond’ is not a member of ‘Sophus’ SE3(Sophus::Quaterniond(1, 0, -rotDelta, rotDelta), ^~~~~~~~~~~ /home/hutiancheng/project/tandem/tandem/src/FullSystem/FullSystem.cpp:482:46: note: suggested alternative: In file included from /usr/local/include/eigen3/Eigen/Geometry:42:0, from /usr/local/include/eigen3/Eigen/Dense:6, from /usr/local/include/sophus/rotation_matrix.hpp:7, from /usr/local/include/sophus/so3.hpp:7, from /usr/local/include/sophus/rxso3.hpp:7, from /usr/local/include/sophus/sim3.hpp:7, from /home/hutiancheng/project/tandem/tandem/src/util/NumType.h:28, from /home/hutiancheng/project/tandem/tandem/src/FullSystem/FullSystem.h:29, from /home/hutiancheng/project/tandem/tandem/src/FullSystem/FullSystem.cpp:32: /usr/local/include/eigen3/Eigen/src/Geometry/Quaternion.h:310:28: note: ‘Eigen::Quaterniond’ typedef Quaternion Quaterniond; ^~~~~~~~~~~ /home/hutiancheng/project/tandem/tandem/src/FullSystem/FullSystem.cpp:485:46: error: ‘Quaterniond’ is not a member of ‘Sophus’ SE3(Sophus::Quaterniond(1, -rotDelta, 0, rotDelta), ^~~~~~~~~~~ /home/hutiancheng/project/tandem/tandem/src/FullSystem/FullSystem.cpp:485:46: note: suggested alternative: In file included from /usr/local/include/eigen3/Eigen/Geometry:42:0, from /usr/local/include/eigen3/Eigen/Dense:6, from /usr/local/include/sophus/rotation_matrix.hpp:7, from /usr/local/include/sophus/so3.hpp:7, from /usr/local/include/sophus/rxso3.hpp:7, from /usr/local/include/sophus/sim3.hpp:7, from /home/hutiancheng/project/tandem/tandem/src/util/NumType.h:28, from /home/hutiancheng/project/tandem/tandem/src/FullSystem/FullSystem.h:29, from /home/hutiancheng/project/tandem/tandem/src/FullSystem/FullSystem.cpp:32: /usr/local/include/eigen3/Eigen/src/Geometry/Quaternion.h:310:28: note: ‘Eigen::Quaterniond’ typedef Quaternion Quaterniond; ^~~~~~~~~~~ /home/hutiancheng/project/tandem/tandem/src/FullSystem/FullSystem.cpp:488:46: error: ‘Quaterniond’ is not a member of ‘Sophus’ SE3(Sophus::Quaterniond(1, rotDelta, -rotDelta, 0), ^~~~~~~~~~~ /home/hutiancheng/project/tandem/tandem/src/FullSystem/FullSystem.cpp:488:46: note: suggested alternative: In file included from /usr/local/include/eigen3/Eigen/Geometry:42:0, from /usr/local/include/eigen3/Eigen/Dense:6, from /usr/local/include/sophus/rotation_matrix.hpp:7, from /usr/local/include/sophus/so3.hpp:7, from /usr/local/include/sophus/rxso3.hpp:7, from /usr/local/include/sophus/sim3.hpp:7, from /home/hutiancheng/project/tandem/tandem/src/util/NumType.h:28, from /home/hutiancheng/project/tandem/tandem/src/FullSystem/FullSystem.h:29, from /home/hutiancheng/project/tandem/tandem/src/FullSystem/FullSystem.cpp:32: /usr/local/include/eigen3/Eigen/src/Geometry/Quaternion.h:310:28: note: ‘Eigen::Quaterniond’ typedef Quaternion Quaterniond; ^~~~~~~~~~~ /home/hutiancheng/project/tandem/tandem/src/FullSystem/FullSystem.cpp:491:46: error: ‘Quaterniond’ is not a member of ‘Sophus’ SE3(Sophus::Quaterniond(1, 0, rotDelta, -rotDelta), ^~~~~~~~~~~ /home/hutiancheng/project/tandem/tandem/src/FullSystem/FullSystem.cpp:491:46: note: suggested alternative: In file included from /usr/local/include/eigen3/Eigen/Geometry:42:0, from /usr/local/include/eigen3/Eigen/Dense:6, from /usr/local/include/sophus/rotation_matrix.hpp:7, from /usr/local/include/sophus/so3.hpp:7, from /usr/local/include/sophus/rxso3.hpp:7, from /usr/local/include/sophus/sim3.hpp:7, from /home/hutiancheng/project/tandem/tandem/src/util/NumType.h:28, from /home/hutiancheng/project/tandem/tandem/src/FullSystem/FullSystem.h:29, from /home/hutiancheng/project/tandem/tandem/src/FullSystem/FullSystem.cpp:32: /usr/local/include/eigen3/Eigen/src/Geometry/Quaternion.h:310:28: note: ‘Eigen::Quaterniond’ typedef Quaternion Quaterniond; ^~~~~~~~~~~ /home/hutiancheng/project/tandem/tandem/src/FullSystem/FullSystem.cpp:494:46: error: ‘Quaterniond’ is not a member of ‘Sophus’ SE3(Sophus::Quaterniond(1, rotDelta, 0, -rotDelta), ^~~~~~~~~~~ /home/hutiancheng/project/tandem/tandem/src/FullSystem/FullSystem.cpp:494:46: note: suggested alternative: In file included from /usr/local/include/eigen3/Eigen/Geometry:42:0, from /usr/local/include/eigen3/Eigen/Dense:6, from /usr/local/include/sophus/rotation_matrix.hpp:7, from /usr/local/include/sophus/so3.hpp:7, from /usr/local/include/sophus/rxso3.hpp:7, from /usr/local/include/sophus/sim3.hpp:7, from /home/hutiancheng/project/tandem/tandem/src/util/NumType.h:28, from /home/hutiancheng/project/tandem/tandem/src/FullSystem/FullSystem.h:29, from /home/hutiancheng/project/tandem/tandem/src/FullSystem/FullSystem.cpp:32: /usr/local/include/eigen3/Eigen/src/Geometry/Quaternion.h:310:28: note: ‘Eigen::Quaterniond’ typedef Quaternion Quaterniond; ^~~~~~~~~~~ /home/hutiancheng/project/tandem/tandem/src/FullSystem/FullSystem.cpp:497:46: error: ‘Quaterniond’ is not a member of ‘Sophus’ SE3(Sophus::Quaterniond(1, -rotDelta, -rotDelta, 0), ^~~~~~~~~~~ /home/hutiancheng/project/tandem/tandem/src/FullSystem/FullSystem.cpp:497:46: note: suggested alternative: In file included from /usr/local/include/eigen3/Eigen/Geometry:42:0, from /usr/local/include/eigen3/Eigen/Dense:6, from /usr/local/include/sophus/rotation_matrix.hpp:7, from /usr/local/include/sophus/so3.hpp:7, from /usr/local/include/sophus/rxso3.hpp:7, from /usr/local/include/sophus/sim3.hpp:7, from /home/hutiancheng/project/tandem/tandem/src/util/NumType.h:28, from /home/hutiancheng/project/tandem/tandem/src/FullSystem/FullSystem.h:29, from /home/hutiancheng/project/tandem/tandem/src/FullSystem/FullSystem.cpp:32: /usr/local/include/eigen3/Eigen/src/Geometry/Quaternion.h:310:28: note: ‘Eigen::Quaterniond’ typedef Quaternion Quaterniond; ^~~~~~~~~~~ /home/hutiancheng/project/tandem/tandem/src/FullSystem/FullSystem.cpp:500:46: error: ‘Quaterniond’ is not a member of ‘Sophus’ SE3(Sophus::Quaterniond(1, 0, -rotDelta, -rotDelta), ^~~~~~~~~~~ /home/hutiancheng/project/tandem/tandem/src/FullSystem/FullSystem.cpp:500:46: note: suggested alternative: In file included from /usr/local/include/eigen3/Eigen/Geometry:42:0, from /usr/local/include/eigen3/Eigen/Dense:6, from /usr/local/include/sophus/rotation_matrix.hpp:7, from /usr/local/include/sophus/so3.hpp:7, from /usr/local/include/sophus/rxso3.hpp:7, from /usr/local/include/sophus/sim3.hpp:7, from /home/hutiancheng/project/tandem/tandem/src/util/NumType.h:28, from /home/hutiancheng/project/tandem/tandem/src/FullSystem/FullSystem.h:29, from /home/hutiancheng/project/tandem/tandem/src/FullSystem/FullSystem.cpp:32: /usr/local/include/eigen3/Eigen/src/Geometry/Quaternion.h:310:28: note: ‘Eigen::Quaterniond’ typedef Quaternion Quaterniond; ^~~~~~~~~~~ /home/hutiancheng/project/tandem/tandem/src/FullSystem/FullSystem.cpp:503:46: error: ‘Quaterniond’ is not a member of ‘Sophus’ SE3(Sophus::Quaterniond(1, -rotDelta, 0, -rotDelta), ^~~~~~~~~~~ /home/hutiancheng/project/tandem/tandem/src/FullSystem/FullSystem.cpp:503:46: note: suggested alternative: In file included from /usr/local/include/eigen3/Eigen/Geometry:42:0, from /usr/local/include/eigen3/Eigen/Dense:6, from /usr/local/include/sophus/rotation_matrix.hpp:7, from /usr/local/include/sophus/so3.hpp:7, from /usr/local/include/sophus/rxso3.hpp:7, from /usr/local/include/sophus/sim3.hpp:7, from /home/hutiancheng/project/tandem/tandem/src/util/NumType.h:28, from /home/hutiancheng/project/tandem/tandem/src/FullSystem/FullSystem.h:29, from /home/hutiancheng/project/tandem/tandem/src/FullSystem/FullSystem.cpp:32: /usr/local/include/eigen3/Eigen/src/Geometry/Quaternion.h:310:28: note: ‘Eigen::Quaterniond’ typedef Quaternion Quaterniond; ^~~~~~~~~~~ /home/hutiancheng/project/tandem/tandem/src/FullSystem/FullSystem.cpp:506:46: error: ‘Quaterniond’ is not a member of ‘Sophus’ SE3(Sophus::Quaterniond(1, -rotDelta, -rotDelta, -rotDelta), ^~~~~~~~~~~ /home/hutiancheng/project/tandem/tandem/src/FullSystem/FullSystem.cpp:506:46: note: suggested alternative: In file included from /usr/local/include/eigen3/Eigen/Geometry:42:0, from /usr/local/include/eigen3/Eigen/Dense:6, from /usr/local/include/sophus/rotation_matrix.hpp:7, from /usr/local/include/sophus/so3.hpp:7, from /usr/local/include/sophus/rxso3.hpp:7, from /usr/local/include/sophus/sim3.hpp:7, from /home/hutiancheng/project/tandem/tandem/src/util/NumType.h:28, from /home/hutiancheng/project/tandem/tandem/src/FullSystem/FullSystem.h:29, from /home/hutiancheng/project/tandem/tandem/src/FullSystem/FullSystem.cpp:32: /usr/local/include/eigen3/Eigen/src/Geometry/Quaternion.h:310:28: note: ‘Eigen::Quaterniond’ typedef Quaternion Quaterniond; ^~~~~~~~~~~ /home/hutiancheng/project/tandem/tandem/src/FullSystem/FullSystem.cpp:509:46: error: ‘Quaterniond’ is not a member of ‘Sophus’ SE3(Sophus::Quaterniond(1, -rotDelta, -rotDelta, rotDelta), ^~~~~~~~~~~ /home/hutiancheng/project/tandem/tandem/src/FullSystem/FullSystem.cpp:509:46: note: suggested alternative: In file included from /usr/local/include/eigen3/Eigen/Geometry:42:0, from /usr/local/include/eigen3/Eigen/Dense:6, from /usr/local/include/sophus/rotation_matrix.hpp:7, from /usr/local/include/sophus/so3.hpp:7, from /usr/local/include/sophus/rxso3.hpp:7, from /usr/local/include/sophus/sim3.hpp:7, from /home/hutiancheng/project/tandem/tandem/src/util/NumType.h:28, from /home/hutiancheng/project/tandem/tandem/src/FullSystem/FullSystem.h:29, from /home/hutiancheng/project/tandem/tandem/src/FullSystem/FullSystem.cpp:32: /usr/local/include/eigen3/Eigen/src/Geometry/Quaternion.h:310:28: note: ‘Eigen::Quaterniond’ typedef Quaternion Quaterniond; ^~~~~~~~~~~ /home/hutiancheng/project/tandem/tandem/src/FullSystem/FullSystem.cpp:512:46: error: ‘Quaterniond’ is not a member of ‘Sophus’ SE3(Sophus::Quaterniond(1, -rotDelta, rotDelta, -rotDelta), ^~~~~~~~~~~ /home/hutiancheng/project/tandem/tandem/src/FullSystem/FullSystem.cpp:512:46: note: suggested alternative: In file included from /usr/local/include/eigen3/Eigen/Geometry:42:0, from /usr/local/include/eigen3/Eigen/Dense:6, from /usr/local/include/sophus/rotation_matrix.hpp:7, from /usr/local/include/sophus/so3.hpp:7, from /usr/local/include/sophus/rxso3.hpp:7, from /usr/local/include/sophus/sim3.hpp:7, from /home/hutiancheng/project/tandem/tandem/src/util/NumType.h:28, from /home/hutiancheng/project/tandem/tandem/src/FullSystem/FullSystem.h:29, from /home/hutiancheng/project/tandem/tandem/src/FullSystem/FullSystem.cpp:32: /usr/local/include/eigen3/Eigen/src/Geometry/Quaternion.h:310:28: note: ‘Eigen::Quaterniond’ typedef Quaternion Quaterniond; ^~~~~~~~~~~ /home/hutiancheng/project/tandem/tandem/src/FullSystem/FullSystem.cpp:515:46: error: ‘Quaterniond’ is not a member of ‘Sophus’ SE3(Sophus::Quaterniond(1, -rotDelta, rotDelta, rotDelta), ^~~~~~~~~~~ /home/hutiancheng/project/tandem/tandem/src/FullSystem/FullSystem.cpp:515:46: note: suggested alternative: In file included from /usr/local/include/eigen3/Eigen/Geometry:42:0, from /usr/local/include/eigen3/Eigen/Dense:6, from /usr/local/include/sophus/rotation_matrix.hpp:7, from /usr/local/include/sophus/so3.hpp:7, from /usr/local/include/sophus/rxso3.hpp:7, from /usr/local/include/sophus/sim3.hpp:7, from /home/hutiancheng/project/tandem/tandem/src/util/NumType.h:28, from /home/hutiancheng/project/tandem/tandem/src/FullSystem/FullSystem.h:29, from /home/hutiancheng/project/tandem/tandem/src/FullSystem/FullSystem.cpp:32: /usr/local/include/eigen3/Eigen/src/Geometry/Quaternion.h:310:28: note: ‘Eigen::Quaterniond’ typedef Quaternion Quaterniond; ^~~~~~~~~~~ /home/hutiancheng/project/tandem/tandem/src/FullSystem/FullSystem.cpp:518:46: error: ‘Quaterniond’ is not a member of ‘Sophus’ SE3(Sophus::Quaterniond(1, rotDelta, -rotDelta, -rotDelta), ^~~~~~~~~~~ /home/hutiancheng/project/tandem/tandem/src/FullSystem/FullSystem.cpp:518:46: note: suggested alternative: In file included from /usr/local/include/eigen3/Eigen/Geometry:42:0, from /usr/local/include/eigen3/Eigen/Dense:6, from /usr/local/include/sophus/rotation_matrix.hpp:7, from /usr/local/include/sophus/so3.hpp:7, from /usr/local/include/sophus/rxso3.hpp:7, from /usr/local/include/sophus/sim3.hpp:7, from /home/hutiancheng/project/tandem/tandem/src/util/NumType.h:28, from /home/hutiancheng/project/tandem/tandem/src/FullSystem/FullSystem.h:29, from /home/hutiancheng/project/tandem/tandem/src/FullSystem/FullSystem.cpp:32: /usr/local/include/eigen3/Eigen/src/Geometry/Quaternion.h:310:28: note: ‘Eigen::Quaterniond’ typedef Quaternion Quaterniond; ^~~~~~~~~~~ /home/hutiancheng/project/tandem/tandem/src/FullSystem/FullSystem.cpp:521:46: error: ‘Quaterniond’ is not a member of ‘Sophus’ SE3(Sophus::Quaterniond(1, rotDelta, -rotDelta, rotDelta), ^~~~~~~~~~~ /home/hutiancheng/project/tandem/tandem/src/FullSystem/FullSystem.cpp:521:46: note: suggested alternative: In file included from /usr/local/include/eigen3/Eigen/Geometry:42:0, from /usr/local/include/eigen3/Eigen/Dense:6, from /usr/local/include/sophus/rotation_matrix.hpp:7, from /usr/local/include/sophus/so3.hpp:7, from /usr/local/include/sophus/rxso3.hpp:7, from /usr/local/include/sophus/sim3.hpp:7, from /home/hutiancheng/project/tandem/tandem/src/util/NumType.h:28, from /home/hutiancheng/project/tandem/tandem/src/FullSystem/FullSystem.h:29, from /home/hutiancheng/project/tandem/tandem/src/FullSystem/FullSystem.cpp:32: /usr/local/include/eigen3/Eigen/src/Geometry/Quaternion.h:310:28: note: ‘Eigen::Quaterniond’ typedef Quaternion Quaterniond; ^~~~~~~~~~~ /home/hutiancheng/project/tandem/tandem/src/FullSystem/FullSystem.cpp:524:46: error: ‘Quaterniond’ is not a member of ‘Sophus’ SE3(Sophus::Quaterniond(1, rotDelta, rotDelta, -rotDelta), ^~~~~~~~~~~ /home/hutiancheng/project/tandem/tandem/src/FullSystem/FullSystem.cpp:524:46: note: suggested alternative: In file included from /usr/local/include/eigen3/Eigen/Geometry:42:0, from /usr/local/include/eigen3/Eigen/Dense:6, from /usr/local/include/sophus/rotation_matrix.hpp:7, from /usr/local/include/sophus/so3.hpp:7, from /usr/local/include/sophus/rxso3.hpp:7, from /usr/local/include/sophus/sim3.hpp:7, from /home/hutiancheng/project/tandem/tandem/src/util/NumType.h:28, from /home/hutiancheng/project/tandem/tandem/src/FullSystem/FullSystem.h:29, from /home/hutiancheng/project/tandem/tandem/src/FullSystem/FullSystem.cpp:32: /usr/local/include/eigen3/Eigen/src/Geometry/Quaternion.h:310:28: note: ‘Eigen::Quaterniond’ typedef Quaternion Quaterniond; ^~~~~~~~~~~ /home/hutiancheng/project/tandem/tandem/src/FullSystem/FullSystem.cpp:527:46: error: ‘Quaterniond’ is not a member of ‘Sophus’ SE3(Sophus::Quaterniond(1, rotDelta, rotDelta, rotDelta), ^~~~~~~~~~~ /home/hutiancheng/project/tandem/tandem/src/FullSystem/FullSystem.cpp:527:46: note: suggested alternative: In file included from /usr/local/include/eigen3/Eigen/Geometry:42:0, from /usr/local/include/eigen3/Eigen/Dense:6, from /usr/local/include/sophus/rotation_matrix.hpp:7, from /usr/local/include/sophus/so3.hpp:7, from /usr/local/include/sophus/rxso3.hpp:7, from /usr/local/include/sophus/sim3.hpp:7, from /home/hutiancheng/project/tandem/tandem/src/util/NumType.h:28, from /home/hutiancheng/project/tandem/tandem/src/FullSystem/FullSystem.h:29, from /home/hutiancheng/project/tandem/tandem/src/FullSystem/FullSystem.cpp:32: /usr/local/include/eigen3/Eigen/src/Geometry/Quaternion.h:310:28: note: ‘Eigen::Quaterniond’ typedef Quaternion Quaterniond; ^~~~~~~~~~~ CMakeFiles/tandem.dir/build.make:209: recipe for target 'CMakeFiles/tandem.dir/src/FullSystem/HessianBlocks.cpp.o' failed make[2]: *** [CMakeFiles/tandem.dir/src/FullSystem/HessianBlocks.cpp.o] Error 1 make[2]: *** Waiting for unfinished jobs.... CMakeFiles/tandem.dir/build.make:573: recipe for target 'CMakeFiles/tandem.dir/src/IOWrapper/Pangolin/PangolinDSOViewer.cpp.o' failed make[2]: *** [CMakeFiles/tandem.dir/src/IOWrapper/Pangolin/PangolinDSOViewer.cpp.o] Error 1 CMakeFiles/tandem.dir/build.make:79: recipe for target 'CMakeFiles/tandem.dir/src/FullSystem/FullSystem.cpp.o' failed make[2]: *** [CMakeFiles/tandem.dir/src/FullSystem/FullSystem.cpp.o] Error 1 CMakeFiles/Makefile2:190: recipe for target 'CMakeFiles/tandem.dir/all' failed make[1]: *** [CMakeFiles/tandem.dir/all] Error 2 Makefile:100: recipe for target 'all' failed make: *** [all] Error 2

    could you give me some device? Thanks a lot!

    opened by autohtc92 4
  • cmake error

    cmake error

    -- The C compiler identification is GNU 8.4.0 -- The CXX compiler identification is GNU 8.4.0 -- Detecting C compiler ABI info -- Detecting C compiler ABI info - done -- Check for working C compiler: /usr/bin/cc - skipped -- Detecting C compile features -- Detecting C compile features - done -- Detecting CXX compiler ABI info -- Detecting CXX compiler ABI info - done -- Check for working CXX compiler: /usr/bin/c++ - skipped -- Detecting CXX compile features -- Detecting CXX compile features - done -- Found Eigen3: /usr/local/include/eigen3 (Required is at least version "2.91.0") -- Looking for pthread.h -- Looking for pthread.h - found -- Performing Test CMAKE_HAVE_LIBC_PTHREAD -- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Failed -- Looking for pthread_create in pthreads -- Looking for pthread_create in pthreads - not found -- Looking for pthread_create in pthread -- Looking for pthread_create in pthread - found -- Found Threads: TRUE
    -- Found Boost: /usr/include (found version "1.67.0") found components: system thread chrono date_time atomic -- Found CUDA: /usr/local/cuda-11.1 (found version "11.1") -- Caffe2: CUDA detected: 11.1 -- Caffe2: CUDA nvcc is: /usr/local/cuda-11.1/bin/nvcc -- Caffe2: CUDA toolkit directory: /usr/local/cuda-11.1 -- Caffe2: Header version is: 11.1 -- Found CUDNN: /usr/local/cuda-11.1/lib64/libcudnn.so
    -- Found cuDNN: v7.6.5 (include: /usr/local/cuda-11.1/include, library: /usr/local/cuda-11.1/lib64/libcudnn.so) -- /usr/local/cuda-11.1/lib64/libnvrtc.so shorthash is 3a20f2b6 -- Autodetected CUDA architecture(s): 7.5 -- Added CUDA NVCC flags for: -gencode;arch=compute_75,code=sm_75 -- Found Torch: /home/haimou/文档/libtorch-cxx11-abi-shared-with-deps-1.9.0+cu111/libtorch/lib/libtorch.so
    -- Found OpenCV: /usr/local (found version "4.2.0") -- The CUDA compiler identification is NVIDIA 11.1.105 -- Detecting CUDA compiler ABI info -- Detecting CUDA compiler ABI info - done -- Check for working CUDA compiler: /usr/local/cuda-11.1/bin/nvcc - skipped -- Detecting CUDA compile features -- Detecting CUDA compile features - done -- Found CUDAToolkit: /usr/local/cuda-11.1/include (found version "11.1.105") -- Found Eigen headers in: /usr/local/include/eigen3 -- Release mode (!= not Debug). Adding -O3;-lineinfo to NVCC FLAGS --- found PANGOLIN, compiling tandem_pangolin library. --- found OpenCV (/usr/local/include/opencv4), compiling tandem_opencv library. --- compiling tandem_dataset. --- not building tandem_demo, since either don't have openCV, Pangolin, or Realsense. --- not building realsense_calib_recorder, since either don't have openCV, Pangolin, or Realsense. -- Configuring done CMake Error at CMakeLists.txt:131 (add_executable): Target "tandem_dataset" links to target "Eigen3::Eigen" but the target was not found. Perhaps a find_package() call is missing for an IMPORTED target, or an ALIAS target is missing?

    CMake Error at CMakeLists.txt:131 (add_executable): Target "tandem_dataset" links to target "Eigen3::Eigen" but the target was not found. Perhaps a find_package() call is missing for an IMPORTED target, or an ALIAS target is missing?

    -- Generating done CMake Generate step failed. Build files cannot be regenerated correctly.

    opened by Ruinmou 4
  • Error with c++11 compiler

    Error with c++11 compiler

    Hello!

    I am trying to run the code, while building tandem I am getting 2 major errors from cuda_coarse_tracker side. I have cuda 11.1, and same cudNN and libtorch version as mentioned in readme.

    (tandem-exp) vanjani@port-2118:~/tandem/tandem/build$ make -jConsolidate compiler generated dependencies of target dr-mvsnet
[  1%] Building CXX object libdr/cuda_coarse_tracker/CMakeFiles/cuda_coarse_tracker.dir/src/cuda_coarse_tracker.cpp.o
[  3%] Building CUDA object libdr/cuda_coarse_tracker/CMakeFiles/cuda_coarse_tracker.dir/src/cuda_coarse_tracker_private.cu.o
Consolidate compiler generated dependencies of target dr-fusion
[  6%] Built target dr-mvsnet
[ 22%] Built target dr-fusion
Consolidate compiler generated dependencies of target dr_mvsnet_test
[ 25%] Built target dr_mvsnet_test
/home/vanjani/tandem/tandem/libdr/cuda_coarse_tracker/src/cuda_coarse_tracker.cpp:9:18: fatal error: cnpy.h: No such file or directory
compilation terminated.
libdr/cuda_coarse_tracker/CMakeFiles/cuda_coarse_tracker.dir/build.make:75: recipe for target 'libdr/cuda_coarse_tracker/CMakeFiles/cuda_coarse_tracker.dir/src/cuda_coarse_tracker.cpp.o' failed
make[2]: *** [libdr/cuda_coarse_tracker/CMakeFiles/cuda_coarse_tracker.dir/src/cuda_coarse_tracker.cpp.o] Error 1
make[2]: *** Waiting for unfinished jobs....
In file included from /usr/include/c++/5/type_traits:35:0,
                 from /home/vanjani/tandem/tandem/libdr/cuda_coarse_tracker/include/private/utils.h:11,
                 from /home/vanjani/tandem/tandem/libdr/cuda_coarse_tracker/include/private/numeric_cuda.h:16,
                 from /home/vanjani/tandem/tandem/libdr/cuda_coarse_tracker/src/cuda_coarse_tracker_private.cu:4:
/usr/include/c++/5/bits/c++0x_warning.h:32:2: error: #error This file requires compiler and library support for the ISO C++ 2011 standard. This support must be enabled with the -std=c++11 or -std=gnu++11 compiler options.
 #error This file requires compiler and library support \
  ^
libdr/cuda_coarse_tracker/CMakeFiles/cuda_coarse_tracker.dir/build.make:89: recipe for target 'libdr/cuda_coarse_tracker/CMakeFiles/cuda_coarse_tracker.dir/src/cuda_coarse_tracker_private.cu.o' failed
make[2]: *** [libdr/cuda_coarse_tracker/CMakeFiles/cuda_coarse_tracker.dir/src/cuda_coarse_tracker_private.cu.o] Error 1
CMakeFiles/Makefile2:274: recipe for target 'libdr/cuda_coarse_tracker/CMakeFiles/cuda_coarse_tracker.dir/all' failed
make[1]: *** [libdr/cuda_coarse_tracker/CMakeFiles/cuda_coarse_tracker.dir/all] Error 2
Makefile:90: recipe for target 'all' failed
make: *** [all] Error 2
(tandem-exp) vanjani@port-2118:~/tandem/tandem/build$

    Could you suggest anything to solve these?

    I am using SET(CMAKE_CXX_FLAGS "-std=c++0x") in CMakeLists.txt

    Thanks,

    Update1: I fixed the cnpy error by cloning the cnpy repo, cmake, make and make install for future reference

    C++11 Compiler error still remains

    (tandem-exp) vanjani@port-2118:~/tandem/tandem/build$ cmake ..     -DCMAKE_BUILD_TYPE=Release     -DCMAKE_PREFIX_PATH=$TANDEM_LIBTORCH_DIR     -DCUDNN_LIBRARY=$TANDEM_CUDNN_LIBRARY     -DCUDNN_INCLUDE_PATH=$TANDEM_CUDNN_INCLUDE_PATH
-- Caffe2: CUDA detected: 11.1
-- Caffe2: CUDA nvcc is: /usr/local/cuda-11.1/bin/nvcc
-- Caffe2: CUDA toolkit directory: /usr/local/cuda-11.1
-- Caffe2: Header version is: 11.1
-- Found cuDNN: v8.0.5  (include: /usr/local/cuda/include, library: /usr/local/cuda-11.1/lib64/libcudnn.so)
-- /usr/local/cuda-11.1/lib64/libnvrtc.so shorthash is 3a20f2b6
-- Autodetected CUDA architecture(s):  6.1
-- Added CUDA NVCC flags for: -gencode;arch=compute_61,code=sm_61
-- Found Eigen headers in: /usr/local/include/eigen3
-- Release mode (!= not Debug). Adding -O3;-lineinfo to NVCC FLAGS
--- found PANGOLIN, compiling tandem_pangolin library.
--- found OpenCV (/usr/local/include;/usr/local/include/opencv), compiling tandem_opencv library.
--- compiling tandem_dataset.
--- not building tandem_demo, since either don't have openCV, Pangolin, or Realsense.
--- not building realsense_calib_recorder, since either don't have openCV, Pangolin, or Realsense.
-- Configuring done
-- Generating done
-- Build files have been written to: /home/vanjani/tandem/tandem/build
(tandem-exp) vanjani@port-2118:~/tandem/tandem/build$ make -jConsolidate compiler generated dependencies of target dr-mvsnet
Consolidate compiler generated dependencies of target dr-fusion
[  1%] Building CXX object libdr/cuda_coarse_tracker/CMakeFiles/cuda_coarse_tracker.dir/src/cuda_coarse_tracker.cpp.o
[  3%] Building CUDA object libdr/cuda_coarse_tracker/CMakeFiles/cuda_coarse_tracker.dir/src/cuda_coarse_tracker_private.cu.o
[ 18%] Built target dr-fusion
[ 22%] Built target dr-mvsnet
Consolidate compiler generated dependencies of target dr_mvsnet_test
[ 25%] Built target dr_mvsnet_test
In file included from /usr/include/c++/5/type_traits:35:0,
                 from /home/vanjani/tandem/tandem/libdr/cuda_coarse_tracker/include/private/utils.h:11,
                 from /home/vanjani/tandem/tandem/libdr/cuda_coarse_tracker/include/private/numeric_cuda.h:16,
                 from /home/vanjani/tandem/tandem/libdr/cuda_coarse_tracker/src/cuda_coarse_tracker_private.cu:4:
/usr/include/c++/5/bits/c++0x_warning.h:32:2: error: #error This file requires compiler and library support for the ISO C++ 2011 standard. This support must be enabled with the -std=c++11 or -std=gnu++11 compiler options.
 #error This file requires compiler and library support \
  ^
libdr/cuda_coarse_tracker/CMakeFiles/cuda_coarse_tracker.dir/build.make:89: recipe for target 'libdr/cuda_coarse_tracker/CMakeFiles/cuda_coarse_tracker.dir/src/cuda_coarse_tracker_private.cu.o' failed
make[2]: *** [libdr/cuda_coarse_tracker/CMakeFiles/cuda_coarse_tracker.dir/src/cuda_coarse_tracker_private.cu.o] Error 1
make[2]: *** Waiting for unfinished jobs....
CMakeFiles/Makefile2:274: recipe for target 'libdr/cuda_coarse_tracker/CMakeFiles/cuda_coarse_tracker.dir/all' failed
make[1]: *** [libdr/cuda_coarse_tracker/CMakeFiles/cuda_coarse_tracker.dir/all] Error 2
Makefile:90: recipe for target 'all' failed
make: *** [all] Error 2
(tandem-exp) vanjani@port-2118:~/tandem/tandem/build$

    This error was coming in dr_mvsnet too but using SET(CMAKE_CXX_FLAGS "-std=c++0x") in Cmakelists.txt solved the issue but somehow it is not helping for cuda_coarse_tracler

    opened by pankhurivanjani 3
  • links to target

    links to target "CUDA::cublas" but the target was not found.

    Hi， when I run

    cmake .. \
   -DCMAKE_BUILD_TYPE=Release \
   -DCMAKE_PREFIX_PATH=$TANDEM_LIBTORCH_DIR \
   -DCUDNN_LIBRARY=$TANDEM_CUDNN_LIBRARY \
   -DCUDNN_INCLUDE_PATH=$TANDEM_CUDNN_INCLUDE_PATH

    there are some errors: -- The C compiler identification is GNU 7.5.0 -- The CXX compiler identification is GNU 7.5.0 -- Check for working C compiler: /usr/bin/cc -- Check for working C compiler: /usr/bin/cc -- works -- Detecting C compiler ABI info -- Detecting C compiler ABI info - done -- Detecting C compile features -- Detecting C compile features - done -- Check for working CXX compiler: /usr/bin/c++ -- Check for working CXX compiler: /usr/bin/c++ -- works -- Detecting CXX compiler ABI info -- Detecting CXX compiler ABI info - done -- Detecting CXX compile features -- Detecting CXX compile features - done -- Looking for pthread.h -- Looking for pthread.h - found -- Looking for pthread_create -- Looking for pthread_create - not found -- Looking for pthread_create in pthreads -- Looking for pthread_create in pthreads - not found -- Looking for pthread_create in pthread -- Looking for pthread_create in pthread - found -- Found Threads: TRUE
    -- Boost version: 1.65.1 -- Found the following Boost libraries: -- system -- thread -- chrono -- date_time -- atomic -- Found CUDA: /usr/local/cuda (found version "11.1") -- Caffe2: CUDA detected: 11.1 -- Caffe2: CUDA nvcc is: /usr/local/cuda/bin/nvcc -- Caffe2: CUDA toolkit directory: /usr/local/cuda -- Caffe2: Header version is: 11.1 -- Found CUDNN: /usr/local/cuda/lib64/libcudnn.so
    -- Found cuDNN: v8.2.1 (include: /usr/local/cuda/include, library: /usr/local/cuda/lib64/libcudnn.so) -- /usr/local/cuda/lib64/libnvrtc.so shorthash is 3a20f2b6 -- Autodetected CUDA architecture(s): 8.6 -- Added CUDA NVCC flags for: -gencode;arch=compute_86,code=sm_86 -- Found Torch: /home/libtorch-cxx11-abi-shared-with-deps-1.9.0+cu111/libtorch/lib/libtorch.so
    -- Found OpenCV: /usr/local (found version "4.5.4") -- Boost version: 1.65.1 -- Found the following Boost libraries: -- system -- thread -- chrono -- date_time -- atomic -- The CUDA compiler identification is NVIDIA 11.1.105 -- Check for working CUDA compiler: /usr/local/cuda/bin/nvcc -- Check for working CUDA compiler: /usr/local/cuda/bin/nvcc -- works -- Detecting CUDA compiler ABI info -- Detecting CUDA compiler ABI info - done -- Found Eigen headers in: /usr/include/eigen3 -- Release mode (!= not Debug). Adding -O3;-lineinfo to NVCC FLAGS --- found PANGOLIN, compiling tandem_pangolin library. --- found OpenCV (/usr/local/include/opencv4), compiling tandem_opencv library. --- compiling tandem_dataset. -- TANDEM DEMO will be compiled -- Realsense Calib Recorded Will be compiled -- Configuring done CMake Error at CMakeLists.txt:115 (add_library): Target "tandem" links to target "CUDA::cublas" but the target was not found. Perhaps a find_package() call is missing for an IMPORTED target, or an ALIAS target is missing?

    CMake Error at CMakeLists.txt:142 (add_executable): Target "tandem_demo" links to target "CUDA::cublas" but the target was not found. Perhaps a find_package() call is missing for an IMPORTED target, or an ALIAS target is missing?

    CMake Error at CMakeLists.txt:131 (add_executable): Target "tandem_dataset" links to target "CUDA::cublas" but the target was not found. Perhaps a find_package() call is missing for an IMPORTED target, or an ALIAS target is missing?

    CMake Error at libdr/cuda_coarse_tracker/CMakeLists.txt:27 (add_library): Target "cuda_coarse_tracker" links to target "CUDA::cublas" but the target was not found. Perhaps a find_package() call is missing for an IMPORTED target, or an ALIAS target is missing?

    CMake Error: Error required internal CMake variable not set, cmake may not be built correctly. Missing variable is: CMAKE_CUDA_DEVICE_LINK_EXECUTABLE CMake Error: Error required internal CMake variable not set, cmake may not be built correctly. Missing variable is: CMAKE_CUDA_DEVICE_LINK_EXECUTABLE -- Generating done

    how to fix it ? Thanks!

    opened by DormouseLi 3
  • Segmentation fault when testing tandem on a custom synthetic data

    Segmentation fault when testing tandem on a custom synthetic data

    Hello,

    I am trying to test Tandem on a custom dataset.

    When it loads the model the program crashes and I get a segmentation fault like this

    (tandem) pankhuri@pankhuri-GE76-Raider-11UH:~/tandem/tandem$ ./build/bin/tandem_dataset preset=gui result_folder=/home/pankhuri/tandem-dataset/results files=/home/pankhuri/tandem-dataset/images calib=/home/pankhuri/tandem-dataset/camera.txt mvsnet_folder=/home/pankhuri/tandem/tandem/exported/tandem mode=1
loading data from /home/pankhuri/tandem-dataset/images!
loading calibration from /home/pankhuri/tandem-dataset/camera.txt!
Loading MVSNet from /home/pankhuri/tandem/tandem/exported/tandem/!
PHOTOMETRIC MODE WITHOUT CALIBRATION!

=============== TANDEM Settings: ===============
	Setting 'gui':
	- no real-time enforcing
	- 2000 active points
	- 5-7 active frames
	- 1-6 LM iteration each KF
	- TSDF fusion: yes
	- dense tracking on cpu (step=1)
	- Pangolin
	  - Fullscreen: 0
	  - Mesh: 1
	  - Smaller Images: 1

Reading Calibration from file /home/pankhuri/tandem-dataset/camera.txt ... found!
Input resolution: 640 480
In: 300.000000 300.000000 480.000000 270.000000 0.000000
Out: Rectify Crop
Output resolution: 640 480
finding CROP optimal new model!
initial range: x: -1.6159 - 0.5352; y: -0.9090 - 0.7036!
iteration 00001: range: x: -1.6078 - 0.5325; y: -0.9090 - 0.7036!
iteration 00002: range: x: -1.5998 - 0.5299; y: -0.9090 - 0.7036!
iteration 00003: range: x: -1.5998 - 0.5299; y: -0.9045 - 0.7000!
iteration 00004: range: x: -1.5998 - 0.5299; y: -0.8999 - 0.6965!
iteration 00005: range: x: -1.5998 - 0.5299; y: -0.8999 - 0.6965!

Rectified Kamera Matrix:
300.051       0 480.015
      0 300.035 270.011
      0       0       1

NO PHOTOMETRIC Calibration!
Reading Photometric Calibration from file 
PhotometricUndistorter: Could not open file!
got 0 images and 0 timestamps and 0 exposures.!
ImageFolderReader: got 0 files in /home/pankhuri/tandem-dataset/images!
using pyramid levels 0 to 3. coarsest resolution: 80 x 60!
START PANGOLIN!

----DRMVSNET Initalizing fusion----

----DRMVSNET Initalizing fusion done----
DrMvsnet torch::cuda::is_vailable == true --> seems good
View Num: 7, ref index: 5
[W BinaryOps.cpp:467] Warning: floor_divide is deprecated, and will be removed in a future version of pytorch. It currently rounds toward 0 (like the 'trunc' function NOT 'floor'). This results in incorrect rounding for negative values.
To keep the current behavior, use torch.div(a, b, rounding_mode='trunc'), or for actual floor division, use torch.div(a, b, rounding_mode='floor'). (function operator())
Correctness:
	Depth correct     : 0, error: 0.0106286
	Confidence correct: 1, error: 0.00296127
Correctness:
	Depth correct     : 0, error: 0.0106182
	Confidence correct: 1, error: 0.00295481
Correctness:
	Depth correct     : 0, error: 0.0106182
	Confidence correct: 1, error: 0.00295481
Correctness:
	Depth correct     : 0, error: 0.0106182
	Confidence correct: 1, error: 0.00295482
Correctness:
	Depth correct     : 0, error: 0.0106182
	Confidence correct: 1, error: 0.00295482
Correctness:
	Depth correct     : 0, error: 0.0106182
	Confidence correct: 1, error: 0.00295482
Correctness:
	Depth correct     : 0, error: 0.0106182
	Confidence correct: 1, error: 0.0029548
Correctness:
	Depth correct     : 0, error: 0.0106182
	Confidence correct: 1, error: 0.00295482
Correctness:
	Depth correct     : 0, error: 0.0106182
	Confidence correct: 1, error: 0.00295482
Performance:
	CallAsync     : 6.832 ms
	Ready         : 0 ms
	GetResult     : 116.539 ms
There has been an error. Do not use the model.
Segmentation fault (core dumped)
(tandem) pankhuri@pankhuri-GE76-Raider-11UH:~/tandem/tandem$

    What could be the reason behind this segmentation fault? Is it dependent on geometric calibration?

    Note:

    It works fine with other dataset like tum rgbd freiburg sequence, although there comes a warning of not loading MVSNet successfully

    (tandem) pankhuri@pankhuri-GE76-Raider-11UH:~/tandem/tandem$ ./build/bin/tandem_dataset preset=gui result_folder=/home/pankhuri/tandem-dataset/results files=/home/pankhuri/rgbd_dataset_freiburg2_xyz/rgb calib=/home/pankhuri/rgbd_dataset_freiburg2_xyz/camera.txt mvsnet_folder=/home/pankhuri/tandem/tandem/exported/tandem mode=1
loading data from /home/pankhuri/rgbd_dataset_freiburg2_xyz/rgb!
loading calibration from /home/pankhuri/rgbd_dataset_freiburg2_xyz/camera.txt!
Loading MVSNet from /home/pankhuri/tandem/tandem/exported/tandem/!
PHOTOMETRIC MODE WITHOUT CALIBRATION!

=============== TANDEM Settings: ===============
	Setting 'gui':
	- no real-time enforcing
	- 2000 active points
	- 5-7 active frames
	- 1-6 LM iteration each KF
	- TSDF fusion: yes
	- dense tracking on cpu (step=1)
	- Pangolin
	  - Fullscreen: 0
	  - Mesh: 1
	  - Smaller Images: 1

Reading Calibration from file /home/pankhuri/rgbd_dataset_freiburg2_xyz/camera.txt ... found!
Input resolution: 640 480
In: 517.300000 516.500000 318.600000 255.300000 0.000000
Out: Rectify Crop
Output resolution: 640 480
finding CROP optimal new model!
initial range: x: -0.6220 - 0.6255; y: -0.4991 - 0.4374!
iteration 00001: range: x: -0.6188 - 0.6224; y: -0.4991 - 0.4374!
iteration 00002: range: x: -0.6158 - 0.6193; y: -0.4991 - 0.4374!
iteration 00003: range: x: -0.6158 - 0.6193; y: -0.4966 - 0.4352!
iteration 00004: range: x: -0.6158 - 0.6193; y: -0.4942 - 0.4331!
iteration 00005: range: x: -0.6158 - 0.6193; y: -0.4942 - 0.4331!

Rectified Kamera Matrix:
517.406       0 318.595
      0 516.592 255.281
      0       0       1

NO PHOTOMETRIC Calibration!
Reading Photometric Calibration from file 
PhotometricUndistorter: Could not open file!
set EXPOSURES to zero!
got 3669 images and 3669 timestamps and 0 exposures.!
ImageFolderReader: got 3669 files in /home/pankhuri/rgbd_dataset_freiburg2_xyz/rgb!
using pyramid levels 0 to 3. coarsest resolution: 80 x 60!
START PANGOLIN!

----DRMVSNET Initalizing fusion----

----DRMVSNET Initalizing fusion done----
DrMvsnet torch::cuda::is_vailable == true --> seems good
View Num: 7, ref index: 5
[W BinaryOps.cpp:467] Warning: floor_divide is deprecated, and will be removed in a future version of pytorch. It currently rounds toward 0 (like the 'trunc' function NOT 'floor'). This results in incorrect rounding for negative values.
To keep the current behavior, use torch.div(a, b, rounding_mode='trunc'), or for actual floor division, use torch.div(a, b, rounding_mode='floor'). (function operator())
Correctness:
	Depth correct     : 0, error: 0.0106286
	Confidence correct: 1, error: 0.00296125
Correctness:
	Depth correct     : 0, error: 0.0106182
	Confidence correct: 1, error: 0.00295482
Correctness:
	Depth correct     : 0, error: 0.0106182
	Confidence correct: 1, error: 0.00295482
Correctness:
	Depth correct     : 0, error: 0.0106182
	Confidence correct: 1, error: 0.00295481
Correctness:
	Depth correct     : 0, error: 0.0106182
	Confidence correct: 1, error: 0.00295481
Correctness:
	Depth correct     : 0, error: 0.0106182
	Confidence correct: 1, error: 0.00295481
Correctness:
	Depth correct     : 0, error: 0.0106182
	Confidence correct: 1, error: 0.0029548
Correctness:
	Depth correct     : 0, error: 0.0106182
	Confidence correct: 1, error: 0.00295482
Correctness:
	Depth correct     : 0, error: 0.0106182
	Confidence correct: 1, error: 0.00295482
Performance:
	CallAsync     : 6.80075 ms
	Ready         : 0 ms
	GetResult     : 116.749 ms
There has been an error. Do not use the model.
Couldn't load MVSNet successfully.INITIALIZE FROM INITIALIZER (2048 pts)!

TANDEM TIMING: ==================
3668 Frames (32.8 fps)
30.48ms per frame; 
111.79s total time; 
======================

DrFusion::SaveMeshToFile volume_->ExtractMesh (485 ms)
DrFusion::SaveMeshToFile mesh.SaveToFile (3587 ms)
Mesh Saving done!
^CCaught signal 2
    opened by pankhurivanjani 2
  • make error about Sophus

    make error about Sophus

    make error about Sophus

    image

    When I fellow the readme in the Sophus folder to make Sophus, something that looks likes make fail appeared.

    image

    Does anyone know how to solve it ? Thanks!

    opened by Bobkk-k 2
  • not able to open

    not able to open "tandem/cva_mvsnet/model.pt"

    Thanks for the opensourcing code. It seems the model.pt is not provided for running your system on euroc dataset.

    After running the command for the given dataset euroc, the system seems to initialize successfully but is not able to open the "model.pt" using cva_mvsnet. The flag "mvsnet_folder=Folder that contains the exported MVSNet." is not very clearly explained. Following is my log from the terminal:

    ~/tandem/tandem$ ./build/bin/tandem_dataset preset=dataset result_folder=./results files=/home/shu/Downloads/euroc_tandem_format_1.1.beta/euroc_tandem_format/V1_01_easy/images calib=/home/shu/Downloads/euroc_tandem_format_1.1.beta/euroc_tandem_format/V1_01_easy/camera.txt mvsnet_folder=/home/shu/Examples/tandem/cva_mvsnet mode=1

    loading data from /home/shu/Downloads/euroc_tandem_format_1.1.beta/euroc_tandem_format/V1_01_easy/images! loading calibration from /home/shu/Downloads/euroc_tandem_format_1.1.beta/euroc_tandem_format/V1_01_easy/camera.txt! Loading MVSNet from /home/shu/Examples/tandem/cva_mvsnet/! PHOTOMETRIC MODE WITHOUT CALIBRATION!

    =============== TANDEM Settings: =============== Setting 'dataset': - no real-time enforcing - 2000 active points - 5-7 active frames - 1-6 LM iteration each KF - TSDF fusion: yes - dense tracking on cpu (step=1) - Pangolin - Fullscreen: 0 - Mesh: 1 - Smaller Images: 1

    Reading Calibration from file /home/shu/Downloads/euroc_tandem_format_1.1.beta/euroc_tandem_format/V1_01_easy/camera.txt ... found! Input resolution: 640 480 In: 436.225021 436.225021 310.162763 256.946041 0.000000 Out: Rectify Crop Output resolution: 640 480 finding CROP optimal new model! initial range: x: -0.7181 - 0.7613; y: -0.5949 - 0.5141! iteration 00001: range: x: -0.7145 - 0.7575; y: -0.5949 - 0.5141! iteration 00002: range: x: -0.7109 - 0.7537; y: -0.5949 - 0.5141! iteration 00003: range: x: -0.7109 - 0.7537; y: -0.5919 - 0.5115! iteration 00004: range: x: -0.7109 - 0.7537; y: -0.5890 - 0.5090! iteration 00005: range: x: -0.7109 - 0.7537; y: -0.5890 - 0.5090!

    Rectified Kamera Matrix: 436.27 0 310.165 0 436.28 256.95 0 0 1

    NO PHOTOMETRIC Calibration! Reading Photometric Calibration from file PhotometricUndistorter: Could not open file! set timestamps and exposures to zero! set EXPOSURES to zero! got 2778 images and 0 timestamps and 0 exposures.! ImageFolderReader: got 2778 files in /home/shu/Downloads/euroc_tandem_format_1.1.beta/euroc_tandem_format/V1_01_easy/images! using pyramid levels 0 to 3. coarsest resolution: 80 x 60!

    ----DRMVSNET Initalizing fusion----

    ----DRMVSNET Initalizing fusion done---- DrMvsnet torch::cuda::is_vailable == true --> seems good terminate called after throwing an instance of 'c10::Error' what(): open file failed, file path: /home/shu/Examples/tandem/cva_mvsnet/model.pt Exception raised from FileAdapter at ../caffe2/serialize/file_adapter.cc:11 (most recent call first): frame #0: c10::Error::Error(c10::SourceLocation, std::__cxx11::basic_string<char, std::char_traits, std::allocator >) + 0x69 (0x7f8ea507c1d9 in /home/shu/Examples/libtorch/lib/libc10.so) frame #1: c10::detail::torchCheckFail(char const*, char const*, unsigned int, std::__cxx11::basic_string<char, std::char_traits, std::allocator > const&) + 0xd2 (0x7f8ea5078812 in /home/shu/Examples/libtorch/lib/libc10.so) frame #2: caffe2::serialize::FileAdapter::FileAdapter(std::__cxx11::basic_string<char, std::char_traits, std::allocator > const&) + 0x1ff (0x7f8ea7ddbc0f in /home/shu/Examples/libtorch/lib/libtorch_cpu.so) frame #3: torch::jit::load(std::__cxx11::basic_string<char, std::char_traits, std::allocator > const&, c10::optionalc10::Device, std::unordered_map<std::__cxx11::basic_string<char, std::char_traits, std::allocator >, std::__cxx11::basic_string<char, std::char_traits, std::allocator >, std::hash<std::__cxx11::basic_string<char, std::char_traits, std::allocator > >, std::equal_to<std::__cxx11::basic_string<char, std::char_traits, std::allocator > >, std::allocator<std::pair<std::__cxx11::basic_string<char, std::char_traits, std::allocator > const, std::__cxx11::basic_string<char, std::char_traits, std::allocator > > > >&) + 0x40 (0x7f8ea92e4b50 in /home/shu/Examples/libtorch/lib/libtorch_cpu.so) frame #4: torch::jit::load(std::__cxx11::basic_string<char, std::char_traits, std::allocator > const&, c10::optionalc10::Device) + 0x6f (0x7f8ea92e4cdf in /home/shu/Examples/libtorch/lib/libtorch_cpu.so) frame #5: + 0x16b827 (0x55a76f9e3827 in ./build/bin/tandem_dataset) frame #6: + 0x163482 (0x55a76f9db482 in ./build/bin/tandem_dataset) frame #7: + 0x3a6bc (0x55a76f8b26bc in ./build/bin/tandem_dataset) frame #8: + 0x2980d (0x55a76f8a180d in ./build/bin/tandem_dataset) frame #9: + 0xbd6df (0x7f8e526046df in /usr/lib/x86_64-linux-gnu/libstdc++.so.6) frame #10: + 0x76db (0x7f8ea55006db in /lib/x86_64-linux-gnu/libpthread.so.0) frame #11: clone + 0x3f (0x7f8e51cc171f in /lib/x86_64-linux-gnu/libc.so.6)

    Aborted (core dumped)

    opened by PeterFWS 2
  • RuntimeError: CUDA out of memory

    RuntimeError: CUDA out of memory

    I will like to start by saying thank you for the amazing project and paper! After I cloned the repository I followed the instructions to set up the environment as described in the read me file. Finally, I downloaded the replica dataset and started the training, but before it even starts it exists with the CUDA out of memory error message. I'm using NVIDIA GeForce RTX 2080 Ti. I will put the full error message below. Any help would be appreciated . Thank you !!

    
GPU available: True, used: True
INFO:lightning:GPU available: True, used: True
No environment variable for node rank defined. Set as 0.
WARNING:lightning:No environment variable for node rank defined. Set as 0.
CUDA_VISIBLE_DEVICES: [0]
INFO:lightning:CUDA_VISIBLE_DEVICES: [0]
Epoch 1:   0%|                                                                                                                                                                     | 0/2714 [00:00<?, ?it/s]Traceback (most recent call last):
  File "train.py", line 128, in <module>
    main(*parse_args(parser))
  File "train.py", line 124, in main
    trainer.fit(model)
  File "/home/tmc/miniconda3/envs/tandem/lib/python3.7/site-packages/pytorch_lightning/trainer/trainer.py", line 859, in fit
    self.single_gpu_train(model)
  File "/home/tmc/miniconda3/envs/tandem/lib/python3.7/site-packages/pytorch_lightning/trainer/distrib_parts.py", line 503, in single_gpu_train
    self.run_pretrain_routine(model)
  File "/home/tmc/miniconda3/envs/tandem/lib/python3.7/site-packages/pytorch_lightning/trainer/trainer.py", line 1015, in run_pretrain_routine
    self.train()
  File "/home/tmc/miniconda3/envs/tandem/lib/python3.7/site-packages/pytorch_lightning/trainer/training_loop.py", line 347, in train
    self.run_training_epoch()
  File "/home/tmc/miniconda3/envs/tandem/lib/python3.7/site-packages/pytorch_lightning/trainer/training_loop.py", line 419, in run_training_epoch
    _outputs = self.run_training_batch(batch, batch_idx)
  File "/home/tmc/miniconda3/envs/tandem/lib/python3.7/site-packages/pytorch_lightning/trainer/training_loop.py", line 597, in run_training_batch
    loss, batch_output = optimizer_closure()
  File "/home/tmc/miniconda3/envs/tandem/lib/python3.7/site-packages/pytorch_lightning/trainer/training_loop.py", line 561, in optimizer_closure
    output_dict = self.training_forward(split_batch, batch_idx, opt_idx, self.hiddens)
  File "/home/tmc/miniconda3/envs/tandem/lib/python3.7/site-packages/pytorch_lightning/trainer/training_loop.py", line 727, in training_forward
    output = self.model.training_step(*args)
  File "/home/tmc/tandem/cva_mvsnet/models/tandem.py", line 101, in training_step
    outputs = self(batch)
  File "/home/tmc/miniconda3/envs/tandem/lib/python3.7/site-packages/torch/nn/modules/module.py", line 550, in __call__
    result = self.forward(*input, **kwargs)
  File "/home/tmc/tandem/cva_mvsnet/models/tandem.py", line 43, in forward
    depth_max=batch['depth_max'],
  File "/home/tmc/miniconda3/envs/tandem/lib/python3.7/site-packages/torch/nn/modules/module.py", line 550, in __call__
    result = self.forward(*input, **kwargs)
  File "/home/tmc/tandem/cva_mvsnet/models/cva_mvsnet.py", line 162, in forward
    volume_gate=self.volume_gates[stage] if self.view_aggregation else None,
  File "/home/tmc/tandem/cva_mvsnet/models/module.py", line 1085, in depth_prediction
    half_pixel_centers=half_pixel_centers
  File "/home/tmc/tandem/cva_mvsnet/models/module.py", line 891, in homo_warping
    warped_src_fea[torch.isnan(warped_src_fea)] = 0
RuntimeError: CUDA out of memory. Tried to allocate 2.34 GiB (GPU 0; 10.76 GiB total capacity; 7.43 GiB already allocated; 740.06 MiB free; 8.94 GiB reserved in total by PyTorch)
Epoch 1:   0%|          | 0/2714 [00:04<?, ?it/s]
    opened by yohannes-taye 2
  • Saving sparse reconstruction

    Saving sparse reconstruction

    Hi. I wanted to ask how we can save the Sparse reconstruction instead of normal one. I nthe GUI we have option show sparse but I am not sure how to save that sparse map.

    opened by pytholic 0
  • Segmentation fault

    Segmentation fault

    Hi and thank you for the repo. I keep getting the following error on 80% of the videos. I have tried different devices like gopro, iphone, drone footage. I also performed checkerboard calibration.

    There has been an error. Do not use the model.
Couldn't load MVSNet successfully.INITIALIZE FROM INITIALIZER (1961 pts)!
I THINK INITIALIZATINO FAILED! Resetting.
RESETTING!
destroyed ThreadReduce
INITIALIZE FROM INITIALIZER (1961 pts)!
Segmentation fault (core dumped)
    opened by pytholic 0
  • Docker and bash support

    Docker and bash support

    Hi. I tested the dockerfile provided by rancheng in an earlier pull request. It works. However, there were some shortcomings.

    I have created a detailed repository with updated instructions, docker support including the docker image and added bash script for faster map generation. It can be found here.

    I am creating this pull request so that you can check it and include it in the main repository,

    Cheers!

    opened by pytholic 0
  • cnpy error when i install

    cnpy error when i install

    when i run make -j i got :

    what should i do to fix it?

    /home/tandem/tandem-master/tandem/libdr/cuda_coarse_tracker/src/cuda_coarse_tracker.cpp: In member function 'Vec6 CudaCoarseTracker::calcRes(const Eigen::Matrix&, float, const Vector2d&, float)': /home/tandem/tandem-master/tandem/libdr/cuda_coarse_tracker/src/cuda_coarse_tracker.cpp:218:64: error: no matching function for call to 'npy_save(const char [25], float [6], )' 218 | cnpy::npy_save("cct_data/nfxfyaffLLb.npy", nfxfyaffLLb, {6}); | ^ In file included from /home/tandem/tandem-master/tandem/libdr/cuda_coarse_tracker/src/cuda_coarse_tracker.cpp:9: /usr/local/include/cnpy.h:64:31: note: candidate: 'template void cnpy::npy_save(std::string, const T*, const unsigned int*, unsigned int, std::string, std::string)' 64 | template void npy_save(std::string fname, const T* data, const unsigned int* shape, const unsigned int ndims, std::string mode = "w", std::string order = "C") { | ^~~~~~~~ /usr/local/include/cnpy.h:64:31: note: template argument deduction/substitution failed: /home/tandem/tandem-master/tandem/libdr/cuda_coarse_tracker/src/cuda_coarse_tracker.cpp:218:64: note: candidate expects 6 arguments, 3 provided 218 | cnpy::npy_save("cct_data/nfxfyaffLLb.npy", nfxfyaffLLb, {6}); | ^ /home/tandem/tandem-master/tandem/libdr/cuda_coarse_tracker/src/cuda_coarse_tracker.cpp:221:87: error: no matching function for call to 'npy_save(const char [48], float [6], )' 221 | cnpy::npy_save("cct_data/setting_huberTH_maxEnergy_cutoffTH.npy", nfxfyaffLLb, {3}); | ^ In file included from /home/tandem/tandem-master/tandem/libdr/cuda_coarse_tracker/src/cuda_coarse_tracker.cpp:9: /usr/local/include/cnpy.h:64:31: note: candidate: 'template void cnpy::npy_save(std::string, const T*, const unsigned int*, unsigned int, std::string, std::string)' 64 | template void npy_save(std::string fname, const T* data, const unsigned int* shape, const unsigned int ndims, std::string mode = "w", std::string order = "C") { | ^~~~~~~~ /usr/local/include/cnpy.h:64:31: note: template argument deduction/substitution failed: /home/tandem/tandem-master/tandem/libdr/cuda_coarse_tracker/src/cuda_coarse_tracker.cpp:221:87: note: candidate expects 6 arguments, 3 provided 221 | cnpy::npy_save("cct_data/setting_huberTH_maxEnergy_cutoffTH.npy", nfxfyaffLLb, {3}); | ^ /home/tandem/tandem-master/tandem/libdr/cuda_coarse_tracker/src/cuda_coarse_tracker.cpp:223:79: error: no matching function for call to 'npy_save(const char [30], float*&, )' 223 | cnpy::npy_save("cct_data/refToNew_Ki_host.npy", refToNew_Ki_host, {16 + 9}); | ^ In file included from /home/tandem/tandem-master/tandem/libdr/cuda_coarse_tracker/src/cuda_coarse_tracker.cpp:9: /usr/local/include/cnpy.h:64:31: note: candidate: 'template void cnpy::npy_save(std::string, const T*, const unsigned int*, unsigned int, std::string, std::string)' 64 | template void npy_save(std::string fname, const T* data, const unsigned int* shape, const unsigned int ndims, std::string mode = "w", std::string order = "C") { | ^~~~~~~~ /usr/local/include/cnpy.h:64:31: note: template argument deduction/substitution failed: /home/tandem/tandem-master/tandem/libdr/cuda_coarse_tracker/src/cuda_coarse_tracker.cpp:223:79: note: candidate expects 6 arguments, 3 provided 223 | cnpy::npy_save("cct_data/refToNew_Ki_host.npy", refToNew_Ki_host, {16 + 9}); | ^ /home/tandem/tandem-master/tandem/libdr/cuda_coarse_tracker/src/cuda_coarse_tracker.cpp:227:65: error: no matching function for call to 'npy_save(const char [18], float*&, )' 227 | cnpy::npy_save("cct_data/pc_u.npy", buf, {(unsigned long) n}); | ^ In file included from /home/tandem/tandem-master/tandem/libdr/cuda_coarse_tracker/src/cuda_coarse_tracker.cpp:9: /usr/local/include/cnpy.h:64:31: note: candidate: 'template void cnpy::npy_save(std::string, const T*, const unsigned int*, unsigned int, std::string, std::string)' 64 | template void npy_save(std::string fname, const T* data, const unsigned int* shape, const unsigned int ndims, std::string mode = "w", std::string order = "C") { | ^~~~~~~~ /usr/local/include/cnpy.h:64:31: note: template argument deduction/substitution failed: /home/tandem/tandem-master/tandem/libdr/cuda_coarse_tracker/src/cuda_coarse_tracker.cpp:227:65: note: candidate expects 6 arguments, 3 provided 227 | cnpy::npy_save("cct_data/pc_u.npy", buf, {(unsigned long) n}); | ^ /home/tandem/tandem-master/tandem/libdr/cuda_coarse_tracker/src/cuda_coarse_tracker.cpp:230:65: error: no matching function for call to 'npy_save(const char [18], float*&, )' 230 | cnpy::npy_save("cct_data/pc_v.npy", buf, {(unsigned long) n}); | ^ In file included from /home/tandem/tandem-master/tandem/libdr/cuda_coarse_tracker/src/cuda_coarse_tracker.cpp:9: /usr/local/include/cnpy.h:64:31: note: candidate: 'template void cnpy::npy_save(std::string, const T*, const unsigned int*, unsigned int, std::string, std::string)' 64 | template void npy_save(std::string fname, const T* data, const unsigned int* shape, const unsigned int ndims, std::string mode = "w", std::string order = "C") { | ^~~~~~~~ /usr/local/include/cnpy.h:64:31: note: template argument deduction/substitution failed: /home/tandem/tandem-master/tandem/libdr/cuda_coarse_tracker/src/cuda_coarse_tracker.cpp:230:65: note: candidate expects 6 arguments, 3 provided 230 | cnpy::npy_save("cct_data/pc_v.npy", buf, {(unsigned long) n}); | ^ /home/tandem/tandem-master/tandem/libdr/cuda_coarse_tracker/src/cuda_coarse_tracker.cpp:233:70: error: no matching function for call to 'npy_save(const char [23], float*&, )' 233 | cnpy::npy_save("cct_data/pc_idepth.npy", buf, {(unsigned long) n}); | ^ In file included from /home/tandem/tandem-master/tandem/libdr/cuda_coarse_tracker/src/cuda_coarse_tracker.cpp:9: /usr/local/include/cnpy.h:64:31: note: candidate: 'template void cnpy::npy_save(std::string, const T*, const unsigned int*, unsigned int, std::string, std::string)' 64 | template void npy_save(std::string fname, const T* data, const unsigned int* shape, const unsigned int ndims, std::string mode = "w", std::string order = "C") { | ^~~~~~~~ /usr/local/include/cnpy.h:64:31: note: template argument deduction/substitution failed: /home/tandem/tandem-master/tandem/libdr/cuda_coarse_tracker/src/cuda_coarse_tracker.cpp:233:70: note: candidate expects 6 arguments, 3 provided 233 | cnpy::npy_save("cct_data/pc_idepth.npy", buf, {(unsigned long) n}); | ^ /home/tandem/tandem-master/tandem/libdr/cuda_coarse_tracker/src/cuda_coarse_tracker.cpp:236:69: error: no matching function for call to 'npy_save(const char [22], float*&, )' 236 | cnpy::npy_save("cct_data/pc_color.npy", buf, {(unsigned long) n}); | ^ In file included from /home/tandem/tandem-master/tandem/libdr/cuda_coarse_tracker/src/cuda_coarse_tracker.cpp:9: /usr/local/include/cnpy.h:64:31: note: candidate: 'template void cnpy::npy_save(std::string, const T*, const unsigned int*, unsigned int, std::string, std::string)' 64 | template void npy_save(std::string fname, const T* data, const unsigned int* shape, const unsigned int ndims, std::string mode = "w", std::string order = "C") { | ^~~~~~~~ /usr/local/include/cnpy.h:64:31: note: template argument deduction/substitution failed: /home/tandem/tandem-master/tandem/libdr/cuda_coarse_tracker/src/cuda_coarse_tracker.cpp:236:69: note: candidate expects 6 arguments, 3 provided 236 | cnpy::npy_save("cct_data/pc_color.npy", buf, {(unsigned long) n}); | ^ /home/tandem/tandem-master/tandem/libdr/cuda_coarse_tracker/src/cuda_coarse_tracker.cpp:241:74: error: no matching function for call to 'npy_save(const char [19], float*&, )' 241 | cnpy::npy_save("cct_data/dInew.npy", buf, {(unsigned long) 3 * w * h}); | ^ In file included from /home/tandem/tandem-master/tandem/libdr/cuda_coarse_tracker/src/cuda_coarse_tracker.cpp:9: /usr/local/include/cnpy.h:64:31: note: candidate: 'template void cnpy::npy_save(std::string, const T*, const unsigned int*, unsigned int, std::string, std::string)' 64 | template void npy_save(std::string fname, const T* data, const unsigned int* shape, const unsigned int ndims, std::string mode = "w", std::string order = "C") { | ^~~~~~~~ /usr/local/include/cnpy.h:64:31: note: template argument deduction/substitution failed: /home/tandem/tandem-master/tandem/libdr/cuda_coarse_tracker/src/cuda_coarse_tracker.cpp:241:74: note: candidate expects 6 arguments, 3 provided 241 | cnpy::npy_save("cct_data/dInew.npy", buf, {(unsigned long) 3 * w * h}); | ^ /home/tandem/tandem-master/tandem/libdr/cuda_coarse_tracker/src/cuda_coarse_tracker.cpp: In member function 'void CudaCoarseTracker::calcG(Eigen::Matrix&, Eigen::Matrix&, float, const Vector2d&)': /home/tandem/tandem-master/tandem/libdr/cuda_coarse_tracker/src/cuda_coarse_tracker.cpp:290:64: error: no matching function for call to 'npy_save(const char [25], float [6], )' 290 | cnpy::npy_save("cct_data/nfxfyaffLLb.npy", nfxfyaffLLb, {6}); | ^ In file included from /home/tandem/tandem-master/tandem/libdr/cuda_coarse_tracker/src/cuda_coarse_tracker.cpp:9: /usr/local/include/cnpy.h:64:31: note: candidate: 'template void cnpy::npy_save(std::string, const T*, const unsigned int*, unsigned int, std::string, std::string)' 64 | template void npy_save(std::string fname, const T* data, const unsigned int* shape, const unsigned int ndims, std::string mode = "w", std::string order = "C") { | ^~~~~~~~ /usr/local/include/cnpy.h:64:31: note: template argument deduction/substitution failed: /home/tandem/tandem-master/tandem/libdr/cuda_coarse_tracker/src/cuda_coarse_tracker.cpp:290:64: note: candidate expects 6 arguments, 3 provided 290 | cnpy::npy_save("cct_data/nfxfyaffLLb.npy", nfxfyaffLLb, {6}); | ^ /home/tandem/tandem-master/tandem/libdr/cuda_coarse_tracker/src/cuda_coarse_tracker.cpp:294:69: error: no matching function for call to 'npy_save(const char [22], float*&, )' 294 | cnpy::npy_save("cct_data/pc_color.npy", buf, {(unsigned long) n}); | ^ In file included from /home/tandem/tandem-master/tandem/libdr/cuda_coarse_tracker/src/cuda_coarse_tracker.cpp:9: /usr/local/include/cnpy.h:64:31: note: candidate: 'template void cnpy::npy_save(std::string, const T*, const unsigned int*, unsigned int, std::string, std::string)' 64 | template void npy_save(std::string fname, const T* data, const unsigned int* shape, const unsigned int ndims, std::string mode = "w", std::string order = "C") { | ^~~~~~~~ /usr/local/include/cnpy.h:64:31: note: template argument deduction/substitution failed: /home/tandem/tandem-master/tandem/libdr/cuda_coarse_tracker/src/cuda_coarse_tracker.cpp:294:69: note: candidate expects 6 arguments, 3 provided 294 | cnpy::npy_save("cct_data/pc_color.npy", buf, {(unsigned long) n}); | ^ /home/tandem/tandem-master/tandem/libdr/cuda_coarse_tracker/src/cuda_coarse_tracker.cpp:297:69: error: no matching function for call to 'npy_save(const char [22], float*&, )' 297 | cnpy::npy_save("cct_data/warped_u.npy", buf, {(unsigned long) n}); | ^ In file included from /home/tandem/tandem-master/tandem/libdr/cuda_coarse_tracker/src/cuda_coarse_tracker.cpp:9: /usr/local/include/cnpy.h:64:31: note: candidate: 'template void cnpy::npy_save(std::string, const T*, const unsigned int*, unsigned int, std::string, std::string)' 64 | template void npy_save(std::string fname, const T* data, const unsigned int* shape, const unsigned int ndims, std::string mode = "w", std::string order = "C") { | ^~~~~~~~ /usr/local/include/cnpy.h:64:31: note: template argument deduction/substitution failed: /home/tandem/tandem-master/tandem/libdr/cuda_coarse_tracker/src/cuda_coarse_tracker.cpp:297:69: note: candidate expects 6 arguments, 3 provided 297 | cnpy::npy_save("cct_data/warped_u.npy", buf, {(unsigned long) n}); | ^ /home/tandem/tandem-master/tandem/libdr/cuda_coarse_tracker/src/cuda_coarse_tracker.cpp:300:69: error: no matching function for call to 'npy_save(const char [22], float*&, )' 300 | cnpy::npy_save("cct_data/warped_v.npy", buf, {(unsigned long) n}); | ^ In file included from /home/tandem/tandem-master/tandem/libdr/cuda_coarse_tracker/src/cuda_coarse_tracker.cpp:9: /usr/local/include/cnpy.h:64:31: note: candidate: 'template void cnpy::npy_save(std::string, const T*, const unsigned int*, unsigned int, std::string, std::string)' 64 | template void npy_save(std::string fname, const T* data, const unsigned int* shape, const unsigned int ndims, std::string mode = "w", std::string order = "C") { | ^~~~~~~~ /usr/local/include/cnpy.h:64:31: note: template argument deduction/substitution failed: /home/tandem/tandem-master/tandem/libdr/cuda_coarse_tracker/src/cuda_coarse_tracker.cpp:300:69: note: candidate expects 6 arguments, 3 provided 300 | cnpy::npy_save("cct_data/warped_v.npy", buf, {(unsigned long) n}); | ^ /home/tandem/tandem-master/tandem/libdr/cuda_coarse_tracker/src/cuda_coarse_tracker.cpp:303:70: error: no matching function for call to 'npy_save(const char [23], float*&, )' 303 | cnpy::npy_save("cct_data/warped_dx.npy", buf, {(unsigned long) n}); | ^ In file included from /home/tandem/tandem-master/tandem/libdr/cuda_coarse_tracker/src/cuda_coarse_tracker.cpp:9: /usr/local/include/cnpy.h:64:31: note: candidate: 'template void cnpy::npy_save(std::string, const T*, const unsigned int*, unsigned int, std::string, std::string)' 64 | template void npy_save(std::string fname, const T* data, const unsigned int* shape, const unsigned int ndims, std::string mode = "w", std::string order = "C") { | ^~~~~~~~ /usr/local/include/cnpy.h:64:31: note: template argument deduction/substitution failed: /home/tandem/tandem-master/tandem/libdr/cuda_coarse_tracker/src/cuda_coarse_tracker.cpp:303:70: note: candidate expects 6 arguments, 3 provided 303 | cnpy::npy_save("cct_data/warped_dx.npy", buf, {(unsigned long) n}); | ^ /home/tandem/tandem-master/tandem/libdr/cuda_coarse_tracker/src/cuda_coarse_tracker.cpp:306:70: error: no matching function for call to 'npy_save(const char [23], float*&, )' 306 | cnpy::npy_save("cct_data/warped_dy.npy", buf, {(unsigned long) n}); | ^ In file included from /home/tandem/tandem-master/tandem/libdr/cuda_coarse_tracker/src/cuda_coarse_tracker.cpp:9: /usr/local/include/cnpy.h:64:31: note: candidate: 'template void cnpy::npy_save(std::string, const T*, const unsigned int*, unsigned int, std::string, std::string)' 64 | template void npy_save(std::string fname, const T* data, const unsigned int* shape, const unsigned int ndims, std::string mode = "w", std::string order = "C") { | ^~~~~~~~ /usr/local/include/cnpy.h:64:31: note: template argument deduction/substitution failed: /home/tandem/tandem-master/tandem/libdr/cuda_coarse_tracker/src/cuda_coarse_tracker.cpp:306:70: note: candidate expects 6 arguments, 3 provided 306 | cnpy::npy_save("cct_data/warped_dy.npy", buf, {(unsigned long) n}); | ^ /home/tandem/tandem-master/tandem/libdr/cuda_coarse_tracker/src/cuda_coarse_tracker.cpp:309:74: error: no matching function for call to 'npy_save(const char [27], float*&, )' 309 | cnpy::npy_save("cct_data/warped_idepth.npy", buf, {(unsigned long) n}); | ^ In file included from /home/tandem/tandem-master/tandem/libdr/cuda_coarse_tracker/src/cuda_coarse_tracker.cpp:9: /usr/local/include/cnpy.h:64:31: note: candidate: 'template void cnpy::npy_save(std::string, const T*, const unsigned int*, unsigned int, std::string, std::string)' 64 | template void npy_save(std::string fname, const T* data, const unsigned int* shape, const unsigned int ndims, std::string mode = "w", std::string order = "C") { | ^~~~~~~~ /usr/local/include/cnpy.h:64:31: note: template argument deduction/substitution failed: /home/tandem/tandem-master/tandem/libdr/cuda_coarse_tracker/src/cuda_coarse_tracker.cpp:309:74: note: candidate expects 6 arguments, 3 provided 309 | cnpy::npy_save("cct_data/warped_idepth.npy", buf, {(unsigned long) n}); | ^ /home/tandem/tandem-master/tandem/libdr/cuda_coarse_tracker/src/cuda_coarse_tracker.cpp:312:76: error: no matching function for call to 'npy_save(const char [29], float*&, )' 312 | cnpy::npy_save("cct_data/warped_residual.npy", buf, {(unsigned long) n}); | ^ In file included from /home/tandem/tandem-master/tandem/libdr/cuda_coarse_tracker/src/cuda_coarse_tracker.cpp:9: /usr/local/include/cnpy.h:64:31: note: candidate: 'template void cnpy::npy_save(std::string, const T*, const unsigned int*, unsigned int, std::string, std::string)' 64 | template void npy_save(std::string fname, const T* data, const unsigned int* shape, const unsigned int ndims, std::string mode = "w", std::string order = "C") { | ^~~~~~~~ /usr/local/include/cnpy.h:64:31: note: template argument deduction/substitution failed: /home/tandem/tandem-master/tandem/libdr/cuda_coarse_tracker/src/cuda_coarse_tracker.cpp:312:76: note: candidate expects 6 arguments, 3 provided 312 | cnpy::npy_save("cct_data/warped_residual.npy", buf, {(unsigned long) n}); | ^ /home/tandem/tandem-master/tandem/libdr/cuda_coarse_tracker/src/cuda_coarse_tracker.cpp:315:74: error: no matching function for call to 'npy_save(const char [27], float*&, )' 315 | cnpy::npy_save("cct_data/warped_weight.npy", buf, {(unsigned long) n}); | ^ In file included from /home/tandem/tandem-master/tandem/libdr/cuda_coarse_tracker/src/cuda_coarse_tracker.cpp:9: /usr/local/include/cnpy.h:64:31: note: candidate: 'template void cnpy::npy_save(std::string, const T*, const unsigned int*, unsigned int, std::string, std::string)' 64 | template void npy_save(std::string fname, const T* data, const unsigned int* shape, const unsigned int ndims, std::string mode = "w", std::string order = "C") { | ^~~~~~~~ /usr/local/include/cnpy.h:64:31: note: template argument deduction/substitution failed: /home/tandem/tandem-master/tandem/libdr/cuda_coarse_tracker/src/cuda_coarse_tracker.cpp:315:74: note: candidate expects 6 arguments, 3 provided 315 | cnpy::npy_save("cct_data/warped_weight.npy", buf, {(unsigned long) n}); | ^ make[2]: *** [libdr/cuda_coarse_tracker/CMakeFiles/cuda_coarse_tracker.dir/build.make:76: libdr/cuda_coarse_tracker/CMakeFiles/cuda_coarse_tracker.dir/src/cuda_coarse_tracker.cpp.o] Error 1 make[2]: *** Waiting for unfinished jobs.... make[1]: *** [CMakeFiles/Makefile2:275: libdr/cuda_coarse_tracker/CMakeFiles/cuda_coarse_tracker.dir/all] Error 2 make[1]: *** Waiting for unfinished jobs....
    opened by GY56723-debug 1
  • About TSDF Fusion

    About TSDF Fusion

    Regarding tsdf map fusion, there is no detailed explanation in the paper. I would like to ask a few questions, which I would be grateful for.

    1. When building a map in real time, how is the size of TSDF map initialized?
    2. Traditional TSDF algorithm can't deal with the occluded surface. Is there any special treatment for TSDF fusion in Tandem? What's the difference between it and conventional TSDF fusion? Thank you very much for your guidance！
    opened by bscbje 0
Owner
TUM Computer Vision Group
TUM Computer Vision Group
GitHub
Python and C++ implementation of "MarkerPose: Robust real-time planar target tracking for accurate stereo pose estimation". Accepted at LXCV @ CVPR 2021.

MarkerPose: Robust real-time planar target tracking for accurate stereo pose estimation This is a PyTorch and LibTorch implementation of MarkerPose: a

Jhacson Meza 47 Nov 18, 2022
Multi-Scale Geometric Consistency Guided Multi-View Stereo

ACMM [News] The code for ACMH is released!!! [News] The code for ACMP is released!!! About ACMM is a multi-scale geometric consistency guided multi-vi

Qingshan Xu 118 Jan 4, 2023
Code for "Multi-View Multi-Person 3D Pose Estimation with Plane Sweep Stereo"

Multi-View Multi-Person 3D Pose Estimation with Plane Sweep Stereo This repository includes the source code for our CVPR 2021 paper on multi-view mult

Jiahao Lin 66 Jan 4, 2023
Code release of paper "Deep Multi-View Stereo gone wild"

Deep MVS gone wild Pytorch implementation of "Deep MVS gone wild" (Paper | website) This repository provides the code to reproduce the experiments of

François Darmon 53 Dec 24, 2022
Python scripts form performing stereo depth estimation using the high res stereo model in PyTorch .

PyTorch-High-Res-Stereo-Depth-Estimation Python scripts form performing stereo depth estimation using the high res stereo model in PyTorch. Stereo dep

Ibai Gorordo 26 Nov 24, 2022
SSL_SLAM2: Lightweight 3-D Localization and Mapping for Solid-State LiDAR (mapping and localization separated) ICRA 2021

SSL_SLAM2 Lightweight 3-D Localization and Mapping for Solid-State LiDAR (Intel Realsense L515 as an example) This repo is an extension work of SSL_SL

Wang Han 王晗 1.3k Jan 8, 2023
RAFT-Stereo: Multilevel Recurrent Field Transforms for Stereo Matching

RAFT-Stereo: Multilevel Recurrent Field Transforms for Stereo Matching This repository contains the source code for our paper: RAFT-Stereo: Multilevel

Princeton Vision & Learning Lab 328 Jan 9, 2023
Pytorch codes for "Self-supervised Multi-view Stereo via Effective Co-Segmentation and Data-Augmentation"

Self-Supervised-MVS This repository is the official PyTorch implementation of our AAAI 2021 paper: "Self-supervised Multi-view Stereo via Effective Co

hongbin_xu 127 Jan 4, 2023
COLMAP - Structure-from-Motion and Multi-View Stereo

COLMAP About COLMAP is a general-purpose Structure-from-Motion (SfM) and Multi-View Stereo (MVS) pipeline with a graphical and command-line interface.

null 4.7k Jan 7, 2023
Planar Prior Assisted PatchMatch Multi-View Stereo

ACMP [News] The code for ACMH is released!!! [News] The code for ACMM is released!!! About This repository contains the code for the paper Planar Prio

Qingshan Xu 127 Dec 31, 2022
[ICCV 2021 Oral] NerfingMVS: Guided Optimization of Neural Radiance Fields for Indoor Multi-view Stereo

NerfingMVS Project Page | Paper | Video | Data NerfingMVS: Guided Optimization of Neural Radiance Fields for Indoor Multi-view Stereo Yi Wei, Shaohui

Yi Wei 369 Dec 24, 2022
MatryODShka: Real-time 6DoF Video View Synthesis using Multi-Sphere Images

Main repo for ECCV 2020 paper MatryODShka: Real-time 6DoF Video View Synthesis using Multi-Sphere Images. visual.cs.brown.edu/matryodshka

Brown University Visual Computing Group 75 Dec 13, 2022
Real-Time SLAM for Monocular, Stereo and RGB-D Cameras, with Loop Detection and Relocalization Capabilities

ORB-SLAM2 Authors: Raul Mur-Artal, Juan D. Tardos, J. M. M. Montiel and Dorian Galvez-Lopez (DBoW2) 13 Jan 2017: OpenCV 3 and Eigen 3.3 are now suppor

Raul Mur-Artal 7.8k Dec 30, 2022
Blender add-on: Add to Cameras menu: View → Camera, View → Add Camera, Camera → View, Previous Camera, Next Camera

Blender add-on: Camera additions In 3D view, it adds these actions to the View|Cameras menu: View → Camera : set the current camera to the 3D view Vie

German Bauer 11 Feb 8, 2022
T-LOAM: Truncated Least Squares Lidar-only Odometry and Mapping in Real-Time

T-LOAM: Truncated Least Squares Lidar-only Odometry and Mapping in Real-Time The first Lidar-only odometry framework with high performance based on tr

Pengwei Zhou 183 Dec 1, 2022
RTS3D: Real-time Stereo 3D Detection from 4D Feature-Consistency Embedding Space for Autonomous Driving

RTS3D: Real-time Stereo 3D Detection from 4D Feature-Consistency Embedding Space for Autonomous Driving (AAAI2021). RTS3D is efficiency and accuracy s

null 71 Nov 29, 2022
The official implementation code of "PlantStereo: A Stereo Matching Benchmark for Plant Surface Dense Reconstruction."

PlantStereo This is the official implementation code for the paper "PlantStereo: A Stereo Matching Benchmark for Plant Surface Dense Reconstruction".

Wang Qingyu 14 Nov 28, 2022
A 3D Dense mapping backend library of SLAM based on taichi-Lang designed for the aerial swarm.

TaichiSLAM This project is a 3D Dense mapping backend library of SLAM based Taichi-Lang, designed for the aerial swarm. Intro Taichi is an efficient d

XuHao 230 Dec 19, 2022
Real-Time-Student-Attendence-System - Real Time Student Attendence System

Real-Time-Student-Attendence-System The Student Attendance Management System Pro

Rounak Das 1 Feb 15, 2022