TSDF++: A Multi-Object Formulation for Dynamic Object Tracking and Reconstruction

If I would add ROS2 support to your project in a forked repo, would you be open to review it as a PR into your repo (either master or special ros2 branch) ?

Hi, everyone, when compile this project in ubuntu 18.04, i met this problem : ~/catkin_ws/devel/lib/librgbd_segmentation.so：undefined reference to‘cv::_OutputArray::_OutputArray(cv::Mat&)’, is there any advice on how to solve this problem ?

Hi, I am trying to use this mapping framework with iTHOR simulator. Currently mapping framework is used with ground truth pose and ground truth pointclouds which are obtained via 3D reprojection of image pixels with ground truth depth information. I run into this issue during my experiments. TSDF++ doesn't fusing previous observations if an object is tracked.

Experiment Setup

1. Robot initially starts with this view and after 0.2 meters left movement, the mesh is generated using /tsdf_plusplus_ros/generate_mesh ROS servise.

2. Then, robot crouches and after 0.2 meters left movement, another mesh is generated. The view of ther crouch action is as follows:

Please note that during this experiment, all objects are stationary. Therefore, ICP part in object tracking is commented out. (Line 408-409 in controller.cc.

Generated Meshes

Whole scene

The following screenshots are the meshes generated after each step of the experiment.

Mesh 1

Mesh 2

As it can be seen, in the second mesh, some parts of the objects is not fused with previous observations, e.g. top part of the first sofa.

Only sofa

In this part, I only published the point cloud of the initial sofa in the environment.

Mesh 1

Mesh 2

Only sofa without object tracking

In this part, I only published the point cloud of the initial sofa in the environment. Additionally, I disabled the object tracking by commenting out the line 425 in controller.cc (map_->transformLayer(segment->object_id_, T_O_S);).

Mesh 1

Mesh 2

In this part, second mesh also contains the previous observations. Could you please help me? Thanks in advance.

Hi, I am trying to build this seemingly wonderful package and can't seem to get over that error message... Does anybody knows what I am supposed to do to fix it ? Thanks in advance.

Errors     << tsdf_plusplus:make /d/qserdel/ros_ws/voxbloxpp_ws/logs/tsdf_plusplus/build.make.004.log                                                                                                     
/d/qserdel/ros_ws/voxbloxpp_ws/src/tsdf-plusplus/tsdf_plusplus/src/core/map.cc: In member function ‘void Map::transformLayer(const ObjectID&, const Transformation&)’:
/d/qserdel/ros_ws/voxbloxpp_ws/src/tsdf-plusplus/tsdf_plusplus/src/core/map.cc:235:25: error: ‘struct std::atomic<bool>’ has no member named ‘set’
  235 |     mo_block->updated().set();
      |                         ^~~
/d/qserdel/ros_ws/voxbloxpp_ws/src/tsdf-plusplus/tsdf_plusplus/src/core/map.cc:322:27: error: ‘struct std::atomic<bool>’ has no member named ‘set’
  322 |       mo_block->updated().set();
      |                           ^~~
/d/qserdel/ros_ws/voxbloxpp_ws/src/tsdf-plusplus/tsdf_plusplus/src/core/map.cc: In member function ‘void Map::removeObject(const ObjectID&)’:
/d/qserdel/ros_ws/voxbloxpp_ws/src/tsdf-plusplus/tsdf_plusplus/src/core/map.cc:356:25: error: ‘struct std::atomic<bool>’ has no member named ‘set’
  356 |     mo_block->updated().set();
      |                         ^~~
/d/qserdel/ros_ws/voxbloxpp_ws/src/tsdf-plusplus/tsdf_plusplus/src/mesh/mesh_integrator.cc: In member function ‘bool MOMeshIntegrator::generateMesh(bool, bool)’:
/d/qserdel/ros_ws/voxbloxpp_ws/src/tsdf-plusplus/tsdf_plusplus/src/mesh/mesh_integrator.cc:58:49: error: ‘Update’ has not been declared
   58 |     map_->getMapLayerPtr()->getAllUpdatedBlocks(Update::kMesh, &all_map_blocks);
      |                                                 ^~~~~~
/d/qserdel/ros_ws/voxbloxpp_ws/src/tsdf-plusplus/tsdf_plusplus/src/mesh/mesh_integrator.cc: In member function ‘void MOMeshIntegrator::generateMeshBlocksFunction(const BlockIndexList&, bool, voxblox::ThreadSafeIndex*)’:
/d/qserdel/ros_ws/voxbloxpp_ws/src/tsdf-plusplus/tsdf_plusplus/src/mesh/mesh_integrator.cc:104:24: error: ‘struct std::atomic<bool>’ has no member named ‘reset’
  104 |       block->updated().reset(Update::kMesh);
      |                        ^~~~~
/d/qserdel/ros_ws/voxbloxpp_ws/src/tsdf-plusplus/tsdf_plusplus/src/mesh/mesh_integrator.cc:104:30: error: ‘Update’ has not been declared
  104 |       block->updated().reset(Update::kMesh);
      |                              ^~~~~~
make[2]: *** [CMakeFiles/tsdf_plusplus.dir/build.make:89: CMakeFiles/tsdf_plusplus.dir/src/core/map.cc.o] Error 1
make[2]: *** Waiting for unfinished jobs....
/d/qserdel/ros_ws/voxbloxpp_ws/src/tsdf-plusplus/tsdf_plusplus/src/integrator/integrator.cc: In member function ‘MOVoxel* Integrator::allocateStorageAndGetVoxelPtr(const GlobalIndex&, voxblox::Block<MOVoxel>::Ptr*, voxblox::BlockIndex*)’:
/d/qserdel/ros_ws/voxbloxpp_ws/src/tsdf-plusplus/tsdf_plusplus/src/integrator/integrator.cc:435:28: error: ‘struct std::atomic<bool>’ has no member named ‘set’
  435 |   (*last_block)->updated().set();

Hi, your work is amazing and thank you very much for the code! Could you please provide some instructions on how to run it? I would really like to test it on some datasets.

When compiling for ROS Noetic on Ubuntu 20.04, I encountered the error

CMake Error at /usr/lib/x86_64-linux-gnu/cmake/Boost-1.71.0/BoostConfig.cmake:117 (find_package): Could not find a package configuration file provided by "boost_signals" (requested version 1.71.0) with any of the following names:

boost_signalsConfig.cmake
boost_signals-config.cmake

Add the installation prefix of "boost_signals" to CMAKE_PREFIX_PATH or set "boost_signals_DIR" to a directory containing one of the above files. If "boost_signals" provides a separate development package or SDK, be sure it has been installed. Call Stack (most recent call first): /usr/lib/x86_64-linux-gnu/cmake/Boost-1.71.0/BoostConfig.cmake:182 (boost_find_component) /usr/share/cmake-3.16/Modules/FindBoost.cmake:443 (find_package) CMakeLists.txt:6 (find_package)

This was solved by simply removing signals from src/pcl_catkin/geometry2/tf2/CMakeLists.txt (see this for context).

Thanks for the amazing work!

Hi there! Thank you for your great work. But how exactly do I need to preprocess the input data before passing it to the tsdf_plusplus_node on the /segment_pointcloud topic? Thanks in advance!