Algorithm to texture 3D reconstructions from multi-view stereo images

Hi, I used MVS to texture my mesh and it really impressed me. Now, I have some multi-tiff files (it means several bands in one tif file) which need to be textured, but I am failed to use the current version MVS to do that. I am wonder whether there are some ways to achieve that, or is it a heavy work to revise the code?

Hi, I have reconstructed mesh models with another application. Now, when I set camera parameters and apply textures on models, a strange shift is visible in all models! I have used the following configuration: 1- Using Lens Distortions K1 and K2 2- Principal pointX=0.5-(my principal pointX / X dimension of sensor) 3- Principal pointY=0.5-(my principal pointY / Y dimension of sensor) 4- Aspect Ratio ~1 5- Focal lenght=Focal length/largest dimension of sensor Moreover, I used tex::image_undistort_bundler and text::image_undistort_vsfm for making undistorted images. I am sure that camera parameters and point clouds have been created with high accuracy, but I cannot find the causes of this problem. Is there any assumption in setting camera parameters that I have missed?

Hi Nils, These days I compile your code of mvs-texturing on windows platform,but there is little problems.Since msvc support openmp2.0, but your code, there are details about openmp3.0 or higher.The code is as follows: void generate_texture_atlases(std::vectorTexturePatch::Ptr * orig_texture_patches, std::vectorTextureAtlas::Ptr * texture_atlases) {

std::list<TexturePatch::ConstPtr> texture_patches;
while (!orig_texture_patches->empty()) {
    //TODO avoid copying
    texture_patches.push_back(orig_texture_patches->back());
    orig_texture_patches->pop_back();
}

std::cout << "\tSorting texture patches... " << std::flush;
/* Improve the bin-packing algorithm efficiency by sorting texture patches
 * in descending order of size. */
texture_patches.sort(comp);
std::cout << "done." << std::endl;

std::size_t const total_num_patches = texture_patches.size();
std::size_t remaining_patches = texture_patches.size();
std::ofstream tty("/dev/tty", std::ios_base::out);

#pragma omp parallel
{
#pragma omp single
{

while (!texture_patches.empty()) {
    unsigned int texture_size = calculate_texture_size(texture_patches);

    texture_atlases->push_back(TextureAtlas::create(texture_size));
    TextureAtlas::Ptr texture_atlas = texture_atlases->back();

    /* Try to insert each of the texture patches into the texture atlas. */
    std::list<TexturePatch::ConstPtr>::iterator it = texture_patches.begin();
    for (; it != texture_patches.end();) {
        std::size_t done_patches = total_num_patches - remaining_patches;
        int precent = static_cast<float>(done_patches)
            / total_num_patches * 100.0f;
        if (total_num_patches > 100
            && done_patches % (total_num_patches / 100) == 0) {

            tty << "\r\tWorking on atlas " << texture_atlases->size() << " "
             << precent << "%... " << std::flush;
        }

        if (texture_atlas->insert(*it)) {
            it = texture_patches.erase(it);
            remaining_patches -= 1;
        } else {
            ++it;
        }
    }

    #pragma omp task
    texture_atlas->finalize();
}

std::cout << "\r\tWorking on atlas " << texture_atlases->size()
    << " 100%... done." << std::endl;
util::WallTimer timer;
std::cout << "\tFinalizing texture atlases... " << std::flush;
#pragma omp taskwait
std::cout << "done. (Took: " << timer.get_elapsed_sec() << "s)" << std::endl;

/* End of single region */
}
/* End of parallel region. */
}

} It shows "task" should openmp directives name when I compile this code.So I ask this code (on task and taskwait) could use other instructions (openmp2.0) replace it?

In newer versions of texrecon, the color mapping is worse than before. As an example, I made a textured 3d reconstruction of the kermit scene (from bundler) with an old texrecon (github of around April 2015) and the current one (github of 23.09.2015). I have seen the same behaviour also with other scenes, however less drastic. Is there a way to have the new features (MVS2 compatibility, reduced memory consumption) and the old color mapping?

Old texrecon:

New texrecon:

Both reconstructions were made with Regard3D (www.regard3d.org, I am the author).

Thanks in advance!

I have a mesh.ply file and .cam and .jpg files in cam folder. I am running this command "./texrecon cam mesh.ply textured " I got this error "Load and prepare mesh: Reading PLY: 478986 verts... 956298 faces... done. Generating texture views: Loading 0%...Could not load image header of /home/indshine-2/Downloads/openMVG_Build/Linux-x86_64-RELEASE/cam/cam/DJI_0001.JPG /home/indshine-2/Downloads/openMVG_Build/Linux-x86_64-RELEASE/cam/cam/DJI_0001.JPG: No such file or directory" Then I included cam/cam folder and moved my .cam and .jpg data in it and again ran. Generating texture views: No proper input scene descriptor given. A input descriptor can be: BUNDLE_FILE - a bundle file (currently onle .nvm files are supported) SCENE_FOLDER - a folder containing images and .cam files MVE_SCENE::EMBEDDING - a mve scene and embedding

Can you tell me what am I missing here? Thank You.

Thanks for sharing your code!

The last version of texrecon (compiled yesterday) crashes for the following dataset: https://www.dropbox.com/s/fvh6wbzgq9zs9ms/texrecon_test_now_complete.zip?dl=0 (Update: one image was missing in the first version of the zip archive. Now it's complete.)

The command I used is:

Do you have any idea what may be the problem?

Regards, Philipp

This is more of a conceptual question that I was hoping to get answered before I embark on it.

I have successfully ran a set of images through a fairly-long SfM pipeline involving OpenMVG, MVE and finally mvs-texturing. The result is a textured OBJ file that I can import into Blender and view. Very cool!

I would now like to manipulate that model programmatically by changing the RGB-color values for a subset of 3D points inside of it. The outcome would be that certain objects or sections inside the final textured model might appear, say, bright red, whereas they are not bright red in the original input images, etc.

I see that MVE's final output is a PLY file (OUTPUT_MESH_CLEAN.ply). I believe this is the main input for mvs-texturing, for example:

/path/to/texrecon MVE::undistorted MVE/OUTPUT_MESH_CLEAN.ply /path/to/output/mymodel

My theory is that I could:

1. Use a PLY reader (in a language of my choosing), combined with my own software to select the 3D points/vertices that I want to color/modify and set their new color values; then
2. Save the modified PLY file as, say, OUT_MESH_CLEAN_MODIFIED.ply; then
3. Load the modified PLY into mvs-texturing like I have already done with the non-modified one produced directly by MVE

Is this the correct course of action, or are there other inputs (for mvs-texturing) besides the PLY file that would also need to be changed in order to have the desired effect?

Thanks in advance!

When processing a large dataset, I will often set up a remote machine with a large mounted drive where the dataset is stored. / is only about 20GB. There are temporary files that are stored to /tmp so when it fills up the volume mvs-texturing will crash. Two questions:

1. Can these files be stored in a temporary folder local to the dataset?
2. Why are these temporary files so much bigger than the actual images? My images are 3-7MB but those in /tmp are 22-25MB.

For reference I think the code in question is here: https://github.com/nmoehrle/mvs-texturing/blob/4fb24145a0201f7d18a1e3eb9ac2eecef624065c/libs/tex/generate_texture_views.cpp#L161

and https://github.com/nmoehrle/mvs-texturing/blob/master/libs/tex/generate_texture_views.cpp#L193

Hi,

im currently working with OpenDroneMap and i ran into an issue (i can work around it but its annoying) stemming from this library

the concrete point of failure is https://github.com/nmoehrle/mvs-texturing/blob/733ff76734c2eff09850b223d16b005203ed2383/libs/tex/generate_texture_views.cpp#L194

this lead to a permission problem.

example: user "a" runs ODM with pictureset "pic_a". at some point the above mentioned piece of code is invoked, and images are copied to /tmp. these images are now owned by "a", and do not get deleted after completion

now user "b" also runs ODM with pictureset "pic_a", and your library tries to overwrite the images that were left in /tmp. this leads to a thrown util::FileException and ODM crashes.

i would propose using a seperate random directory in tmp each time the program is run (seperate directory so several users could process the same imageset at the same time, if necessary), and also deleting it after the program is done, so the tmp directory doesnt fill up

i would do it myself, but i am not proficient enough in c++.

if the thing with the directory is too much effort, it would be enough (for my purposes) if the images got deleted after the program is done

I have test the n-channel branch with a 10 channels file, and the results are showed below.

First, the textured result with the single band file as input images, it is very pretty.

Below is the textured result with 10 channels tif file as input images.

1. the result of the first band, it looks ok.

but, the textured result for other channels seems not very good, 2. the result for the second band

The result for the fourth band

The result for the tenth band

I've been struggling to make a build for Windows and been experiencing same issues described here earlier. I did try Andres fork with no success. Here's example of few errors I'm getting:

'vertex_idx': index variable in OpenMP 'for' statement must have signed integral type

'edge_idx': index variable in OpenMP 'for' statement must have signed integral type 

'capture': unexpected token encountered on OpenMP 'atomic' directive

Expression following '#pragma omp atomic' has improper form

'i': index variable in OpenMP 'for' statement must have signed integral type

and so on...

Any ideas on what may cause these errors?

Hi, I am new to mvs-texturing and What I want to do is generate a texture for my .ply mesh file which generated from colmap

As the website http://www.gcc.tu-darmstadt.de/home/proj/texrecon/ is not available now, so all the information I have is the prompt from executing ./texrecon that tells me how to set the arguments:

./texrecon (built on Sep  7 2022, 10:55:32)

Textures a mesh given images in form of a 3D scene.

Usage: ./texrecon [options] IN_SCENE IN_MESH OUT_PREFIX

IN_SCENE := (SCENE_FOLDER | BUNDLE_FILE | MVE_SCENE::EMBEDDING)

....

As I understood, IN_SCENE can be either a SCENE_FOLDER or a BUNDLE_FILE or MVE_SCENE::EMBEDDING, I used a .nvm file (exported from colmap) as IN_SCENE;

As for IN_MESH , I used fused.ply generated from colmap, thus I put the command as ./texrecon ~/mvs-texturing/test/tutorial3.nvm ~/mvs-texturing/test/fused.ply recon, however it gave following errors:

./texrecon (built on Sep  7 2022, 10:55:32)
Load and prepare mesh:
Reading PLY: 101042 verts... done.
Generating texture views:
NVM: Loading file...
NVM: Number of views: 0
NVM: Number of features: 101042
NVM: 101042 strange points not seem by any camera!
No proper input scene descriptor given.
A input descriptor can be:
BUNDLE_FILE - a bundle file (currently onle .nvm files are supported)
SCENE_FOLDER - a folder containing images and .cam files
MVE_SCENE::EMBEDDING - a mve scene and embedding

Hello, has anyone else had a use case like this?

I have an issue with the .obj file that I generate using the texrecon function call. When I am trying to view the textured model in Swift, using the following code:

let scene = SCNScene(named: "HeadModel3D.obj") self.sceneView.scene = scene

The issue is nothing ends up being displayed in the preview window. There is no indication the model is present. This doesn't happen with other textured .obj files I try to use online; those ones work just fine and display with the color textures successfully. I was wondering if there was a specific setting about the .objs generated with MVS Texturing that makes it not work.

Below are the parameters I passed to texrecon.

'--outlier_removal=gauss_damping' '--tone_mapping=none' '--data_term=gmi' '--no_intermediate_results' '-Seye_mask' '-Veye_mask/views_per_segment.json' 'extracted_highres_kfs/normalized_calibration.json' 'fused_point_cloud_normals_meshed_smoothed.ply' 'dense_reconstructions/HeadModel3D'

when I was building the project, I met: 1:

/home/will/soft/mvs-texturing/elibs/mapmap/mapmap/header/color.h:13:42: fatal error: oneapi/tbb/concurrent_vector.h: No such file or directory
 #include <oneapi/tbb/concurrent_vector.h>

I have checked the directory:/usr/include and /usr/local/include, but never find "oneapi" library, even using $:locate oneapi. but I have tbb in /usr/include (It maybe come from "sudo apt-get install libtbb-dev libtbb2"). And I found there is no difference between my tbb files' name and oneapi/tbb files' name. so I tried to delete all "oneapi/" in error files.But here is new error: error:" tbb::feader" has not been declared. Then I compared and exactly found there are some difference between them. 2: So I installed the oneTBB from [https://github.com/oneapi-src/oneTBB]. And now there are three tbb in my environment:/usr/include/tbb(from libtbb),and /usr/local/include/tbb /usr/local/include/oneapi/tbb And then the lateset error appears:

/home/will/soft/mvs-texturing/elibs/mapmap/mapmap/source/graph.impl.h:58:9 error:reference to 'tbb' is ambigous
tbb::concurrent_vector<complet> * complet_out)

this error also appears after I remove the libtbb(delete /usr/include/tbb). How can I resolve it? Thanks for a lot.

When I was building the project, I met such a mistake: /usr/include/c++/9/tuple:875:3: error: template specialization with C linkage 875 | template<> | ^~~~~~~~ In file included from /usr/include/features.h:461, from /usr/include/x86_64-linux-gnu/c++/9/bits/os_defines.h:39, from /usr/include/x86_64-linux-gnu/c++/9/bits/c++config.h:528, from /usr/include/c++/9/iostream:38, from file_system.cc:10: /usr/include/string.h:28:1: note: ‘extern "C"’ linkage started here 28 | __BEGIN_DECLS | ^~~~~~~~~~~~~ In file included from /usr/include/c++/9/bits/unique_ptr.h:37, from /usr/include/c++/9/bits/locale_conv.h:41, from /usr/include/c++/9/locale:43, from /usr/include/c++/9/iomanip:43, from ./strings.h:15, from /usr/include/string.h:432, from /usr/include/c++/9/cstring:42, from file_system.cc:15: /usr/include/c++/9/tuple:892:3: error: template with C linkage 892 | template<typename _T1, typename _T2> | ^~~~~~~~ In file included from /usr/include/features.h:461, from /usr/include/x86_64-linux-gnu/c++/9/bits/os_defines.h:39, from /usr/include/x86_64-linux-gnu/c++/9/bits/c++config.h:528, from /usr/include/c++/9/iostream:38, from file_system.cc:10: /usr/include/string.h:28:1: note: ‘extern "C"’ linkage started here 28 | __BEGIN_DECLS | ^~~~~~~~~~~~~ In file included from /usr/include/c++/9/bits/unique_ptr.h:37, from /usr/include/c++/9/bits/locale_conv.h:41, from /usr/include/c++/9/locale:43, from /usr/include/c++/9/iomanip:43, from ./strings.h:15, from /usr/include/string.h:432, from /usr/include/c++/9/cstring:42, from file_system.cc:15: /usr/include/c++/9/tuple:1271:3: error: template with C linkage 1271 | template<typename... _Elements> | ^~~~~~~~ In file included from /usr/include/features.h:461, from /usr/include/x86_64-linux-gnu/c++/9/bits/os_defines.h:39, from /usr/include/x86_64-linux-gnu/c++/9/bits/c++config.h:528, from /usr/include/c++/9/iostream:38, from file_system.cc:10: /usr/include/string.h:28:1: note: ‘extern "C"’ linkage started here 28 | __BEGIN_DECLS | ^~~~~~~~~~~~~ In file included from /usr/include/c++/9/bits/unique_ptr.h:37, from /usr/include/c++/9/bits/locale_conv.h:41, from /usr/include/c++/9/locale:43, from /usr/include/c++/9/iomanip:43, from ./strings.h:15, from /usr/include/string.h:432, from /usr/include/c++/9/cstring:42, from file_system.cc:15: /usr/include/c++/9/tuple:1284:3: error: template with C linkage 1284 | template<std::size_t __i, typename _Head, typename... _Tail> | ^~~~~~~~ In file included from /usr/include/features.h:461, from /usr/include/x86_64-linux-gnu/c++/9/bits/os_defines.h:39, from /usr/include/x86_64-linux-gnu/c++/9/bits/c++config.h:528, from /usr/include/c++/9/iostream:38, from file_system.cc:10: /usr/include/string.h:28:1: note: ‘extern "C"’ linkage started here 28 | __BEGIN_DECLS | ^~~~~~~~~~~~~ In file included from /usr/include/c++/9/bits/unique_ptr.h:37, from /usr/include/c++/9/bits/locale_conv.h:41, from /usr/include/c++/9/locale:43, from /usr/include/c++/9/iomanip:43, from ./strings.h:15, from /usr/include/string.h:432, from /usr/include/c++/9/cstring:42, from file_system.cc:15: /usr/include/c++/9/tuple:1291:3: error: template with C linkage 1291 | template<typename _Head, typename... _Tail> | ^~~~~~~~ In file included from /usr/include/features.h:461, from /usr/include/x86_64-linux-gnu/c++/9/bits/os_defines.h:39, from /usr/include/x86_64-linux-gnu/c++/9/bits/c++config.h:528, from /usr/include/c++/9/iostream:38, from file_system.cc:10: /usr/include/string.h:28:1: note: ‘extern "C"’ linkage started here 28 | __BEGIN_DECLS | ^~~~~~~~~~~~~ In file included from /usr/include/c++/9/bits/unique_ptr.h:37, from /usr/include/c++/9/bits/locale_conv.h:41, from /usr/include/c++/9/locale:43, from /usr/include/c++/9/iomanip:43, from ./strings.h:15, from /usr/include/string.h:432, from /usr/include/c++/9/cstring:42, from file_system.cc:15: /usr/include/c++/9/tuple:1300:3: error: template with C linkage 1300 | template<size_t __i> | ^~~~~~~~ In file included from /usr/include/features.h:461, from /usr/include/x86_64-linux-gnu/c++/9/bits/os_defines.h:39, from /usr/include/x86_64-linux-gnu/c++/9/bits/c++config.h:528, from /usr/include/c++/9/iostream:38, from file_system.cc:10: