An Efficient Implementation of Analytic Mesh Algorithm for 3D Iso-surface Extraction from Neural Networks

Overview

AnalyticMesh

Analytic Marching is an exact meshing solution from neural networks. Compared to standard methods, it completely avoids geometric and topological errors that result from insufficient sampling, by means of mathematically guaranteed analysis.

This repository gives an implementation of Analytic Marching algorithm. This algorithm is initially proposed in our conference paper Analytic Marching: An Analytic Meshing Solution from Deep Implicit Surface Networks, then finally improved in our journal paper: Learning and Meshing from Deep Implicit Surface Networks Using an Efficient Implementation of Analytic Marching.

Our codes provide web pages for manipulating your models via graphic interface, and a backend for giving full control of the algorithm by writing python codes.

Installation

First please download our codes:

git clone https://github.com/Karbo123/AnalyticMesh.git --depth=1
cd AnalyticMesh
export AMROOT=`pwd`

Backend

Backend gives a python binding of analytic marching. You can write simple python codes in your own project after compiling the backend.

Our implementation supports pytorch, and possibly also other deep learning frameworks (e.g. tensorflow), but we do not test other frameworks yet.

Requirements:

Compilation:

cd $AMROOT/backend
mkdir build && cd build
cmake ..
make -j8
cd ..

If your pytorch version < 1.5.1, you may need to fix cpp extension compile failure on some envs.

Make sure compiled library can pass the tests. Run:

CUDA_VISIBLE_DEVICES=0 PYTHONDONTWRITEBYTECODE=1 pytest -s -p no:warnings -p no:cacheprovider

It will generate some files under folder $AMROOT/backend/tmp. Generally, those generated meshes (.ply) are watertight, you can check with meshlab.

If it passes all the tests, you can finally link to somewhere so that python can find it:

ln -s $AMROOT `python -c 'import site; print(site.getsitepackages()[0])'`

Frontend

We also provide an easy-to-use interactive interface to apply analytic marching to your input network model by just clicking your mouse. To use the web interface, you may follow steps below to install.

Requirement:

Before compiling, you may need to modify the server information given in file frontend/pages/src/assets/index.js. Then you can compile those files by running:

cd $AMROOT/frontend/pages
npm install
npm run build

The $AMROOT/frontend/pages/dist directory is ready to be deployed. If you want to deploy web pages to a server, please additionally follow these instructions.

To start the server, simply run:

cd $AMROOT/frontend && python server.py

You can open the interface via either opening file $AMROOT/frontend/pages/dist/index.html on your local machine or opening the url to which the page is deployed.

Demo

We provide some samples in $AMROOT/examples, you can try them.

Here we show a simple example (which is from $AMROOT/examples/2_polytope.py):

import os
import torch
from AnalyticMesh import save_model, load_model, AnalyticMarching

class MLPPolytope(torch.nn.Module):
    def __init__(self):
        super(MLPPolytope, self).__init__()
        self.linear0 = torch.nn.Linear(3, 14)
        self.linear1 = torch.nn.Linear(14, 1)
        with torch.no_grad(): # here we give the weights explicitly since training takes time
            weight0 = torch.tensor([[ 1,  1,  1],
                                    [-1, -1, -1],
                                    [ 0,  1,  1],
                                    [ 0, -1, -1],
                                    [ 1,  0,  1],
                                    [-1,  0, -1],
                                    [ 1,  1,  0],
                                    [-1, -1,  0],
                                    [ 1,  0,  0],
                                    [-1,  0,  0],
                                    [ 0,  1,  0],
                                    [ 0, -1,  0],
                                    [ 0,  0,  1],
                                    [ 0,  0, -1]], dtype=torch.float32)
            bias0 = torch.zeros(14)
            weight1 = torch.ones([14], dtype=torch.float32).unsqueeze(0)
            bias1 = torch.tensor([-2], dtype=torch.float32)

            add_noise = lambda x: x + torch.randn_like(x) * (1e-7)
            self.linear0.weight.copy_(add_noise(weight0))
            self.linear0.bias.copy_(add_noise(bias0))
            self.linear1.weight.copy_(add_noise(weight1))
            self.linear1.bias.copy_(add_noise(bias1))

    def forward(self, x):
        return self.linear1(torch.relu(self.linear0(x)))


if __name__ == "__main__":
    #### save onnx
    DIR = os.path.dirname(os.path.abspath(__file__)) # the directory to save files
    onnx_path = os.path.join(DIR, "polytope.onnx")
    save_model(MLPPolytope(), onnx_path) # we save the model as onnx format
    print(f"we save onnx to: {onnx_path}")

    #### save ply
    ply_path = os.path.join(DIR, "polytope.ply")
    model = load_model(onnx_path) # load as a specific model
    AnalyticMarching(model, ply_path) # do analytic marching
    print(f"we save ply to: {ply_path}")

API

We mainly provide the following two ways to use analytic marching:

  • Web interface (provides an easy-to-use graphic interface)
  • Python API (gives more detailed control)
  1. Web interface

    You should compile both the backend and frontend to use this web interface. Its usage is detailed in the user guide on the web page.

  2. Python API

    It's very simple to use, just three lines of code.

    from AnalyticMesh import load_model, AnalyticMarching 
    model = load_model(load_onnx_path) 
    AnalyticMarching(model, save_ply_path)

    If results are not satisfactory, you may need to change default values of the AnalyticMarching function.

    To obtain an onnx model file, you can just use the save_model function we provide.

    from AnalyticMesh import save_model
    save_model(your_custom_nn_module, save_onnx_path)

Some tips:

  • It is highly recommended that you try dichotomy first as the initialization method.
  • If CUDA runs out of memory, try setting voxel_configs. It will partition the space and solve them serially.
  • More details are commented in our source codes.

Use Analytic Marching in your own project

There are generally three ways to use Analytic Marching.

  1. Directly representing a single shape by a multi-layer perceptron. For a single object, you can simply represent the shape as a single network. For example, you can directly fit a point cloud by a multi-layer perceptron. In this way, the weights of the network uniquely determine the shape.
  2. Generating the weights of multi-layer perceptron from a hyper-network. To learn from multiple shapes, one can use hyper-network to generate the weights of multi-layer perceptron in a learnable manner.
  3. Re-parameterizing the latent code into the bias of the first layer. To learn from multiple shapes, we can condition the network with a latent code input at the first layer (e.g. 3+256 -> 512 -> 512 -> 1). Note that the concatenated latent code can be re-parameterized and combined into the bias of the first layer. More specifically, the computation of the first layer can be re-parameterized as , where the newly computed bias is .

About

This repository is mainly maintained by Jiabao Lei (backend) and Yongyi Su (frontend). If you have any question, feel free to create an issue on github.

If you find our works useful, please consider citing our papers.

@inproceedings{
    Lei2020,
    title = {Analytic Marching: An Analytic Meshing Solution from Deep Implicit Surface Networks},
    author = {Jiabao Lei and Kui Jia},
    booktitle = {International Conference on Machine Learning 2020 {ICML-20}},
    year = {2020},
    month = {7}
}

@misc{
    Lei2021,
    title={Learning and Meshing from Deep Implicit Surface Networks Using an Efficient Implementation of Analytic Marching}, 
    author={Jiabao Lei and Kui Jia and Yi Ma},
    year={2021},
    eprint={2106.10031},
    archivePrefix={arXiv},
    primaryClass={cs.CV}
}

Contact: [email protected]

You might also like...
Multiview Neural Surface Reconstruction by Disentangling Geometry and Appearance
Multiview Neural Surface Reconstruction by Disentangling Geometry and Appearance

Multiview Neural Surface Reconstruction by Disentangling Geometry and Appearance Project Page | Paper | Data This repository contains an implementatio

Subdivision-based Mesh Convolutional Networks
Subdivision-based Mesh Convolutional Networks

Subdivision-based Mesh Convolutional Networks The official implementation of SubdivNet in our paper, Subdivion-based Mesh Convolutional Networks Requi

PyTorch Implementation of [1611.06440] Pruning Convolutional Neural Networks for Resource Efficient Inference

PyTorch implementation of [1611.06440 Pruning Convolutional Neural Networks for Resource Efficient Inference] This demonstrates pruning a VGG16 based

Implementation of Memory-Efficient Neural Networks with Multi-Level Generation, ICCV 2021
Implementation of Memory-Efficient Neural Networks with Multi-Level Generation, ICCV 2021

Memory-Efficient Multi-Level In-Situ Generation (MLG) By Jiaqi Gu, Hanqing Zhu, Chenghao Feng, Mingjie Liu, Zixuan Jiang, Ray T. Chen and David Z. Pan

A PyTorch port of the Neural 3D Mesh Renderer
A PyTorch port of the Neural 3D Mesh Renderer

Neural 3D Mesh Renderer (CVPR 2018) This repo contains a PyTorch implementation of the paper Neural 3D Mesh Renderer by Hiroharu Kato, Yoshitaka Ushik

Code For TDEER: An Efficient Translating Decoding Schema for Joint Extraction of Entities and Relations (EMNLP2021)
Code For TDEER: An Efficient Translating Decoding Schema for Joint Extraction of Entities and Relations (EMNLP2021)

TDEER (WIP) Code For TDEER: An Efficient Translating Decoding Schema for Joint Extraction of Entities and Relations (EMNLP2021) Overview TDEER is an e

FAMIE is a comprehensive and efficient active learning (AL) toolkit for multilingual information extraction (IE)

FAMIE: A Fast Active Learning Framework for Multilingual Information Extraction

Implementation for the
Implementation for the "Surface Reconstruction from 3D Line Segments" paper.

Surface Reconstruction from 3D Line Segments Surface reconstruction from 3d line segments. Langlois, P. A., Boulch, A., & Marlet, R. In 2019 Internati

The official implementation code of
The official implementation code of "PlantStereo: A Stereo Matching Benchmark for Plant Surface Dense Reconstruction."

PlantStereo This is the official implementation code for the paper "PlantStereo: A Stereo Matching Benchmark for Plant Surface Dense Reconstruction".

Comments
  • reserved identifier violation

    reserved identifier violation

    I would like to point out that identifiers like “__MLP_H__” and “__POLYMESH_H__do not fit to the expected naming convention of the C++ language standard. Would you like to adjust your selection for unique names?

    opened by elfring 3
  • What cuda version do you use? error: calling a __host__ function(

    What cuda version do you use? error: calling a __host__ function("cudaMemcpyAsync") ... is not allowed.

    Hi! First of all, thanks for the impressive works! I'm trying to compile the backend. The cmake step has successfully passed, but I ran into a bug when running the make step.

    [ 33%] Building CUDA object CMakeFiles/cuam.dir/src/cuam_kernel.cu.o
    In file included from /home/PJLAB/guojianfei/ai_ws/shape/AnalyticMesh/backend/src/cuam_kernel.cu:12:0:
    /home/PJLAB/guojianfei/ai_ws/shape/AnalyticMesh/backend/inc/polymesh.h:342:2: warning: #warning It may contain degenerated faces (such as `A-B-C-C-D-A`) [-Wcpp]
     #warning It may contain degenerated faces (such as `A-B-C-C-D-A`)
      ^
    In file included from /home/PJLAB/guojianfei/ai_ws/shape/AnalyticMesh/backend/src/cuam_kernel.cu:13:0:
    /home/PJLAB/guojianfei/ai_ws/shape/AnalyticMesh/backend/inc/kernel.h:595:2: warning: #warning `UseCublas` branch of function `solve_3x3_Batched` is not implemented. [-Wcpp]
     #warning `UseCublas` branch of function `solve_3x3_Batched` is not implemented.
      ^
    /home/PJLAB/guojianfei/ai_ws/shape/AnalyticMesh/backend/inc/kernel.h(1469): error: calling a __host__ function("cudaMemcpyAsync") from a __global__ function("inferNewStates_kernel<double> ") is not allowed
    
    /home/PJLAB/guojianfei/ai_ws/shape/AnalyticMesh/backend/inc/kernel.h(1469): error: identifier "cudaMemcpyAsync" is undefined in device code
    
    /home/PJLAB/guojianfei/ai_ws/shape/AnalyticMesh/backend/inc/kernel.h(1469): error: calling a __host__ function("cudaMemcpyAsync") from a __global__ function("inferNewStates_kernel<float> ") is not allowed
    
    /home/PJLAB/guojianfei/ai_ws/shape/AnalyticMesh/backend/inc/kernel.h(1469): error: identifier "cudaMemcpyAsync" is undefined in device code
    
    4 errors detected in the compilation of "/tmp/tmpxft_00004759_00000000-6_cuam_kernel.cpp1.ii".
    CMakeFiles/cuam.dir/build.make:89: recipe for target 'CMakeFiles/cuam.dir/src/cuam_kernel.cu.o' failed
    make[2]: *** [CMakeFiles/cuam.dir/src/cuam_kernel.cu.o] Error 1
    CMakeFiles/Makefile2:99: recipe for target 'CMakeFiles/cuam.dir/all' failed
    make[1]: *** [CMakeFiles/cuam.dir/all] Error 2
    Makefile:90: recipe for target 'all' failed
    make: *** [all] Error 2
    

    I've tried google the error info, but surprisingly there are no relative results. Do you have any clue?

    OS: ubuntu16 Graphics card: GTX1660s CUDA: 10.0 cudnn: 7.6.5

    opened by ventusff 2
Owner
Karbo
Karbo
A repo that contains all the mesh keys needed for mesh backend, along with a code example of how to use them in python

Mesh-Keys A repo that contains all the mesh keys needed for mesh backend, along with a code example of how to use them in python Have been seeing alot

Joseph 53 Dec 13, 2022
Mesh Graphormer is a new transformer-based method for human pose and mesh reconsruction from an input image

MeshGraphormer ✨ ✨ This is our research code of Mesh Graphormer. Mesh Graphormer is a new transformer-based method for human pose and mesh reconsructi

Microsoft 251 Jan 8, 2023
CoSMA: Convolutional Semi-Regular Mesh Autoencoder. From Paper "Mesh Convolutional Autoencoder for Semi-Regular Meshes of Different Sizes"

Mesh Convolutional Autoencoder for Semi-Regular Meshes of Different Sizes Implementation of CoSMA: Convolutional Semi-Regular Mesh Autoencoder arXiv p

Fraunhofer SCAI 10 Oct 11, 2022
Given a 2D triangle mesh, we could randomly generate cloud points that fill in the triangle mesh

generate_cloud_points Given a 2D triangle mesh, we could randomly generate cloud points that fill in the triangle mesh. Run python disp_mesh.py Or you

Peng Yu 2 Dec 24, 2021
AI Face Mesh: This is a simple face mesh detection program based on Artificial intelligence.

AI Face Mesh: This is a simple face mesh detection program based on Artificial Intelligence which made with Python. It's able to detect 468 different

Md. Rakibul Islam 1 Jan 13, 2022
Code for Iso-Points: Optimizing Neural Implicit Surfaces with Hybrid Representations

Implementation for Iso-Points (CVPR 2021) Official code for paper Iso-Points: Optimizing Neural Implicit Surfaces with Hybrid Representations paper |

Yifan Wang 66 Nov 8, 2022
Convert game ISO and archives to CD CHD for emulation on Linux.

tochd Convert game ISO and archives to CD CHD for emulation. Author: Tuncay D. Source: https://github.com/thingsiplay/tochd Releases: https://github.c

Tuncay 20 Jan 2, 2023
A PyTorch implementation of "Cluster-GCN: An Efficient Algorithm for Training Deep and Large Graph Convolutional Networks" (KDD 2019).

ClusterGCN ⠀⠀ A PyTorch implementation of "Cluster-GCN: An Efficient Algorithm for Training Deep and Large Graph Convolutional Networks" (KDD 2019). A

Benedek Rozemberczki 697 Dec 27, 2022
The official implementation of NeMo: Neural Mesh Models of Contrastive Features for Robust 3D Pose Estimation [ICLR-2021]. https://arxiv.org/pdf/2101.12378.pdf

NeMo: Neural Mesh Models of Contrastive Features for Robust 3D Pose Estimation [ICLR-2021] Release Notes The offical PyTorch implementation of NeMo, p

Angtian Wang 76 Nov 23, 2022
Neural Surface Maps

Neural Surface Maps Official implementation of Neural Surface Maps - Luca Morreale, Noam Aigerman, Vladimir Kim, Niloy J. Mitra [Paper] [Project Page]

Luca Morreale 49 Dec 13, 2022