Neural implicit reconstruction experiments for the Vector Neuron paper

Related tags

Deep Learning vnn-neural-implicits
Overview

Neural Implicit Reconstruction with Vector Neurons

This repository contains code for the neural implicit reconstruction experiments in the paper Vector Neurons: A General Framework for SO(3)-Equivariant Networks. Code for classification and segmentation experiments can be found here.

[Project] [Paper]

Preparation

The code structure follows Occupancy Networks. Please follow their instructions to prepare the data and install the dependencies. Run

python generate_random_rotation.py

to precompute the random rotations for all input pointclouds.

Usage

To train and evaluate the networks, please run these two commands

python train.py CONFIG.yaml
python eval.py CONFIG.yaml

The configuration files are, for VN-OccNet,

configs/equinet/vnn_pointnet_resnet_resnet_ROTATION.yaml

for the vanilla OccNet baseline,

configs/pointcloud/onet_resnet_ROTATION.yaml

and for vanilla PointNet encoder + invariant decoder,

configs/equinet/inner_baseline_resnet_ROTATION.yaml

ROTATION can be chosen from aligned (no rotations) and so3 (with precomputed random rotations). We also provide two settings rot-rand (generate random rotations on the fly during training) and pca (apply PCA pre-alignment the the input pointclouds), which are not reported in the paper.

Citation

Please cite this paper if you want to use it in your work,

@article{deng2021vn,
  title={Vector Neurons: a general framework for SO(3)-equivariant networks},
  author={Deng, Congyue and Litany, Or and Duan, Yueqi and Poulenard, Adrien and Tagliasacchi, Andrea and Guibas, Leonidas},
  journal={arXiv preprint arXiv:2104.12229},
  year={2021}
}

License

MIT License

Acknowledgement

The structure of this codebase is borrowed from Occupancy Networks.

You might also like...
Unofficial Tensorflow 2 implementation of the paper Implicit Neural Representations with Periodic Activation Functions
Unofficial Tensorflow 2 implementation of the paper Implicit Neural Representations with Periodic Activation Functions

Siren: Implicit Neural Representations with Periodic Activation Functions The unofficial Tensorflow 2 implementation of the paper Implicit Neural Repr

Seyma Yucer 2 Jun 27, 2022
Open source repository for the code accompanying the paper 'Non-Rigid Neural Radiance Fields Reconstruction and Novel View Synthesis of a Deforming Scene from Monocular Video'.
Open source repository for the code accompanying the paper 'Non-Rigid Neural Radiance Fields Reconstruction and Novel View Synthesis of a Deforming Scene from Monocular Video'.

Non-Rigid Neural Radiance Fields This is the official repository for the project "Non-Rigid Neural Radiance Fields: Reconstruction and Novel View Synt

Facebook Research 296 Dec 29, 2022
The code for the CVPR 2021 paper Neural Deformation Graphs, a novel approach for globally-consistent deformation tracking and 3D reconstruction of non-rigid objects.
The code for the CVPR 2021 paper Neural Deformation Graphs, a novel approach for globally-consistent deformation tracking and 3D reconstruction of non-rigid objects.

Neural Deformation Graphs Project Page | Paper | Video Neural Deformation Graphs for Globally-consistent Non-rigid Reconstruction Aljaž Božič, Pablo P

Aljaz Bozic 134 Dec 16, 2022
This is the code repository implementing the paper
This is the code repository implementing the paper "TreePartNet: Neural Decomposition of Point Clouds for 3D Tree Reconstruction".

TreePartNet This is the code repository implementing the paper "TreePartNet: Neural Decomposition of Point Clouds for 3D Tree Reconstruction". Depende

刘彦超 34 Nov 30, 2022
Code for paper ECCV 2020 paper: Who Left the Dogs Out? 3D Animal Reconstruction with Expectation Maximization in the Loop.
Code for paper ECCV 2020 paper: Who Left the Dogs Out? 3D Animal Reconstruction with Expectation Maximization in the Loop.

Who Left the Dogs Out? Evaluation and demo code for our ECCV 2020 paper: Who Left the Dogs Out? 3D Animal Reconstruction with Expectation Maximization

Benjamin Biggs 29 Dec 28, 2022
Simple command line tool for text to image generation using OpenAI's CLIP and Siren (Implicit neural representation network)
Simple command line tool for text to image generation using OpenAI's CLIP and Siren (Implicit neural representation network)

Deep Daze mist over green hills shattered plates on the grass cosmic love and attention a time traveler in the crowd life during the plague meditative

Phil Wang 4.4k Jan 3, 2023
Implicit Graph Neural Networks

Implicit Graph Neural Networks This repository is the official PyTorch implementation of "Implicit Graph Neural Networks". Fangda Gu*, Heng Chang*, We

Heng Chang 48 Nov 29, 2022
Code for Iso-Points: Optimizing Neural Implicit Surfaces with Hybrid Representations
Code for Iso-Points: Optimizing Neural Implicit Surfaces with Hybrid Representations

Implementation for Iso-Points (CVPR 2021) Official code for paper Iso-Points: Optimizing Neural Implicit Surfaces with Hybrid Representations paper |

Yifan Wang 66 Nov 8, 2022
Pytorch implementation of COIN, a framework for compression with implicit neural representations 🌸
Pytorch implementation of COIN, a framework for compression with implicit neural representations 🌸

COIN 🌟 This repo contains a Pytorch implementation of COIN: COmpression with Implicit Neural representations, including code to reproduce all experim

Emilien Dupont 104 Dec 14, 2022
Comments
  • Issue with environment.yaml file

    Issue with environment.yaml file

    Hello @FlyingGiraffe,

    Thank you for sharing the code. I used environment.yaml file given in this repository to create conda environment. However, I am getting following error while importing pytorch:

    ImportError: /anaconda3/envs/mesh_funcspace/lib/python3.6/site-packages/torch/lib/libmkldnn.so.0: undefined symbol: cblas_sgemm_alloc

    Could you please share the requirements.txt file? It would be helpful to create virtualenv and use pip to install required packages rather than creating conda environment.

    Thank you, Supriya

    opened by supriya-gdptl 3
  • What is T in DecoderInner?

    What is T in DecoderInner?

    Hello @FlyingGiraffe

    I am sorry to disturb you but in line 53 of file /im2mesh/vnn_onet/models/decoder_inner.py I do not understand to what correspond the T dimension of p. For my understanding p is a batch of 3D points and therefore we have Batch_size=batch size, T=?, D=3 Thank you, Julien.

    opened by Julien-Gustin 0
  • random rotation for inputs and points

    random rotation for inputs and points

    Hi, If I understand correctly, here a random rotation is applied on a data, where data is occupancy data i.e. points in space, their occ, and inputs (for point clouds the actual PC). However, it seems that you are applying different rotations for the points and inputs. Doesn't it create points and inputs which are not aligned?

    opened by orenkatzir 0
  • VNResnetBlockFC weight initialization

    VNResnetBlockFC weight initialization

    Hello @FlyingGiraffe,

    Thank you for sharing your nice work! I have a question regarding the VNResnetBlockFC. What is the intuition behind initially setting the weights of fc1 to zero?

    Thank you!

    Best, Philipp

    opened by philippreiser 1
Owner
Congyue Deng
CS PhD student at Stanford, advised by Leonidas Guibas | Previous: math undergrad at Tsinghua
Congyue Deng
GitHub
Vector AI — A platform for building vector based applications. Encode, query and analyse data using vectors.

Vector AI is a framework designed to make the process of building production grade vector based applications as quickly and easily as possible. Create

Vector AI 267 Dec 23, 2022
[ICCV'21] UNISURF: Unifying Neural Implicit Surfaces and Radiance Fields for Multi-View Reconstruction

UNISURF: Unifying Neural Implicit Surfaces and Radiance Fields for Multi-View Reconstruction Project Page | Paper | Supplementary | Video This reposit

null 331 Dec 28, 2022
Neuron Merging: Compensating for Pruned Neurons (NeurIPS 2020)

Neuron Merging: Compensating for Pruned Neurons Pytorch implementation of Neuron Merging: Compensating for Pruned Neurons, accepted at 34th Conference

Woojeong Kim 33 Dec 30, 2022
Deep Implicit Moving Least-Squares Functions for 3D Reconstruction

DeepMLS: Deep Implicit Moving Least-Squares Functions for 3D Reconstruction This repository contains the implementation of the paper: Deep Implicit Mo

null 103 Dec 22, 2022
A Planar RGB-D SLAM which utilizes Manhattan World structure to provide optimal camera pose trajectory while also providing a sparse reconstruction containing points, lines and planes, and a dense surfel-based reconstruction.

ManhattanSLAM Authors: Raza Yunus, Yanyan Li and Federico Tombari ManhattanSLAM is a real-time SLAM library for RGB-D cameras that computes the camera

null 117 Dec 28, 2022
"MST++: Multi-stage Spectral-wise Transformer for Efficient Spectral Reconstruction" (CVPRW 2022) & (Winner of NTIRE 2022 Challenge on Spectral Reconstruction from RGB)

MST++: Multi-stage Spectral-wise Transformer for Efficient Spectral Reconstruction (CVPRW 2022) Yuanhao Cai, Jing Lin, Zudi Lin, Haoqian Wang, Yulun Z

Yuanhao Cai 274 Jan 5, 2023
The LaTeX and Python code for generating the paper, experiments' results and visualizations reported in each paper is available (whenever possible) in the paper's directory

This repository contains the software implementation of most algorithms used or developed in my research. The LaTeX and Python code for generating the

João Fonseca 3 Jan 3, 2023
Code to reproduce the experiments from our NeurIPS 2021 paper " The Limitations of Large Width in Neural Networks: A Deep Gaussian Process Perspective"

Code To run: python runner.py new --save <SAVE_NAME> --data <PATH_TO_DATA_DIR> --dataset <DATASET> --model <model_name> [options] --n 1000 - train - t

Geoff Pleiss 5 Dec 12, 2022
Code for the paper "Implicit Representations of Meaning in Neural Language Models"

Implicit Representations of Meaning in Neural Language Models Preliminaries Create and set up a conda environment as follows: conda create -n state-pr

Belinda Li 39 Nov 3, 2022
Official code release for ICCV 2021 paper SNARF: Differentiable Forward Skinning for Animating Non-rigid Neural Implicit Shapes.

Official code release for ICCV 2021 paper SNARF: Differentiable Forward Skinning for Animating Non-rigid Neural Implicit Shapes.

null 235 Dec 26, 2022