This is a work in progress reimplementation of Instant Neural Graphics Primitives

Penn

Last update: Sep 1, 2022

Related tags

Deep Learning neural_hash_encoding

Overview

Neural Hash Encoding

This is a work in progress reimplementation of Instant Neural Graphics Primitives Currently this can train an implicit representation of a gigapixel image using a multires hash encoding.

FYI: This is brand new -- most parameters in the training script are hard coded right now

Check out results in viz

Setup:

Download the Tokyo image

wget -O tokyo.jpg https://live.staticflickr.com/1859/29314390837_b39ae4876e_o_d.jpg

Convert to numpy binary format for faster reading (1s w/ .npz vs 14s with .jpg)

from PIL import Image
Image.MAX_IMAGE_PIXELS = 10**10

img = np.asarray(Image.open("tokyo.jpg"))   # Abount 3.5 gb
np.save("tokyo.npy", img)

Train:

python src/train_image.py

Implementation Notes (From the Paper)

Architecture

In all tasks, except for NeRF which we will describe later, we use an MLP with two hidden layers that have a width of 64 neurons and rectified linear unit (ReLU)

4. Initialization

Initialize hash table entries with uniform distribution [-1e-4, 1e-4]

4. Training

Optimizer
- Adam: β1 = 0.9, β2 = 0.99, ϵ = 1e−15
- Learning rate: 1e-2 (ref: tiny-cuda-nn)
Regularization:
- L2: 10e-6 Applied to the MLP weigths not the hash table weights

we skip Adam steps for hash table entries whose gradient is exactly 0. This saves ∼10% performance when gradients are sparse

You might also like...

PyTorch reimplementation of minimal-hand (CVPR2020)

Minimal Hand Pytorch Unofficial PyTorch reimplementation of minimal-hand (CVPR2020). you can also find in youtube or bilibili bare hand youtube or bil

228 Dec 29, 2022

PyTorch reimplementation of the paper Involution: Inverting the Inherence of Convolution for Visual Recognition [CVPR 2021].

Involution: Inverting the Inherence of Convolution for Visual Recognition Unofficial PyTorch reimplementation of the paper Involution: Inverting the I

100 Dec 1, 2022

PyTorch reimplementation of hand-biomechanical-constraints (ECCV2020)

Hand Biomechanical Constraints Pytorch Unofficial PyTorch reimplementation of Hand-Biomechanical-Constraints (ECCV2020). This project reimplement foll

59 Dec 20, 2022

A PyTorch Reimplementation of TecoGAN: Temporally Coherent GAN for Video Super-Resolution

TecoGAN-PyTorch Introduction This is a PyTorch reimplementation of TecoGAN: Temporally Coherent GAN for Video Super-Resolution (VSR). Please refer to

165 Dec 17, 2022

a reimplementation of Optical Flow Estimation using a Spatial Pyramid Network in PyTorch

pytorch-spynet This is a personal reimplementation of SPyNet [1] using PyTorch. Should you be making use of this work, please cite the paper according

269 Jan 2, 2023

An official reimplementation of the method described in the INTERSPEECH 2021 paper - Speech Resynthesis from Discrete Disentangled Self-Supervised Representations.

Speech Resynthesis from Discrete Disentangled Self-Supervised Representations Implementation of the method described in the Speech Resynthesis from Di

253 Jan 6, 2023

Comments

add support for nd interpolation
Re-names:

HashArray2D -> HashArray

Interpolate2D -> Interpolate

DenseEncodingLevel2D -> DenseEncodingLevel

HashEncodingLevel2D -> HashEncodingLevel

MultiResEncoding2D -> MultiResEncoding

Adds map_coordinates adapted from jax.scipy.ndimage.map_coordinates for n-dimensional interpolation
opened by jenkspt 0
Added PyTorch version with NeuS training

Check out my PyTorch version that implements the MultiresHashEncoding: https://github.com/jenkspt/multires-hash-encoding-pytorch And the NeuS fork that replaces the NeRF-like modules with MultiresHashEncodings: https://github.com/jenkspt/NeuS/tree/hash And discussion here: https://github.com/Totoro97/NeuS/issues/65

opened by jenkspt 0
add NeRF training
Adapt jax code from sarafridov/plenoxels

Replace the plenoxel sparse grid with HashArray of

Replace spherical harmonic eval_sh with MLP

The instant neural graphics nerf training is more similar to plenoxel than original nerf, since plenoxel uses trilinear interpolation on a sparse grid.
opened by jenkspt 0

This is a work in progress reimplementation of Instant Neural Graphics Primitives

Related tags

Overview

Neural Hash Encoding

Setup:

Train:

Implementation Notes (From the Paper)

Architecture

4. Initialization

4. Training

You might also like...

PyTorch reimplementation of minimal-hand (CVPR2020)

PyTorch reimplementation of the paper Involution: Inverting the Inherence of Convolution for Visual Recognition [CVPR 2021].

PyTorch reimplementation of hand-biomechanical-constraints (ECCV2020)

A PyTorch Reimplementation of TecoGAN: Temporally Coherent GAN for Video Super-Resolution

a reimplementation of Optical Flow Estimation using a Spatial Pyramid Network in PyTorch

An official reimplementation of the method described in the INTERSPEECH 2021 paper - Speech Resynthesis from Discrete Disentangled Self-Supervised Representations.

PyTorch reimplementation of REALM and ORQA

Reimplementation of the paper "Attention, Learn to Solve Routing Problems!" in jax/flax.

a reimplementation of UnFlow in PyTorch that matches the official TensorFlow version

Comments

add support for nd interpolation

Added PyTorch version with NeuS training

add NeRF training

Owner

Penn

Optimized primitives for collective multi-GPU communication

Reimplementation of the paper `Human Attention Maps for Text Classification: Do Humans and Neural Networks Focus on the Same Words? (ACL2020)`

Instant Real-Time Example-Based Style Transfer to Facial Videos

Instant-Teaching: An End-to-End Semi-Supervised Object Detection Framework

Torch-ngp - A pytorch implementation of the hash encoder proposed in instant-ngp

Instant-nerf-pytorch - NeRF trained SUPER FAST in pytorch

Collection of in-progress libraries for entity neural networks.

NPBG++: Accelerating Neural Point-Based Graphics

[ECCV 2020] Reimplementation of 3DDFAv2, including face mesh, head pose, landmarks, and more.

text_recognition_toolbox: The reimplementation of a series of classical scene text recognition papers with Pytorch in a uniform way.