Instant-nerf-pytorch - NeRF trained SUPER FAST in pytorch

Last update: Nov 22, 2022

Related tags

Deep Learning instant-nerf-pytorch

Overview

instant-nerf-pytorch

This is WORK IN PROGRESS, please feel free to contribute via pull request.

We are trying to make NeRF train super fast in pytorch by using pytorch bindings for Instant-NGP.

Current Progress:

Code is implemented and runs, but cannot achieve super good results.
Per iteration it is ~3.5x faster than the nerf-pytorch code it is built upon.
VERY quickly (1 min) gets up to ~20 PSNR (this is MUCH faster, even in iteration count than the normal NeRF).
But doesn't really get above ~25 PSNR even when training for a long time.
There is a bug where running the 'fine' network doesn't work, results above are only for coarse network (e.g. N_importance = 0), and speed comparisons also to only course network, this might be the reason the PSNR doesn't get super high.

How to get running:

Install tiny-cuda-nn (https://github.com/NVlabs/tiny-cuda-nn).
Download, install dependencies and run this code.

Links to sources:

The code is based upon nerf-pytorch (https://github.com/yenchenlin/nerf-pytorch).
We are trying to make this faster, by replacing the torch MLP and Pos_enc with those from tiny-cuda-nn (https://github.com/NVlabs/tiny-cuda-nn).
This should then become a pytorch version of the (cuda) code for Instant-NGP (https://github.com/NVlabs/instant-ngp)

Authors:

Jonathon Luiten
Kangle Deng

Notes:

Specify --backbone ngp to enable Instant-NGP (already done in configs/fern_ngp.txt).

You might also like...

A PyTorch implementation of NeRF (Neural Radiance Fields) that reproduces the results.

NeRF-pytorch NeRF (Neural Radiance Fields) is a method that achieves state-of-the-art results for synthesizing novel views of complex scenes. Here are

3.2k Jan 8, 2023

Pytorch implementation for A-NeRF: Articulated Neural Radiance Fields for Learning Human Shape, Appearance, and Pose

A-NeRF: Articulated Neural Radiance Fields for Learning Human Shape, Appearance, and Pose Paper | Website | Data A-NeRF: Articulated Neural Radiance F

172 Dec 22, 2022

Unofficial pytorch-lightning implement of Mip-NeRF

mipnerf_pl Unofficial pytorch-lightning implement of Mip-NeRF, Here are some results generated by this repository (pre-trained models are provided bel

159 Dec 23, 2022

This repository contains a pytorch implementation of "HeadNeRF: A Real-time NeRF-based Parametric Head Model (CVPR 2022)".

HeadNeRF: A Real-time NeRF-based Parametric Head Model This repository contains a pytorch implementation of "HeadNeRF: A Real-time NeRF-based Parametr

294 Jan 1, 2023

(Arxiv 2021) NeRF--: Neural Radiance Fields Without Known Camera Parameters

NeRF--: Neural Radiance Fields Without Known Camera Parameters Project Page | Arxiv | Colab Notebook | Data Zirui Wang¹, Shangzhe Wu², Weidi Xie², Min

411 Dec 26, 2022

PlenOctrees: NeRF-SH Training & Conversion

PlenOctrees Official Repo: NeRF-SH training and conversion This repository contains code to train NeRF-SH and to extract the PlenOctree, constituting

323 Dec 29, 2022

Unofficial & improved implementation of NeRF--: Neural Radiance Fields Without Known Camera Parameters

[Unofficial code-base] NeRF--: Neural Radiance Fields Without Known Camera Parameters [ Project | Paper | Official code base ] ⬅️ Thanks the original

239 Dec 22, 2022

Mip-NeRF: A Multiscale Representation for Anti-Aliasing Neural Radiance Fields.

This repository contains the code release for Mip-NeRF: A Multiscale Representation for Anti-Aliasing Neural Radiance Fields. This implementation is written in JAX, and is a fork of Google's JaxNeRF implementation. Contact Jon Barron if you encounter any issues.

625 Dec 30, 2022

Code release for DS-NeRF (Depth-supervised Neural Radiance Fields)

Depth-supervised NeRF: Fewer Views and Faster Training for Free Project | Paper | YouTube Pytorch implementation of our method for learning neural rad

524 Jan 8, 2023

Comments

Trying to backward through the graph a second time, but the saved intermediate results have already been freed. Specify retain_graph=True when calling backward the first time.

when I use the config, fern_ngp, it raise error, I add retain_graph, but still _module_function_backwardBackward.forward: expected Tensor or tuple of Tensor (got NoneType) for return value 0 may you tell me how to fix it? torch 1.7.1 cu11

opened by chensjtu 0

Instant-nerf-pytorch - NeRF trained SUPER FAST in pytorch

Related tags

Overview

instant-nerf-pytorch

Current Progress:

How to get running:

Links to sources:

Authors:

Notes:

You might also like...

A PyTorch implementation of NeRF (Neural Radiance Fields) that reproduces the results.

Pytorch implementation for A-NeRF: Articulated Neural Radiance Fields for Learning Human Shape, Appearance, and Pose

Unofficial pytorch-lightning implement of Mip-NeRF

This repository contains a pytorch implementation of "HeadNeRF: A Real-time NeRF-based Parametric Head Model (CVPR 2022)".

(Arxiv 2021) NeRF--: Neural Radiance Fields Without Known Camera Parameters

PlenOctrees: NeRF-SH Training & Conversion

Unofficial & improved implementation of NeRF--: Neural Radiance Fields Without Known Camera Parameters

Mip-NeRF: A Multiscale Representation for Anti-Aliasing Neural Radiance Fields.

Code release for DS-NeRF (Depth-supervised Neural Radiance Fields)

Comments

Trying to backward through the graph a second time, but the saved intermediate results have already been freed. Specify retain_graph=True when calling backward the first time.

Owner

HashNeRF-pytorch - Pure PyTorch Implementation of NVIDIA paper on Instant Training of Neural Graphics primitives

Torch-ngp - A pytorch implementation of the hash encoder proposed in instant-ngp

Instant Real-Time Example-Based Style Transfer to Facial Videos

Instant-Teaching: An End-to-End Semi-Supervised Object Detection Framework

Annotate datasets with a semi-trained or fully trained YOLOv5 model

A collection of pre-trained StyleGAN2 models trained on different datasets at different resolution.

Pre-trained Deep Learning models and demos (high quality and extremely fast)

Fast and Context-Aware Framework for Space-Time Video Super-Resolution (VCIP 2021)

NeRF Meta-Learning with PyTorch

This repository contains a PyTorch implementation of "AD-NeRF: Audio Driven Neural Radiance Fields for Talking Head Synthesis".