Dynamic hair modeling from monocular videos using deep neural networks

Last update: Oct 18, 2022

Related tags

Deep Learning Dynamic-Hair

Overview

Dynamic Hair Modeling

The source code of the networks for our paper "Dynamic hair modeling from monocular videos using deep neural networks" (SIGGRAPH ASIA 2019)

We propose a novel framework for dynamic hair modeling from monocular videos. We use two networks HairSpatNet and HairTempNet to separately predict hair geometry and hair motion. The entire framework is as follows:

Improvments

For HairSpatNet, we removed instance normalization and the discriminator to speed up training process and reduce memory cost. We found that the fine-grained details imposed by the discriminator would be obliterated by the space-time optimization afterwards.
For motion prediction, we redesigned a network named HairWarpNet to directly predict flow based on the 3D fields (similar to the regression of optical flow). It is more reasonable and achieves better results than HairTempNet.
There are more designs of toVoxel modules.
You can check other research directions in folder OtherResearch.

Prerequisites

Linux
Python 3.6
NVIDIA GPU + CUDA 10.0 + cuDNN 7.5
tensorflow-gpu 1.13.1

Getting Started

Conda installation:

# 1. Create a conda virtual environment.
conda create -n dhair python=3.6 -y
conda activate dhair

# 2. Install dependency
pip install -r requirement.txt

You can run the scripts in the Script folder to train/test your models.

Citation

If you find this useful for your research, please cite the following paper.

@article{yang2019dynamic,
  title={Dynamic hair modeling from monocular videos using deep neural networks},
  author={Yang, Lingchen and Shi, Zefeng and Zheng, Youyi and Zhou, Kun},
  journal={ACM Transactions on Graphics (TOG)},
  volume={38},
  number={6},
  pages={1--12},
  year={2019},
}

Comments

How should I prepare my input?

Thank you so much for releasing the code of the project! I have a problem preparing the input data.

I put a mp4 video file into test_video_dir but it didn't work out. Then I converted the video into a sequence of 130 jpg images, starting from "frame001.jpg", which also ended in "ValueError: invalid literal for int() with base 10: '001.jpg'".

So my question is: what kind of input should I prepare exactly?

opened by qiqigit 1

Complex-Valued Neural Networks (CVNN)Complex-Valued Neural Networks (CVNN)

Complex-Valued Neural Networks (CVNN) Done by @NEGU93 - J. Agustin Barrachina Using this library, the only difference with a Tensorflow code is that y

1 Nov 12, 2021

A framework that constructs deep neural networks, autoencoders, logistic regressors, and linear networks

A framework that constructs deep neural networks, autoencoders, logistic regressors, and linear networks without the use of any outside machine learning libraries - all from scratch.

2 Nov 14, 2022

Open source repository for the code accompanying the paper 'Non-Rigid Neural Radiance Fields Reconstruction and Novel View Synthesis of a Deforming Scene from Monocular Video'.

Non-Rigid Neural Radiance Fields This is the official repository for the project "Non-Rigid Neural Radiance Fields: Reconstruction and Novel View Synt

296 Dec 29, 2022

Cancer-and-Tumor-Detection-Using-Inception-model - In this repo i am gonna show you how i did cancer/tumor detection in lungs using deep neural networks, specifically here the Inception model by google.

Cancer-and-Tumor-Detection-Using-Inception-model In this repo i am gonna show you how i did cancer/tumor detection in lungs using deep neural networks

1 Jan 1, 2022

Bayesian-Torch is a library of neural network layers and utilities extending the core of PyTorch to enable the user to perform stochastic variational inference in Bayesian deep neural networks

Bayesian-Torch is a library of neural network layers and utilities extending the core of PyTorch to enable the user to perform stochastic variational inference in Bayesian deep neural networks. Bayesian-Torch is designed to be flexible and seamless in extending a deterministic deep neural network architecture to corresponding Bayesian form by simply replacing the deterministic layers with Bayesian layers.

210 Jan 4, 2023

An implementation demo of the ICLR 2021 paper Neural Attention Distillation: Erasing Backdoor Triggers from Deep Neural Networks in PyTorch.

Neural Attention Distillation This is an implementation demo of the ICLR 2021 paper Neural Attention Distillation: Erasing Backdoor Triggers from Deep

84 Jan 4, 2023

DeepHyper: Scalable Asynchronous Neural Architecture and Hyperparameter Search for Deep Neural Networks

What is DeepHyper? DeepHyper is a software package that uses learning, optimization, and parallel computing to automate the design and development of

214 Jan 8, 2023

This repository contains notebook implementations of the following Neural Process variants: Conditional Neural Processes (CNPs), Neural Processes (NPs), Attentive Neural Processes (ANPs).

The Neural Process Family This repository contains notebook implementations of the following Neural Process variants: Conditional Neural Processes (CN

892 Dec 28, 2022

[CVPR 2021] Monocular depth estimation using wavelets for efficiency

Single Image Depth Prediction with Wavelet Decomposition Michaël Ramamonjisoa, Michael Firman, Jamie Watson, Vincent Lepetit and Daniyar Turmukhambeto

205 Jan 2, 2023

Dynamic hair modeling from monocular videos using deep neural networks

Related tags

Overview

Dynamic Hair Modeling

Improvments

Prerequisites

Getting Started

Citation

You might also like...

Complex-Valued Neural Networks (CVNN)Complex-Valued Neural Networks (CVNN)

A framework that constructs deep neural networks, autoencoders, logistic regressors, and linear networks

Open source repository for the code accompanying the paper 'Non-Rigid Neural Radiance Fields Reconstruction and Novel View Synthesis of a Deforming Scene from Monocular Video'.

Cancer-and-Tumor-Detection-Using-Inception-model - In this repo i am gonna show you how i did cancer/tumor detection in lungs using deep neural networks, specifically here the Inception model by google.

Bayesian-Torch is a library of neural network layers and utilities extending the core of PyTorch to enable the user to perform stochastic variational inference in Bayesian deep neural networks

An implementation demo of the ICLR 2021 paper Neural Attention Distillation: Erasing Backdoor Triggers from Deep Neural Networks in PyTorch.

DeepHyper: Scalable Asynchronous Neural Architecture and Hyperparameter Search for Deep Neural Networks

This repository contains notebook implementations of the following Neural Process variants: Conditional Neural Processes (CNPs), Neural Processes (NPs), Attentive Neural Processes (ANPs).

[CVPR 2021] Monocular depth estimation using wavelets for efficiency

Comments

How should I prepare my input?

Owner

Reproducing code of hair style replacement method from Barbershorp.

Implementation of the "Point 4D Transformer Networks for Spatio-Temporal Modeling in Point Cloud Videos" paper.

Code and datasets for the paper "Combining Events and Frames using Recurrent Asynchronous Multimodal Networks for Monocular Depth Prediction" (RA-L, 2021)

Search and filter videos based on objects that appear in them using convolutional neural networks

Dynamic vae - Dynamic VAE algorithm is used for anomaly detection of battery data

Tensors and Dynamic neural networks in Python with strong GPU acceleration

Tensors and Dynamic neural networks in Python with strong GPU acceleration

Multi-Object Tracking in Satellite Videos with Graph-Based Multi-Task Modeling

Efficient neural networks for analog audio effect modeling

[CIKM 2019] Code and dataset for "Fi-GNN: Modeling Feature Interactions via Graph Neural Networks for CTR Prediction"