Implementation of "Meta-rPPG: Remote Heart Rate Estimation Using a Transductive Meta-Learner"

Eugene Lee

Last update: Dec 13, 2022

Related tags

Deep Learning Meta-rPPG

Overview

Meta-rPPG: Remote Heart Rate Estimation Using a Transductive Meta-Learner

This repository is the official implementation of Meta-rPPG: Remote Heart Rate Estimation Using a Transductive Meta-Learner that has been accepted to ECCV 2020.

Heatmap Visualization

Left to right:

Cropped input image
End-to-end trained model (baseline)
Meta-rPPG (transducive inference)
Top to down: rPPG signal, Power Spectral Density (PSD), Predicted and ground truth heart rate

Requirements

To install requirements:

pip install -r requirements.txt

All experiments can be run on a single NVIDIA GTX1080Ti GPU.

The code was tested with python3.6 the following software versions:

Software	version
cuDNN	7.6.5
Pytorch	1.5.0
CUDA	10.2

Training

Training Data Preparation

Download training data (example.pth) from Google Drive. Due to privacy issue (face images), provided data contains only a subset of the entire training data, i.e. contains faces of the authors of this paper.

Move example.pth to data/ directory:

mv example.pth data/

Begin Training

To begin training, run:

python3 train.py

Validation Data

Validation data can be requested from:

MAHNOB-HCI

UBFC-rPPG

Contributing

If you find this work useful, consider citing our work using the following bibTex:

@inproceedings{lee2020meta,
  title={Meta-rPPG: Remote Heart Rate Estimation Using a Transductive Meta-Learner},
  author={Lee, Eugene and Chen, Evan and Lee, Chen-Yi},
  booktitle={European Conference on Computer Vision (ECCV)},
  year={2020}
}

Comments

learning phase

Thanks for releasing the code for the meta-rPPG paper! Very interesting paper!

I am wondering where is the code for the learning phase described in the algrotihm 1 (Training of Meta-Learner)? Is it 'def fewshot_learning' whic is currently unifished? If so, does the trainng pipeline work without it?

Thanks!

opened by xliucs 7
Where do the benchmarks for other methods on UBFC come from?

In the paper there is a comparison of Meta-rPPG with other methods, but not all of the referenced authors list MAE, RMSE and r on UBFC. Could you please tell me whether you have trained these models on your own data as well?

opened by aptlin 1
Duplicate few-shot updates of the encoder

Thanks so much for releasing the code for the great paper!

At the moment the encoder seems to get updated twice during validation, judging from the following two loops:

https://github.com/eugenelet/Meta-rPPG/blob/bab00d97731d07966eaa3ca700be153fac0bd50a/model/main_model.py#L367

https://github.com/eugenelet/Meta-rPPG/blob/bab00d97731d07966eaa3ca700be153fac0bd50a/model/main_model.py#L376

Is it a bug? Am I missing anything?

opened by aptlin 1
What should be the input for the method set_input_for_test?

Based on the example.pth, I thought it should be a tensor with size (1, 60, 3, 64, 64) representing the input returned by the test SlideWindowDataLoader, but when I try to use this method with model.test(), it gives me the following error: "Expected hidden[0] size (4, 1, 60), got (4, 3, 60)". So I would like to know what should be the shape. Also, if possible, could tell me the diference between model.fewshot_test() and model.test()?

opened by LucasLO1 0
PreProcessing Issue
It is calling baseline process function, after checking baseline process it seems you are doing mean subtraction, mask multiplication and butter processing, but when I checked the output It is actually only doing mean subtraction and not mask multiplication and butter etc. Is it intentional or is it a bug in code ? Thanks

self.baseline_procress(inputs, masks.clone()) ppg = self.quantify(ppg)
opened by AnmolDuaAI 0
Pretrained model

Hi, Thank you for releasing the code for Meta-rPPG, very interesting paper indeed. However, It would be really helpful if you could provided access to the custom training data-set or if privacy is a concern you could maybe upload the pretrained model.

Thank you.

opened by ridasaleem0 0
train on VIPL dataset

Hi @eugenelet @EvanChen022kan, Thanks for your work, I know that BVP is the signal that results when highpass filtering the PPG signal, so can I use BVP in VIPL dataset as ground truth label in model? How do you normalize ppg label in example.pth file?

opened by chuan298 0
UBFC Video - In which part of the code we need to pass it ?

Hi All,

I'm working on this meta-rPPG. I got access for UBFC dataset for validation.

But I'm not sure how to pass the bunch of subject.

In the current work, author directly passed .pth format.

Please help me with it

opened by keerthanas18 0

Owner

Eugene Lee

Live. Code. Hack.

GitHub

ALBERT-pytorch-implementation - ALBERT pytorch implementation

ALBERT-pytorch-implementation developing... 모델의 개념이해를 돕기 위한 구현물로 현재 변수명을 상세히 적었고

3 Oct 6, 2022

Numenta Platform for Intelligent Computing is an implementation of Hierarchical Temporal Memory (HTM), a theory of intelligence based strictly on the neuroscience of the neocortex.

NuPIC Numenta Platform for Intelligent Computing The Numenta Platform for Intelligent Computing (NuPIC) is a machine intelligence platform that implem

6.3k Dec 30, 2022

PyTorch implementation of neural style transfer algorithm

neural-style-pt This is a PyTorch implementation of the paper A Neural Algorithm of Artistic Style by Leon A. Gatys, Alexander S. Ecker, and Matthias

770 Jan 2, 2023

PyTorch implementation of DeepDream algorithm

neural-dream This is a PyTorch implementation of DeepDream. The code is based on neural-style-pt. Here we DeepDream a photograph of the Golden Gate Br

121 Nov 5, 2022

The project is an official implementation of our CVPR2019 paper "Deep High-Resolution Representation Learning for Human Pose Estimation"

Deep High-Resolution Representation Learning for Human Pose Estimation (CVPR 2019) News [2020/07/05] A very nice blog from Towards Data Science introd

3.9k Jan 5, 2023

Image-to-Image Translation with Conditional Adversarial Networks (Pix2pix) implementation in keras

pix2pix-keras Pix2pix implementation in keras. Original paper: Image-to-Image Translation with Conditional Adversarial Networks (pix2pix) Paper Author

141 Dec 30, 2022

Python implementation of cover trees, near-drop-in replacement for scipy.spatial.kdtree

This is a Python implementation of cover trees, a data structure for finding nearest neighbors in a general metric space (e.g., a 3D box with periodic

28 Nov 25, 2022

Home repository for the Regularized Greedy Forest (RGF) library. It includes original implementation from the paper and multithreaded one written in C++, along with various language-specific wrappers.

Regularized Greedy Forest Regularized Greedy Forest (RGF) is a tree ensemble machine learning method described in this paper. RGF can deliver better r

364 Dec 28, 2022

Implementation of Restricted Boltzmann Machine (RBM) and its variants in Tensorflow

xRBM Library Implementation of Restricted Boltzmann Machine (RBM) and its variants in Tensorflow Installation Using pip: pip install xrbm Examples Tut

55 Dec 29, 2022

A fast Evolution Strategy implementation in Python

Evostra: Evolution Strategy for Python Evolution Strategy (ES) is an optimization technique based on ideas of adaptation and evolution. You can learn

251 Dec 8, 2022

🌳 A Python-inspired implementation of the Optimum-Path Forest classifier.

OPFython: A Python-Inspired Optimum-Path Forest Classifier Welcome to OPFython. Note that this implementation relies purely on the standard LibOPF. Th

30 Jan 4, 2023

Implementation of Geometric Vector Perceptron, a simple circuit for 3d rotation equivariance for learning over large biomolecules, in Pytorch. Idea proposed and accepted at ICLR 2021

Geometric Vector Perceptron Implementation of Geometric Vector Perceptron, a simple circuit with 3d rotation equivariance for learning over large biom

59 Nov 24, 2022

Official implementation of AAAI-21 paper "Label Confusion Learning to Enhance Text Classification Models"

Description: This is the official implementation of our AAAI-21 accepted paper Label Confusion Learning to Enhance Text Classification Models. The str

101 Nov 25, 2022

Official PyTorch implementation for paper Context Matters: Graph-based Self-supervised Representation Learning for Medical Images

Context Matters: Graph-based Self-supervised Representation Learning for Medical Images Official PyTorch implementation for paper Context Matters: Gra

49 Nov 23, 2022

PyTorch implementation of "Conformer: Convolution-augmented Transformer for Speech Recognition" (INTERSPEECH 2020)

PyTorch implementation of Conformer: Convolution-augmented Transformer for Speech Recognition. Transformer models are good at capturing content-based

565 Jan 4, 2023

An essential implementation of BYOL in PyTorch + PyTorch Lightning

Essential BYOL A simple and complete implementation of Bootstrap your own latent: A new approach to self-supervised Learning in PyTorch + PyTorch Ligh

48 Sep 27, 2022

The official implementation of NeMo: Neural Mesh Models of Contrastive Features for Robust 3D Pose Estimation [ICLR-2021]. https://arxiv.org/pdf/2101.12378.pdf

NeMo: Neural Mesh Models of Contrastive Features for Robust 3D Pose Estimation [ICLR-2021] Release Notes The offical PyTorch implementation of NeMo, p

76 Nov 23, 2022

A PyTorch re-implementation of the paper 'Exploring Simple Siamese Representation Learning'. Reproduced the 67.8% Top1 Acc on ImageNet.

Exploring simple siamese representation learning This is a PyTorch re-implementation of the SimSiam paper on ImageNet dataset. The results match that

72 Nov 9, 2022

PyTorch implementation of "A Full-Band and Sub-Band Fusion Model for Real-Time Single-Channel Speech Enhancement."

FullSubNet This Git repository for the official PyTorch implementation of "A Full-Band and Sub-Band Fusion Model for Real-Time Single-Channel Speech E

357 Jan 4, 2023