Code for the paper "PortraitNet: Real-time portrait segmentation network for mobile device" @ CAD&Graphics2019

Last update: Dec 1, 2022

Overview

PortraitNet

Code for the paper "PortraitNet: Real-time portrait segmentation network for mobile device". @ CAD&Graphics 2019

Introduction

We propose a real-time portrait segmentation model, called PortraitNet, that can run effectively and efficiently on mobile device. PortraitNet is based on a lightweight U-shape architecture with two auxiliary losses at the training stage, while no additional cost is required at the testing stage for portrait inference.

Portrait segmentation applications on mobile device.

Experimental setup

Requirements

python 2.7
PyTorch 0.3.0.post4
Jupyter Notebook
pip install easydict matplotlib tqdm opencv-python scipy pyyaml numpy

Download datasets

EG1800 Since several image URL links are invalid in the original EG1800 dataset, we finally use 1447 images for training and 289 images for validation.
Supervise-Portrait Supervise-Portrait is a portrait segmentation dataset collected from the public human segmentation dataset Supervise.ly using the same data process as EG1800.

Training

Network Architecture

Overview of PortraitNet.

Training Steps

Download the datasets (EG1800 or Supervise-Portriat). If you want to training at your own dataset, you need to modify data/datasets.py and data/datasets_portraitseg.py.
Prepare training/testing files, like data/select_data/eg1800_train.txt and data/select_data/eg1800_test.txt.
Select and modify the parameters in the folder of config.
Start the training with single gpu:

cd myTrain
python2.7 train.py

Testing

In the folder of myTest:

you can use EvalModel.ipynb to test on testing datasets.
you can use VideoTest.ipynb to test on a single image or video.

Visualization

Using tensorboard to visualize the training process:

cd path_to_save_model
tensorboard --logdir='./log'

Download models

from Dropbox:

mobilenetv2_eg1800_with_two_auxiliary_losses(Training on EG1800 with two auxiliary losses)
mobilenetv2_supervise_portrait_with_two_auxiliary_losses(Training on Supervise-Portrait with two auxiliary losses)
mobilenetv2_total_with_prior_channel(Training on Human with prior channel)

from Baidu Cloud:

mobilenetv2_eg1800_with_two_auxiliary_losses(Training on EG1800 with two auxiliary losses)
mobilenetv2_supervise_portrait_with_two_auxiliary_losses(Training on Supervise-Portrait with two auxiliary losses)
mobilenetv2_total_with_prior_channel(Training on Human with prior channel)

Comments

About mask file structure

Hello How are you? Thanks for contributing this project. I found a strange part in your code. I know that background is 0 and foreground is 1 in case of portrait mask. Is it right?

opened by rose-jinyang 1
Add a License

Hello,

You seem to have a good work here, could you please add a license to the code you are distributing? https://help.github.com/en/github/creating-cloning-and-archiving-repositories/licensing-a-repository "You're under no obligation to choose a license. However, without a license, the default copyright laws apply, meaning that you retain all rights to your source code and no one may reproduce, distribute, or create derivative works from your work. If you're creating an open source project, we strongly encourage you to include an open source license. The Open Source Guide provides additional guidance on choosing the correct license for your project."

Thanks

opened by LucioC 1
model .par

I use the tar command as below...

tar xvf mobilenetv2_total_with_prior_channel.tar

The error message as follows

tar: This does not look like a tar archive tar: Skipping to next header tar: A lone zero block at 290 tar: Exiting with failure status due to previous errors

How can I get model from .tar file ? Please help me. Thanks

opened by comihuang1030 0
Model tar not extracted

Dear dong-x16, Thank you for sharing your model code. But I can't extract the model tar files that stored in Baidu cloud.

When I run tar xvf, following error occurs.

$ tar -xvf mobilenetv2_eg1800_with_two_auxiliary_losses.tar tar: This does not look like a tar archive tar: Skipping to next header tar: Exiting with failure status due to previous errors

All 3 model tars are same. Please check it again.

Best regards.

opened by nauti16 0

some images repeated in Supervisely_face

when processing the images in supervisely_face_train_easy.txt and supervisely_face_test_easy.txt, I found these following images appear both in the train set and the test set.

[('Supervisely_face/SuperviselyPerson_ds4/img/pexels-photo-785667.png',
  'Supervisely_face/SuperviselyPerson_ds2/img/pexels-photo-785667.png'),
 ('Supervisely_face/SuperviselyPerson_ds5/img/pexels-photo-356170.png',
  'Supervisely_face/SuperviselyPerson_ds1/img/pexels-photo-356170.png'),
 ('Supervisely_face/SuperviselyPerson_ds5/img/pexels-photo-681294.png',
  'Supervisely_face/SuperviselyPerson_ds1/img/pexels-photo-681294.png'),
 ('Supervisely_face/SuperviselyPerson_ds5/img/pexels-photo-681294.png',
  'Supervisely_face/SuperviselyPerson_ds1/img/pexels-photo-681294.png'),
 ('Supervisely_face/SuperviselyPerson_ds5/img/pexels-photo-681294.png',
  'Supervisely_face/SuperviselyPerson_ds1/img/pexels-photo-681294.png'),
 ('Supervisely_face/SuperviselyPerson_ds5/img/pexels-photo-681294.png',
  'Supervisely_face/SuperviselyPerson_ds1/img/pexels-photo-681294.png'),
 ('Supervisely_face/SuperviselyPerson_ds2/img/pexels-photo-221212.png',
  'Supervisely_face/SuperviselyPerson_ds4/img/pexels-photo-221212.png'),
 ('Supervisely_face/SuperviselyPerson_ds3/img/pexels-photo-681294.png',
  'Supervisely_face/SuperviselyPerson_ds1/img/pexels-photo-681294.png'),
 ('Supervisely_face/SuperviselyPerson_ds3/img/pexels-photo-681294.png',
  'Supervisely_face/SuperviselyPerson_ds1/img/pexels-photo-681294.png'),
 ('Supervisely_face/SuperviselyPerson_ds3/img/pexels-photo-681294.png',
  'Supervisely_face/SuperviselyPerson_ds1/img/pexels-photo-681294.png'),
 ('Supervisely_face/SuperviselyPerson_ds3/img/pexels-photo-681294.png',
  'Supervisely_face/SuperviselyPerson_ds1/img/pexels-photo-681294.png'),
 ('Supervisely_face/SuperviselyPerson_ds5/img/pexels-photo-365373.png',
  'Supervisely_face/SuperviselyPerson_ds1/img/pexels-photo-365373.png'),
 ('Supervisely_face/SuperviselyPerson_ds2/img/pexels-photo-432506.png',
  'Supervisely_face/SuperviselyPerson_ds5/img/pexels-photo-432506.png'),
 ('Supervisely_face/SuperviselyPerson_ds5/img/pexels-photo-316680.png',
  'Supervisely_face/SuperviselyPerson_ds2/img/pexels-photo-316680.png'),
 ('Supervisely_face/SuperviselyPerson_ds1/img/pexels-photo-569163.png',
  'Supervisely_face/SuperviselyPerson_ds5/img/pexels-photo-569163.png'),
 ('Supervisely_face/SuperviselyPerson_ds5/img/light-sunset-man-trees.png',
  'Supervisely_face/SuperviselyPerson_ds2/img/light-sunset-man-trees.png')]

In each tuple, the first dir appears in the train set and the second one appears in the test set. Though in different folder, the images are actually the same.

opened by jonny4929 0

resume

Thanks for contributing this project. When I was training, I changed the value of 'resume' to continue training, but an error occurred. I don’t know if there are any details that I didn’t notice.

opened by Danelljian 1
Model training

Hello How are you? Thanks for contributing this project. I am training a new model with config model_mobilenetv2_with_two_auxiliary_losses and AISegment dataset for 8 days. But it is strange that the best model is not updated for 7 days. What do u think the reason?

opened by rose-jinyang 3
The mIoU is low

Thanks for your sharing @dong-x16 .

I have an issue. When I train my model without the prior channel, the mIoU of the model can reach 94%, but when I add prior channel, the mIoU of the model is much lower than without the prior channel, which even cannot reach 90%.

Could you give me a piece of advice about how to set the parameters of every loss?

Thanks a lot.

opened by jackieyung 10
model's dimension doesn't match checkpoint's dimension?

HI,参照myTest目录下的video测试demo脚本, 下载页面提供的模型链接文件将 154行：File = os.path.join(exp_args.model_root, 'model_best.pth.tar') 改成 bestModelFile = os.path.join(exp_args.model_root, 'mobilenetv2_total_with_prior_channel.tar') 运行后产生如下错误

videotest.py:101: YAMLLoadWarning: calling yaml.load() without Loader=... is deprecated, as the default Loader is unsafe. Please read https://msg.pyyaml.org/load for full details. cf = load(cont) finish load config file ... Traceback (most recent call last): File "videotest.py", line 158, in netmodel_video.load_state_dict(checkpoint_video['state_dict']) File "/home/hcruise/.local/lib/python2.7/site-packages/torch/nn/modules/module.py", line 487, in load_state_dict .format(name, own_state[name].size(), param.size())) RuntimeError: While copying the parameter named stage0.0.weight, whose dimensions in the model are torch.Size([32, 4, 3, 3]) and whose dimensions in the checkpoint are torch.Size([32, 3, 3, 3]).

这是什么情况，我是安装主页上说的安装的pytorch 0.3.0post4, 及torchvision 0.2.0 为什么load模型后又说自己的模型dimension不对呢？

opened by chithize 0

Owner

GitHub

Inference code for "StylePeople: A Generative Model of Fullbody Human Avatars" paper. This code is for the part of the paper describing video-based avatars.

NeuralTextures This is repository with inference code for paper "StylePeople: A Generative Model of Fullbody Human Avatars" (CVPR21). This code is for

Visual Understanding Lab @ Samsung AI Center Moscow

18 Oct 6, 2022

This is the official source code for SLATE. We provide the code for the model, the training code, and a dataset loader for the 3D Shapes dataset. This code is implemented in Pytorch.

SLATE This is the official source code for SLATE. We provide the code for the model, the training code and a dataset loader for the 3D Shapes dataset.

66 Dec 26, 2022

Code for paper ECCV 2020 paper: Who Left the Dogs Out? 3D Animal Reconstruction with Expectation Maximization in the Loop.

Who Left the Dogs Out? Evaluation and demo code for our ECCV 2020 paper: Who Left the Dogs Out? 3D Animal Reconstruction with Expectation Maximization

29 Dec 28, 2022

TensorFlow code for the neural network presented in the paper: "Structural Language Models of Code" (ICML'2020)

SLM: Structural Language Models of Code This is an official implementation of the model described in: "Structural Language Models of Code" [PDF] To ap

73 Nov 6, 2022

Code for the prototype tool in our paper "CoProtector: Protect Open-Source Code against Unauthorized Training Usage with Data Poisoning".

CoProtector Code for the prototype tool in our paper "CoProtector: Protect Open-Source Code against Unauthorized Training Usage with Data Poisoning".

1 Oct 26, 2021

Code to use Augmented Shapiro Wilks Stopping, as well as code for the paper "Statistically Signifigant Stopping of Neural Network Training"

This codebase is being actively maintained, please create and issue if you have issues using it Basics All data files are included under losses and ea

32 Nov 9, 2021

Code for our method RePRI for Few-Shot Segmentation. Paper at http://arxiv.org/abs/2012.06166

Region Proportion Regularized Inference (RePRI) for Few-Shot Segmentation In this repo, we provide the code for our paper : "Few-Shot Segmentation Wit

138 Dec 12, 2022

Code for ACM MM 2020 paper "NOH-NMS: Improving Pedestrian Detection by Nearby Objects Hallucination"

NOH-NMS: Improving Pedestrian Detection by Nearby Objects Hallucination The offical implementation for the "NOH-NMS: Improving Pedestrian Detection by

64 Nov 11, 2022

Official TensorFlow code for the forthcoming paper

~ Efficient-CapsNet ~ Are you tired of over inflated and overused convolutional neural networks? You're right! It's time for CAPSULES :)

203 Jan 8, 2023

This is the code for the paper "Contrastive Clustering" (AAAI 2021)

Contrastive Clustering (CC) This is the code for the paper "Contrastive Clustering" (AAAI 2021) Dependency python>=3.7 pytorch>=1.6.0 torchvision>=0.8

210 Dec 30, 2022

Code for the paper Learning the Predictability of the Future

Learning the Predictability of the Future Code from the paper Learning the Predictability of the Future. Website of the project in hyperfuture.cs.colu

Computer Vision Lab at Columbia University

139 Nov 18, 2022

PyTorch code for the paper: FeatMatch: Feature-Based Augmentation for Semi-Supervised Learning

FeatMatch: Feature-Based Augmentation for Semi-Supervised Learning This is the PyTorch implementation of our paper: FeatMatch: Feature-Based Augmentat

43 Nov 19, 2022

Code for the paper A Theoretical Analysis of the Repetition Problem in Text Generation

A Theoretical Analysis of the Repetition Problem in Text Generation This repository share the code for the paper "A Theoretical Analysis of the Repeti

37 Nov 21, 2022

Code for our ICASSP 2021 paper: SA-Net: Shuffle Attention for Deep Convolutional Neural Networks

SA-Net: Shuffle Attention for Deep Convolutional Neural Networks (paper) By Qing-Long Zhang and Yu-Bin Yang [State Key Laboratory for Novel Software T

199 Jan 8, 2023

Open source repository for the code accompanying the paper 'Non-Rigid Neural Radiance Fields Reconstruction and Novel View Synthesis of a Deforming Scene from Monocular Video'.

Non-Rigid Neural Radiance Fields This is the official repository for the project "Non-Rigid Neural Radiance Fields: Reconstruction and Novel View Synt

296 Dec 29, 2022