The BCNet related data and inference model.

Last update: Dec 12, 2022

Related tags

Overview

BCNet

This repository includes the some source code and related dataset of paper BCNet: Learning Body and Cloth Shape from A Single Image, ECCV 2020, https://arxiv.org/abs/2004.00214.

Authors: Boyi Jiang, Juyong Zhang, Yang Hong, Jinhao Luo, Ligang Liu, and Hujun Bao.

Note that all of the code and dataset can be only used for research purposes. If you are interested in business purposes/for-profit use, please contact Juyong Zhang (the corresponding author, email: [email protected]).

Dataset

1. Synthetic Dataset

Download and visualization the Synthetic Dataset follow the instruction in body_garment_dataset.

2. HD Texture Dataset

For now, we are unable to release the full training data due to the restriction of commertial scans.

Inference

1. Install

Download the trained model and mv to the "models" folder.
Download the tmps data in body_garment_dataset.
This code is compatible with python 3.7 and pytorch 1.6. In addition, the following packages are required: numpy, torch_geometric 1.5, openmesh, opencv... You can create an anaconda environment called BCNet with the required dependencies by running:

conda env create -f environment.yml
conda activate BCNet

2. Usage

You can generate the results in recs for example images by running the code:

cd code
python infer.py --inputs ../images --save-root ../recs

Citation

Please cite the following paper if it helps your research:

@inproceedings{jiang2020bcnet,
  title={BCNet: Learning Body and Cloth Shape from A Single Image},
  author={Jiang, Boyi and Zhang, Juyong and Hong, Yang and Luo, Jinhao and Liu, Ligang and Bao, Hujun},
  booktitle={European Conference on Computer Vision},
  year={2020},
  organization={Springer}
}

Comments

BCNet with male or female SMPL model

Hi,

Is possible to use BCNet with male or female SMPL models, instead neutral model? What format should be used? Why do you use a '*.txt' file instead of a *.pkl file to import SMPL model? Any explanation will be helpful for me.

Thank for the great work!

opened by Luciano07 5
Comparison with Existing Methods

Thanks for sharing your work!

I have two questions in your Table 2: 1) why not test all methods on BUFF and DW dataset? 2) The corner mark 3 said 'We did not test [38, 7] on this dataset as most of the samples are not A-pose.' However, DW dataset is from MGN.

opened by zhangyahu1 4
About the evaluation on BCNet

Thanks for sharing your work! It helps me a lot! I have a question that how we can compare the BCNet with other work, which mentioned in 5.2, " Because our model predicts separate body and garment meshes, we extract the outer surface of all meshes as the proxy to do registration and error computing for a fair comparison. "

I don't know how to extract the outer surface of the result of BCNet, could you show the code or the method?

Thank you very much!

opened by mirror58229 2
About Garment PCA

我在进行一些实验的时候发现 alpha(衣物的 PCA)随机取一些值会得到不太合理的结果，就像图里所示。我在论文里没有找到关于Garment PCA的生成过程（类似于SMPL的统计学习过程），请问PCA基是怎么获得的呢，或者如何得到一个比较符合真实情况的alpha呢（比较光滑，不会出现渗透等等）？

opened by Luckyalum 2
camera intrinsic matrix

Thank you for your code. As mentioned in the paper, all training data share a common camera intrinsic matrix. Can you provide the intrinsic matrix of the camera? I want to use projection to generate multi-view images. Thank you.

opened by Luckyalum 2
RuntimeError: Error(s) in loading state_dict for ImageReconstructModel:

Hi @jby1993 Thanks for the awesome work and making the code public . I tried running the model for python3.7 and pytorch1.6 , but still I am getting this issue . Can you please help me resoslve this ?

Thanks Trilok

Traceback (most recent call last): File "infer.py", line 116, in <module> net.load_state_dict(torch.load('../models/garNet.pth',map_location='cpu'),True) File "/home/trilok/.local/lib/python3.7/site-packages/torch/nn/modules/module.py", line 1045, in load_state_dict self.__class__.__name__, "\n\t".join(error_msgs))) RuntimeError: Error(s) in loading state_dict for ImageReconstructModel: Missing key(s) in state_dict: "skinWsNet.ress.0.conv1.att_l", "skinWsNet.ress.0.conv1.att_r", "skinWsNet.ress.0.conv1.lin_l.weight", "skinWsNet.ress.0.conv1.lin_r.weight", "skinWsNet.ress.0.conv2.att_l", "skinWsNet.ress.0.conv2.att_r", "skinWsNet.ress.0.conv2.lin_l.weight", "skinWsNet.ress.0.conv2.lin_r.weight", "skinWsNet.ress.1.conv1.att_l", "skinWsNet.ress.1.conv1.att_r", "skinWsNet.ress.1.conv1.lin_l.weight", "skinWsNet.ress.1.conv1.lin_r.weight", "skinWsNet.ress.1.conv2.att_l", "skinWsNet.ress.1.conv2.att_r", "skinWsNet.ress.1.conv2.lin_l.weight", "skinWsNet.ress.1.conv2.lin_r.weight", "skinWsNet.ress.2.conv1.att_l", "skinWsNet.ress.2.conv1.att_r", "skinWsNet.ress.2.conv1.lin_l.weight", "skinWsNet.ress.2.conv1.lin_r.weight", "skinWsNet.ress.2.conv2.att_l", "skinWsNet.ress.2.conv2.att_r", "skinWsNet.ress.2.conv2.lin_l.weight", "skinWsNet.ress.2.conv2.lin_r.weight", "skinWsNet.ress.3.conv1.att_l", "skinWsNet.ress.3.conv1.att_r", "skinWsNet.ress.3.conv1.lin_l.weight", "skinWsNet.ress.3.conv1.lin_r.weight", "skinWsNet.ress.3.conv2.att_l", "skinWsNet.ress.3.conv2.att_r", "skinWsNet.ress.3.conv2.lin_l.weight", "skinWsNet.ress.3.conv2.lin_r.weight". Unexpected key(s) in state_dict: "skinWsNet.ress.0.conv1.lin.weight", "skinWsNet.ress.0.conv1.att_i", "skinWsNet.ress.0.conv1.att_j", "skinWsNet.ress.0.conv2.lin.weight", "skinWsNet.ress.0.conv2.att_i", "skinWsNet.ress.0.conv2.att_j", "skinWsNet.ress.1.conv1.lin.weight", "skinWsNet.ress.1.conv1.att_i", "skinWsNet.ress.1.conv1.att_j", "skinWsNet.ress.1.conv2.lin.weight", "skinWsNet.ress.1.conv2.att_i", "skinWsNet.ress.1.conv2.att_j", "skinWsNet.ress.2.conv1.lin.weight", "skinWsNet.ress.2.conv1.att_i", "skinWsNet.ress.2.conv1.att_j", "skinWsNet.ress.2.conv2.lin.weight", "skinWsNet.ress.2.conv2.att_i", "skinWsNet.ress.2.conv2.att_j", "skinWsNet.ress.3.conv1.lin.weight", "skinWsNet.ress.3.conv1.att_i", "skinWsNet.ress.3.conv1.att_j", "skinWsNet.ress.3.conv2.lin.weight", "skinWsNet.ress.3.conv2.att_i", "skinWsNet.ress.3.conv2.att_j".

opened by trilokpadhi 1
FileNotFoundError: [Errno 2] No such file or directory: '/~/BCNet/code/module/../../smpl_pytorch/model/neutral_smpl_with_cocoplus_reg.txt'

Traceback (most recent call last): File "infer.py", line 115, in net=M.ImageReconstructModel(skinWsNet,True) File "/home/asimniazi63/Desktop/BCNet/code/module/ImageReconstructModel.py", line 426, in init self.smpl=SMPL(osp.join(osp.dirname(file),'../../smpl_pytorch/model/neutral_smpl_with_cocoplus_reg.txt'),obj_saveable=True) File "../smpl_pytorch/SMPL.py", line 18, in init with open(model_path, 'r') as reader: FileNotFoundError: [Errno 2] No such file or directory: '/home/asimniazi63/Desktop/BCNet/code/module/../../smpl_pytorch/model/neutral_smpl_with_cocoplus_reg.txt'

opened by asimniazi63 1
How to get the texture?

Thank you for your code! Could you please tell me how to get the texture of the initial input image that can be transferred to the 3D garment. Is there any possibility to get a Garment Wardrobe with the code, including the texture and 3D garment, and then dress the SMPL.

opened by LSY1111 1
how to train the network

sorry to bother you, I am trying to construct a new dataset, but I don't know how to train the network. I know the training dataset can't be exposed, But can you provide some introduction about how to train the three networks?

opened by neew123 4
size mismatch for garPcapsLayers.1.components: copying a param with shape torch.Size([64, 12774]) from checkpoint, the shape in current model is torch.Size([64, 8454]).

~/BCNet/code$ python infer.py --inputs ../images --save-root ../recs Traceback (most recent call last): File "infer.py", line 117, in net.load_state_dict(tor, True) File "/opt/conda/lib/python3.7/site-packages/torch/nn/modules/module.py", line 1224, in load_state_dict self.class.name, "\n\t".join(error_msgs))) RuntimeError: Error(s) in loading state_dict for ImageReconstructModel: Missing key(s) in state_dict: "skinWsNet.ress.0.conv1.att_l", "skinWsNet.ress.0.conv1.att_r", "skinWsNet.ress.0.conv1.lin_l.weight", "skinWsNet.ress.0.conv1.lin_r.weight", "skinWsNet.ress.0.conv2.att_l", "skinWsNet.ress.0.conv2.att_r", "skinWsNet.ress.0.conv2.lin_l.weight", "skinWsNet.ress.0.conv2.lin_r.weight", "skinWsNet.ress.1.conv1.att_l", "skinWsNet.ress.1.conv1.att_r", "skinWsNet.ress.1.conv1.lin_l.weight", "skinWsNet.ress.1.conv1.lin_r.weight", "skinWsNet.ress.1.conv2.att_l", "skinWsNet.ress.1.conv2.att_r", "skinWsNet.ress.1.conv2.lin_l.weight", "skinWsNet.ress.1.conv2.lin_r.weight", "skinWsNet.ress.2.conv1.att_l", "skinWsNet.ress.2.conv1.att_r", "skinWsNet.ress.2.conv1.lin_l.weight", "skinWsNet.ress.2.conv1.lin_r.weight", "skinWsNet.ress.2.conv2.att_l", "skinWsNet.ress.2.conv2.att_r", "skinWsNet.ress.2.conv2.lin_l.weight", "skinWsNet.ress.2.conv2.lin_r.weight", "skinWsNet.ress.3.conv1.att_l", "skinWsNet.ress.3.conv1.att_r", "skinWsNet.ress.3.conv1.lin_l.weight", "skinWsNet.ress.3.conv1.lin_r.weight", "skinWsNet.ress.3.conv2.att_l", "skinWsNet.ress.3.conv2.att_r", "skinWsNet.ress.3.conv2.lin_l.weight", "skinWsNet.ress.3.conv2.lin_r.weight". Unexpected key(s) in state_dict: "skinWsNet.ress.0.conv1.lin.weight", "skinWsNet.ress.0.conv1.att_i", "skinWsNet.ress.0.conv1.att_j", "skinWsNet.ress.0.conv2.lin.weight", "skinWsNet.ress.0.conv2.att_i", "skinWsNet.ress.0.conv2.att_j", "skinWsNet.ress.1.conv1.lin.weight", "skinWsNet.ress.1.conv1.att_i", "skinWsNet.ress.1.conv1.att_j", "skinWsNet.ress.1.conv2.lin.weight", "skinWsNet.ress.1.conv2.att_i", "skinWsNet.ress.1.conv2.att_j", "skinWsNet.ress.2.conv1.lin.weight", "skinWsNet.ress.2.conv1.att_i", "skinWsNet.ress.2.conv1.att_j", "skinWsNet.ress.2.conv2.lin.weight", "skinWsNet.ress.2.conv2.att_i", "skinWsNet.ress.2.conv2.att_j", "skinWsNet.ress.3.conv1.lin.weight", "skinWsNet.ress.3.conv1.att_i", "skinWsNet.ress.3.conv1.att_j", "skinWsNet.ress.3.conv2.lin.weight", "skinWsNet.ress.3.conv2.att_i", "skinWsNet.ress.3.conv2.att_j". size mismatch for garPcapsLayers.1.mean: copying a param with shape torch.Size([12774]) from checkpoint, the shape in current model is torch.Size([8454]). size mismatch for garPcapsLayers.1.components: copying a param with shape torch.Size([64, 12774]) from checkpoint, the shape in current model is torch.Size([64, 8454]). size mismatch for garPcapsLayers.1.edge_index: copying a param with shape torch.Size([2, 25224]) from checkpoint, the shape in current model is torch.Size([2, 16572]). size mismatch for garPcapsLayers.1.face_index: copying a param with shape torch.Size([8352, 3]) from checkpoint, the shape in current model is torch.Size([5468, 3]). size mismatch for garPcapsLayers.1.vf_findex: copying a param with shape torch.Size([25056]) from checkpoint, the shape in current model is torch.Size([16404]). size mismatch for garPcapsLayers.1.vf_vindex: copying a param with shape torch.Size([25056]) from checkpoint, the shape in current model is torch.Size([16404]). size mismatch for garDisplacementLayers.1.res1.conv1.spiral_adj: copying a param with shape torch.Size([1, 4259, 20]) from checkpoint, the shape in current model is torch.Size([1, 2819, 20]). size mismatch for garDisplacementLayers.1.res1.conv2.spiral_adj: copying a param with shape torch.Size([1, 4259, 20]) from checkpoint, the shape in current model is torch.Size([1, 2819, 20]). size mismatch for garDisplacementLayers.1.ress.0.conv1.spiral_adj: copying a param with shape torch.Size([1, 4259, 20]) from checkpoint, the shape in current model is torch.Size([1, 2819, 20]). size mismatch for garDisplacementLayers.1.ress.0.conv2.spiral_adj: copying a param with shape torch.Size([1, 4259, 20]) from checkpoint, the shape in current model is torch.Size([1, 2819, 20]). size mismatch for garDisplacementLayers.1.ress.1.conv1.spiral_adj: copying a param with shape torch.Size([1, 4259, 20]) from checkpoint, the shape in current model is torch.Size([1, 2819, 20]). size mismatch for garDisplacementLayers.1.ress.1.conv2.spiral_adj: copying a param with shape torch.Size([1, 4259, 20]) from checkpoint, the shape in current model is torch.Size([1, 2819, 20]). size mismatch for garDisplacementLayers.1.ress.2.conv1.spiral_adj: copying a param with shape torch.Size([1, 4259, 20]) from checkpoint, the shape in current model is torch.Size([1, 2819, 20]). size mismatch for garDisplacementLayers.1.ress.2.conv2.spiral_adj: copying a param with shape torch.Size([1, 4259, 20]) from checkpoint, the shape in current model is torch.Size([1, 2819, 20]). size mismatch for garDisplacementLayers.5.res1.conv1.spiral_adj: copying a param with shape torch.Size([1, 2819, 20]) from checkpoint, the shape in current model is torch.Size([1, 5405, 20]). size mismatch for garDisplacementLayers.5.res1.conv2.spiral_adj: copying a param with shape torch.Size([1, 2819, 20]) from checkpoint, the shape in current model is torch.Size([1, 5405, 20]). size mismatch for garDisplacementLayers.5.ress.0.conv1.spiral_adj: copying a param with shape torch.Size([1, 2819, 20]) from checkpoint, the shape in current model is torch.Size([1, 5405, 20]). size mismatch for garDisplacementLayers.5.ress.0.conv2.spiral_adj: copying a param with shape torch.Size([1, 2819, 20]) from checkpoint, the shape in current model is torch.Size([1, 5405, 20]). size mismatch for garDisplacementLayers.5.ress.1.conv1.spiral_adj: copying a param with shape torch.Size([1, 2819, 20]) from checkpoint, the shape in current model is torch.Size([1, 5405, 20]). size mismatch for garDisplacementLayers.5.ress.1.conv2.spiral_adj: copying a param with shape torch.Size([1, 2819, 20]) from checkpoint, the shape in current model is torch.Size([1, 5405, 20]). size mismatch for garDisplacementLayers.5.ress.2.conv1.spiral_adj: copying a param with shape torch.Size([1, 2819, 20]) from checkpoint, the shape in current model is torch.Size([1, 5405, 20]). size mismatch for garDisplacementLayers.5.ress.2.conv2.spiral_adj: copying a param with shape torch.Size([1, 2819, 20]) from checkpoint, the shape in current model is torch.Size([1, 5405, 20]).

opened by Shakhbozboy 1
How can I do texture stitching?

I really appreciate your paper and open code. In the paper, there are some figures where the meshes are reconstructed with texture. Is there any method for texture stitching? If it is impossible to share code for texture stitching, can you give me some advice?

Thank you

opened by Minsoo2022 2

The BCNet related data and inference model.

Related tags

Overview

BCNet

Dataset

1. Synthetic Dataset

2. HD Texture Dataset

Inference

1. Install

2. Usage

Citation

Comments

Owner

Torchserve server using a YoloV5 model running on docker with GPU and static batch inference to perform production ready inference.

PyTorch-LIT is the Lite Inference Toolkit (LIT) for PyTorch which focuses on easy and fast inference of large models on end-devices.

Monocular 3D pose estimation. OpenVINO. CPU inference or iGPU (OpenCL) inference.

Exploit Camera Raw Data for Video Super-Resolution via Hidden Markov Model Inference

This package proposes simplified exporting pytorch models to ONNX and TensorRT, and also gives some base interface for model inference.

This repository is related to an Arabic tutorial, within the tutorial we discuss the common data structure and algorithms and their worst and best case for each, then implement the code using Python.

This is the unofficial code of Deep Dual-resolution Networks for Real-time and Accurate Semantic Segmentation of Road Scenes. which achieve state-of-the-art trade-off between accuracy and speed on cityscapes and camvid, without using inference acceleration and extra data

HNECV: Heterogeneous Network Embedding via Cloud model and Variational inference

PyTorch implementation of the end-to-end coreference resolution model with different higher-order inference methods.

Inference code for "StylePeople: A Generative Model of Fullbody Human Avatars" paper. This code is for the part of the paper describing video-based avatars.

Python wrapper class for OpenVINO Model Server. User can submit inference request to OVMS with just a few lines of code

Collect super-resolution related papers, data, repositories

Related resources for our EMNLP 2021 paper

In this project we investigate the performance of the SetCon model on realistic video footage. Therefore, we implemented the model in PyTorch and tested the model on two example videos.

Step by Step on how to create an vision recognition model using LOBE.ai, export the model and run the model in an Azure Function

torchlm is aims to build a high level pipeline for face landmarks detection, it supports training, evaluating, exporting, inference(Python/C++) and 100+ data augmentations

A scientific and useful toolbox, which contains practical and effective long-tail related tricks with extensive experimental results

Python Library for learning (Structure and Parameter) and inference (Statistical and Causal) in Bayesian Networks.

For holding anime-related object classification and detection models