OpenMMLab 3D Human Parametric Model Toolbox and Benchmark

Overview

Documentation actions codecov PyPI LICENSE Percentage of issues still open

Introduction

English | 简体中文

MMHuman3D is an open source PyTorch-based codebase for the use of 3D human parametric models in computer vision and computer graphics. It is a part of the OpenMMLab project.

The main branch works with PyTorch 1.7+.

mmhuman3d.demo.mp4

Major Features

  • Reproducing popular methods with a modular framework

    MMHuman3D reimplements popular methods, allowing users to reproduce SOTAs with one line of code. The modular framework is convenient for rapid prototyping: the users may attempt various hyperparameter settings and even network architectures, without actually modifying the code.

  • Supporting various datasets with a unified data convention

    With the help of a convention toolbox, a unified data format HumanData is used to align all supported datasets. Preprocessed data files are also available.

  • Versatile visualization toolbox

    A suite of differentiale visualization tools for human parametric model rendering (including part segmentation, depth map and point clouds) and conventional 2D/3D keypoints are available.

Benchmark and Model Zoo

More details can be found in model_zoo.md.

Supported methods:

(click to collapse)

Supported datasets:

(click to collapse)

We will keep up with the latest progress of the community, and support more popular methods and frameworks.

If you have any feature requests, please feel free to leave a comment in the wishlist.

Get Started

Please see getting_started.md for the basic usage of MMHuman3D.

License

This project is released under the Apache 2.0 license. Some supported methods may carry additional licenses.

Citation

If you find this project useful in your research, please consider cite:

@misc{mmhuman3d,
    title={OpenMMLab 3D Human Parametric Model Toolbox and Benchmark},
    author={MMHuman3D Contributors},
    howpublished = {\url{https://github.com/open-mmlab/mmhuman3d}},
    year={2021}
}

Contributing

We appreciate all contributions to improve MMHuman3D. Please refer to CONTRIBUTING.md for the contributing guideline.

Acknowledgement

MMHuman3D is an open source project that is contributed by researchers and engineers from both the academia and the industry. We appreciate all the contributors who implement their methods or add new features, as well as users who give valuable feedbacks. We wish that the toolbox and benchmark could serve the growing research community by providing a flexible toolkit to reimplement existing methods and develop their own new models.

Projects in OpenMMLab

  • MMCV: OpenMMLab foundational library for computer vision.
  • MIM: MIM Installs OpenMMLab Packages.
  • MMClassification: OpenMMLab image classification toolbox and benchmark.
  • MMDetection: OpenMMLab detection toolbox and benchmark.
  • MMDetection3D: OpenMMLab next-generation platform for general 3D object detection.
  • MMSegmentation: OpenMMLab semantic segmentation toolbox and benchmark.
  • MMAction2: OpenMMLab next-generation action understanding toolbox and benchmark.
  • MMTracking: OpenMMLab video perception toolbox and benchmark.
  • MMPose: OpenMMLab pose estimation toolbox and benchmark.
  • MMEditing: OpenMMLab image and video editing toolbox.
  • MMOCR: A Comprehensive Toolbox for Text Detection, Recognition and Understanding.
  • MMGeneration: OpenMMLab next-generation toolbox for generative models.
  • MMFlow: OpenMMLab optical flow toolbox and benchmark.
  • MMFewShot: OpenMMLab FewShot Learning Toolbox and Benchmark.
  • MMHuman3D: OpenMMLab 3D Human Parametric Model Toolbox and Benchmark.
Comments
  • [Feature] Support STAR as an OpenMM model

    [Feature] Support STAR as an OpenMM model

    Implementation (integration) of the STAR model into MMHuman3D (resolves #148).

    Summary

    Copy and slightly adapt the original PyTorch implementation of the STAR model.

    Changes

    Compared to the original source code, I added explicit argument lists with their types. The model path can be loaded by specifying the directory where the gendered models are or by directly providing the path to the model.

    Discussion

    In contrast to SMPL model, at the moment we cannot simply import the STAR model and then potentially extend it, because the original repository does not properly expose STAR class to be imported. Therefore, it seems that the best solution was to copy and adapt the code.

    The original license is now explicitly added as it feels somewhat improper to just copy the code, slightly adapt and then put another license. If we decide that it is OK, I will remove the second license.

    Future Work (Body Models Refactoring)

    I noticed that the extended SMPL classes abuse the original SMPL in several places, for example:

    1. The override implementation of the forward() method returns a Python dictionary instead SMPLOutput or ModelOutput as it is done in the original class. This forces the default dictionary syntax, such as model['vertices'] instead of model.vertices. Also, it creates code redundancy that we see in several places where the outputs are recreated as dictionaries, such as here and here. The proper solution is to use an (extended) data class definition such as this.
    2. The argument list for extended classes can be simplified. Instead of explicitly providing arguments, even repeating the arguments of the original method, the arguments can be kept as (*args, **kwargs) such as in SPIN or least we should avoid repeating the arguments from the original methods.
    3. The keypoint (re)mapping logic should be moved from the body models implementations as the keypoints have nothing to do with the body models once they are extracted from the vertices.
    4. The gendered models such as this seem redundant as the original SMPL model already accepts the gender parameter.
    5. Improve unit tests (at least for the code coverage).

    We can discuss these and other improvements in a separate issue.

    opened by kristijanbartol 31
  • why Intrinsic Matrix is 4 x 4

    why Intrinsic Matrix is 4 x 4

    @caizhongang why Intrinsic Matrix K defined as

    K = [
            [fx,   0,   px,   0],
            [0,   fy,   py,   0],
            [0,    0,    0,   1],
            [0,    0,    1,   0],
        ]
    

    rather than

    K = [
            [fx,   0,   px],
            [0,   fy,   py],
            [0,    0,    1]
        ]
    
    opened by akk-123 10
  • Why there exists so many difference between provided training log and config file of hybrik?

    Why there exists so many difference between provided training log and config file of hybrik?

    https://openmmlab-share.oss-cn-hangzhou.aliyuncs.com/mmhuman3d/models/hybrik/20211109_164017.log?versionId=CAEQHhiBgICdvonS6xciIDdiNGYzY2Q3N2NiMTQ5MzdhOTZjYjEwZDM0ZjI3ODU1

    opened by YongtaoGe 9
  • How to get the intrinsics and extrinsics of the demo output?

    How to get the intrinsics and extrinsics of the demo output?

    Thank you for your great work! I run the single-person demo according to page, and I tried hmr and vibe. the outputs are the same format. just like the following:

    ['__key_strict__', '__data_len__', '__keypoints_compressed__', 'smpl',
     'verts', 'pred_cams', 'bboxes_xyxy', 'image_path', 'person_id', 'frame_id']
    

    I got the ouput which is correctly visualization, but I have some (maybe simple) questions: The input images are not square but rectangle, but the outputs pred_cams is of (frames_num,3) stand for s,tx,ty for each image. I guess the pred_cams is 'weak perspective camera parameters in cropped image space (s,tx,ty)' , but I don't know how to get the cropped images. By using the pred_cams (s,tx,ty) to calculate the extrinsics of original images seems not good. So I wander to know how to get cropped images? and how to get cams of (sx,sy,tx,ty)?

    opened by Miles629 8
  • when I take hybirk as a example to Inference / Demo, take the errror

    when I take hybirk as a example to Inference / Demo, take the errror

    File "demo/estimate_smpl.py", line 296, in main(args) File "demo/estimate_smpl.py", line 195, in main verts, K0, img_index = single_person_with_mmdet(args, frames_iter) File "demo/estimate_smpl.py", line 65, in single_person_with_mmdet mesh_results = inference_model( File "/home/bixueting/code/mmhuman3d/mmhuman3d/apis/inference.py", line 139, in inference_model inference_pipeline = [LoadImage()] + cfg.inference_pipeline File "/home/bixueting/anaconda3/envs/mmhuman3d/lib/python3.8/site-packages/mmcv/utils/config.py", line 507, in getattr return getattr(self._cfg_dict, name) File "/home/bixueting/anaconda3/envs/mmhuman3d/lib/python3.8/site-packages/mmcv/utils/config.py", line 48, in getattr raise ex AttributeError: 'ConfigDict' object has no attribute 'inference_pipeline'

    opened by bixueting 8
  • Wishlist

    Wishlist

    Please leave a comment on what new method/feature/dataset you would like us to add in MMHuman3D.

    The most upvoted suggestions will be prioritized!

    Update: we have moved our Wishlist to this discussion. See you there!

    opened by caizhongang 8
  • KeyError: 'features' at HMR demo

    KeyError: 'features' at HMR demo

    Demo Command:

    python demo/etimate_smpl_video.py configs/hmr/resnet50_hmr_pw3d.py data/checkpoints/resnet50_hmr_pw3d-04f40f58_20211201.pth --single_person_demo --det_config demo/mmdetection_cfg/faster_rcnn_r50_fpn_coco.py --det_checkpoint data/checkpoints/detection/faster_rcnn_r50_fpn_1x_coco_20200130-047c8118.pth --input_path work_dir/video.mp4 --output work_dir/results/ --show_path work_dir/results/ --smooth_type savgol --draw_bbox

    Get the following error @inference:

      File "demo/etimate_smpl_video.py", line 337, in <module>
        main(args)
      File "demo/etimate_smpl_video.py", line 241, in main
        single_person_with_mmdet(args, frames_iter)
      File "demo/etimate_smpl_video.py", line 91, in single_person_with_mmdet
        mesh_results = inference_video_based_model(
      File "mmhuman3d/mmhuman3d/apis/inference.py", line 244, in inference_video_based_model
        feature_sequences = _collate_feature_sequence(input_features,
      File "mmhuman3d/apis/inference.py", line 465, in _collate_feature_sequence
        C = extracted_features[target_frame][0]['features'].shape[0]
    KeyError: 'features'
    
    

    Same error for SPIN

    P. S. The the config for both hmr & spin is missing data.test.seq_len.

    opened by rlleshi 7
  • [Fix] Bug fixes in data converters and transforms

    [Fix] Bug fixes in data converters and transforms

    Resolves #67

    • Inplace renaming operation to obtain correct image path in agora converter
    • Use tuple instead of string check in agora converter
    • Retrieve correct image path using id in posetrack converter
    • Fix inconsistency is cv2.inwrite image path and stored image path in surreal converter
    • Fix H36M cam_param to store a dictionary of camera parameters

    Resolves #70

    • Fix type in instances where bbox_xywh is bbox_xyxy
    • Split _bbox_expand to bbox_expand (only bbox expansion) and xyxy2xywh (xyxy conversion to xywh)

    Resolves #79 issue of keypoints2d_loss not converging

    • Make a copy of keypoints2d instead of directly modifying when applying augmentations

    Resolves #49 .

    opened by pangyyyyy 7
  • cam_param is absent in HumanData.SUPPORTED_KEYS.

    cam_param is absent in HumanData.SUPPORTED_KEYS.

    I want to convert Human36M dataset and I use the following script

    python tools/convert_datasets.py --datasets h36m_p1 --root_path ../data/datasets/ --output_path ../data/processed_datasets
    

    Then I get the error "cam_param is absent in HumanData.SUPPORTED_KEYS."

    opened by YongtaoGe 7
  • about

    about "find_unused_parameters=True " in training

    File "/root/anaconda3/envs/zxh/lib/python3.8/site-packages/torch/nn/parallel/distributed.py", line 528, in forward self.reducer.prepare_for_backward([]) RuntimeError: Expected to have finished reduction in the prior iteration before starting a new one. This error indicates that your module has parameters that were not used in producing loss. You can enable unused parameter detection by (1) passing the keyword argument find_unused_parameters=True to torch.nn.parallel.DistributedDataParallel; (2) making sure all forward function outputs participate in calculating loss. If you already have done the above two steps, then the distributed data parallel module wasn't able to locate the output tensors in the return value of your module's forward function. Please include the loss function and the structure of the return value of forward of your module when reporting this issue (e.g. list, dict, iterable). Traceback (most recent call last):

    THX!

    opened by Gzzgz 6
  • Error on torch1.3.0

    Error on torch1.3.0

    I install torch1.13.0+cuda116 and mmcv using:

    conda install pytorch==1.13.0 torchvision pytorch-cuda=11.6 -c pytorch -c nvidia
    pip install "mmcv-full>=1.3.17,<=1.5.3" -f https://download.openmmlab.com/mmcv/dist/cu116/torch1.13/index.html
    

    but I got error

    Traceback (most recent call last):
      File "/home/my/Pose/mmhuman3d/tools/train.py", line 188, in <module>
        main()
      File "/home/my/Pose/mmhuman3d/tools/train.py", line 135, in main
        env_info_dict = collect_env()
      File "/home/my/Pose/mmhuman3d/mmhuman3d/utils/collect_env.py", line 9, in collect_env
        env_info = collect_base_env()
      File "/home/my/anaconda3/envs/mmhuman3d_torch113/lib/python3.9/site-packages/mmcv/utils/env.py", line 112, in collect_env
        from mmcv.ops import get_compiler_version, get_compiling_cuda_version
      File "/home/my/anaconda3/envs/mmhuman3d_torch113/lib/python3.9/site-packages/mmcv/ops/__init__.py", line 2, in <module>
        from .active_rotated_filter import active_rotated_filter
      File "/home/my/anaconda3/envs/mmhuman3d_torch113/lib/python3.9/site-packages/mmcv/ops/active_rotated_filter.py", line 10, in <module>
        ext_module = ext_loader.load_ext(
      File "/home/my/anaconda3/envs/mmhuman3d_torch113/lib/python3.9/site-packages/mmcv/utils/ext_loader.py", line 13, in load_ext
        ext = importlib.import_module('mmcv.' + name)
      File "/home/my/anaconda3/envs/mmhuman3d_torch113/lib/python3.9/importlib/__init__.py", line 127, in import_module
        return _bootstrap._gcd_import(name[level:], package, level)
    ImportError: /home/my/anaconda3/envs/mmhuman3d_torch113/lib/python3.9/site-packages/mmcv/_ext.cpython-39-x86_64-linux-gnu.so: undefined symbol: _ZN2at6mm_outERNS_6TensorERKS0_S3_
    
    opened by flyyyyer 0
  • [Feature] Add MultiHumanData

    [Feature] Add MultiHumanData

    This PR mainly focuses on adding a new data structure for multi-human training.

    • [x] Add a new data structure for multi-human training
    • [x] Update the corresponding document
    • [x] Update unitest

    Note: This PR shouldn't be merged until #257 and #264 are merged.

    opened by ttxskk 1
  • ValueError: Unknown CUDA arch (8.6) or GPU not supported

    ValueError: Unknown CUDA arch (8.6) or GPU not supported

    I am having trouble getting mmhuman3d to install on Ubuntu 18.04 with an RTX 3070 GPU using conda.

    I am trying to execute

    pip install -v -e .

    as described in the installation documentation. I followed every step despite using newer versions for Pytorch (1.13) and CUDA (11.6) and Torchvision as these changes are necessary due to RTX 30xx compatiblity issues.

    However, setup fails and this error is thrown: /home/nico90/anaconda3/envs/open-mmlab/lib/python3.8/site-packages/setuptools/command/easy_install.py:144: EasyInstallDeprecationWarning: easy_install command is deprecated. Use build and pip and other standards-based tools. warnings.warn( /home/nico90/anaconda3/envs/open-mmlab/lib/python3.8/site-packages/setuptools/command/install.py:34: SetuptoolsDeprecationWarning: setup.py install is deprecated. Use build and pip and other standards-based tools. warnings.warn( Traceback (most recent call last): File "", line 2, in File "", line 34, in File "/home/nico90/mmhuman3d/setup.py", line 117, in setup( File "/home/nico90/anaconda3/envs/open-mmlab/lib/python3.8/site-packages/setuptools/init.py", line 87, in setup return distutils.core.setup(**attrs) File "/home/nico90/anaconda3/envs/open-mmlab/lib/python3.8/site-packages/setuptools/_distutils/core.py", line 185, in setup return run_commands(dist) File "/home/nico90/anaconda3/envs/open-mmlab/lib/python3.8/site-packages/setuptools/_distutils/core.py", line 201, in run_commands dist.run_commands() File "/home/nico90/anaconda3/envs/open-mmlab/lib/python3.8/site-packages/setuptools/_distutils/dist.py", line 968, in run_commands self.run_command(cmd) File "/home/nico90/anaconda3/envs/open-mmlab/lib/python3.8/site-packages/setuptools/dist.py", line 1217, in run_command super().run_command(command) File "/home/nico90/anaconda3/envs/open-mmlab/lib/python3.8/site-packages/setuptools/_distutils/dist.py", line 987, in run_command cmd_obj.run() File "/home/nico90/anaconda3/envs/open-mmlab/lib/python3.8/site-packages/setuptools/command/develop.py", line 34, in run self.install_for_development() File "/home/nico90/anaconda3/envs/open-mmlab/lib/python3.8/site-packages/setuptools/command/develop.py", line 114, in install_for_development self.run_command('build_ext') File "/home/nico90/anaconda3/envs/open-mmlab/lib/python3.8/site-packages/setuptools/_distutils/cmd.py", line 319, in run_command self.distribution.run_command(command) File "/home/nico90/anaconda3/envs/open-mmlab/lib/python3.8/site-packages/setuptools/dist.py", line 1217, in run_command super().run_command(command) File "/home/nico90/anaconda3/envs/open-mmlab/lib/python3.8/site-packages/setuptools/_distutils/dist.py", line 987, in run_command cmd_obj.run() File "/home/nico90/anaconda3/envs/open-mmlab/lib/python3.8/site-packages/setuptools/command/build_ext.py", line 84, in run _build_ext.run(self) File "/home/nico90/anaconda3/envs/open-mmlab/lib/python3.8/site-packages/Cython/Distutils/old_build_ext.py", line 186, in run _build_ext.build_ext.run(self) File "/home/nico90/anaconda3/envs/open-mmlab/lib/python3.8/site-packages/setuptools/_distutils/command/build_ext.py", line 346, in run self.build_extensions() File "/home/nico90/anaconda3/envs/open-mmlab/lib/python3.8/site-packages/torch/utils/cpp_extension.py", line 372, in build_extensions build_ext.build_extensions(self) File "/home/nico90/anaconda3/envs/open-mmlab/lib/python3.8/site-packages/Cython/Distutils/old_build_ext.py", line 195, in build_extensions _build_ext.build_ext.build_extensions(self) File "/home/nico90/anaconda3/envs/open-mmlab/lib/python3.8/site-packages/setuptools/_distutils/command/build_ext.py", line 466, in build_extensions self._build_extensions_serial() File "/home/nico90/anaconda3/envs/open-mmlab/lib/python3.8/site-packages/setuptools/_distutils/command/build_ext.py", line 492, in _build_extensions_serial self.build_extension(ext) File "/home/nico90/anaconda3/envs/open-mmlab/lib/python3.8/site-packages/setuptools/command/build_ext.py", line 246, in build_extension _build_ext.build_extension(self, ext) File "/home/nico90/anaconda3/envs/open-mmlab/lib/python3.8/site-packages/setuptools/_distutils/command/build_ext.py", line 547, in build_extension objects = self.compiler.compile( File "/home/nico90/anaconda3/envs/open-mmlab/lib/python3.8/site-packages/setuptools/_distutils/ccompiler.py", line 599, in compile self._compile(obj, src, ext, cc_args, extra_postargs, pp_opts) File "/home/nico90/anaconda3/envs/open-mmlab/lib/python3.8/site-packages/torch/utils/cpp_extension.py", line 288, in unix_wrap_compile "'-fPIC'"] + cflags + _get_cuda_arch_flags(cflags) File "/home/nico90/anaconda3/envs/open-mmlab/lib/python3.8/site-packages/torch/utils/cpp_extension.py", line 1027, in _get_cuda_arch_flags raise ValueError("Unknown CUDA arch ({}) or GPU not supported".format(arch)) ValueError: Unknown CUDA arch (8.6) or GPU not supported error: subprocess-exited-with-error

    × python setup.py develop did not run successfully.
    │ exit code: 1
    ╰─> See above for output.
    
    note: This error originates from a subprocess, and is likely not a problem with pip.
    full command: /home/nico90/anaconda3/envs/open-mmlab/bin/python -c '
    exec(compile('"'"''"'"''"'"'
    # This is <pip-setuptools-caller> -- a caller that pip uses to run setup.py
    #
    # - It imports setuptools before invoking setup.py, to enable projects that directly
    #   import from `distutils.core` to work with newer packaging standards.
    # - It provides a clear error message when setuptools is not installed.
    # - It sets `sys.argv[0]` to the underlying `setup.py`, when invoking `setup.py` so
    #   setuptools doesn'"'"'t think the script is `-c`. This avoids the following warning:
    #     manifest_maker: standard file '"'"'-c'"'"' not found".
    # - It generates a shim setup.py, for handling setup.cfg-only projects.
    import os, sys, tokenize
    
    try:
        import setuptools
    except ImportError as error:
        print(
            "ERROR: Can not execute `setup.py` since setuptools is not available in "
            "the build environment.",
            file=sys.stderr,
        )
        sys.exit(1)
    
    __file__ = %r
    sys.argv[0] = __file__
    
    if os.path.exists(__file__):
        filename = __file__
        with tokenize.open(__file__) as f:
            setup_py_code = f.read()
    else:
        filename = "<auto-generated setuptools caller>"
        setup_py_code = "from setuptools import setup; setup()"
    
    exec(compile(setup_py_code, filename, "exec"))
    '"'"''"'"''"'"' % ('"'"'/home/nico90/mmhuman3d/setup.py'"'"',), "<pip-setuptools-caller>", "exec"))' develop --no-deps
    cwd: /home/nico90/mmhuman3d/
    

    error: subprocess-exited-with-error

    × python setup.py develop did not run successfully. │ exit code: 1 ╰─> See above for output.

    note: This error originates from a subprocess, and is likely not a problem with pip.

    torch.cuda.is_available() is returning True

    Does anybody have a fix at hand?

    opened by nicolasfischoeder 0
  • Refactor or Camera/Visualization/Renderer System.

    Refactor or Camera/Visualization/Renderer System.

    Please write your suggestions in below comments. I will rearrange them and summarize a plan list.

    Cameras.

    • [ ] Add verbose switch in renderer.

    Renderer.

    • [ ] Add DecoMR uv render.
    • [ ] Simplify visualize_smpl.
    • [ ] Accelerate visualize_smpl.
    • [ ] Add all the tutorials for different datasets or algorithms.

    The PR is now open. #296

    opened by JasonBoy1 3
Releases(v0.10.0)
  • v0.10.0(Oct 12, 2022)

    Highlights

    • Add webcam demo and real-time renderer
    • Update dataloader to speed up training
    • Add balanced MSE loss for imbalanced HMR training

    New Algorithms

    • Add balanced MSE loss for imbalanced HMR training (added by paper authors)

    New Features

    • Add persistent_workers in dataloader to reduce data time in training
    • Add webcam demo and real-time renderer
    • Support mmhuman3d installation without torch

    Documentation

    • Add smc file description
    • Update install.md to address dependency conflicts
    • Update GTA-Human, main readme, and install guide
    • Update doc to avoid mmcls and mmtrack installation issues

    Bug Fixes

    • Fix error raise messages in estimate_smpl.py
    • Fix ImportError caused by missing __init__.py
    • Fix config not found when installed by 'pip install git+...'
    • Fix GTA-Human config path error
    • Fix bugs in guas1d config
    • Fix bug for the real-time renderer

    Many thanks to this release's contributors! @ykk648 @haofanwang @woo1 @mingyuan-zhang @ttxskk @kimren227 @caizhongang @LazyBusyYang @yl-1993

    Source code(tar.gz)
    Source code(zip)
  • v0.9.0(Jul 13, 2022)

    Highlights

    • Support SMPL-X estimation with ExPose for simultaneous recovery of face, hands and body
    • Support new body model STAR
    • Release of GTA-Human dataset with SPIN-FT (51.98 mm) and PARE-FT (46.84 mm) baselines! (Official)
    • Refactor registration and improve performance of SPIN to 57.54 mm

    * Values are PA-MPJPE on 3DPW test

    New Algorithm

    • Support training and evaluation of SMPL-X estimation method ExPose
    • Add a general architecture for expressive human modelling
    • Add data preparation and preprocessing tools
    • Add evaluation metrics

    New Body Models

    • Support new body model STAR by adapting the official PyTorch implementation into MMHuman3D
    • Support FLAME and MANO needed by ExPose

    New Dataset

    • Official release of our large-scale synthetic dataset GTA-Human that contains 1.4M SMPL annotations
    • Add strong baselines SPIN-FT and PARE-FT baselines, which achieve 51.98 mm and 46.84 mm PA-MPJPE on 3DPW test respectively

    Refactors

    • Refactor registration
      • Allow more flexible registration configurations
      • Improve performance of SPIN to 57.54 mm PA-MPJPE on 3DPW test

    Documentation

    • Add documation for installing MMHuman3D on Windows

    Bug Fixes

    • Fix missing arguments in SMPLify pipeline
    • Adjust import structure in core, data and utils, removing unnecessary dependency on PyTorch3D, and achieving up to 2.6x speed-up during initialization

    Many thanks to this release's contributors!

    @kristijanbartol @mingyuan-zhang @Coach257 @caizhongang @LazyBusyYang @ttxskk

    Source code(tar.gz)
    Source code(zip)
  • v0.8.0(Jun 1, 2022)

    Highlights

    • Support SmoothNet (added by paper authors)
    • Fix circular import and up to 2.5x speed up in module initialization
    • Add documentations in Chinese

    New Algorithms

    • Support SmoothNet as a smoothing method, improving inference precision for image- and video-based models

    Refactors

    • Refactor estimate_smpl.py with compressed pose and shape to avoid possible shape errors and waste of space for multi-person visualization

    Documentations

    • Add Chinese version for 12 documentations

    Bug Fixes

    • Fix circular import in models, conventions, and cameras, and achieve up to 2.5x speed-up in module initialization by importing only the necessary builders
    • Fix bugs in PARE configs
    • Fix wrong type and value in convert_kps

    Many thanks to this release's contributors!

    @Naive-Bayes @ykk648 @juxuan27 @LazyBusyYang

    Source code(tar.gz)
    Source code(zip)
  • v0.7.0(Apr 29, 2022)

    Highlights

    • Support PARE (better than the official implementation)
    • Support DeciWatch (added by paper authors)
    • Add GTA-Human HMR baseline (official release)
    • Support saving inference results

    New Algorithms

    • Support PARE with 49.35 mm PA-MPJPE on 3DPW, better than official implementation (50.9 mm)
    • Support DeciWatch for both smoothing and up to 10x inference speed-up, added by authors of the paper
    • Official release of GTA-Human HMR baseline with 11.82 mm PA-MPJPE improvement on 3DPW than the original HMR

    New Features

    • Support saving inference results in HumanData format

    Refactors

    • Merge HMR's and SPIN's H36M converter so the same preprocessed file is used now

    Bug Fixes

    • Fix bugs in H36M and MPI-INF-3DHP data converters
    • Fix typo in smpl visualization tool
    • Fix in_ndc and is_perspective arguments in camera module

    Many thanks to this release's contributors! @WYJSJTU @juxuan27 @mingyuan-zhang @ttxskk @YongtaoGe @LazyBusyYang @pangyyyyy

    Source code(tar.gz)
    Source code(zip)
  • v0.6.0(Apr 1, 2022)

    Highlights

    • Add HumanDataCache that requires 96% less RAM during training
    • Refactor differentiable renderers and support UV map rendering
    • Support slice/concat operations for HumanData

    New Features

    • Add HumanDataCache to address OOM issue when training using HumanData (e.g., HMR training requires 96% less RAM)
    • Refactor differentiable renderers and support UV map rendering
    • Support slice/concat operations for HumanData
    • Upgrade SMPLify to support more losses, output HumanData, and more
    • Refactor keypoint mapping and support mask-free conversion

    Bug Fixes

    • Fix critical bugs in data converters (for SURREAL, AGORA and more)
    • Fix vertices-related bug in visualization toolbox
    • Fix shape and device in tensor2dict
    • Fix small bugs in HybrIK
    • Fix OOM error for demo
    • Fix doc format issue
    • Remove duplicate limbs in HumanData Update dependency (e.g. cdflib, ffmpeg)
    Source code(tar.gz)
    Source code(zip)
  • v0.5.0(Feb 11, 2022)

    Highlights

    • Support new data structure SMC for new dataset HuMMan that will soon be released
    • Support for multi-GPU training/testing without slurm
    • Support training-time validation and additional metrics such as PVE
    • Bug fixes in data augmentation for more stable training
    • Stronger HybrIK baseline (PA-MPJPE 49.02 on 3DPW)

    New Datasets

    • Add data converter for HuMMan dataset
    • Add support for training with HuMMan dataset

    New Features

    • Add a new data structure SMC with SMCReader
    • Add support for multi-GPU training/testing without slurm
    • Add eval hook that supports validation during training and additional metrics such as PVE
    • Support rigid transformation of SMPL parameters
    • Add video-based inference pipeline to support VIBE demo
    • CameraParameter accepts numpy and torch tensor and K, R, T can be obtained by a single method

    Better Methods

    • Reproduce a stronger HybrIK baseline (PA-MPJPE 49.02 on 3DPW)

    Bug Fixes

    • Fix a bug in data augmentation that causes keypoint2d loss not converging
    • Fix a bug in SMPL rotation augmentation
    • Fix bugs in data converters
    • Fix GPU memory wastage due to unnecessary initialization

    Documentation

    • Use shared menu from OpenMMLab theme
    • Add metafile
    Source code(tar.gz)
    Source code(zip)
  • v0.4.0(Jan 4, 2022)

    Refactoring

    • Registration-based methods are moved to models/, under the new module abstraction named “registrants”
    • Body model wrappers have their own module abstraction named “body_models”

    Datasets

    • Add support for GTA-Human dataset
    • Add support for datasets needed by VIBE

    New features

    • Upgrade SMPLify: batch size adaptation and use standard camera module
    • Upgrade camera: helper functions such as concat, parameter conversion and value type check
    • Upgrade renderer: refactored to use registry, refactored smpl visualization

    Documents

    • Update README, tutorials, and readthedocs

    Bug fixes:

    • Unit tests: test_cache that raises time comparison error
    • Visualization: holes in rendered mesh
    • Data converters: image paths, and H36M and LSP

    CICD

    • Deploy workflow on GitHub
    • Add codedev reports in workflow
    Source code(tar.gz)
    Source code(zip)
  • v0.3.0(Dec 3, 2021)

    Main Features

    • Supports registration-based methods: SMPLify and SMPLify-X
    • Supports regression-based methods: HMR, SPIN, VIBE, and HybrIK
    • Supports 16 datasets with the unified data format HumanData
    • Supports differentiable visualization of parametric models

    The actual commit id for this tag is https://github.com/open-mmlab/mmhuman3d/commit/8255b065db4f46d0ab436d35be4bff27b71009a7

    Source code(tar.gz)
    Source code(zip)
OpenMMLab's Next Generation Video Understanding Toolbox and Benchmark

Introduction English | 简体中文 MMAction2 is an open-source toolbox for video understanding based on PyTorch. It is a part of the OpenMMLab project. The m

OpenMMLab 2.7k Jan 7, 2023
OpenMMLab Pose Estimation Toolbox and Benchmark.

Introduction English | 简体中文 MMPose is an open-source toolbox for pose estimation based on PyTorch. It is a part of the OpenMMLab project. The master b

OpenMMLab 2.8k Dec 31, 2022
OpenMMLab Image and Video Editing Toolbox

Introduction MMEditing is an open source image and video editing toolbox based on PyTorch. It is a part of the OpenMMLab project. The master branch wo

OpenMMLab 3.9k Jan 4, 2023
Mmrotate - OpenMMLab Rotated Object Detection Benchmark

OpenMMLab website HOT OpenMMLab platform TRY IT OUT ?? Documentation | ??️ Insta

OpenMMLab 1.2k Jan 4, 2023
[CVPR2021] UAV-Human: A Large Benchmark for Human Behavior Understanding with Unmanned Aerial Vehicles

UAV-Human Official repository for CVPR2021: UAV-Human: A Large Benchmark for Human Behavior Understanding with Unmanned Aerial Vehicle Paper arXiv Res

null 129 Jan 4, 2023
Convert openmmlab (not only mmdetection) series model to tensorrt

MMDet to TensorRT This project aims to convert the mmdetection model to TensorRT model end2end. Focus on object detection for now. Mask support is exp

JinTian 4 Dec 17, 2021
This repository contains a pytorch implementation of "HeadNeRF: A Real-time NeRF-based Parametric Head Model (CVPR 2022)".

HeadNeRF: A Real-time NeRF-based Parametric Head Model This repository contains a pytorch implementation of "HeadNeRF: A Real-time NeRF-based Parametr

null 294 Jan 1, 2023
Mixup for Supervision, Semi- and Self-Supervision Learning Toolbox and Benchmark

OpenSelfSup News Downstream tasks now support more methods(Mask RCNN-FPN, RetinaNet, Keypoints RCNN) and more datasets(Cityscapes). 'GaussianBlur' is

AI Lab, Westlake University 332 Jan 3, 2023
Ultra-lightweight human body posture key point CNN model. ModelSize:2.3MB HUAWEI P40 NCNN benchmark: 6ms/img,

Ultralight-SimplePose Support NCNN mobile terminal deployment Based on MXNET(>=1.5.1) GLUON(>=0.7.0) framework Top-down strategy: The input image is t

null 223 Dec 27, 2022
An open-access benchmark and toolbox for electricity price forecasting

epftoolbox The epftoolbox is the first open-access library for driving research in electricity price forecasting. Its main goal is to make available a

null 97 Dec 5, 2022
MIM: MIM Installs OpenMMLab Packages

MIM provides a unified API for launching and installing OpenMMLab projects and their extensions, and managing the OpenMMLab model zoo.

OpenMMLab 254 Jan 4, 2023
OpenMMLab Computer Vision Foundation

English | 简体中文 Introduction MMCV is a foundational library for computer vision research and supports many research projects as below: MMCV: OpenMMLab

OpenMMLab 4.6k Jan 9, 2023
Some pre-commit hooks for OpenMMLab projects

pre-commit-hooks Some pre-commit hooks for OpenMMLab projects. Using pre-commit-hooks with pre-commit Add this to your .pre-commit-config.yaml - rep

OpenMMLab 16 Nov 29, 2022
An example to implement a new backbone with OpenMMLab framework.

Backbone example on OpenMMLab framework English | 简体中文 Introduction This is an template repo about how to use OpenMMLab framework to develop a new bac

Ma Zerun 22 Dec 29, 2022
Python scripts for performing 3D human pose estimation using the Mobile Human Pose model in ONNX.

Python scripts for performing 3D human pose estimation using the Mobile Human Pose model in ONNX.

Ibai Gorordo 99 Dec 31, 2022
pytorch implementation of "Contrastive Multiview Coding", "Momentum Contrast for Unsupervised Visual Representation Learning", and "Unsupervised Feature Learning via Non-Parametric Instance-level Discrimination"

Unofficial implementation: MoCo: Momentum Contrast for Unsupervised Visual Representation Learning (Paper) InsDis: Unsupervised Feature Learning via N

Zhiqiang Shen 16 Nov 4, 2020
A parametric soroban written with CADQuery.

A parametric soroban written in CADQuery The purpose of this project is to demonstrate how "code CAD" can be intuitive to learn. See soroban.py for a

Lee 4 Aug 13, 2022
The personal repository of the work: *DanceNet3D: Music Based Dance Generation with Parametric Motion Transformer*.

DanceNet3D The personal repository of the work: DanceNet3D: Music Based Dance Generation with Parametric Motion Transformer. Dataset and Results Pleas

南嘉Nanga 36 Dec 21, 2022