HyperPose is a library for building high-performance custom pose estimation applications.

Overview


FeaturesDocumentationQuick StartPerformanceAccuracyLicense

HyperPose

HyperPose is a library for building high-performance custom pose estimation applications.

Features

HyperPose has two key features:

  • High-performance pose estimation with CPUs/GPUs: HyperPose achieves real-time pose estimation through a high-performance pose estimation engine. This engine implements numerous system optimisations: pipeline parallelism, model inference with TensorRT, CPU/GPU hybrid scheduling, and many others. These optimisations contribute to up to 10x higher FPS compared to OpenPose, TF-Pose and OpenPifPaf.
  • Flexibility for developing custom pose estimation models: HyperPose provides high-level Python APIs to develop pose estimation models. HyperPose users can:
    • Customise training, evaluation, visualisation, pre-processing and post-processing in pose estimation.
    • Customise model architectures (e.g., OpenPose, Pifpaf, PoseProposal Network) and training datasets.
    • Speed up training with multiple GPUs.

Demo

新宝岛 with HyperPose (Lightweight OpenPose model)

Quick Start

The HyperPose library contains two parts:

  • A C++ library for high-performance pose estimation model inference.
  • A Python library for developing custom pose estimation models.

C++ inference library

The easiest way to use the inference library is through a Docker image. Pre-requisites for this image:

Run this command to check if pre-requisites are ready:

wget https://raw.githubusercontent.com/tensorlayer/hyperpose/master/scripts/test_docker.py -qO- | python

Once pre-requisites are ready, pull the HyperPose docker:

docker pull tensorlayer/hyperpose

We provide 4 examples within this image (The following commands have been tested on Ubuntu 18.04):

# [Example 1]: Doing inference on given video, copy the output.avi to the local path.
docker run --name quick-start --gpus all tensorlayer/hyperpose --runtime=stream
docker cp quick-start:/hyperpose/build/output.avi .
docker rm quick-start


# [Example 2](X11 server required to see the imshow window): Real-time inference.
# You may need to install X11 server locally:
# sudo apt install xorg openbox xauth
xhost +; docker run --rm --gpus all -e DISPLAY=$DISPLAY -v /tmp/.X11-unix:/tmp/.X11-unix tensorlayer/hyperpose --imshow


# [Example 3]: Camera + imshow window
xhost +; docker run --name pose-camera --rm --gpus all -e DISPLAY=$DISPLAY -v /tmp/.X11-unix:/tmp/.X11-unix --device=/dev/video0:/dev/video0 tensorlayer/hyperpose --source=camera --imshow
# To quit this image, please type `docker kill pose-camera` in another terminal.


# [Dive into the image]
xhost +; docker run --rm --gpus all -it -e DISPLAY=$DISPLAY -v /tmp/.X11-unix:/tmp/.X11-unix --device=/dev/video0:/dev/video0 --entrypoint /bin/bash tensorlayer/hyperpose
# For users that cannot access a camera or X11 server. You may also use:
# docker run --rm --gpus all -it --entrypoint /bin/bash tensorlayer/hyperpose

For more usage regarding the command line flags, please visit here.

Python training library

We recommend using the Python training library within an Anaconda environment. The below quick-start has been tested with these environments:

OS NVIDIA Driver CUDA Toolkit GPU
Ubuntu 18.04 410.79 10.0 Tesla V100-DGX
Ubuntu 18.04 440.33.01 10.2 Tesla V100-DGX
Ubuntu 18.04 430.64 10.1 TITAN RTX
Ubuntu 18.04 430.26 10.2 TITAN XP
Ubuntu 16.04 430.50 10.1 RTX 2080Ti

Once Anaconda is installed, run below Bash commands to create a virtual environment:

# Create virtual environment (choose yes)
conda create -n hyperpose python=3.7
# Activate the virtual environment, start installation
conda activate hyperpose
# Install cudatoolkit and cudnn library using conda
conda install cudatoolkit=10.0.130
conda install cudnn=7.6.0

We then clone the repository and install the dependencies listed in requirements.txt:

git clone https://github.com/tensorlayer/hyperpose.git && cd hyperpose
pip install -r requirements.txt

We demonstrate how to train a custom pose estimation model with HyperPose. HyperPose APIs contain three key modules: Config, Model and Dataset, and their basic usages are shown below.

from hyperpose import Config, Model, Dataset

# Set model name to distinguish models (necessary)
Config.set_model_name("MyLightweightOpenPose")

# Set model type, model backbone and dataset
Config.set_model_type(Config.MODEL.LightweightOpenpose)
Config.set_model_backbone(Config.BACKBONE.Vggtiny)
Config.set_dataset_type(Config.DATA.MSCOCO)

# Set single-node training or parallel-training
Config.set_train_type(Config.TRAIN.Single_train)

config = Config.get_config()
model = Model.get_model(config)
dataset = Dataset.get_dataset(config)

# Start the training process
Model.get_train(config)(model, dataset)

The full training program is listed here. To evaluate the trained model, you can use the evaluation program here. More information about the training library can be found here.

Documentation

The APIs of the HyperPose training library and the inference library are described in the Documentation.

Performance

We compare the prediction performance of HyperPose with OpenPose 1.6, TF-Pose and OpenPifPaf 0.12. The test-bed has Ubuntu18.04, 1070Ti GPU, Intel i7 CPU (12 logic cores).

HyperPose Configuration DNN Size Input Size HyperPose Baseline
OpenPose (VGG) 209.3MB 656 x 368 27.32 FPS 8 FPS (OpenPose)
OpenPose (TinyVGG) 34.7 MB 384 x 256 124.925 FPS N/A
OpenPose (MobileNet) 17.9 MB 432 x 368 84.32 FPS 8.5 FPS (TF-Pose)
OpenPose (ResNet18) 45.0 MB 432 x 368 62.52 FPS N/A
OpenPifPaf (ResNet50) 97.6 MB 432 x 368 44.16 FPS 14.5 FPS (OpenPifPaf)

Accuracy

We evaluate the accuracy of pose estimation models developed by HyperPose. The environment is Ubuntu16.04, with 4 V100-DGXs and 24 Intel Xeon CPU. The training procedure takes 1~2 weeks using 1 V100-DGX for each model. (If you don't want to train from scratch, you could use our pre-trained backbone models)

HyperPose Configuration DNN Size Input Size Evaluate Dataset Accuracy-hyperpose (Iou=0.50:0.95) Accuracy-original (Iou=0.50:0.95)
OpenPose (VGG19) 199 MB 432 x 368 MSCOCO2014 (random 1160 images) 57.0 map 58.4 map
LightweightOpenPose (Dilated MobileNet) 17.7 MB 432 x 368 MSCOCO2017(all 5000 img.) 46.1 map 42.8 map
LightweightOpenPose (MobileNet-Thin) 17.4 MB 432 x 368 MSCOCO2017 (all 5000 img.) 44.2 map 28.06 map (MSCOCO2014)
LightweightOpenPose (tiny VGG) 23.6 MB 432 x 368 MSCOCO2017 (all 5000 img.) 47.3 map -
LightweightOpenPose (ResNet50) 42.7 MB 432 x 368 MSCOCO2017 (all 5000 img.) 48.2 map -
PoseProposal (ResNet18) 45.2 MB 384 x 384 MPII (all 2729 img.) 54.9 map (PCKh) 72.8 map (PCKh)

License

HyperPose is open-sourced under the Apache 2.0 license.

Comments
  • Error build on windows 10

    Error build on windows 10

    I followed step suggested, but I have issue to build on windows. I tried any different compiler (VS19, VS17, VS15), on x84 on x64. There are some suggestion how to fix that?

    opened by EnricoBeltramo 26
  • why I have so low FPS?

    why I have so low FPS?

    I test it on Windows10+CUDA10.0+cudnn7.6+TensorRT7+opencv3.4, one 1080ti@E5-2620 v4, and run .\build\Release\example-stream-detector.exe --model_file=data\models\hao28-600000-256x384.uff -input_height 256 -input_width 384 --image_files=xxx , the final ouput is: // inferenced 1638 images of 256 x 384, took 124.27s, mean: 75.87ms, FPS: 13.180601, buffer size: 4, use f16: 1, gauss kernel size: 17

    why the FPS just have 13.8? if I do not config the tensorRT with right way or others reason? use f16 or not is the same.

    opened by lwcode 19
  • Training very CPU intensive

    Training very CPU intensive

    First off, awesome library, some great work here!

    Just a question/comment : I'm currently running training the hao28_experimental model, and I've found training is very CPU intensive. To the point that my GPUs are sitting idle much of the time, and all of my CPU cores are under heavy load constantly. Is this due to data augmentation? Would you recommend disabling some augmentation steps to remove this CPU bottleneck?

    opened by jpapon 18
  • tx2 testing

    tx2 testing

    Hi sir ! I have successfully compiled the project on TX2, the image tested is 384x256, There are two people in the picture, the running script is: run-uff-cpp.sh, I have closed the drawing result and the save function, but the display speed is only 4fps. Did not reach the 13FPS in the project introduction, is there any problem with my operation? The results of the operation and the pictures I put in the attachment are: The function that closes the image is shown below, I changed true to false "" bool draw_humans = true; if (draw_humans) { cv::Mat resized_image(cv::Size(width, height), CV_8UC(3), p.hwc_ptr); handle(resized_image, humans); }

    ""

    opened by drt040312 17
  • Converting model to .trt for use on Jetson AGX/DeepStream

    Converting model to .trt for use on Jetson AGX/DeepStream

    I was wondering how I can use the OpenPose models here on my AGX with DeepStream. I've installed the OpenPose_coco.onnx model so far but I'm not really sure how to go about doing this.

    opened by ConnorM17 11
  • do max pooling with cudnn

    do max pooling with cudnn

    // inferenced 80 images of 368 x 432, took 5.682237s, mean: 71.027957ms, FPS: 14.078963, batch size: 4
    
            summary of tracer_ctx_t::global (24.766687s)
    --------------------------------------------------------------------------------
       count      cumulative (s)               %       mean (ms)    call site
    --------------------------------------------------------------------------------
           1           24.766370          100.00      24766.3698    main
           1           18.450263           74.50      18450.2631    create_engine
          20            5.682039           22.94        284.1020    one_batch
          20            2.434639            9.83        121.7319    batch run tensorRT
          20            2.434452            9.83        121.7226    execute
          80            2.390308            9.65         29.8789    run paf_process
          80            2.389776            9.65         29.8722    operator()
          20            2.388232            9.64        119.4116    context->execute
          80            1.594498            6.44         19.9312    <24get_peak_map_gpu_op_implIfE>::operator()
          80            0.844929            3.41         10.5616    smooth
         160            0.494559            2.00          3.0910    resize_area
          80            0.484601            1.96          6.0575    draw_results
           1            0.439320            1.77        439.3196    createHandle
          80            0.307898            1.24          3.8487    basic_cuda_tensor::toHost
          80            0.297529            1.20          3.7191    operator()::internal
          80            0.285934            1.15          3.5742    select_peaks
          80            0.233318            0.94          2.9165    inplace_select_peaks
          80            0.201792            0.81          2.5224    basic_cuda_tensor::fromHost
          20            0.033176            0.13          1.6588    copy input from host
          20            0.032801            0.13          1.6401    cuda_buffer_t::fromHost
           7            0.019365            0.08          2.7664    tensor_t
          20            0.012523            0.05          0.6262    copy output to host
          40            0.012187            0.05          0.3047    cuda_buffer_t::toHost
          80            0.008985            0.04          0.1123    getAllConnections
           1            0.001368            0.01          1.3682    createBuffers_
           3            0.001317            0.01          0.4390    safeCudaMalloc
          80            0.000493            0.00          0.0062    getHumans
          80            0.000270            0.00          0.0034    generate output
           2            0.000019            0.00          0.0097    basic_cuda_tensor
    
    opened by lgarithm 11
  • Pre-trained HyperPose models (.npz) for the new Tensorflow 2 Implementation

    Pre-trained HyperPose models (.npz) for the new Tensorflow 2 Implementation

    Hello. Thank you very much for this amazing project, and thank you for the efforts to upgrade Hyperpose to Tensorflow 2. In my actual project, I only need the pre-trained openpose model; however, as mentioned in this issue #291, the trained models uploaded here are not compatible with the new version of Hyperpose. Therefore, I would like to ask if you could please upload the newly trained models compatible with TF2? At least I need the original openpose trained model.

    On the other hand, I started to train the model in my GPU-limited PC, and I am observing something strange.

    image

    The L2_loss of the model started in approximately 2.12. Then it decreases until approximately 1.75 but suddenly (nearly the 50.000 iterations) it started to increase, and now it is around 2.68. I used the following configuration:

    • model_type: Openpose
    • model_backbone: Default
    • dataset_type: MSCOCO
    • train_type: single_train
    • learning_rate: 4e-5
    • kf_optimizer: Sync_avg

    It is basically the same configuration provided here

    In summary, I would like to know if this behavior is normal since I am using the configuration provided in the Config folder (it is loaded by default). Also, if you (or anybody in this great community :) ) could please provide the trained models for the new TF2 implementation, it could be awesome. Thank you in advance!

    opened by carlosh93 9
  • The C++ Inference Library of OpenPose-Plus v2.0.0

    The C++ Inference Library of OpenPose-Plus v2.0.0

    Features

    • New abstraction with better performance
      • Operator API: For users to manage detailed DNN inference and post-processing. (~90 FPS in video processing)
      • Stream API: Based on Operator API, we made a scheduler to do stream processing, provide end2end video processing. You can have it with very simple C++ stream operators like << or >> to set the input/output streams. (~120FPS in video processing).
    • ONNX model parser support.
    • Documentation.

    The performance was tested on 1070 Ti with 6 physical core CPU.

    WIP

    • Documentation.
      • [x] Doxygen Comments.
      • [x] Doxygen to Sphinx ~~via Breathe~~.
      • [ ] Deployment.
    • Post-Processing.
      • [ ] Proposal Network(NMS).
    • Benchmark over new models.
      • [x] OpenPose model.
      • [x] Lightweight OpenPose.
      • [ ] Proposal Network
    • MISC.
      • [x] Tests on ONNX parser.
      • [x] Add serialized TensorRT Engine file.
      • [x] Fix merge conflicts
      • [ ] CIs.
      • [ ] Ensure the scripts and docker running.
      • [ ] Provide a end2end simple docker application to generate a processed video.

    Looking for preliminary reviews and contributions.

    opened by ganler 9
  • example-inference-1 not running. Pre trained Model error

    example-inference-1 not running. Pre trained Model error

    While running python3 examples/example-inference-1.py --path-to-npz models/pretrained-models/models/openpose-plus/hao28-pose600000.npz --images data/mscoco2017/val2017/ --limit 1

    Following error is encountered File "/usr/local/lib/python3.5/dist-packages/numpy/lib/npyio.py", line 440, in load return pickle.load(fid, **pickle_kwargs) _pickle.UnpicklingError: invalid load key, 'v'.

    opened by PrinceP 9
  • Convert models that has BN layer to UFF format

    Convert models that has BN layer to UFF format

    • [x] support channel_first in BN (https://github.com/tensorlayer/tensorlayer/commit/4e6f768bd2d0c0f27c2385ce7f541b848deb7953)
    • [x] Convert a model that has BN layer to UFF
    opened by lgarithm 9
  • make -j$(proc) error

    make -j$(proc) error

    my environment is
    NVIDIA-SMI 440.33.01 Driver Version: 440.33.01 CUDA Version: 10.2 Ubuntu 18.04 Pycharm community TensorRT7

    when in make -j$(proc),

    /home/piai/gflags-2.2.2/build/hyperpose/src/tensorrt.cpp: In function ‘nvinfer1::ICudaEngine* hyperpose::dnn::create_uff_engine(const string&, cv::Size, const string&, const std::vector<std::__cxx11::basic_string >&, int, nvinfer1::DataType)’: /home/piai/gflags-2.2.2/build/hyperpose/src/tensorrt.cpp:144:69: warning: ‘DimsCHW’ is deprecated [-Wdeprecated-declarations] nvinfer1::DimsCHW(3, input_size.height, input_size.width), ^ In file included from /home/piai/gflags-2.2.2/build/hyperpose/src/tensorrt.cpp:8:0: /usr/include/x86_64-linux-gnu/NvInfer.h:206:22: note: declared here class TRT_DEPRECATED DimsCHW : public Dims3

    /usr/bin/ld: warning: libcudnn.so.7, needed by /usr/lib/gcc/x86_64-linux-gnu/7/../../../x86_64-linux-gnu/libnvinfer.so, may conflict with libcudnn.so.8 [ 96%] Built target example.stream_api_video_paf

    (base) piai@piai-Precision-7920-Tower:~/gflags-2.2.2/build/hyperpose/build$ ./example.operator_api_batched_images_paf Add file: ../data/media/COCO_val2014_000000000544.jpg into batch. Add file: ../data/media/COCO_val2014_000000000328.jpg into batch. Add file: ../data/media/COCO_val2014_000000000488.jpg into batch. Add file: ../data/media/COCO_val2014_000000000623.jpg into batch. Add file: ../data/media/COCO_val2014_000000000536.jpg into batch. Add file: ../data/media/COCO_val2014_000000000569.jpg into batch. Add file: ../data/media/COCO_val2014_000000000459.jpg into batch. Add file: ../data/media/COCO_val2014_000000000428.jpg into batch. Add file: ../data/media/COCO_val2014_000000000589.jpg into batch. Add file: ../data/media/COCO_val2014_000000000338.jpg into batch. Add file: ../data/media/COCO_val2014_000000000360.jpg into batch. Add file: ../data/media/COCO_val2014_000000000564.jpg into batch. Add file: ../data/media/COCO_val2014_000000000257.jpg into batch. Add file: ../data/media/COCO_val2014_000000000241.jpg into batch. Add file: ../data/media/COCO_val2014_000000000192.jpg into batch. Add file: ../data/media/COCO_val2014_000000000395.jpg into batch. Add file: ../data/media/COCO_val2014_000000000415.jpg into batch. Add file: ../data/media/COCO_val2014_000000000294.jpg into batch. Add file: ../data/media/COCO_val2014_000000000474.jpg into batch. Add file: ../data/media/COCO_val2014_000000000357.jpg into batch. [HyperPose::EXAMPLE] Batch shape: [20, 3, 256, 384] ERROR: UffParser: Unsupported number of graph 0 ERROR: Failed to parse Uff in data type: float32

    this errors and warnings happen.

    how can i fix it? it worked until yesterday, but i modified hyperpose.cmake, it doesn't work.

    thank you.

    help wanted question 
    opened by MichelleJin12 8
  • Bump tensorflow from 2.3.1 to 2.9.3

    Bump tensorflow from 2.3.1 to 2.9.3

    Bumps tensorflow from 2.3.1 to 2.9.3.

    Release notes

    Sourced from tensorflow's releases.

    TensorFlow 2.9.3

    Release 2.9.3

    This release introduces several vulnerability fixes:

    TensorFlow 2.9.2

    Release 2.9.2

    This releases introduces several vulnerability fixes:

    ... (truncated)

    Changelog

    Sourced from tensorflow's changelog.

    Release 2.9.3

    This release introduces several vulnerability fixes:

    Release 2.8.4

    This release introduces several vulnerability fixes:

    ... (truncated)

    Commits
    • a5ed5f3 Merge pull request #58584 from tensorflow/vinila21-patch-2
    • 258f9a1 Update py_func.cc
    • cd27cfb Merge pull request #58580 from tensorflow-jenkins/version-numbers-2.9.3-24474
    • 3e75385 Update version numbers to 2.9.3
    • bc72c39 Merge pull request #58482 from tensorflow-jenkins/relnotes-2.9.3-25695
    • 3506c90 Update RELEASE.md
    • 8dcb48e Update RELEASE.md
    • 4f34ec8 Merge pull request #58576 from pak-laura/c2.99f03a9d3bafe902c1e6beb105b2f2417...
    • 6fc67e4 Replace CHECK with returning an InternalError on failing to create python tuple
    • 5dbe90a Merge pull request #58570 from tensorflow/r2.9-7b174a0f2e4
    • Additional commits viewable in compare view

    Dependabot compatibility score

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
    • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    • @dependabot use these labels will set the current labels as the default for future PRs for this repo and language
    • @dependabot use these reviewers will set the current reviewers as the default for future PRs for this repo and language
    • @dependabot use these assignees will set the current assignees as the default for future PRs for this repo and language
    • @dependabot use this milestone will set the current milestone as the default for future PRs for this repo and language

    You can disable automated security fix PRs for this repo from the Security Alerts page.

    dependencies 
    opened by dependabot[bot] 0
  • Trained weights of Openpose-Resnet18

    Trained weights of Openpose-Resnet18

    Thank you for your implementation for several pose models.

    Where can I find the pretrained checkpoints of OpenPose-Resnet18 or Resnet50?

    I want to evaluate the speed and performance of OpenPose with Resnet18/50 backbone, but It is not in the google drive for pretrained weights.

    I'll wait for your reply.

    opened by shawn615 0
  • Bump numpy from 1.19.5 to 1.22.0 in /docs

    Bump numpy from 1.19.5 to 1.22.0 in /docs

    Bumps numpy from 1.19.5 to 1.22.0.

    Release notes

    Sourced from numpy's releases.

    v1.22.0

    NumPy 1.22.0 Release Notes

    NumPy 1.22.0 is a big release featuring the work of 153 contributors spread over 609 pull requests. There have been many improvements, highlights are:

    • Annotations of the main namespace are essentially complete. Upstream is a moving target, so there will likely be further improvements, but the major work is done. This is probably the most user visible enhancement in this release.
    • A preliminary version of the proposed Array-API is provided. This is a step in creating a standard collection of functions that can be used across application such as CuPy and JAX.
    • NumPy now has a DLPack backend. DLPack provides a common interchange format for array (tensor) data.
    • New methods for quantile, percentile, and related functions. The new methods provide a complete set of the methods commonly found in the literature.
    • A new configurable allocator for use by downstream projects.

    These are in addition to the ongoing work to provide SIMD support for commonly used functions, improvements to F2PY, and better documentation.

    The Python versions supported in this release are 3.8-3.10, Python 3.7 has been dropped. Note that 32 bit wheels are only provided for Python 3.8 and 3.9 on Windows, all other wheels are 64 bits on account of Ubuntu, Fedora, and other Linux distributions dropping 32 bit support. All 64 bit wheels are also linked with 64 bit integer OpenBLAS, which should fix the occasional problems encountered by folks using truly huge arrays.

    Expired deprecations

    Deprecated numeric style dtype strings have been removed

    Using the strings "Bytes0", "Datetime64", "Str0", "Uint32", and "Uint64" as a dtype will now raise a TypeError.

    (gh-19539)

    Expired deprecations for loads, ndfromtxt, and mafromtxt in npyio

    numpy.loads was deprecated in v1.15, with the recommendation that users use pickle.loads instead. ndfromtxt and mafromtxt were both deprecated in v1.17 - users should use numpy.genfromtxt instead with the appropriate value for the usemask parameter.

    (gh-19615)

    ... (truncated)

    Commits

    Dependabot compatibility score

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
    • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    • @dependabot use these labels will set the current labels as the default for future PRs for this repo and language
    • @dependabot use these reviewers will set the current reviewers as the default for future PRs for this repo and language
    • @dependabot use these assignees will set the current assignees as the default for future PRs for this repo and language
    • @dependabot use this milestone will set the current milestone as the default for future PRs for this repo and language

    You can disable automated security fix PRs for this repo from the Security Alerts page.

    dependencies 
    opened by dependabot[bot] 0
  • Bump numpy from 1.16.4 to 1.22.0

    Bump numpy from 1.16.4 to 1.22.0

    Bumps numpy from 1.16.4 to 1.22.0.

    Release notes

    Sourced from numpy's releases.

    v1.22.0

    NumPy 1.22.0 Release Notes

    NumPy 1.22.0 is a big release featuring the work of 153 contributors spread over 609 pull requests. There have been many improvements, highlights are:

    • Annotations of the main namespace are essentially complete. Upstream is a moving target, so there will likely be further improvements, but the major work is done. This is probably the most user visible enhancement in this release.
    • A preliminary version of the proposed Array-API is provided. This is a step in creating a standard collection of functions that can be used across application such as CuPy and JAX.
    • NumPy now has a DLPack backend. DLPack provides a common interchange format for array (tensor) data.
    • New methods for quantile, percentile, and related functions. The new methods provide a complete set of the methods commonly found in the literature.
    • A new configurable allocator for use by downstream projects.

    These are in addition to the ongoing work to provide SIMD support for commonly used functions, improvements to F2PY, and better documentation.

    The Python versions supported in this release are 3.8-3.10, Python 3.7 has been dropped. Note that 32 bit wheels are only provided for Python 3.8 and 3.9 on Windows, all other wheels are 64 bits on account of Ubuntu, Fedora, and other Linux distributions dropping 32 bit support. All 64 bit wheels are also linked with 64 bit integer OpenBLAS, which should fix the occasional problems encountered by folks using truly huge arrays.

    Expired deprecations

    Deprecated numeric style dtype strings have been removed

    Using the strings "Bytes0", "Datetime64", "Str0", "Uint32", and "Uint64" as a dtype will now raise a TypeError.

    (gh-19539)

    Expired deprecations for loads, ndfromtxt, and mafromtxt in npyio

    numpy.loads was deprecated in v1.15, with the recommendation that users use pickle.loads instead. ndfromtxt and mafromtxt were both deprecated in v1.17 - users should use numpy.genfromtxt instead with the appropriate value for the usemask parameter.

    (gh-19615)

    ... (truncated)

    Commits

    Dependabot compatibility score

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
    • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    • @dependabot use these labels will set the current labels as the default for future PRs for this repo and language
    • @dependabot use these reviewers will set the current reviewers as the default for future PRs for this repo and language
    • @dependabot use these assignees will set the current assignees as the default for future PRs for this repo and language
    • @dependabot use this milestone will set the current milestone as the default for future PRs for this repo and language

    You can disable automated security fix PRs for this repo from the Security Alerts page.

    dependencies 
    opened by dependabot[bot] 0
  • Conversion of trained model(.npz) to tflite

    Conversion of trained model(.npz) to tflite

    @syoyo @sukhodolin @luomai @lgarithm @Yuyue Hi, I have ran the script of the export_tflite.py but meet the issue of 'BatchDataset' object is not callable. May I get your suggestion to solve this issue? Thanks.

    2022-04-14 12:03:01.103757: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library cudart64_110.dll C:\Users\tonyh\anaconda3\lib\site-packages\numpy_distributor_init.py:30: UserWarning: loaded more than 1 DLL from .libs: C:\Users\tonyh\anaconda3\lib\site-packages\numpy.libs\libopenblas.EL2C6PLE4ZYW3ECEVIV3OXXGRN2NRFM2.gfortran-win_amd64.dll C:\Users\tonyh\anaconda3\lib\site-packages\numpy.libs\libopenblas.NOIJJG62EMASZI6NYURL6JBKM4EVBGM7.gfortran-win_amd64.dll warnings.warn("loaded more than 1 DLL from .libs:" 2022-04-14 12:03:03.919437: I tensorflow/compiler/jit/xla_cpu_device.cc:41] Not creating XLA devices, tf_xla_enable_xla_devices not set 2022-04-14 12:03:03.920201: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library nvcuda.dll 2022-04-14 12:03:03.936544: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1720] Found device 0 with properties: pciBusID: 0000:02:00.0 name: NVIDIA GeForce RTX 3090 computeCapability: 8.6 coreClock: 1.695GHz coreCount: 82 deviceMemorySize: 24.00GiB deviceMemoryBandwidth: 871.81GiB/s 2022-04-14 12:03:03.936735: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library cudart64_110.dll 2022-04-14 12:03:03.941809: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library cublas64_11.dll 2022-04-14 12:03:03.949328: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library cublasLt64_11.dll 2022-04-14 12:03:03.955103: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library cufft64_10.dll 2022-04-14 12:03:03.957090: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library curand64_10.dll 2022-04-14 12:03:03.962445: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library cusolver64_10.dll 2022-04-14 12:03:03.965118: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library cusparse64_11.dll 2022-04-14 12:03:03.965988: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library cudnn64_8.dll 2022-04-14 12:03:03.966194: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1862] Adding visible gpu devices: 0 [INFO]: Configuration initialized! [MODEL] INFO: Setting Resnet18_backbone! [MODEL] INFO: Enable model backbone pretraining:False 2022-04-14 12:03:04.587777: I tensorflow/core/platform/cpu_feature_guard.cc:142] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations: AVX2 To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags. 2022-04-14 12:03:04.588737: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1720] Found device 0 with properties: pciBusID: 0000:02:00.0 name: NVIDIA GeForce RTX 3090 computeCapability: 8.6 coreClock: 1.695GHz coreCount: 82 deviceMemorySize: 24.00GiB deviceMemoryBandwidth: 871.81GiB/s 2022-04-14 12:03:04.590697: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library cudart64_110.dll 2022-04-14 12:03:04.592058: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library cublas64_11.dll 2022-04-14 12:03:04.593235: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library cublasLt64_11.dll 2022-04-14 12:03:04.594663: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library cufft64_10.dll 2022-04-14 12:03:04.596044: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library curand64_10.dll 2022-04-14 12:03:04.597845: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library cusolver64_10.dll 2022-04-14 12:03:04.599085: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library cusparse64_11.dll 2022-04-14 12:03:04.600392: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library cudnn64_8.dll 2022-04-14 12:03:04.601523: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1862] Adding visible gpu devices: 0 2022-04-14 12:03:05.265789: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1261] Device interconnect StreamExecutor with strength 1 edge matrix: 2022-04-14 12:03:05.265974: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1267] 0 2022-04-14 12:03:05.267897: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1280] 0: N 2022-04-14 12:03:05.269206: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1406] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 21821 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 3090, pci bus id: 0000:02:00.0, compute capability: 8.6) 2022-04-14 12:03:05.272014: I tensorflow/compiler/jit/xla_gpu_device.cc:99] Not creating XLA devices, tf_xla_enable_xla_devices not set [MODEL] INFO: Using LightweightOpenpose model arch!

    [INFO]: Welcome to Hyperpose Development Platform!

    ==================================================================================================== [INFO]: Variable Definition: [INFO]: parts: the joints of human body, Enum class [INFO]: limbs: the limbs of human body, List of tuple. example: [(joint index 1, joint index 2),...] [INFO]: colors: the visualize color for each parts, List. example: [(0,0,255),...] (optional) [INFO]: n_parts: number of human body joints, int. example: n_parts=len(parts) [INFO]: n_limbs: number of human body limbs, int. example: n_limbs=len(limbs) [INFO]: hin: height of the model input image, int. example: 368 [INFO]: win: width of the model input image, int. example: 368 [INFO]: hout: height of model output heatmap, int. example: 46 [INFO]: wout: wout of model output heatmap, int. example: 46

    ==================================================================================================== [INFO]: Object Definition: [INFO]: config: a object contains all the configurations used to assemble the model, dataset, and pipeline. easydict object. return by the Config.get_config function.

    [INFO]: model: a neural network takes in the image and output the calculated activation map. BasicModel object. have forward, cal_loss, infer(optional) functions. custom: users could inherit the Model.BasicModel class for customization. example: please refer to Model.LightWeightOpenPose class for details.

    [INFO]: dataset: a dataset generator provides train and evaluate dataset. Base_dataset object. have get_train_dataset and get_eval_dataset functions. custom: users could inherit the Dataset.BasicDataset class for customizationn example: please refer to Datset.CocoDataset class for details.

    [INFO]: augmentor: a data augmentor that takes in the image, key point annotation, mask and perform affine transformation for data augmentation. BasicAumentor object. have process and process_only_image functions. custom: users could inherit the Model.BasicAugmentor class for customization. example: please refer to Model.BasicAugmentor class for details.

    [INFO]: preprocessor: a data preprocessor that takes in the image, key point annotation and mask to produce the target heatmap for model to calculate loss and learn. BasicPreProcessor object. have process function. custom: users could inherit the Model.BasicPreProcessor class for customizationn example: please refer to Model.openpose.PreProcessor class for details.

    [INFO]: postprocessor: a data postprocessor that takes in the predicted heatmaps and infer the human body joints and limbs. have process function. BasicPostProcessor object. custom: users could inherit the Model.BasicPostProcessor class for customization example: please refer to the Model.openpose.PostProcessor class for details.

    [INFO]: visualizer: a visualizer that takes in the predicted heatmaps and output visualization images for train and evaluation. have visualize and visualize_comapre functions. BasicVisualizer object. custom: users could inherit the Model.BasicVisualizer class for customization. example: please refer to the Model.openpose.Visualizer class for details.

    ==================================================================================================== [INFO]: Development platform basic usage: 1.Use the sets APIs of Config module to configure the pipeline, choose the algorithm type, the neural network backbone, the dataset etc. that best fit your application scenario. 2.Use the get_model API of Model module to get the configured model, use get_dataset API of dataset module to get the configured dataset, use the get_train API of Model module to get the configured train procedure. Then start training! Check the loss values and sample training result images during training. 3.Use the get_eval API of Model module to get the configured evaluation procedure. evaluate the model you trained. 4.Eport the model to .pb, .onnx, .tflite formats for deployment. [INFO]: Development platform custom usage: Hyperpose enables users to custom model, dataset, augmentor, preprocessor, postprocessor and visualizer. Users could inherit the corresponding basic class(mentioned above), and implement corresponding the member functions required according to the function annotation, then use Config.set_custom_xxx APIs to set the custom component. [INFO]: Additional features: 1.Parallel distributed training with Kungfu. 2.Domain adaption to leverage unlabled data. 3.Neural network backbone pretraining. [INFO]: Currently all the procedures are uniformed to be channels_first data format. [INFO]: Currently all model weights are saved in npz_dict format.

    ==================================================================================================== [DATA] INFO: Using MSCOCO dataset! [DATA] INFO: Using MSCOCO dataset version:2017 [DATA] INFO: generating official evaluating data... loading annotations into memory... Done (t=0.22s) creating index... index created! Overall get 5000 valid pose images from ./data\mscoco2017\val2017 and ./data\mscoco2017\annotations\person_keypoints_val2017.json [DATA] INFO: total 5000 official evaluating data generated! [DATA] INFO: filtering non-exist images and targets [DATA] INFO: filtering finished! total 5000 images and targets left, 0 invalid found. [DATA] INFO: eval dataset generation finished! test rep_dataset:<BatchDataset shapes: , types: tf.float32> 2022-04-14 12:03:10.341282: I tensorflow/core/grappler/devices.cc:69] Number of eligible GPUs (core count >= 8, compute capability >= 0.0): 1 2022-04-14 12:03:10.341886: I tensorflow/core/grappler/clusters/single_machine.cc:356] Starting new session 2022-04-14 12:03:10.345371: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1720] Found device 0 with properties: pciBusID: 0000:02:00.0 name: NVIDIA GeForce RTX 3090 computeCapability: 8.6 coreClock: 1.695GHz coreCount: 82 deviceMemorySize: 24.00GiB deviceMemoryBandwidth: 871.81GiB/s 2022-04-14 12:03:10.345519: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library cudart64_110.dll 2022-04-14 12:03:10.346483: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library cublas64_11.dll 2022-04-14 12:03:10.347721: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library cublasLt64_11.dll 2022-04-14 12:03:10.348919: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library cufft64_10.dll 2022-04-14 12:03:10.350544: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library curand64_10.dll 2022-04-14 12:03:10.352452: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library cusolver64_10.dll 2022-04-14 12:03:10.353376: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library cusparse64_11.dll 2022-04-14 12:03:10.354696: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library cudnn64_8.dll 2022-04-14 12:03:10.357520: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1862] Adding visible gpu devices: 0 2022-04-14 12:03:10.358586: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1261] Device interconnect StreamExecutor with strength 1 edge matrix: 2022-04-14 12:03:10.359677: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1267] 0 2022-04-14 12:03:10.360801: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1280] 0: N 2022-04-14 12:03:10.361998: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1406] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 21821 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 3090, pci bus id: 0000:02:00.0, compute capability: 8.6) 2022-04-14 12:03:10.363133: I tensorflow/compiler/jit/xla_gpu_device.cc:99] Not creating XLA devices, tf_xla_enable_xla_devices not set 2022-04-14 12:03:10.644173: I tensorflow/core/grappler/optimizers/meta_optimizer.cc:928] Optimization results for grappler item: graph_to_optimize function_optimizer: Graph size after: 1511 nodes (1303), 3016 edges (2807), time = 51.623ms. function_optimizer: function_optimizer did nothing. time = 1.284ms.

    2022-04-14 12:03:14.000137: W tensorflow/compiler/mlir/lite/python/tf_tfl_flatbuffer_helpers.cc:316] Ignored output_format. 2022-04-14 12:03:14.000269: W tensorflow/compiler/mlir/lite/python/tf_tfl_flatbuffer_helpers.cc:319] Ignored drop_control_dependency. 2022-04-14 12:03:14.125133: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1720] Found device 0 with properties: pciBusID: 0000:02:00.0 name: NVIDIA GeForce RTX 3090 computeCapability: 8.6 coreClock: 1.695GHz coreCount: 82 deviceMemorySize: 24.00GiB deviceMemoryBandwidth: 871.81GiB/s 2022-04-14 12:03:14.125440: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library cudart64_110.dll 2022-04-14 12:03:14.128525: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library cublas64_11.dll 2022-04-14 12:03:14.129714: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library cublasLt64_11.dll 2022-04-14 12:03:14.130943: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library cufft64_10.dll 2022-04-14 12:03:14.132017: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library curand64_10.dll 2022-04-14 12:03:14.133044: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library cusolver64_10.dll 2022-04-14 12:03:14.134062: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library cusparse64_11.dll 2022-04-14 12:03:14.135088: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library cudnn64_8.dll 2022-04-14 12:03:14.136167: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1862] Adding visible gpu devices: 0 2022-04-14 12:03:14.137155: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1261] Device interconnect StreamExecutor with strength 1 edge matrix: 2022-04-14 12:03:14.138268: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1267] 0 2022-04-14 12:03:14.139320: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1280] 0: N 2022-04-14 12:03:14.140509: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1406] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 21821 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 3090, pci bus id: 0000:02:00.0, compute capability: 8.6) 2022-04-14 12:03:14.141507: I tensorflow/compiler/jit/xla_gpu_device.cc:99] Not creating XLA devices, tf_xla_enable_xla_devices not set INFO: Created TensorFlow Lite delegate for select TF ops. 2022-04-14 12:03:16.153224: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1720] Found device 0 with properties: pciBusID: 0000:02:00.0 name: NVIDIA GeForce RTX 3090 computeCapability: 8.6 coreClock: 1.695GHz coreCount: 82 deviceMemorySize: 24.00GiB deviceMemoryBandwidth: 871.81GiB/s 2022-04-14 12:03:16.154270: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library cudart64_110.dll 2022-04-14 12:03:16.156074: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library cublas64_11.dll 2022-04-14 12:03:16.157709: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library cublasLt64_11.dll 2022-04-14 12:03:16.158826: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library cufft64_10.dll 2022-04-14 12:03:16.159961: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library curand64_10.dll 2022-04-14 12:03:16.161424: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library cusolver64_10.dll 2022-04-14 12:03:16.162684: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library cusparse64_11.dll 2022-04-14 12:03:16.163717: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library cudnn64_8.dll 2022-04-14 12:03:16.164830: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1862] Adding visible gpu devices: 0 2022-04-14 12:03:16.165863: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1261] Device interconnect StreamExecutor with strength 1 edge matrix: 2022-04-14 12:03:16.166728: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1267] 0 2022-04-14 12:03:16.167719: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1280] 0: N 2022-04-14 12:03:16.167826: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1406] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 21821 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 3090, pci bus id: 0000:02:00.0, compute capability: 8.6) 2022-04-14 12:03:16.168712: I tensorflow/compiler/jit/xla_gpu_device.cc:99] Not creating XLA devices, tf_xla_enable_xla_devices not set INFO: TfLiteFlexDelegate delegate: 1 nodes delegated out of 192 nodes with 1 partitions.

    Traceback (most recent call last): File "export_tflite.py", line 38, in tflite_model_quant = converter.convert() File "C:\Users\tonyh\anaconda3\lib\site-packages\tensorflow\lite\python\lite.py", line 1117, in convert return super(TFLiteConverterV2, self).convert() File "C:\Users\tonyh\anaconda3\lib\site-packages\tensorflow\lite\python\lite.py", line 942, in convert return super(TFLiteFrozenGraphConverterV2, File "C:\Users\tonyh\anaconda3\lib\site-packages\tensorflow\lite\python\lite.py", line 632, in convert result = self._calibrate_quantize_model(result, **flags) File "C:\Users\tonyh\anaconda3\lib\site-packages\tensorflow\lite\python\lite.py", line 459, in _calibrate_quantize_model return calibrate_quantize.calibrate_and_quantize( File "C:\Users\tonyh\anaconda3\lib\site-packages\tensorflow\lite\python\optimize\calibrator.py", line 93, in calibrate_and_quantize for sample in dataset_gen(): TypeError: 'BatchDataset' object is not callable

    opened by maketo97 0
  • Issue of converting the trained NPZ model into ProtoBuf format

    Issue of converting the trained NPZ model into ProtoBuf format

    @syoyo @sukhodolin @luomai @lgarithm @Yuyue

    Hi, I do the conversion of lightweight openpose vggtiny given by the authors using the 'export_pb.py' but encounter the error. May you help me in this issue? Thanks in advance.

    Log file for the execution of 'export_pb.py":-

    2022-04-07 16:04:34.939061: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library cudart64_101.dll 2022-04-07 16:04:37.196428: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library nvcuda.dll 2022-04-07 16:04:38.231690: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1716] Found device 0 with properties: pciBusID: 0000:01:00.0 name: NVIDIA GeForce GTX 1660 Ti computeCapability: 7.5 coreClock: 1.59GHz coreCount: 24 deviceMemorySize: 6.00GiB deviceMemoryBandwidth: 268.26GiB/s 2022-04-07 16:04:38.231824: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library cudart64_101.dll 2022-04-07 16:04:38.234714: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library cublas64_10.dll 2022-04-07 16:04:38.237753: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library cufft64_10.dll 2022-04-07 16:04:38.238956: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library curand64_10.dll 2022-04-07 16:04:38.242075: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library cusolver64_10.dll 2022-04-07 16:04:38.243890: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library cusparse64_10.dll 2022-04-07 16:04:38.250656: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library cudnn64_7.dll 2022-04-07 16:04:38.250867: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1858] Adding visible gpu devices: 0 [INFO]: Configuration initialized! [MODEL] INFO: Setting Vggtiny_backbone! [MODEL] INFO: Enable model backbone pretraining:False 2022-04-07 16:04:38.713381: I tensorflow/core/platform/cpu_feature_guard.cc:142] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN)to use the following CPU instructions in performance-critical operations: AVX2 To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags. 2022-04-07 16:04:38.720480: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x27c9f0da7d0 initialized for platform Host (this does not guarantee that XLA will be used). Devices: 2022-04-07 16:04:38.720618: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): Host, Default Version 2022-04-07 16:04:39.231945: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1716] Found device 0 with properties: pciBusID: 0000:01:00.0 name: NVIDIA GeForce GTX 1660 Ti computeCapability: 7.5 coreClock: 1.59GHz coreCount: 24 deviceMemorySize: 6.00GiB deviceMemoryBandwidth: 268.26GiB/s 2022-04-07 16:04:39.232143: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library cudart64_101.dll 2022-04-07 16:04:39.232955: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library cublas64_10.dll 2022-04-07 16:04:39.233246: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library cufft64_10.dll 2022-04-07 16:04:39.234553: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library curand64_10.dll 2022-04-07 16:04:39.234594: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library cusolver64_10.dll 2022-04-07 16:04:39.234623: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library cusparse64_10.dll 2022-04-07 16:04:39.234866: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library cudnn64_7.dll 2022-04-07 16:04:39.234949: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1858] Adding visible gpu devices: 0 2022-04-07 16:04:39.791163: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1257] Device interconnect StreamExecutor with strength 1 edge matrix: 2022-04-07 16:04:39.791302: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1263] 0 2022-04-07 16:04:39.791786: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1276] 0: N 2022-04-07 16:04:39.792406: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1402] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 4743 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce GTX 1660 Ti, pci bus id: 0000:01:00.0, compute capability: 7.5) 2022-04-07 16:04:39.796529: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x27cc1277d60 initialized for platform CUDA (this does not guarantee that XLA will be used). Devices: 2022-04-07 16:04:39.796586: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): NVIDIA GeForce GTX 1660 Ti, Compute Capability 7.5 [MODEL] INFO: Using LightweightOpenpose model arch!

    [INFO]: Welcome to Hyperpose Development Platform!

    ==================================================================================================== [INFO]: Variable Definition: [INFO]: parts: the joints of human body, Enum class [INFO]: limbs: the limbs of human body, List of tuple. example: [(joint index 1, joint index 2),...] [INFO]: colors: the visualize color for each parts, List. example: [(0,0,255),...] (optional) [INFO]: n_parts: number of human body joints, int. example: n_parts=len(parts) [INFO]: n_limbs: number of human body limbs, int. example: n_limbs=len(limbs) [INFO]: hin: height of the model input image, int. example: 368 [INFO]: win: width of the model input image, int. example: 368 [INFO]: hout: height of model output heatmap, int. example: 46 [INFO]: wout: wout of model output heatmap, int. example: 46

    ==================================================================================================== [INFO]: Object Definition: [INFO]: config: a object contains all the configurations used to assemble the model, dataset, and pipeline. easydict object. return by the Config.get_config function.

    [INFO]: model: a neural network takes in the image and output the calculated activation map. BasicModel object. have forward, cal_loss, infer(optional) functions. custom: users could inherit the Model.BasicModel class for customization. example: please refer to Model.LightWeightOpenPose class for details.

    [INFO]: dataset: a dataset generator provides train and evaluate dataset. Base_dataset object. have get_train_dataset and get_eval_dataset functions. custom: users could inherit the Dataset.BasicDataset class for customizationn example: please refer to Datset.CocoDataset class for details.

    [INFO]: augmentor: a data augmentor that takes in the image, key point annotation, mask and perform affine transformation for data augmentation. BasicAumentor object. have process and process_only_image functions. custom: users could inherit the Model.BasicAugmentor class for customization. example: please refer to Model.BasicAugmentor class for details.

    [INFO]: preprocessor: a data preprocessor that takes in the image, key point annotation and mask to produce the target heatmap for model to calculate loss and learn. BasicPreProcessor object. have process function. custom: users could inherit the Model.BasicPreProcessor class for customizationn example: please refer to Model.openpose.PreProcessor class for details.

    [INFO]: postprocessor: a data postprocessor that takes in the predicted heatmaps and infer the human body joints and limbs. have process function. BasicPostProcessor object. custom: users could inherit the Model.BasicPostProcessor class for customization example: please refer to the Model.openpose.PostProcessor class for details.

    [INFO]: visualizer: a visualizer that takes in the predicted heatmaps and output visualization images for train and evaluation. have visualize and visualize_comapre functions. BasicVisualizer object. custom: users could inherit the Model.BasicVisualizer class for customization. example: please refer to the Model.openpose.Visualizer class for details.

    ==================================================================================================== [INFO]: Development platform basic usage: 1.Use the sets APIs of Config module to configure the pipeline, choose the algorithm type, the neural network backbone, the dataset etc. that best fit your application scenario. 2.Use the get_model API of Model module to get the configured model, use get_dataset API of dataset module to get the configured dataset, use the get_train API of Model module to get the configured train procedure. Then start training! Check the loss values and sample training result images during training. 3.Use the get_eval API of Model module to get the configured evaluation procedure. evaluate the model you trained. 4.Eport the model to .pb, .onnx, .tflite formats for deployment. [INFO]: Development platform custom usage: Hyperpose enables users to custom model, dataset, augmentor, preprocessor, postprocessor and visualizer. Users could inherit the corresponding basic class(mentioned above), and implement corresponding the member functions required according to the function annotation, then use Config.set_custom_xxx APIs to set the custom component. [INFO]: Additional features: 1.Parallel distributed training with Kungfu. 2.Domain adaption to leverage unlabled data. 3.Neural network backbone pretraining. [INFO]: Currently all the procedures are uniformed to be channels_first data format. [INFO]: Currently all model weights are saved in npz_dict format.

    ==================================================================================================== export_batch_size=None export_h=None export_w=None Exporting model MyLightweightOpenpose from ./save_dir/MyLightweightOpenpose/model_dir/newest_model.npz... Traceback (most recent call last): File "export_pb.py", line 87, in export_model.load_weights(input_path) File "C:\Users\User\anaconda3\lib\site-packages\tensorlayer\models\core.py", line 958, in load_weights utils.load_and_assign_npz(filepath, self) File "C:\Users\User\anaconda3\lib\site-packages\tensorlayer\files\utils.py", line 2045, in load_and_assign_npz weights = load_npz(name=name) File "C:\Users\User\anaconda3\lib\site-packages\tensorlayer\files\utils.py", line 1985, in load_npz return d['params'] File "C:\Users\User\anaconda3\lib\site-packages\numpy\lib\npyio.py", line 266, in getitem raise KeyError("%s is not a file in the archive" % key) KeyError: 'params is not a file in the archive'

    opened by maketo97 0
Releases(v2.2.0)
  • v2.2.0(Jun 30, 2021)

    • OpenPifPaf model support (Python Training & C++ Inference), 3.1x SPEED UP 🚀🚀🚀
    • Docker container updated to 2.2.0: https://hub.docker.com/repository/docker/tensorlayer/hyperpose
      • support CUDA 10 series (low driver requirement) and advanced CUDA 10.2 - CuDNN 8 - TensorRT 8 series.
    • Pip installation! https://pypi.org/project/hyperpose/
    • Add a bunch of new pre-trained models, please check: https://drive.google.com/drive/folders/1w9EjMkrjxOmMw3Rf6fXXkiv_ge7M99jR
      • OpenPifPaf, OpenPose-VGGv2, OpenPose-MobileNet
    • Heavily refactored documentation for a better reading experience! https://hyperpose.readthedocs.io/en/latest/index.html
    Source code(tar.gz)
    Source code(zip)
  • 2.2.0-alpha(Jun 26, 2021)

    • OpenPifPaf model support (Python Training + C++ Inference);
    • Docker container updated to 2.2.0: https://hub.docker.com/repository/docker/tensorlayer/hyperpose
    • Add a bunch of new pre-trained models, please check: https://drive.google.com/drive/folders/1w9EjMkrjxOmMw3Rf6fXXkiv_ge7M99jR
    • Refactored documents.
    Source code(tar.gz)
    Source code(zip)
  • 2.1.0(Aug 30, 2020)

    • Docker support: https://hub.docker.com/repository/docker/tensorlayer/hyperpose
    • HyperPose command-line tool: hyperpose-cli
    • Add functionality to keep the original aspect ratio(this is good for precision).
    • New README.
    • Visualization:
      • Line size varies with person size.
      • Translucent connection line in hyperpose-cli.
    Source code(tar.gz)
    Source code(zip)
  • 2.0.0(Aug 17, 2020)

    Key Features:

    • Python Training Module
      • User-defined model arch
      • MS COCO + MPII dataset support
      • User-defined dataset filter and training pipeline
    • C++ Accelerated Prediction
      • Operator API
      • Stream API
    • More model supports
      • OpenPose
      • Lightweight-OpenPose
      • PoseProposal Network
    • Documentation
    • More pretrained models(Continuous updating)
    Source code(tar.gz)
    Source code(zip)
  • 1.0.0(May 3, 2020)

    The 1st version of OpenPose-Plus provides the functionality to do low-level inference, local training, and distributed training. The inference API of the 1st version is quite straightforward and low-level(without much high-level abstraction).

    As a new version of OpenPose-Plus is upcoming, we made the old version a release.

    Source code(tar.gz)
    Source code(zip)
Owner
TensorLayer Community
An open community to promote AI technology.
TensorLayer Community
Simple Pose: Rethinking and Improving a Bottom-up Approach for Multi-Person Pose Estimation

SimplePose Code and pre-trained models for our paper, “Simple Pose: Rethinking and Improving a Bottom-up Approach for Multi-Person Pose Estimation”, a

Jia Li 256 Dec 24, 2022
Repository for the paper "PoseAug: A Differentiable Pose Augmentation Framework for 3D Human Pose Estimation", CVPR 2021.

PoseAug: A Differentiable Pose Augmentation Framework for 3D Human Pose Estimation Code repository for the paper: PoseAug: A Differentiable Pose Augme

Pyjcsx 328 Dec 17, 2022
This repository contains codes of ICCV2021 paper: SO-Pose: Exploiting Self-Occlusion for Direct 6D Pose Estimation

SO-Pose This repository contains codes of ICCV2021 paper: SO-Pose: Exploiting Self-Occlusion for Direct 6D Pose Estimation This paper is basically an

shangbuhuan 52 Nov 25, 2022
Python scripts for performing 3D human pose estimation using the Mobile Human Pose model in ONNX.

Python scripts for performing 3D human pose estimation using the Mobile Human Pose model in ONNX.

Ibai Gorordo 99 Dec 31, 2022
Web service for facial landmark detection, head pose estimation, facial action unit recognition, and eye-gaze estimation based on OpenFace 2.0

OpenGaze: Web Service for OpenFace Facial Behaviour Analysis Toolkit Overview OpenFace is a fantastic tool intended for computer vision and machine le

Sayom Shakib 4 Nov 3, 2022
OpenFace – a state-of-the art tool intended for facial landmark detection, head pose estimation, facial action unit recognition, and eye-gaze estimation.

OpenFace 2.2.0: a facial behavior analysis toolkit Over the past few years, there has been an increased interest in automatic facial behavior analysis

Tadas Baltrusaitis 5.8k Dec 31, 2022
The project is an official implementation of our CVPR2019 paper "Deep High-Resolution Representation Learning for Human Pose Estimation"

Deep High-Resolution Representation Learning for Human Pose Estimation (CVPR 2019) News [2020/07/05] A very nice blog from Towards Data Science introd

Leo Xiao 3.9k Jan 5, 2023
Deep High-Resolution Representation Learning for Human Pose Estimation

Deep High-Resolution Representation Learning for Human Pose Estimation (accepted to CVPR2019) News If you are interested in internship or research pos

HRNet 167 Dec 27, 2022
Re-implementation of the Noise Contrastive Estimation algorithm for pyTorch, following "Noise-contrastive estimation: A new estimation principle for unnormalized statistical models." (Gutmann and Hyvarinen, AISTATS 2010)

Noise Contrastive Estimation for pyTorch Overview This repository contains a re-implementation of the Noise Contrastive Estimation algorithm, implemen

Denis Emelin 42 Nov 24, 2022
API for RL algorithm design & testing of BCA (Building Control Agent) HVAC on EnergyPlus building energy simulator by wrapping their EMS Python API

RL - EmsPy (work In Progress...) The EmsPy Python package was made to facilitate Reinforcement Learning (RL) algorithm research for developing and tes

null 20 Jan 5, 2023
A fast, scalable, high performance Gradient Boosting on Decision Trees library, used for ranking, classification, regression and other machine learning tasks for Python, R, Java, C++. Supports computation on CPU and GPU.

Website | Documentation | Tutorials | Installation | Release Notes CatBoost is a machine learning method based on gradient boosting over decision tree

CatBoost 6.9k Jan 4, 2023
A fast, scalable, high performance Gradient Boosting on Decision Trees library, used for ranking, classification, regression and other machine learning tasks for Python, R, Java, C++. Supports computation on CPU and GPU.

Website | Documentation | Tutorials | Installation | Release Notes CatBoost is a machine learning method based on gradient boosting over decision tree

CatBoost 5.7k Feb 12, 2021
PPLNN is a Primitive Library for Neural Network is a high-performance deep-learning inference engine for efficient AI inferencing

PPLNN is a Primitive Library for Neural Network is a high-performance deep-learning inference engine for efficient AI inferencing

null 943 Jan 7, 2023
Bytedance Inc. 2.5k Jan 6, 2023
🔥🔥High-Performance Face Recognition Library on PaddlePaddle & PyTorch🔥🔥

face.evoLVe: High-Performance Face Recognition Library based on PaddlePaddle & PyTorch Evolve to be more comprehensive, effective and efficient for fa

Zhao Jian 3.1k Jan 2, 2023
🔥🔥High-Performance Face Recognition Library on PaddlePaddle & PyTorch🔥🔥

face.evoLVe: High-Performance Face Recognition Library based on PaddlePaddle & PyTorch Evolve to be more comprehensive, effective and efficient for fa

Zhao Jian 3.1k Jan 4, 2023
Vector AI — A platform for building vector based applications. Encode, query and analyse data using vectors.

Vector AI is a framework designed to make the process of building production grade vector based applications as quickly and easily as possible. Create

Vector AI 267 Dec 23, 2022
《Unsupervised 3D Human Pose Representation with Viewpoint and Pose Disentanglement》(ECCV 2020) GitHub: [fig9]

Unsupervised 3D Human Pose Representation [Paper] The implementation of our paper Unsupervised 3D Human Pose Representation with Viewpoint and Pose Di

null 42 Nov 24, 2022
Example-custom-ml-block-keras - Custom Keras ML block example for Edge Impulse

Custom Keras ML block example for Edge Impulse This repository is an example on

Edge Impulse 8 Nov 2, 2022