A deep learning object detector framework written in Python for supporting Land Search and Rescue Missions.

Overview

AIR-logo

AIR: Aerial Inspection RetinaNet for supporting Land Search and Rescue Missions

DOI Docker Pulls

AIR is a deep learning based object detection solution to automate the aerial drone footage inspection task frequently carried out during search and rescue (SAR) operations with drone units. It provides a fast, convenient and reliable way to augment aerial, high-resolution image inspection for clues about human presence by highlighting relevant image regions with bounding boxes, as done in the image below. With the assistance of AIR, SAR missions with aerial drone searches can likely be carried out much faster before, and with considerably higher success rate.

This code repository is based on the master's thesis work by Pasi Pyrrö ORCID iD icon from Aalto University, School of Science with the title "AIR: Aerial Inspection RetinaNet for Land Search and Rescue Missions". This thesis work was funded by Accenture.

AIR-example

Key features

  • Supports both video and images
  • Highly accurate deep learning detector custom built for this application
  • Decent processing speed despite the high-resolution images and a very deep neural network used
  • Supports object tracking with kalman filter
  • Implements the novel MOB bounding box postprocessing algorithm
  • Implements the SAR-APD evaluation algorithm for real-world, value-oriented metric reporting in SAR aerial person detection applications
  • Plethora of options for customization (e.g., in bounding box postprocessing, image preprocessing, speed vs accuracy trade-offs, etc.)

Results on HERIDAL test set

In the table below are listed the results of the AIR detector and other notable state-of-the-art methods on the HERIDAL test set. Note that the FPS metric is a rough estimate based on the reported inference speed and hardware setup by the original authors, and should be taken as mostly directional. We did not test any of the other methods ourselves.

Method Precision Recall AP FPS
Mean shift segmentation method [1] 18.7 74.7 - -
Saliency guided VGG16 [2] 34.8 88.9 - -
Faster R-CNN [3] 67.3 88.3 86.1 1
Two-stage multimodel CNN [4] 68.9 94.7 - 0.1
SSD [4] 4.3 94.4 - -
AIR with NMS (ours) 90.1 86.1 84.6 1

It turns out AIR achieves state-of-the-art results in precison and inference speed while having comparable recall to the strongest competitors!

You can check out the full details of AIR evaluation in this Wandb report.

Bounding Box Aggregation options

NMS-vs-MOB

AIR implements both NMS and MOB algorithms for bounding box prediction postprocessing. The image above shows the main difference: MOB (c) merges bounding boxes (a) instead of eliminating them like NMS (b). Thus, choosing MOB can produce visually more pleasing predictions. Moreover, MOB is less sensitive to the choice of condifence score threshold, making it more robust under unseen data. AIR also comes with a custom SAR-APD evaluation scheme that truthfully ranks MOB-equipped object detector performance (as standard object detection metrics, such as VOC2012 AP, do not like MOB very much).

Should you choose MOB postprocessing and SAR-APD evaluation, you should see similar evaluation results (e.g., after running bash evaluate.sh)

NMS-vs-MOB

All metrics increased by over 4 percentage points from NMS & VOC2012 results with no loss of prediction visual quality, neat!

Hardware Requirements

  • x86 or x64 processor architecture (at least for the Docker installation, and if trying to install natively on ARM64 architecture, expect a lot of compiling python packages from source)
  • NVIDIA® GPU with CUDA® capability 3.5 or higher (recommended)
  • ~8 GB of RAM or more

Software Requirements

If using containers:

If using native installation:

  • Python 3.6
  • pip and setuptools
  • FFmpeg (recommended)
  • Linux or Mac OS (not tested on Windows)

Quick install instructions using Docker

  • Clone this repo
  • Either start docker CPU environment by running: bash start-air-cpu-env.sh docker
  • Or start docker GPU environment by running: bash start-air-gpu-env.sh docker
  • Inside docker container, build the AIR project by running: python setup.py build_ext --inplace

Quick install instructions using Singularity

  • Clone this repo
  • You might wanna specify your SINGULARITY_CACHEDIR env variable prior to installation, if so edit the shell scripts used in the next steps
  • Either start docker CPU environment by running: bash start-air-cpu-env.sh singularity
  • Or start docker GPU environment by running: bash start-air-gpu-env.sh singularity
  • Go to mounted directory by typing cd AIR
  • Build the AIR project by running: python setup.py build_ext --inplace

Quick native install instructions

  • Clone this repo
  • To build the AIR project for CPU, run the command: /usr/bin/python3 -m pip install air-detector[cpu]
  • To build the AIR project for GPU, run the command: /usr/bin/python3 -m pip install air-detector[gpu]

Download data and the trained model

Quick demos

Once everything is set up (installation and asset downloads), you might wanna try out these cool and simple demos to get the hang of using the AIR detector.

Running inference on HERIDAL test image folder

  • In the project root, run the command: /bin/bash infer.sh
  • Check data/predictions/dauntless-sweep-2_resnet152_pascal-enclose-inference/ folder for the output images

Evaluating AIR performance on HERIDAL test set (can be slow on CPU)

  • In the project root, run the command: /bin/bash evaluate.sh
  • Check data/predictions/dauntless-sweep-2_resnet152_pascal-enclose-sar-apd-eval/ folder for the output images

SAR-APD video detection with image output

  • In the project root, run the command: /bin/bash convert-model.sh
  • Then start the inferencing by running: /usr/bin/python3 video_detect.py -c mob_cpu_images
  • Check data/videos/Ylojarvi-gridiajo-two-guys-moving_air_output/ folder for the output images

SAR-APD video detection with video output

  • In the project root, run the command: /bin/bash convert-model.sh
  • Then start the inferencing by running: /usr/bin/python3 video_detect.py -c mob_gpu_tracking
  • Check data/videos/Ylojarvi-gridiajo-two-guys-moving_air_output_compressed.mov output video

Wandb support

  • AIR uses experiment tracking software by Weight & Biases
  • Wandb enables you to view the whole public experimentation record for AIR
  • AIR experiments can be controlled with the following environment variables:
    • WANDB_MODE="disabled"
      • disables all wandb functionality, required if not logged in to wandb
    • WANDB_MODE="dryrun"
      • wandb logging works as normal, but nothing is uploaded to cloud
    • WANDB_API_KEY=<your_api_key>
      • needed to interact with wandb web UI, you might want to put this in your ~/.bashrc or ~/.zshrc so that it's automatically included into the Docker envs. On most linux based systems, you can achieve this by running this shell command with your Wandb API key:
        echo "export WANDB_API_KEY=<your_api_key>" >> "~/.${SHELL/\/bin\//}rc"; exec $SHELL
    • WANDB_DIR=~/wandb
      • select where local log files are stored
    • WANDB_PROJECT="air-testing"
      • select your wandb cloud project
    • WANDB_ENTITY="ML-Mike"
      • select your wandb account username
    • WANDB_RUN_GROUP="cool-experiments"
      • Semantic group within a project to log experiments runs to
  • More info in wandb documentation (not all env variables are supported though)

Project folder structure

  • airutils: air utility functions that augment the standard keras-retinanet framework to incorporate aerial person detection (APD) support to it.
  • config: configuration files for video_detect.py script (easier to use than the CLI in most cases)
  • data: input and output data for the AIR detector, divided into subfolders:
    • /datasets (input for training and evaluation)
    • /images (input for general inference)
    • /predictions (output of the AIR detector)
    • /videos (input for video_detect.py)
  • keras-retinanet: a fork of the keras-retinanet repository with some AIR modifications
  • models: contains trained object detection models (both trainable and inference models)

General toubleshooting

  • Try setting the AIR_VERBOSE=1 enviroment variable to see full TF logs
  • If using docker, make sure you're allocating enough memory for the container (like at least 8 GB)
  • If some script simply gets "killed", it's a clear indicator that you have too little memory allocated
  • Tracker options in video_detect.py might need to be recalibrated for each use case for best performance
  • If you edit any of the bash scripts in the root folder (e.g., evaluate.sh), make sure there are no whitespace after linebreaks '\', bash can be picky about these things... Also avoid commenting out any command line parameters in those scripts, just delete the whole line outright
  • The actual python scripts corresponding to the bash scripts are in keras_retinanet/keras_retinanet/bin folder and they can be called directly (or you can try the installed console scripts (e.g., air-evaluate) if you ran pip install .) with approproate parameters (examples can be found in those bash scripts)
  • Using the command line parameter -h or --help usually helps (pun intended)
  • If you find a bug or have a complicated question, open a new issue here

Acknowledgements

  • Kudos to Aalto Science-IT project for providing the compute hardware for training and testing the AIR detector
  • Big thanks to contributors of keras-retinanet, and their marvellous programming efforts that have allowed AIR to exists in the first place!
  • Huge credit to the authors of the HERIDAL dataset [2], for which AIR equally well owes its very existence
  • Also thanks to Ari Nissinen from Finnish Voluntary Rescue Service (VAPEPA) for providing video material used in development and testing of the AIR detector
  • Lastly, thanks to Hassan Naseri from Accenture for instructing with the thesis work and providing valuable insights throughout the whole AIR project

How to cite

@MastersThesis{pyrro2021air, 
	title={{AIR:} {Aerial} Inspection RetinaNet for Land Search and Rescue Missions}, 
	author={Pyrr{\"o}, Pasi and Naseri, Hassan and Jung, Alexander},
	school={Aalto University, School of Science},
	year={2021}
}

Licenses

References

[1] TURIC, H., DUJMIC, H., AND PAPIC, V. Two-stage segmentation of aerial images for search and rescue. Information Technology and Control 39, 2 (2010).

[2] BOŽIC-ŠTULIC, D., MARUŠIC, Ž., AND GOTOVAC, S. Deep learning approach in aerial imagery for supporting land search and rescue missions. International Journal of Computer Vision 127, 9 (2019), 1256–1278.

[3] MARUŠIC, Ž., BOŽIC-ŠTULIC, D., GOTOVAC, S., AND MARUŠIC, T. Region proposal approach for human detection on aerial imagery. In 2018 3rd International Conference on Smart and Sustainable Technologies (SpliTech) (2018), IEEE, pp. 1–6.

[4] VASIC, M. K., AND PAPIC, V. Multimodel deep learning for person detection in aerial images. Electronics 9, 9 (2020), 1459.

Comments
  • cudnn failed to initialyze

    cudnn failed to initialyze

    Hello i tried to install the docker-gpu images, i've download the weights and the data, but when i launch /bin/bash infer.sh i get this response :

    root@rbena-GF63-Thin-10SCSR:/home# /bin/bash infer.sh 
    Using TensorFlow backend.
    2022-03-01 23:46:15.554674: I tensorflow/core/platform/cpu_feature_guard.cc:142] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA
    2022-03-01 23:46:15.578708: I tensorflow/core/platform/profile_utils/cpu_utils.cc:94] CPU Frequency: 2499950000 Hz
    2022-03-01 23:46:15.579340: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x561a790 initialized for platform Host (this does not guarantee that XLA will be used). Devices:
    2022-03-01 23:46:15.579364: I tensorflow/compiler/xla/service/service.cc:176]   StreamExecutor device (0): Host, Default Version
    2022-03-01 23:46:15.580946: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcuda.so.1
    2022-03-01 23:46:15.636357: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:983] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
    2022-03-01 23:46:15.636553: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x53a4460 initialized for platform CUDA (this does not guarantee that XLA will be used). Devices:
    2022-03-01 23:46:15.636569: I tensorflow/compiler/xla/service/service.cc:176]   StreamExecutor device (0): NVIDIA GeForce GTX 1650 Ti with Max-Q Design, Compute Capability 7.5
    2022-03-01 23:46:15.636762: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:983] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
    2022-03-01 23:46:15.636888: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1618] Found device 0 with properties: 
    name: NVIDIA GeForce GTX 1650 Ti with Max-Q Design major: 7 minor: 5 memoryClockRate(GHz): 1.2
    pciBusID: 0000:01:00.0
    2022-03-01 23:46:15.637043: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.0
    2022-03-01 23:46:15.637856: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10.0
    2022-03-01 23:46:15.638563: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10.0
    2022-03-01 23:46:15.638766: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10.0
    2022-03-01 23:46:15.639774: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10.0
    2022-03-01 23:46:15.640478: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10.0
    2022-03-01 23:46:15.642728: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7
    2022-03-01 23:46:15.642815: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:983] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
    2022-03-01 23:46:15.642967: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:983] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
    2022-03-01 23:46:15.643040: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1746] Adding visible gpu devices: 0
    2022-03-01 23:46:15.643072: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.0
    2022-03-01 23:46:15.643622: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1159] Device interconnect StreamExecutor with strength 1 edge matrix:
    2022-03-01 23:46:15.643634: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1165]      0 
    2022-03-01 23:46:15.643639: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1178] 0:   N 
    2022-03-01 23:46:15.643714: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:983] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
    2022-03-01 23:46:15.643833: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:983] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
    2022-03-01 23:46:15.643934: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1304] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 2857 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce GTX 1650 Ti with Max-Q Design, pci bus id: 0000:01:00.0, compute capability: 7.5)
    Loading model, this may take a second...
    tracking <tf.Variable 'Variable:0' shape=(15, 4) dtype=float32> anchors
    tracking <tf.Variable 'Variable_1:0' shape=(15, 4) dtype=float32> anchors
    tracking <tf.Variable 'Variable_2:0' shape=(15, 4) dtype=float32> anchors
    tracking <tf.Variable 'Variable_3:0' shape=(15, 4) dtype=float32> anchors
    tracking <tf.Variable 'Variable_4:0' shape=(15, 4) dtype=float32> anchors
    Running inference on image folder: /home/data/images/test
    Running network: N/A% (0 of 4) |                                                                                                                                     | Elapsed Time: 0:00:00 ETA:  --:--:--2022-03-01 23:46:40.231228: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7
    2022-03-01 23:46:40.548887: E tensorflow/stream_executor/cuda/cuda_dnn.cc:329] Could not create cudnn handle: CUDNN_STATUS_INTERNAL_ERROR
    2022-03-01 23:46:40.554431: E tensorflow/stream_executor/cuda/cuda_dnn.cc:329] Could not create cudnn handle: CUDNN_STATUS_INTERNAL_ERROR
    Traceback (most recent call last):
      File "keras_retinanet/keras_retinanet/bin/infer.py", line 215, in <module>
        main()
      File "keras_retinanet/keras_retinanet/bin/infer.py", line 192, in main
        profile=args.profile
      File "keras_retinanet/keras_retinanet/bin/../../keras_retinanet/utils/eval.py", line 250, in get_detections
        max_inflation_factor=max_inflation_factor
      File "keras_retinanet/keras_retinanet/bin/../../keras_retinanet/utils/eval.py", line 134, in run_inference_on_image
        boxes, scores, labels = model.predict_on_batch(np.expand_dims(image, axis=0))[:3]
      File "/usr/local/lib/python3.6/dist-packages/keras/engine/training.py", line 1580, in predict_on_batch
        outputs = self.predict_function(ins)
      File "/usr/local/lib/python3.6/dist-packages/tensorflow_core/python/keras/backend.py", line 3476, in __call__
        run_metadata=self.run_metadata)
      File "/usr/local/lib/python3.6/dist-packages/tensorflow_core/python/client/session.py", line 1472, in __call__
        run_metadata_ptr)
    tensorflow.python.framework.errors_impl.UnknownError: 2 root error(s) found.
      (0) Unknown: Failed to get convolution algorithm. This is probably because cuDNN failed to initialize, so try looking to see if a warning log message was printed above.
    	 [[{{node conv1/convolution}}]]
    	 [[filtered_detections/map/while/LoopCond/_9551]]
      (1) Unknown: Failed to get convolution algorithm. This is probably because cuDNN failed to initialize, so try looking to see if a warning log message was printed above.
    	 [[{{node conv1/convolution}}]]
    0 successful operations.
    0 derived errors ignored.
    
    

    inside the container I checked under the cuda repository and noticed that the cudnn librairies where not inside , so i downloaded it and copy it inside cuda/include and cuda/lib64 but i've got the same result. I am on ubuntu 18.04 i have a GeForce GTX 1650 , nvidia-smi return cuda 11.6 nvida-driver 510 Can you help me on that ? Thank you

    help wanted 
    opened by mrba59 7
  • Box detect group of person

    Box detect group of person

    Hi, I've runned inference on my eval dataset, the results are really good and i congrat you on that it's really impressive. I just have one question. The inference detect 15 person on this image , as you can see some box contains more that one person but they are detected as one, maybe there is a parameter taking account group of person ? Is there any chance to return one box for each person ?

    frame22401-idx-7

    opened by mrba59 3
  • Dataset unavailable

    Dataset unavailable

    Hi, nice work! Unfortunately i can't reach out the HERIDAL download page. If i try from here: http://ipsar.fesb.unist.hr/HERIDAL%20database.html It prompts an error.

    Is there any chance you have an offline dump you can share with me somehow?

    Thanks

    opened by lodm94 1
  • ModuleNotFoundError: No module named 'keras.api'

    ModuleNotFoundError: No module named 'keras.api'

    When I ran !/bin/bash convert-model.sh in Google Colab, it resulted in:

    2022-03-22 09:32:42.586979: E tensorflow/stream_executor/cuda/cuda_driver.cc:271] failed call to cuInit: CUDA_ERROR_NO_DEVICE: no CUDA-capable device is detected
    Traceback (most recent call last):
      File "keras_retinanet/keras_retinanet/bin/convert_model.py", line 103, in <module>
        main()
      File "keras_retinanet/keras_retinanet/bin/convert_model.py", line 79, in main
        model = models.load_model(args.model_in, backbone_name=args.backbone)
      File "keras_retinanet/keras_retinanet/bin/../../keras_retinanet/models/__init__.py", line 87, in load_model
        return keras.models.load_model(filepath, custom_objects=backbone(backbone_name).custom_objects, compile=compile)
      File "keras_retinanet/keras_retinanet/bin/../../keras_retinanet/models/__init__.py", line 57, in backbone
        from .resnet import ResNetBackbone as b
      File "keras_retinanet/keras_retinanet/bin/../../keras_retinanet/models/resnet.py", line 18, in <module>
        from keras.utils import get_file
    ImportError: cannot import name 'get_file' from 'keras.utils' (/usr/local/lib/python3.7/dist-packages/keras/utils/__init__.py)
    

    After some googling I found out that I should install kashgari, so I ran

    pip install kashgari

    And when I came back to !/bin/bash convert-model.sh, I had:

    Using TensorFlow backend.
    Traceback (most recent call last):
      File "keras_retinanet/keras_retinanet/bin/convert_model.py", line 33, in <module>
        from ..utils.config import read_config_file, parse_anchor_parameters
      File "keras_retinanet/keras_retinanet/bin/../../keras_retinanet/utils/config.py", line 19, in <module>
        import keras
      File "/usr/local/lib/python3.7/dist-packages/keras/__init__.py", line 3, in <module>
        from . import utils
      File "/usr/local/lib/python3.7/dist-packages/keras/utils/__init__.py", line 26, in <module>
        from .vis_utils import model_to_dot
      File "/usr/local/lib/python3.7/dist-packages/keras/utils/vis_utils.py", line 7, in <module>
        from ..models import Model
      File "/usr/local/lib/python3.7/dist-packages/keras/models.py", line 12, in <module>
        from .engine.training import Model
      File "/usr/local/lib/python3.7/dist-packages/keras/engine/__init__.py", line 8, in <module>
        from .training import Model
      File "/usr/local/lib/python3.7/dist-packages/keras/engine/training.py", line 14, in <module>
        from . import training_utils
      File "/usr/local/lib/python3.7/dist-packages/keras/engine/training_utils.py", line 17, in <module>
        from .. import metrics as metrics_module
      File "/usr/local/lib/python3.7/dist-packages/keras/metrics.py", line 1850, in <module>
        BaseMeanIoU = tf.keras.metrics.MeanIoU
      File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/util/lazy_loader.py", line 58, in __getattr__
        module = self._load()
      File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/util/lazy_loader.py", line 41, in _load
        module = importlib.import_module(self.__name__)
      File "/usr/lib/python3.7/importlib/__init__.py", line 127, in import_module
        return _bootstrap._gcd_import(name[level:], package, level)
    ModuleNotFoundError: No module named 'keras.api'
    

    Am I doing something wrong? Is there a way to address this error?

    help wanted 
    opened by dinarakhay 1
  • Bump opencv-python from 4.1.0.25 to 4.2.0.32

    Bump opencv-python from 4.1.0.25 to 4.2.0.32

    ⚠️ Dependabot is rebasing this PR ⚠️

    Rebasing might not happen immediately, so don't worry if this takes some time.

    Note: if you make any changes to this PR yourself, they will take precedence over the rebase.


    Bumps opencv-python from 4.1.0.25 to 4.2.0.32.

    Release notes

    Sourced from opencv-python's releases.

    4.2.0.32

    OpenCV version 4.2.0.

    Changes:

    • macOS environment updated from xcode8.3 to xcode 9.4
    • macOS uses now Qt 5 instead of Qt 4
    • Nasm version updated to Docker containers
    • multibuild updated

    Fixes:

    • don't use deprecated brew tap-pin, instead refer to the full package name when installing #267
    • replace get_config_var() with get_config_vars() in setup.py #274
    • add workaround for DLL errors in Windows Server #264

    4.1.2.30

    OpenCV version 4.1.2.

    Changes:

    • Python 3.8 builds added to the build matrix
    • Support for Python 3.4 builds dropped (Python 3.4 is in EOL)
    • multibuild updated
    • minor build logic changes
    • Docker images rebuilt

    Notes:

    Please note that Python 2.7 enters into EOL phase in January 2020. opencv-python Python 2.7 wheels won't be provided after that.

    4.1.1.26

    OpenCV version 4.1.1.

    Changes:

    ... (truncated)

    Commits

    Dependabot compatibility score

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
    • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    • @dependabot use these labels will set the current labels as the default for future PRs for this repo and language
    • @dependabot use these reviewers will set the current reviewers as the default for future PRs for this repo and language
    • @dependabot use these assignees will set the current assignees as the default for future PRs for this repo and language
    • @dependabot use this milestone will set the current milestone as the default for future PRs for this repo and language

    You can disable automated security fix PRs for this repo from the Security Alerts page.

    dependencies 
    opened by dependabot[bot] 1
  • Bump pillow from 6.2.1 to 8.3.2

    Bump pillow from 6.2.1 to 8.3.2

    ⚠️ Dependabot is rebasing this PR ⚠️

    Rebasing might not happen immediately, so don't worry if this takes some time.

    Note: if you make any changes to this PR yourself, they will take precedence over the rebase.


    Bumps pillow from 6.2.1 to 8.3.2.

    Release notes

    Sourced from pillow's releases.

    8.3.2

    https://pillow.readthedocs.io/en/stable/releasenotes/8.3.2.html

    Security

    • CVE-2021-23437 Raise ValueError if color specifier is too long [hugovk, radarhere]

    • Fix 6-byte OOB read in FliDecode [wiredfool]

    Python 3.10 wheels

    • Add support for Python 3.10 #5569, #5570 [hugovk, radarhere]

    Fixed regressions

    • Ensure TIFF RowsPerStrip is multiple of 8 for JPEG compression #5588 [kmilos, radarhere]

    • Updates for ImagePalette channel order #5599 [radarhere]

    • Hide FriBiDi shim symbols to avoid conflict with real FriBiDi library #5651 [nulano]

    8.3.1

    https://pillow.readthedocs.io/en/stable/releasenotes/8.3.1.html

    Changes

    8.3.0

    https://pillow.readthedocs.io/en/stable/releasenotes/8.3.0.html

    Changes

    ... (truncated)

    Changelog

    Sourced from pillow's changelog.

    8.3.2 (2021-09-02)

    • CVE-2021-23437 Raise ValueError if color specifier is too long [hugovk, radarhere]

    • Fix 6-byte OOB read in FliDecode [wiredfool]

    • Add support for Python 3.10 #5569, #5570 [hugovk, radarhere]

    • Ensure TIFF RowsPerStrip is multiple of 8 for JPEG compression #5588 [kmilos, radarhere]

    • Updates for ImagePalette channel order #5599 [radarhere]

    • Hide FriBiDi shim symbols to avoid conflict with real FriBiDi library #5651 [nulano]

    8.3.1 (2021-07-06)

    • Catch OSError when checking if fp is sys.stdout #5585 [radarhere]

    • Handle removing orientation from alternate types of EXIF data #5584 [radarhere]

    • Make Image.array take optional dtype argument #5572 [t-vi, radarhere]

    8.3.0 (2021-07-01)

    • Use snprintf instead of sprintf. CVE-2021-34552 #5567 [radarhere]

    • Limit TIFF strip size when saving with LibTIFF #5514 [kmilos]

    • Allow ICNS save on all operating systems #4526 [baletu, radarhere, newpanjing, hugovk]

    • De-zigzag JPEG's DQT when loading; deprecate convert_dict_qtables #4989 [gofr, radarhere]

    • Replaced xml.etree.ElementTree #5565 [radarhere]

    ... (truncated)

    Commits
    • 8013f13 8.3.2 version bump
    • 23c7ca8 Update CHANGES.rst
    • 8450366 Update release notes
    • a0afe89 Update test case
    • 9e08eb8 Raise ValueError if color specifier is too long
    • bd5cf7d FLI tests for Oss-fuzz crash.
    • 94a0cf1 Fix 6-byte OOB read in FliDecode
    • cece64f Add 8.3.2 (2021-09-02) [CI skip]
    • e422386 Add release notes for Pillow 8.3.2
    • 08dcbb8 Pillow 8.3.2 supports Python 3.10 [ci skip]
    • Additional commits viewable in compare view

    Dependabot compatibility score

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
    • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    • @dependabot use these labels will set the current labels as the default for future PRs for this repo and language
    • @dependabot use these reviewers will set the current reviewers as the default for future PRs for this repo and language
    • @dependabot use these assignees will set the current assignees as the default for future PRs for this repo and language
    • @dependabot use this milestone will set the current milestone as the default for future PRs for this repo and language

    You can disable automated security fix PRs for this repo from the Security Alerts page.

    dependencies 
    opened by dependabot[bot] 1
  • Bump opencv-contrib-python from 4.1.0.25 to 4.2.0.32

    Bump opencv-contrib-python from 4.1.0.25 to 4.2.0.32

    ⚠️ Dependabot is rebasing this PR ⚠️

    Rebasing might not happen immediately, so don't worry if this takes some time.

    Note: if you make any changes to this PR yourself, they will take precedence over the rebase.


    Bumps opencv-contrib-python from 4.1.0.25 to 4.2.0.32.

    Release notes

    Sourced from opencv-contrib-python's releases.

    4.2.0.32

    OpenCV version 4.2.0.

    Changes:

    • macOS environment updated from xcode8.3 to xcode 9.4
    • macOS uses now Qt 5 instead of Qt 4
    • Nasm version updated to Docker containers
    • multibuild updated

    Fixes:

    • don't use deprecated brew tap-pin, instead refer to the full package name when installing #267
    • replace get_config_var() with get_config_vars() in setup.py #274
    • add workaround for DLL errors in Windows Server #264

    4.1.2.30

    OpenCV version 4.1.2.

    Changes:

    • Python 3.8 builds added to the build matrix
    • Support for Python 3.4 builds dropped (Python 3.4 is in EOL)
    • multibuild updated
    • minor build logic changes
    • Docker images rebuilt

    Notes:

    Please note that Python 2.7 enters into EOL phase in January 2020. opencv-python Python 2.7 wheels won't be provided after that.

    4.1.1.26

    OpenCV version 4.1.1.

    Changes:

    ... (truncated)

    Commits

    Dependabot compatibility score

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
    • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    • @dependabot use these labels will set the current labels as the default for future PRs for this repo and language
    • @dependabot use these reviewers will set the current reviewers as the default for future PRs for this repo and language
    • @dependabot use these assignees will set the current assignees as the default for future PRs for this repo and language
    • @dependabot use this milestone will set the current milestone as the default for future PRs for this repo and language

    You can disable automated security fix PRs for this repo from the Security Alerts page.

    dependencies 
    opened by dependabot[bot] 1
  • Unable to get working with GPU: OOM and/or hangs

    Unable to get working with GPU: OOM and/or hangs

    First of all, thanks for putting this together! I'm trying to integrate it into my volunteer search & rescue team's RPAS/drone workflow, and we're all very excited about the initial results.

    While the results under CPU mode are great, they are a bit slow for our purposes (using 4000x2250px images). Our team computer has a 3060 GPU (Windows 11, i7 CPU, 32GB RAM), which we hope can speed up processing, but we're unable to get it working in GPU mode.

    When running the default GPU Docker install, we get OOM errors:

    root@docker-desktop:/home# /bin/bash infer.sh
    Using TensorFlow backend.
    2022-06-16 02:32:07.062753: I tensorflow/core/platform/cpu_feature_guard.cc:142] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA
    2022-06-16 02:32:07.068259: I tensorflow/core/platform/profile_utils/cpu_utils.cc:94] CPU Frequency: 2687995000 Hz
    2022-06-16 02:32:07.069773: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x57d5a40 initialized for platform Host (this does not guarantee that XLA will be used). Devices:
    2022-06-16 02:32:07.069800: I tensorflow/compiler/xla/service/service.cc:176]   StreamExecutor device (0): Host, Default Version
    2022-06-16 02:32:07.072249: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcuda.so.1
    2022-06-16 02:32:08.733246: E tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:969] could not open file to read NUMA node: /sys/bus/pci/devices/0000:01:00.0/numa_node
    Your kernel may have been built without NUMA support.
    2022-06-16 02:32:08.733462: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x570f500 initialized for platform CUDA (this does not guarantee that XLA will be used). Devices:
    2022-06-16 02:32:08.733513: I tensorflow/compiler/xla/service/service.cc:176]   StreamExecutor device (0): NVIDIA GeForce RTX 3060 Laptop GPU, Compute Capability 8.6
    2022-06-16 02:32:08.738143: E tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:969] could not open file to read NUMA node: /sys/bus/pci/devices/0000:01:00.0/numa_node
    Your kernel may have been built without NUMA support.
    2022-06-16 02:32:08.738184: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1618] Found device 0 with properties:
    name: NVIDIA GeForce RTX 3060 Laptop GPU major: 8 minor: 6 memoryClockRate(GHz): 1.702
    pciBusID: 0000:01:00.0
    2022-06-16 02:32:08.738376: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.0
    2022-06-16 02:32:08.739257: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10.0
    2022-06-16 02:32:08.740589: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10.0
    2022-06-16 02:32:08.740821: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10.0
    2022-06-16 02:32:08.742371: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10.0
    2022-06-16 02:32:08.743383: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10.0
    2022-06-16 02:32:08.745999: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7
    2022-06-16 02:32:08.746592: E tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:969] could not open file to read NUMA node: /sys/bus/pci/devices/0000:01:00.0/numa_node
    Your kernel may have been built without NUMA support.
    2022-06-16 02:32:08.747027: E tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:969] could not open file to read NUMA node: /sys/bus/pci/devices/0000:01:00.0/numa_node
    Your kernel may have been built without NUMA support.
    2022-06-16 02:32:08.747050: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1746] Adding visible gpu devices: 0
    2022-06-16 02:32:08.747094: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.0
    2022-06-16 02:32:08.747495: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1159] Device interconnect StreamExecutor with strength 1 edge matrix:
    2022-06-16 02:32:08.747514: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1165]      0
    2022-06-16 02:32:08.747532: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1178] 0:   N
    2022-06-16 02:32:08.748081: E tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:969] could not open file to read NUMA node: /sys/bus/pci/devices/0000:01:00.0/numa_node
    Your kernel may have been built without NUMA support.
    2022-06-16 02:32:08.748109: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1387] Could not identify NUMA node of platform GPU id 0, defaulting to 0.  Your kernel may not have been built with NUMA support.
    2022-06-16 02:32:08.748516: E tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:969] could not open file to read NUMA node: /sys/bus/pci/devices/0000:01:00.0/numa_node
    Your kernel may have been built without NUMA support.
    2022-06-16 02:32:08.748577: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1304] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 4857 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 3060 Laptop GPU, pci bus id: 0000:01:00.0, compute capability: 8.6)
    Loading model, this may take a second...
    2022-06-16 02:36:14.483391: E tensorflow/stream_executor/cuda/cuda_driver.cc:893] failed to alloc 1073741824 bytes on host: CUDA_ERROR_OUT_OF_MEMORY: out of memory
    2022-06-16 02:36:14.483439: W ./tensorflow/core/common_runtime/gpu/gpu_host_allocator.h:44] could not allocate pinned host memory of size: 1073741824
    2022-06-16 02:36:14.576914: E tensorflow/stream_executor/cuda/cuda_driver.cc:893] failed to alloc 966367744 bytes on host: CUDA_ERROR_OUT_OF_MEMORY: out of memory
    2022-06-16 02:36:14.576962: W ./tensorflow/core/common_runtime/gpu/gpu_host_allocator.h:44] could not allocate pinned host memory of size: 966367744
    2022-06-16 02:36:14.671003: E tensorflow/stream_executor/cuda/cuda_driver.cc:893] failed to alloc 869731072 bytes on host: CUDA_ERROR_OUT_OF_MEMORY: out of memory
    2022-06-16 02:36:14.671047: W ./tensorflow/core/common_runtime/gpu/gpu_host_allocator.h:44] could not allocate pinned host memory of size: 869731072
    2022-06-16 02:36:14.770382: E tensorflow/stream_executor/cuda/cuda_driver.cc:893] failed to alloc 782758144 bytes on host: CUDA_ERROR_OUT_OF_MEMORY: out of memory
    2022-06-16 02:36:14.770424: W ./tensorflow/core/common_runtime/gpu/gpu_host_allocator.h:44] could not allocate pinned host memory of size: 782758144
    2022-06-16 02:36:14.864264: E tensorflow/stream_executor/cuda/cuda_driver.cc:893] failed to alloc 704482304 bytes on host: CUDA_ERROR_OUT_OF_MEMORY: out of memory
    2022-06-16 02:36:14.864312: W ./tensorflow/core/common_runtime/gpu/gpu_host_allocator.h:44] could not allocate pinned host memory of size: 704482304
    2022-06-16 02:36:14.957833: E tensorflow/stream_executor/cuda/cuda_driver.cc:893] failed to alloc 634034176 bytes on host: CUDA_ERROR_OUT_OF_MEMORY: out of memory
    2022-06-16 02:36:14.957889: W ./tensorflow/core/common_runtime/gpu/gpu_host_allocator.h:44] could not allocate pinned host memory of size: 634034176
    2022-06-16 02:36:15.053452: E tensorflow/stream_executor/cuda/cuda_driver.cc:893] failed to alloc 570630912 bytes on host: CUDA_ERROR_OUT_OF_MEMORY: out of memory
    2022-06-16 02:36:15.053498: W ./tensorflow/core/common_runtime/gpu/gpu_host_allocator.h:44] could not allocate pinned host memory of size: 570630912
    2022-06-16 02:36:15.150759: E tensorflow/stream_executor/cuda/cuda_driver.cc:893] failed to alloc 513568000 bytes on host: CUDA_ERROR_OUT_OF_MEMORY: out of memory
    2022-06-16 02:36:15.150802: W ./tensorflow/core/common_runtime/gpu/gpu_host_allocator.h:44] could not allocate pinned host memory of size: 513568000
    2022-06-16 02:36:15.243685: E tensorflow/stream_executor/cuda/cuda_driver.cc:893] failed to alloc 462211328 bytes on host: CUDA_ERROR_OUT_OF_MEMORY: out of memory
    2022-06-16 02:36:15.243726: W ./tensorflow/core/common_runtime/gpu/gpu_host_allocator.h:44] could not allocate pinned host memory of size: 462211328
    2022-06-16 02:36:15.335976: E tensorflow/stream_executor/cuda/cuda_driver.cc:893] failed to alloc 415990272 bytes on host: CUDA_ERROR_OUT_OF_MEMORY: out of memory
    2022-06-16 02:36:15.336028: W ./tensorflow/core/common_runtime/gpu/gpu_host_allocator.h:44] could not allocate pinned host memory of size: 415990272
    2022-06-16 02:36:15.427373: E tensorflow/stream_executor/cuda/cuda_driver.cc:893] failed to alloc 374391296 bytes on host: CUDA_ERROR_OUT_OF_MEMORY: out of memory
    2022-06-16 02:36:15.427417: W ./tensorflow/core/common_runtime/gpu/gpu_host_allocator.h:44] could not allocate pinned host memory of size: 374391296
    2022-06-16 02:36:15.520526: E tensorflow/stream_executor/cuda/cuda_driver.cc:893] failed to alloc 336952320 bytes on host: CUDA_ERROR_OUT_OF_MEMORY: out of memory
    2022-06-16 02:36:15.520570: W ./tensorflow/core/common_runtime/gpu/gpu_host_allocator.h:44] could not allocate pinned host memory of size: 336952320
    2022-06-16 02:36:15.612661: E tensorflow/stream_executor/cuda/cuda_driver.cc:893] failed to alloc 303257088 bytes on host: CUDA_ERROR_OUT_OF_MEMORY: out of memory
    2022-06-16 02:36:15.612720: W ./tensorflow/core/common_runtime/gpu/gpu_host_allocator.h:44] could not allocate pinned host memory of size: 303257088
    2022-06-16 02:36:15.704694: E tensorflow/stream_executor/cuda/cuda_driver.cc:893] failed to alloc 272931584 bytes on host: CUDA_ERROR_OUT_OF_MEMORY: out of memory
    2022-06-16 02:36:15.704741: W ./tensorflow/core/common_runtime/gpu/gpu_host_allocator.h:44] could not allocate pinned host memory of size: 272931584
    2022-06-16 02:36:15.798110: E tensorflow/stream_executor/cuda/cuda_driver.cc:893] failed to alloc 245638656 bytes on host: CUDA_ERROR_OUT_OF_MEMORY: out of memory
    2022-06-16 02:36:15.798153: W ./tensorflow/core/common_runtime/gpu/gpu_host_allocator.h:44] could not allocate pinned host memory of size: 245638656
    2022-06-16 02:36:15.889580: E tensorflow/stream_executor/cuda/cuda_driver.cc:893] failed to alloc 221074944 bytes on host: CUDA_ERROR_OUT_OF_MEMORY: out of memory
    2022-06-16 02:36:15.889628: W ./tensorflow/core/common_runtime/gpu/gpu_host_allocator.h:44] could not allocate pinned host memory of size: 221074944
    2022-06-16 02:36:15.984319: E tensorflow/stream_executor/cuda/cuda_driver.cc:893] failed to alloc 198967552 bytes on host: CUDA_ERROR_OUT_OF_MEMORY: out of memory
    2022-06-16 02:36:15.984362: W ./tensorflow/core/common_runtime/gpu/gpu_host_allocator.h:44] could not allocate pinned host memory of size: 198967552
    2022-06-16 02:36:16.077288: E tensorflow/stream_executor/cuda/cuda_driver.cc:893] failed to alloc 179070976 bytes on host: CUDA_ERROR_OUT_OF_MEMORY: out of memory
    2022-06-16 02:36:16.077332: W ./tensorflow/core/common_runtime/gpu/gpu_host_allocator.h:44] could not allocate pinned host memory of size: 179070976
    2022-06-16 02:36:16.169940: E tensorflow/stream_executor/cuda/cuda_driver.cc:893] failed to alloc 161164032 bytes on host: CUDA_ERROR_OUT_OF_MEMORY: out of memory
    2022-06-16 02:36:16.169981: W ./tensorflow/core/common_runtime/gpu/gpu_host_allocator.h:44] could not allocate pinned host memory of size: 161164032
    2022-06-16 02:36:16.264281: E tensorflow/stream_executor/cuda/cuda_driver.cc:893] failed to alloc 145047808 bytes on host: CUDA_ERROR_OUT_OF_MEMORY: out of memory
    2022-06-16 02:36:16.264328: W ./tensorflow/core/common_runtime/gpu/gpu_host_allocator.h:44] could not allocate pinned host memory of size: 145047808
    tracking <tf.Variable 'Variable:0' shape=(15, 4) dtype=float32> anchors
    tracking <tf.Variable 'Variable_1:0' shape=(15, 4) dtype=float32> anchors
    tracking <tf.Variable 'Variable_2:0' shape=(15, 4) dtype=float32> anchors
    tracking <tf.Variable 'Variable_3:0' shape=(15, 4) dtype=float32> anchors
    tracking <tf.Variable 'Variable_4:0' shape=(15, 4) dtype=float32> anchors
    Running inference on image folder: /home/data/images/test
    Running network: N/A% (0 of 138) |                                          | Elapsed Time: 0:00:00 ETA:  --:--:--2022-06-16 02:36:21.094051: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7
    2022-06-16 02:48:27.323880: W tensorflow/stream_executor/cuda/redzone_allocator.cc:312] Internal: ptxas exited with non-zero error code 65280, output: ptxas fatal   : Value 'sm_86' is not defined for option 'gpu-name'
    
    Relying on driver to perform ptx compilation. This message will be only logged once.
    2022-06-16 02:48:27.773454: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10.0
    2022-06-16 02:49:29.518638: W tensorflow/core/common_runtime/bfc_allocator.cc:305] Garbage collection: deallocate free memory regions (i.e., allocations) so that we can re-allocate a larger region to avoid OOM due to memory fragmentation. If you see this message frequently, you are running near the threshold of the available device memory and re-allocation may incur great performance overhead. You may try smaller batch sizes to observe the performance impact. Set TF_ENABLE_GPU_GARBAGE_COLLECTION=false if you'd like to disable this feature.
    Traceback (most recent call last):
      File "keras_retinanet/keras_retinanet/bin/infer.py", line 215, in <module>
        main()
      File "keras_retinanet/keras_retinanet/bin/infer.py", line 192, in main
        profile=args.profile
      File "keras_retinanet/keras_retinanet/bin/../../keras_retinanet/utils/eval.py", line 250, in get_detections
        max_inflation_factor=max_inflation_factor
      File "keras_retinanet/keras_retinanet/bin/../../keras_retinanet/utils/eval.py", line 168, in run_inference_on_image
        max_inflation_factor=max_inflation_factor)
      File "keras_retinanet/keras_retinanet/bin/../../keras_retinanet/utils/../../../airutils/mob.py", line 365, in merge_boxes_per_label
        iou_threshold, max_iterations, top_k, max_inflation_factor, merge_mode)
      File "keras_retinanet/keras_retinanet/bin/../../keras_retinanet/utils/../../../airutils/mob.py", line 276, in merge_overlapping_boxes
        X = 1. - compute_overlap(boxes, boxes)
      File "compute_overlap.pyx", line 27, in compute_overlap.compute_overlap
        cdef np.ndarray[double, ndim=2] overlaps = np.zeros((N, K), dtype=np.float64)
    MemoryError: Unable to allocate array with shape (100000, 100000) and data type float64
    .....
    

    I've tried some extreme configuration options to lessen the memory burden, but we still get the OOM errors:

        --image_min_side 100 
        --image_max_side 150 
        --image_tiling_dim 8 
    

    I found that by setting compile=False in the below line of keras_retinanet\keras_retinanet\bin\infer.py, I avoid getting the Cuda OOM errors, but the process just hangs and eventually errors out the same. model = models.load_model(args.model, backbone_name=args.backbone, compile=False)

    Complete output with compile=False:

    root@docker-desktop:/home# /bin/bash infer.sh
    Using TensorFlow backend.
    2022-06-16 02:09:13.780103: I tensorflow/core/platform/cpu_feature_guard.cc:142] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA
    2022-06-16 02:09:13.786143: I tensorflow/core/platform/profile_utils/cpu_utils.cc:94] CPU Frequency: 2687995000 Hz
    2022-06-16 02:09:13.787804: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x5b70d30 initialized for platform Host (this does not guarantee that XLA will be used). Devices:
    2022-06-16 02:09:13.787838: I tensorflow/compiler/xla/service/service.cc:176]   StreamExecutor device (0): Host, Default Version
    2022-06-16 02:09:13.790658: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcuda.so.1
    2022-06-16 02:09:15.386150: E tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:969] could not open file to read NUMA node: /sys/bus/pci/devices/0000:01:00.0/numa_node
    Your kernel may have been built without NUMA support.
    2022-06-16 02:09:15.386309: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x58fbfd0 initialized for platform CUDA (this does not guarantee that XLA will be used). Devices:
    2022-06-16 02:09:15.386346: I tensorflow/compiler/xla/service/service.cc:176]   StreamExecutor device (0): NVIDIA GeForce RTX 3060 Laptop GPU, Compute Capability 8.6
    2022-06-16 02:09:15.389332: E tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:969] could not open file to read NUMA node: /sys/bus/pci/devices/0000:01:00.0/numa_node
    Your kernel may have been built without NUMA support.
    2022-06-16 02:09:15.389373: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1618] Found device 0 with properties:
    name: NVIDIA GeForce RTX 3060 Laptop GPU major: 8 minor: 6 memoryClockRate(GHz): 1.702
    pciBusID: 0000:01:00.0
    2022-06-16 02:09:15.389623: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.0
    2022-06-16 02:09:15.390896: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10.0
    2022-06-16 02:09:15.392376: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10.0
    2022-06-16 02:09:15.392657: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10.0
    2022-06-16 02:09:15.395000: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10.0
    2022-06-16 02:09:15.396610: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10.0
    2022-06-16 02:09:15.399907: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7
    2022-06-16 02:09:15.400535: E tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:969] could not open file to read NUMA node: /sys/bus/pci/devices/0000:01:00.0/numa_node
    Your kernel may have been built without NUMA support.
    2022-06-16 02:09:15.401007: E tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:969] could not open file to read NUMA node: /sys/bus/pci/devices/0000:01:00.0/numa_node
    Your kernel may have been built without NUMA support.
    2022-06-16 02:09:15.401035: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1746] Adding visible gpu devices: 0
    2022-06-16 02:09:15.401082: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.0
    2022-06-16 02:09:15.401523: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1159] Device interconnect StreamExecutor with strength 1 edge matrix:
    2022-06-16 02:09:15.401542: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1165]      0
    2022-06-16 02:09:15.401581: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1178] 0:   N
    2022-06-16 02:09:15.402103: E tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:969] could not open file to read NUMA node: /sys/bus/pci/devices/0000:01:00.0/numa_node
    Your kernel may have been built without NUMA support.
    2022-06-16 02:09:15.402132: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1387] Could not identify NUMA node of platform GPU id 0, defaulting to 0.  Your kernel may not have been built with NUMA support.
    2022-06-16 02:09:15.402663: E tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:969] could not open file to read NUMA node: /sys/bus/pci/devices/0000:01:00.0/numa_node
    Your kernel may have been built without NUMA support.
    2022-06-16 02:09:15.402747: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1304] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 4857 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 3060 Laptop GPU, pci bus id: 0000:01:00.0, compute capability: 8.6)
    Loading model, this may take a second...
    tracking <tf.Variable 'Variable:0' shape=(15, 4) dtype=float32> anchors
    tracking <tf.Variable 'Variable_1:0' shape=(15, 4) dtype=float32> anchors
    tracking <tf.Variable 'Variable_2:0' shape=(15, 4) dtype=float32> anchors
    tracking <tf.Variable 'Variable_3:0' shape=(15, 4) dtype=float32> anchors
    tracking <tf.Variable 'Variable_4:0' shape=(15, 4) dtype=float32> anchors
    Running inference on image folder: /home/data/images/test
    Running network: N/A% (0 of 138) |                                               | Elapsed Time: 0:00:00 ETA:  --:--:--2022-06-16 02:13:11.539626: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7
    2022-06-16 02:25:21.996183: W tensorflow/stream_executor/cuda/redzone_allocator.cc:312] Internal: ptxas exited with non-zero error code 65280, output: ptxas fatal   : Value 'sm_86' is not defined for option 'gpu-name'
    
    Relying on driver to perform ptx compilation. This message will be only logged once.
    2022-06-16 02:25:22.378005: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10.0
    2022-06-16 02:26:24.908769: W tensorflow/core/common_runtime/bfc_allocator.cc:305] Garbage collection: deallocate free memory regions (i.e., allocations) so that we can re-allocate a larger region to avoid OOM due to memory fragmentation. If you see this message frequently, you are running near the threshold of the available device memory and re-allocation may incur great performance overhead. You may try smaller batch sizes to observe the performance impact. Set TF_ENABLE_GPU_GARBAGE_COLLECTION=false if you'd like to disable this feature.
    Traceback (most recent call last):
      File "keras_retinanet/keras_retinanet/bin/infer.py", line 216, in <module>
      File "keras_retinanet/keras_retinanet/bin/infer.py", line 193, in main
        )
      File "keras_retinanet/keras_retinanet/bin/../../keras_retinanet/utils/eval.py", line 250, in get_detections
        max_inflation_factor=max_inflation_factor
      File "keras_retinanet/keras_retinanet/bin/../../keras_retinanet/utils/eval.py", line 168, in run_inference_on_image
        max_inflation_factor=max_inflation_factor)
      File "keras_retinanet/keras_retinanet/bin/../../keras_retinanet/utils/../../../airutils/mob.py", line 365, in merge_boxes_per_label
        iou_threshold, max_iterations, top_k, max_inflation_factor, merge_mode)
      File "keras_retinanet/keras_retinanet/bin/../../keras_retinanet/utils/../../../airutils/mob.py", line 276, in merge_overlapping_boxes
        X = 1. - compute_overlap(boxes, boxes)
      File "compute_overlap.pyx", line 27, in compute_overlap.compute_overlap
        cdef np.ndarray[double, ndim=2] overlaps = np.zeros((N, K), dtype=np.float64)
    MemoryError: Unable to allocate array with shape (100000, 100000) and data type float64
    

    In case it helps:

    root@docker-desktop:/home# nvidia-smi
    Thu Jun 16 02:29:41 2022
    +-----------------------------------------------------------------------------+
    | NVIDIA-SMI 510.73.08    Driver Version: 512.96       CUDA Version: 11.6     |
    |-------------------------------+----------------------+----------------------+
    | GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
    | Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
    |                               |                      |               MIG M. |
    |===============================+======================+======================|
    |   0  NVIDIA GeForce ...  On   | 00000000:01:00.0 Off |                  N/A |
    | N/A   47C    P0    24W /  N/A |      0MiB /  6144MiB |      0%      Default |
    |                               |                      |                  N/A |
    +-------------------------------+----------------------+----------------------+
    
    +-----------------------------------------------------------------------------+
    | Processes:                                                                  |
    |  GPU   GI   CI        PID   Type   Process name                  GPU Memory |
    |        ID   ID                                                   Usage      |
    |=============================================================================|
    |  No running processes found                                                 |
    +-----------------------------------------------------------------------------+
    
    

    Any suggestions would be very much appreciated, as I think I'm fully out of ideas and we're really keen to make this work!

    opened by duncanenman 2
Releases(0.0.2)
  • 0.0.2(Jan 14, 2022)

  • 0.0.1(Nov 9, 2021)

Owner
Accenture
Accenture Github site
Accenture
Lane follower: Lane-detector (OpenCV) + Object-detector (YOLO5) + CAN-bus

Lane Follower This code is for the lane follower, including perception and control, as shown below. Environment Hardware Industrial Camera Intel-NUC(1

Siqi Fan 3 Jul 7, 2022
Convert Mission Planner (ArduCopter) Waypoint Missions to Litchi CSV Format to execute on DJI Drones

Mission Planner to Litchi Convert Mission Planner (ArduCopter) Waypoint Surveys to Litchi CSV Format to execute on DJI Drones Litchi doesn't support S

Yaros 24 Dec 9, 2022
HeartRate detector with ArduinoandPython - Use Arduino and Python create a heartrate detector.

Syllabus of Contents Syllabus of Contents Introduction Of Project Features Develop With Python code introduction Installation License Developer Contac

null 1 Jan 5, 2022
Video lie detector using xgboost - A video lie detector using OpenFace and xgboost

video_lie_detector_using_xgboost a video lie detector using OpenFace and xgboost

null 2 Jan 11, 2022
This YoloV5 based model is fit to detect people and different types of land vehicles, and displaying their density on a fitted map, according to their coordinates and detected labels.

This YoloV5 based model is fit to detect people and different types of land vehicles, and displaying their density on a fitted map, according to their

Liron Bdolah 8 May 22, 2022
A whale detector design for the Kaggle whale-detector challenge!

CNN (InceptionV1) + STFT based Whale Detection Algorithm So, this repository is my PyTorch solution for the Kaggle whale-detection challenge. The obje

Tarin Ziyaee 92 Sep 28, 2021
Imposter-detector-2022 - HackED 2022 Team 3IQ - 2022 Imposter Detector

HackED 2022 Team 3IQ - 2022 Imposter Detector By Aneeljyot Alagh, Curtis Kan, Jo

Joshua Ji 3 Aug 20, 2022
LoveDA: A Remote Sensing Land-Cover Dataset for Domain Adaptive Semantic Segmentation (NeurIPS2021 Benchmark and Dataset Track)

LoveDA: A Remote Sensing Land-Cover Dataset for Domain Adaptive Semantic Segmentation by Junjue Wang, Zhuo Zheng, Ailong Ma, Xiaoyan Lu, and Yanfei Zh

Kingdrone 174 Dec 22, 2022
The Agriculture Domain of ERPNext comes with features to record crops and land

Agriculture The Agriculture Domain of ERPNext comes with features to record crops and land, track plant, soil, water, weather analytics, and even trac

Frappe 21 Jan 2, 2023
LoveDA: A Remote Sensing Land-Cover Dataset for Domain Adaptive Semantic Segmentation

LoveDA: A Remote Sensing Land-Cover Dataset for Domain Adaptive Semantic Segmentation by Junjue Wang, Zhuo Zheng, Ailong Ma, Xiaoyan Lu, and Yanfei Zh

Payphone 8 Nov 21, 2022
Pytorch implementation of Supporting Clustering with Contrastive Learning, NAACL 2021

Supporting Clustering with Contrastive Learning SCCL (NAACL 2021) Dejiao Zhang, Feng Nan, Xiaokai Wei, Shangwen Li, Henghui Zhu, Kathleen McKeown, Ram

null 231 Jan 5, 2023
Deep Text Search is an AI-powered multilingual text search and recommendation engine with state-of-the-art transformer-based multilingual text embedding (50+ languages).

Deep Text Search - AI Based Text Search & Recommendation System Deep Text Search is an AI-powered multilingual text search and recommendation engine w

null 19 Sep 29, 2022
Realtime Face Anti Spoofing with Face Detector based on Deep Learning using Tensorflow/Keras and OpenCV

Realtime Face Anti-Spoofing Detection ?? Realtime Face Anti Spoofing Detection with Face Detector to detect real and fake faces Please star this repo

Prem Kumar 86 Aug 3, 2022
Deformable DETR is an efficient and fast-converging end-to-end object detector.

Deformable DETR: Deformable Transformers for End-to-End Object Detection.

null 2k Jan 5, 2023
LiDAR R-CNN: An Efficient and Universal 3D Object Detector

LiDAR R-CNN: An Efficient and Universal 3D Object Detector Introduction This is the official code of LiDAR R-CNN: An Efficient and Universal 3D Object

TuSimple 295 Jan 5, 2023
Playing around with FastAPI and streamlit to create a YoloV5 object detector

FastAPI-Streamlit-based-YoloV5-detector Playing around with FastAPI and streamlit to create a YoloV5 object detector It turns out that a User Interfac

null 2 Jan 20, 2022
ViDT: An Efficient and Effective Fully Transformer-based Object Detector

ViDT: An Efficient and Effective Fully Transformer-based Object Detector by Hwanjun Song1, Deqing Sun2, Sanghyuk Chun1, Varun Jampani2, Dongyoon Han1,

NAVER AI 262 Dec 27, 2022
A simple, fast, and efficient object detector without FPN

You Only Look One-level Feature (YOLOF), CVPR2021 A simple, fast, and efficient object detector without FPN. This repo provides an implementation for

null 789 Jan 9, 2023
Model search is a framework that implements AutoML algorithms for model architecture search at scale

Model search (MS) is a framework that implements AutoML algorithms for model architecture search at scale. It aims to help researchers speed up their exploration process for finding the right model architecture for their classification problems (i.e., DNNs with different types of layers).

Google 3.2k Dec 31, 2022