Deep Learning Datasets Maker is a QGIS plugin to make datasets creation easier for raster and vector data.

Last update: Dec 15, 2022

Overview

Deep Learning Dataset Maker

Deep Learning Datasets Maker is a QGIS plugin to make datasets creation easier for raster and vector data.

How to use

Download and install QGIS and clone the repo :

git clone [email protected]:deepbands/deep-learning-datasets-maker.git

Copy folder named deep-learning-datasets-maker in QGIS configuration folder and choose the plugin from plugin manager in QGIS (If not appeared restart QGIS).
- You can know this folder from QGIS Setting Menu at the top-left of QGIS UI Settings > User Profiles > Open Active Profile Folder .
- Go to python/plugins then paste the deep-learning-datasets-maker folder.
- Full path should be like : C:\Users\$USER\AppData\Roaming\QGIS\QGIS3\profiles\default\python\plugins\deep-learning-datasets-maker.
Open QGIS, load your raster and vector data then select the output paths for rasterized, images and labels then click ok.

TODO

Fix: If vector layer saved in memory not in file, rasterize can't work.
Fix: Splitiing Image Size.

Comments

Error running Quick Queries with QuickOSM recently started on Mac

I have been trying to run a query that I have previously had no problem running at all and have run into this error every time. Here are my system specs and whatnot:

And this is the dialogue from the QuickOSM Panel

2022-03-19T18:47:06     INFO    All OSM objects with the key 'natural'='water' in the canvas or layer extent are going to be downloaded.
2022-03-19T18:47:06     INFO    Query: natural_water
2022-03-19T18:47:06     INFO    Encoded URL: https://lz4.overpass-api.de/api/interpreter?data=[out:xml] [timeout:25];%0A(%0A node[%22natural%22%3D%22water%22]( 27.49988,-82.45185,27.55754,-82.35723);%0A way[%22natural%22%3D%22water%22]( 27.49988,-82.45185,27.55754,-82.35723);%0A relation[%22natural%22%3D%22water%22]( 27.49988,-82.45185,27.55754,-82.35723);%0A);%0A(._;%3E;);%0Aout body;&info=QgisQuickOSMPlugin
2022-03-19T18:47:09     INFO    Request completed
2022-03-19T18:47:09     INFO    Checking OSM file content /private/var/folders/8s/cldnh_0n2wx39hw880qwh0lh0000gp/T/request-tNoEkc.osm
2022-03-19T18:47:09     INFO    The OSM file is: /private/var/folders/8s/cldnh_0n2wx39hw880qwh0lh0000gp/T/request-tNoEkc.osm
2022-03-19T18:47:09     CRITICAL   A critical error occurred, this is the traceback:
2022-03-19T18:47:09     CRITICAL    
2022-03-19T18:47:09     CRITICAL    base_processing_panel.py
2022-03-19T18:47:09     CRITICAL    Error: Algorithm qgis:checkvalidity not found
             
2022-03-19T18:47:09     CRITICAL     File "/Users/Cecilia1/Library/Application Support/QGIS/QGIS3/profiles/default/python/plugins/QuickOSM/ui/base_processing_panel.py", line 47, in run
              self._run()
             
              File "/Users/Cecilia1/Library/Application Support/QGIS/QGIS3/profiles/default/python/plugins/QuickOSM/ui/quick_query_panel.py", line 423, in _run
              num_layers = process_quick_query(
             
              File "/Users/Cecilia1/Library/Application Support/QGIS/QGIS3/profiles/default/python/plugins/QuickOSM/core/process.py", line 318, in process_quick_query
              return process_query(
             
              File "/Users/Cecilia1/Library/Application Support/QGIS/QGIS3/profiles/default/python/plugins/QuickOSM/core/process.py", line 257, in process_query
              return open_file(
             
              File "/Users/Cecilia1/Library/Application Support/QGIS/QGIS3/profiles/default/python/plugins/QuickOSM/core/process.py", line 106, in open_file
              layers = osm_parser.processing_parse()
             
              File "/Users/Cecilia1/Library/Application Support/QGIS/QGIS3/profiles/default/python/plugins/QuickOSM/core/parser/osm_parser.py", line 133, in processing_parse
              validity = processing.run(
             
              File "/Applications/QGIS-LTR.app/Contents/MacOS/../Resources/python/plugins/processing/tools/general.py", line 108, in run
              return Processing.runAlgorithm(algOrName, parameters, onFinish, feedback, context)
             
              File "/Applications/QGIS-LTR.app/Contents/MacOS/../Resources/python/plugins/processing/core/Processing.py", line 171, in runAlgorithm
              raise QgsProcessingException(msg)

Finally: The Processing panel shows this dialogue each time I try a query.

Error: Algorithm qgis:checkvalidity not found

I have made sure my language is american english and that the startup style of the app is Macintosh. If anyone could please please help me I would be so appreciative. This function is so important to the research I am doing for my senior thesis. Thank you so much for taking time to read this.

opened by cecilia-hampton 1

Init create COCO

@Youssef-Harby The function is implemented, please check and use it. If you can, it can be merged into develop. But at present, the cutting image is repeated and can be modified 😃

opened by geoyee 1
Error running in QGIS

Using on a tiff image with 4 bands RGB and alpha QGIS 3.22.4

2022-03-10T15:24:22 WARNING Traceback (most recent call last): File "C:\Users/Chris/AppData/Roaming/QGIS/QGIS3\profiles\default/python/plugins\deep-learning-datasets-maker\split_rs_data.py", line 315, in run rasterize(ras_path, vec_path, output) File "C:\Users/Chris/AppData/Roaming/QGIS/QGIS3\profiles\default/python/plugins\deep-learning-datasets-maker\utils\rasterize.py", line 11, in rasterize lyr = vec_ds.GetLayer() AttributeError: 'NoneType' object has no attribute 'GetLayer'

opened by Chris-Has 1
Couldn't load plugin 'deep-learning-datasets-maker' due to an error when calling its classFactory() method

Couldn't load plugin 'deep-learning-datasets-maker' due to an error when calling its classFactory() method

subprocess.CalledProcessError: Command '['python3', '-m', 'pip', 'install', 'Cython', 'scikit-image', 'Pillow', 'pycocotools']' returned non-zero exit status 1. Traceback (most recent call last): File "C:/Users/pprt/AppData/Roaming/QGIS/QGIS3\profiles\default/python/plugins\deep-learning-datasets-maker\utils\COCO\pycococreatortools\pycococreatortools.py", line 10, in from pycocotools import mask File "C:/PROGRA~1/QGIS3~1.4/apps/qgis-ltr/./python\qgis\utils.py", line 685, in _import mod = _builtin_import(name, globals, locals, fromlist, level) ModuleNotFoundError: No module named 'pycocotools'

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "C:/PROGRA~1/QGIS3~1.4/apps/qgis-ltr/./python\qgis\utils.py", line 335, in startPlugin plugins[packageName] = package.classFactory(iface) File "C:/Users/pprt/AppData/Roaming/QGIS/QGIS3\profiles\default/python/plugins\deep-learning-datasets-maker_init_.py", line 35, in classFactory from .split_rs_data import SplitRSData File "C:/PROGRA~1/QGIS3~1.4/apps/qgis-ltr/./python\qgis\utils.py", line 685, in _import mod = _builtin_import(name, globals, locals, fromlist, level) File "C:/Users/pprt/AppData/Roaming/QGIS/QGIS3\profiles\default/python/plugins\deep-learning-datasets-maker\split_rs_data.py", line 46, in from .utils.COCO import clip_from_file, slice, from_mask_to_coco File "C:/PROGRA~1/QGIS3~1.4/apps/qgis-ltr/./python\qgis\utils.py", line 685, in _import mod = builtin_import(name, globals, locals, fromlist, level) File "C:/Users/pprt/AppData/Roaming/QGIS/QGIS3\profiles\default/python/plugins\deep-learning-datasets-maker\utils\COCO_init.py", line 1, in from .shape_to_coco import clip_from_file, slice, from_mask_to_coco File "C:/PROGRA~1/QGIS3~1.4/apps/qgis-ltr/./python\qgis\utils.py", line 685, in _import mod = _builtin_import(name, globals, locals, fromlist, level) File "C:/Users/pprt/AppData/Roaming/QGIS/QGIS3\profiles\default/python/plugins\deep-learning-datasets-maker\utils\COCO\shape_to_coco.py", line 16, in from .pycococreatortools import * File "C:/PROGRA~1/QGIS3~1.4/apps/qgis-ltr/./python\qgis\utils.py", line 685, in _import mod = builtin_import(name, globals, locals, fromlist, level) File "C:/Users/pprt/AppData/Roaming/QGIS/QGIS3\profiles\default/python/plugins\deep-learning-datasets-maker\utils\COCO\pycococreatortools_init.py", line 1, in from .pycococreatortools import * File "C:/PROGRA~1/QGIS3~1.4/apps/qgis-ltr/./python\qgis\utils.py", line 685, in _import mod = _builtin_import(name, globals, locals, fromlist, level) File "C:/Users/pprt/AppData/Roaming/QGIS/QGIS3\profiles\default/python/plugins\deep-learning-datasets-maker\utils\COCO\pycococreatortools\pycococreatortools.py", line 16, in ["python3", '-m', 'pip', 'install', 'Cython', 'scikit-image', 'Pillow', 'pycocotools']) File "C:\PROGRA~1\QGIS3~1.4\apps\Python37\lib\subprocess.py", line 328, in check_call raise CalledProcessError(retcode, cmd) subprocess.CalledProcessError: Command '['python3', '-m', 'pip', 'install', 'Cython', 'scikit-image', 'Pillow', 'pycocotools']' returned non-zero exit status 1.

Python version: 3.7.0 (v3.7.0:1bf9cc5093, Jun 27 2018, 04:59:51) [MSC v.1914 64 bit (AMD64)] QGIS version: 3.4.13-Madeira Madeira, 567300ccf1

Python Path: C:/PROGRA~1/QGIS3~1.4/apps/qgis-ltr/./python C:/Users/pprt/AppData/Roaming/QGIS/QGIS3\profiles\default/python C:/Users/pprt/AppData/Roaming/QGIS/QGIS3\profiles\default/python/plugins C:/PROGRA~1/QGIS3~1.4/apps/qgis-ltr/./python/plugins C:\Users\pprt\matterport\models\research C:\Users\pprt\matterport\models\research\slim C:\Program Files\QGIS 3.4\bin\python37.zip C:\PROGRA~1\QGIS3~1.4\apps\Python37\DLLs C:\PROGRA~1\QGIS3~1.4\apps\Python37\lib C:\Program Files\QGIS 3.4\bin C:\Users\pprt\AppData\Roaming\Python\Python37\site-packages C:\Users\pprt\AppData\Roaming\Python\Python37\site-packages\tf_unet-0.1.2-py3.7.egg C:\PROGRA~1\QGIS3~1.4\apps\Python37 C:\PROGRA~1\QGIS3~1.4\apps\Python37\lib\site-packages C:\PROGRA~1\QGIS3~1.4\apps\Python37\lib\site-packages\win32 C:\PROGRA~1\QGIS3~1.4\apps\Python37\lib\site-packages\win32\lib C:\PROGRA~1\QGIS3~1.4\apps\Python37\lib\site-packages\Pythonwin C:/Users/pprt/AppData/Roaming/QGIS/QGIS3\profiles\default/python
enhancement

opened by aldinorizaldy 3
An error occurred during execution of following code

An error occurred during execution of following code: pyplugin_installer.instance().installPlugin('deep-learning-datasets-maker', stable=False)

Traceback (most recent call last): File "", line 1, in File "/usr/share/qgis/python/pyplugin_installer/installer.py", line 333, in installPlugin self.processDependencies(plugin["id"]) File "/usr/share/qgis/python/pyplugin_installer/installer.py", line 682, in processDependencies dlg = QgsPluginDependenciesDialog(plugin_id, to_install, to_upgrade, not_found) File "/usr/share/qgis/python/pyplugin_installer/qgsplugindependenciesdialog.py", line 92, in init _make_row(data, i, name) File "/usr/share/qgis/python/pyplugin_installer/qgsplugindependenciesdialog.py", line 63, in _make_row widget.use_stable_version = data['use_stable_version'] KeyError: 'use_stable_version'

Python version: 3.8.10 (default, Nov 26 2021, 20:14:08) [GCC 9.3.0]

QGIS version: 3.22.3-Białowieża 'Białowieża', 1628765ec7

Python path: ['/usr/share/qgis/python', '/home/robotics/.local/share/QGIS/QGIS3/profiles/default/python', '/home/robotics/.local/share/QGIS/QGIS3/profiles/default/python/plugins', '/usr/share/qgis/python/plugins', '/usr/lib/python38.zip', '/usr/lib/python3.8', '/usr/lib/python3.8/lib-dynload', '/usr/local/lib/python3.8/dist-packages', '/usr/lib/python3/dist-packages', '/home/robotics/.local/share/QGIS/QGIS3/profiles/default/python', '/home/robotics/.local/share/QGIS/QGIS3/profiles/default/python/plugins/DeepLearningTools']
bug

opened by makamkkumar 1

Releases(0.2.1)

0.2.1(Jan 19, 2022)
Update Requirements pycocotools needs Cython to be installed on Windows

What's Changed

update new version v0.2.1 by @Youssef-Harby in https://github.com/deepbands/deep-learning-datasets-maker/pull/4

Full Changelog: https://github.com/deepbands/deep-learning-datasets-maker/compare/0.2...0.2.1
Source code(tar.gz)
Source code(zip)
deep-learning-datasets-maker.zip(59.95 KB)
0.2(Dec 25, 2021)
Support COCO export dataset format

Update UI

Fix some bugs

Replace OpenCV with GDAL/OGR

https://plugins.qgis.org/plugins/deep-learning-datasets-maker/version/0.2/
Source code(tar.gz)
Source code(zip)
deep-learning-datasets-maker.zip(63.00 KB)
Experimental(Dec 12, 2021)

https://plugins.qgis.org/plugins/deep-learning-datasets-maker/version/0.1/
Source code(tar.gz)
Source code(zip)
deep-learning-datasets-maker.zip(44.95 KB)

Deep Learning Datasets Maker is a QGIS plugin to make datasets creation easier for raster and vector data.

Related tags

Overview

Deep Learning Dataset Maker

How to use

TODO

Comments

Error running Quick Queries with QuickOSM recently started on Mac

Init create COCO

Error running in QGIS

Couldn't load plugin 'deep-learning-datasets-maker' due to an error when calling its classFactory() method

An error occurred during execution of following code

Releases(0.2.1)

0.2.1(Jan 19, 2022)

What's Changed

0.2(Dec 25, 2021)

Experimental(Dec 12, 2021)

Owner

deepbands

buildseg is a building extraction plugin of QGIS based on PaddlePaddle.

buildseg is a building extraction plugin of QGIS based on PaddlePaddle.

rastrainer is a QGIS plugin to training remote sensing semantic segmentation model based on PaddlePaddle.

Implementation of "RaScaNet: Learning Tiny Models by Raster-Scanning Image" from CVPR 2021.

Apply a perspective transformation to a raster image inside Inkscape (no need to use an external software such as GIMP or Krita).

A series of convenience functions to make basic image processing operations such as translation, rotation, resizing, skeletonization, and displaying Matplotlib images easier with OpenCV and Python.

This project aim to create multi-label classification annotation tool to boost annotation speed and make it more easier.

Python-experiments - A Repository which contains python scripts to automate things and make your life easier with python

Agile SVG maker for python

AITUS - An atomatic notr maker for CYTUS

ktrain is a Python library that makes deep learning and AI more accessible and easier to apply

code and data for paper "GIANT: Scalable Creation of a Web-scale Ontology"

Make a Turtlebot3 follow a figure 8 trajectory and create a robot arm and make it follow a trajectory

Reference implementation of code generation projects from Facebook AI Research. General toolkit to apply machine learning to code, from dataset creation to model training and evaluation. Comes with pretrained models.

Implementation of Geometric Vector Perceptron, a simple circuit for 3d rotation equivariance for learning over large biomolecules, in Pytorch. Idea proposed and accepted at ICLR 2021

Geometric Vector Perceptron --- a rotation-equivariant GNN for learning from biomolecular structure

[SIGGRAPH Asia 2021] DeepVecFont: Synthesizing High-quality Vector Fonts via Dual-modality Learning.

[SIGGRAPH Asia 2021] DeepVecFont: Synthesizing High-quality Vector Fonts via Dual-modality Learning.