An API-first distributed deployment system of deep learning models using timeseries data to analyze and predict systems behaviour

Equinor

Last update: Dec 27, 2022

Related tags

Deep Learning machine-learning scalable end-to-end distributed-computing yaml-configuration kubernetes-deployment out-of-the-box

Overview

Gordo

Building thousands of models with timeseries data to monitor systems.

Table of content

About
Examples
Install
Uninstall
Developer manual
- How to prepare working environment
- How to run tests locally

About

Gordo fulfills the role of inhaling config files and supplying components to the pipeline of:

Fetching data
Training model
Serving model

Examples

See our example notebooks for how to develop with gordo locally.

Install

pip install --upgrade gordo

Bleeding edge:
pip install git+https://github.com/equinor/gordo.git

Uninstall

pip uninstall gordo

Developer manual

This section will explain how to start development of Gordo.

How to prepare working environment

install requirements

# create and activate virtualenv. Note: you should use python3.7 (project's tensorflow version is not compatible with python3.8)
# then:
pip install --upgrade pip
pip install --upgrade pip-tools
pip install -r requirements/full_requirements.txt
pip install -r requirements/test_requirements.txt

How to run tests locally

Tests system requirements

To run tests it's required for your system to has (note: commands might differ from your OS):

running docker process;
available 5432 port for postgres container.

Run tests

List of commands to run tests can be found here.
Running of tests takes some time, so it's faster to run tests in parallel:

# example
pytest tests/gordo/client/test_client.py --ignore benchmarks --cov-report= --no-cov -n auto -m 'not dockertest' 
# or if you have multiple python versions and they're not resolved properly:
python3.7 -m pytest ...

How to run tests in debug mode

Note: this example is for Pycharm IDE to use breakpoints in the code of the tests.
On the configuration setup for test running add to Additional arguments: in pytest section following string: --ignore benchmarks --cov-report= --no-cov

Comments

Add MLflow logging to model builder
Will close #441

Problem :airplane:

The DS team would like to have additional logging of metrics for various configuration of deployed models for post-analysis.

Solution :rocket:

Add MLFlow logging to the the CLI build command and gordo_components.builder.build_model::build_model.

This includes:

[x] Add a method that authenticates to AzureML and sets MLFlow logging to a wokspace. The configuration is passed to the CLI build as a dict, via the env var MLFLOW_CONFIG. Default behavior is to require interactive authentication and log MLFlow locally.

[x] Set the workspace (with auth) at the CLI level.

[x] Log the model_id (changed from cache_id) at CLI level.

[x] Log metadata at build_model level

[x] Add test for set_mlflow_tracking

[x] Add test for build_model
opened by ryanjdillon 15
Bump scikit-learn from 0.21.3 to 0.22
Bumps scikit-learn from 0.21.3 to 0.22.

Commits

5f3c3f0 DOC Mention intel conda channel in installation doc (#14247)

f135e26 bump version to 0.22

eb0f6eb DOC Pre 0.22 release (#15735)

4246521 FIX add pos_label when computing AP in plot_precision_recall_curve (#15739)

0a56df6 bump version to 0.22rc3

dbcc93e MNT Fix some easy-to-make typos (#15720)

aebd762 DOC Typo in the error message of _binary_clf_curve (#15703)

2329915 MNT fix filtering of examples file to run by sphinx (#15680)

5fab501 MNT Include all pxd files in the package (#15626)

9318adb pep8 on test_plot_confusion_matrix.py

Additional commits viewable in compare view

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

Dependabot commands and options

You can trigger Dependabot actions by commenting on this PR:

@dependabot rebase will rebase this PR

@dependabot recreate will recreate this PR, overwriting any edits that have been made to it

@dependabot merge will merge this PR after your CI passes on it

@dependabot squash and merge will squash and merge this PR after your CI passes on it

@dependabot cancel merge will cancel a previously requested merge and block automerging

@dependabot reopen will reopen this PR if it is closed

@dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually

@dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)

@dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)

@dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)

@dependabot use these labels will set the current labels as the default for future PRs for this repo and language

@dependabot use these reviewers will set the current reviewers as the default for future PRs for this repo and language

@dependabot use these assignees will set the current assignees as the default for future PRs for this repo and language

@dependabot use this milestone will set the current milestone as the default for future PRs for this repo and language

@dependabot badge me will comment on this PR with code to add a "Dependabot enabled" badge to your readme

Additionally, you can set the following in your Dependabot dashboard:

Update frequency (including time of day and day of week)

Pull request limits (per update run and/or open at any time)

Automerge options (never/patch/minor, and dev/runtime dependencies)

Out-of-range updates (receive only lockfile updates, if desired)

Security updates (receive only security updates, if desired)

dependencies
opened by dependabot-preview[bot] 10
Added optional requests.Session injection parameter

To use gordo-client with azure authentication, we want to decorate the session with some auth headers.

By allowing the caller to inject their own session object, this can happen without gordo-client ever having to know or care about it.

opened by mrdeveloperdude 9
TimeSeriesDataset: support multiple aggregation methods

Now the TimeSeriesDataset supports an optional argument aggregation_methods with default value mean. It can be either a single aggregation method or a list of aggregation methods. Any aggregation method supported by pandas is supported. If multiple aggregation methods are provided then the resulting dataframe contains a multi-level column index with the tag-name as the top level, and the aggregation method as the second level. If a single aggregation method is provided then only the first level (tag-name) is used.

This closes #278

opened by epa095 8
Handling inf & -inf values

How to handle inf & -inf values in data ? +-inf values are not treated as null by default in pandas, https://github.com/pandas-dev/pandas/issues/2858 Ideally inf value should be replaced by a large positive number and -inf should be replaced by negative large number, like we do in dataframe:

df.replace([np.inf, -np.inf],[np.finfo(np.float32).max,np.finfo(np.float32).min])

Choose np.float64 if that's the datatype.

Would be nice to do this using a step in Pipeline or some under the hood pre-processing step like we do fillna during reading.

opened by choukha 7
[WIP] New model initializers
First try on integrating the LSTM into gordo!

There are 2 functions for now:

create_lstm_model: builds the lstm autoencoder similarly to FFN, I tried to follow Miles' synthax as close as possible

exctract_decoder: takes the model_config from the AE and builds a model based only on the decoder part (for prediction)

Next step will be the fit and predict functions which are a bit more complicated than the one available with keras, but I think there is enough work on those two functions already :)
opened by Lezinou 7
Add assets to dataset resolvers

Adds resolving of assets in tilstandomatic CSV files that are not currently supported.

As per the issue on gordo-infrastructure: https://github.com/equinor/gordo-infrastructure/issues/508

opened by ryanjdillon 6
Bump pytest-mock from 1.11.2 to 1.12.1
Bumps pytest-mock from 1.11.2 to 1.12.1.

Release notes

Sourced from pytest-mock's releases.

1.12.0 (2019-11-19)

Now all patch functions also raise a ValueError when used as a context-manager. Thanks @AlexGascon for the PR (#168).

Changelog

Sourced from pytest-mock's changelog.

1.12.1 (2019-11-20)

Fix error if mocker.patch is used in code where the source file is not available, for example stale .pyc files (#169).

1.12.0 (2019-11-19)

Now all patch functions also raise a ValueError when used as a context-manager. Thanks @AlexGascon for the PR (#168).

Commits

d3d4c4a Handle 'mocker.patch' being used without source code

420fe64 Prepare release 1.12.0 (#170)

dc2e973 Prepare release 1.12.0

845876e Raise error if any version of patch is used as a context manager (#168)

ed8216b Add testing for Python 3.8 (#166)

d3d10a5 Testing exception on patch as context manager

6587f79 Raise error on every patch context manager

1c5e8e9 Update CHANGELOG.rst

3fb27bf Add testing for Python 3.8

19a4e4d Raise an error if mocker is used in a with context (#165)

See full diff in compare view

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

Dependabot commands and options

You can trigger Dependabot actions by commenting on this PR:

@dependabot rebase will rebase this PR

@dependabot recreate will recreate this PR, overwriting any edits that have been made to it

@dependabot merge will merge this PR after your CI passes on it

@dependabot squash and merge will squash and merge this PR after your CI passes on it

@dependabot cancel merge will cancel a previously requested merge and block automerging

@dependabot reopen will reopen this PR if it is closed

@dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually

@dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)

@dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)

@dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)

@dependabot use these labels will set the current labels as the default for future PRs for this repo and language

@dependabot use these reviewers will set the current reviewers as the default for future PRs for this repo and language

@dependabot use these assignees will set the current assignees as the default for future PRs for this repo and language

@dependabot use this milestone will set the current milestone as the default for future PRs for this repo and language

@dependabot badge me will comment on this PR with code to add a "Dependabot enabled" badge to your readme

Additionally, you can set the following in your Dependabot dashboard:

Update frequency (including time of day and day of week)

Pull request limits (per update run and/or open at any time)

Automerge options (never/patch/minor, and dev/runtime dependencies)

Out-of-range updates (receive only lockfile updates, if desired)

Security updates (receive only security updates, if desired)

dependencies
opened by dependabot-preview[bot] 6
Bump gunicorn from 19.9.0 to 20.0.0
Bumps gunicorn from 19.9.0 to 20.0.0.

Commits

a05a447 Change maintainers (#2147)

c1f17a0 doc enhancements.

83e07e5 Fixed linting error in changelog

a1bb7a9 add 20.0 changelog

9edf8e8 add 2018 changelog doc

21d6287 Fix fdopen RuntimeWarning in 3.8 (#2146)

3fcb551 Merge pull request #2144 from Djailla/py38

c67f4bb Merge pull request #2143 from Djailla/coverage

f0a9c6b [tests] Remove old coverage required version

7d0c8f9 [py38] Python 3.8 is now official, use it by default

Additional commits viewable in compare view

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

Dependabot will merge this PR once it's up-to-date and CI passes on it, as requested by @ryanjdillon.

Dependabot commands and options

You can trigger Dependabot actions by commenting on this PR:

@dependabot rebase will rebase this PR

@dependabot recreate will recreate this PR, overwriting any edits that have been made to it

@dependabot merge will merge this PR after your CI passes on it

@dependabot squash and merge will squash and merge this PR after your CI passes on it

@dependabot cancel merge will cancel a previously requested merge and block automerging

@dependabot reopen will reopen this PR if it is closed

@dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually

@dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)

@dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)

@dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)

@dependabot use these labels will set the current labels as the default for future PRs for this repo and language

@dependabot use these reviewers will set the current reviewers as the default for future PRs for this repo and language

@dependabot use these assignees will set the current assignees as the default for future PRs for this repo and language

@dependabot use this milestone will set the current milestone as the default for future PRs for this repo and language

@dependabot badge me will comment on this PR with code to add a "Dependabot enabled" badge to your readme

Additionally, you can set the following in your Dependabot dashboard:

Update frequency (including time of day and day of week)

Pull request limits (per update run and/or open at any time)

Automerge options (never/patch/minor, and dev/runtime dependencies)

Out-of-range updates (receive only lockfile updates, if desired)

Security updates (receive only security updates, if desired)

dependencies
opened by dependabot-preview[bot] 6
Bump numpy from 1.17.2 to 1.17.3
Bumps numpy from 1.17.2 to 1.17.3.

Release notes

Sourced from numpy's releases.

v1.17.3

NumPy 1.17.3 Release Notes

This release contains fixes for bugs reported against NumPy 1.17.2 along with a some documentation improvements. The Python versions supported in this release are 3.5-3.8.

Downstream developers should use Cython >= 0.29.13 for Python 3.8 support and OpenBLAS >= 3.7 to avoid errors on the Skylake architecture.

Highlights

Wheels for Python 3.8

Boolean matmul fixed to use booleans instead of integers.

Compatibility notes

The seldom used PyArray_DescrCheck macro has been changed/fixed.

Contributors

A total of 7 people contributed to this release. People with a "+" by their names contributed a patch for the first time.

Allan Haldane

Charles Harris

Kevin Sheppard

Matti Picus

Ralf Gommers

Sebastian Berg

Warren Weckesser

Pull requests merged

A total of 12 pull requests were merged for this release.

#14456: MAINT: clean up pocketfft modules inside numpy.fft namespace.

#14463: BUG: random.hypergeometic assumes npy_long is npy_int64, hung...

#14502: BUG: random: Revert gh-14458 and refix gh-14557.

#14504: BUG: add a specialized loop for boolean matmul.

#14506: MAINT: Update pytest version for Python 3.8

... (truncated)

Commits

ff3df08 REL: NumPy 1.14.5 release.

89208b4 Merge pull request #14704 from charris/prepare-1.17.3-release

f6725ad REL: Prepare for 1.17.3 release.

acb10b0 Merge pull request #14687 from charris/backport-14605

7758dd1 BUG: properly define PyArray_DescrCheck

51826db Merge pull request #14578 from charris/backport-14531

dc441ee BUG: random: Create a legacy implementation of random.binomial.

df93b2b Merge pull request #14563 from rgommers/bport-fftlicense

5504e6d DOC: remove note about Pocketfft license file (non-existing here).

2053327 Merge pull request #14519 from charris/backport-14498

Additional commits viewable in compare view

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

Dependabot commands and options

You can trigger Dependabot actions by commenting on this PR:

@dependabot rebase will rebase this PR

@dependabot recreate will recreate this PR, overwriting any edits that have been made to it

@dependabot merge will merge this PR after your CI passes on it

@dependabot squash and merge will squash and merge this PR after your CI passes on it

@dependabot cancel merge will cancel a previously requested merge and block automerging

@dependabot reopen will reopen this PR if it is closed

@dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually

@dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)

@dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)

@dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)

@dependabot use these labels will set the current labels as the default for future PRs for this repo and language

@dependabot use these reviewers will set the current reviewers as the default for future PRs for this repo and language

@dependabot use these assignees will set the current assignees as the default for future PRs for this repo and language

@dependabot use this milestone will set the current milestone as the default for future PRs for this repo and language

@dependabot badge me will comment on this PR with code to add a "Dependabot enabled" badge to your readme

Additionally, you can set the following in your Dependabot dashboard:

Update frequency (including time of day and day of week)

Pull request limits (per update run and/or open at any time)

Automerge options (never/patch/minor, and dev/runtime dependencies)

Out-of-range updates (receive only lockfile updates, if desired)

Security updates (receive only security updates, if desired)

dependencies
opened by dependabot-preview[bot] 6
Error: command 'g++' failed with exit status 1 on Mac OS Mojave

When run pip install --upgrade gordo-components on Mac OS Mojave, It will have following error

clang: warning: libstdc++ is deprecated; move to libc++ with a minimum deployment target of OS X 10.9 [-Wdeprecated] clang: warning: libstdc++ is deprecated; move to libc++ with a minimum deployment target of OS X 10.9 [-Wdeprecated] ld: library not found for -lstdc++ clang: error: linker command failed with exit code 1 (use -v to see invocation) error: command 'g++' failed with exit status 1

Mac OS Mojave version：10.14.4 to 10.14.5

The problem is that Xcode, especially Xcode 10.x, has not installed everything, so ensure the command line tools are installed, type this in a terminal shell:

xcode-select --install

also start XCode and ensure all the required installation is installed ( you should get prompted if it is not.) and since XCode 10 does not install the full Mac OS SDK, run the installer at

open /Library/Developer/CommandLineTools/Packages/macOS_SDK_headers_for_macOS_10.14.pkg

as this package is not installed by XCode 10.

It is better to update this to Read.me

opened by jieliu218 6
[Snyk] Security upgrade gitpython from 3.1.29 to 3.1.30
Snyk has created this PR to fix one or more vulnerable packages in the `pip` dependencies of this project.

Changes included in this PR

Changes to the following files to upgrade the vulnerable dependencies to a fixed version:

requirements/full_requirements.txt

⚠️ Warning

paramiko 2.12.0 requires pynacl, which is not installed. ndg-httpsclient 0.5.1 requires PyOpenSSL, which is not installed. msrest 0.7.0 requires azure-core, which is not installed. msal 1.20.0 requires PyJWT, which is not installed. msal-extensions 1.0.0 requires portalocker, which is not installed. knack 0.10.1 requires pygments, which is not installed. knack 0.10.1 requires tabulate, which is not installed. Keras-Preprocessing 1.1.2 requires numpy, which is not installed. Jinja2 2.11.3 requires MarkupSafe, which is not installed. google-auth 2.15.0 requires rsa, which is not installed. databricks-cli 0.17.4 requires pyjwt, which is not installed. databricks-cli 0.17.4 requires tabulate, which is not installed. catboost 0.26.1 requires numpy, which is not installed. catboost 0.26.1 requires matplotlib, which is not installed. catboost 0.26.1 requires scipy, which is not installed. catboost 0.26.1 requires graphviz, which is not installed. catboost 0.26.1 requires plotly, which is not installed. catboost 0.26.1 requires pandas, which is not installed. adal 1.2.7 requires PyJWT, which is not installed.

Vulnerabilities that will be fixed

By pinning:

Severity | Priority Score (*) | Issue | Upgrade | Breaking Change | Exploit Maturity :-------------------------:|-------------------------|:-------------------------|:-------------------------|:-------------------------|:------------------------- | 619/1000
Why? Has a fix available, CVSS 8.1 | Remote Code Execution (RCE)
SNYK-PYTHON-GITPYTHON-3113858 | gitpython:
3.1.29 -> 3.1.30
| No | No Known Exploit

(*) Note that the real score may have changed since the PR was raised.

Some vulnerabilities couldn't be fully fixed and so Snyk will still find them when the project is tested again. This may be because the vulnerability existed within more than one direct dependency, but not all of the affected dependencies could be upgraded.

Check the changes in this PR to ensure they won't cause issues with your project.

Note: You are seeing this because you or someone else with access to this repository has authorized Snyk to open fix PRs.

For more information: 🧐 View latest project report

🛠 Adjust project settings

📚 Read more about Snyk's upgrade and patch logic

Learn how to fix vulnerabilities with free interactive lessons:

🦉 Remote Code Execution (RCE)
opened by snyk-bot 0
[Snyk] Security upgrade setuptools from 39.0.1 to 65.5.1
This PR was automatically created by Snyk using the credentials of a real user.

Snyk has created this PR to fix one or more vulnerable packages in the `pip` dependencies of this project.

Changes included in this PR

Changes to the following files to upgrade the vulnerable dependencies to a fixed version:

requirements/test_requirements.txt

⚠️ Warning

Jinja2 2.11.3 requires MarkupSafe, which is not installed.

Vulnerabilities that will be fixed

By pinning:

Severity | Priority Score (*) | Issue | Upgrade | Breaking Change | Exploit Maturity :-------------------------:|-------------------------|:-------------------------|:-------------------------|:-------------------------|:------------------------- | 551/1000
Why? Recently disclosed, Has a fix available, CVSS 5.3 | Regular Expression Denial of Service (ReDoS)
SNYK-PYTHON-SETUPTOOLS-3180412 | setuptools:
39.0.1 -> 65.5.1
| No | No Known Exploit

(*) Note that the real score may have changed since the PR was raised.

Some vulnerabilities couldn't be fully fixed and so Snyk will still find them when the project is tested again. This may be because the vulnerability existed within more than one direct dependency, but not all of the affected dependencies could be upgraded.

Check the changes in this PR to ensure they won't cause issues with your project.

Note: You are seeing this because you or someone else with access to this repository has authorized Snyk to open fix PRs.

For more information: 🧐 View latest project report

🛠 Adjust project settings

📚 Read more about Snyk's upgrade and patch logic

Learn how to fix vulnerabilities with free interactive lessons:

🦉 Regular Expression Denial of Service (ReDoS)
opened by o-fedorov 0
[Snyk] Security upgrade setuptools from 39.0.1 to 65.5.1
This PR was automatically created by Snyk using the credentials of a real user.

Snyk has created this PR to fix one or more vulnerable packages in the `pip` dependencies of this project.

Changes included in this PR

Changes to the following files to upgrade the vulnerable dependencies to a fixed version:

requirements/full_requirements.txt

⚠️ Warning

paramiko 2.12.0 requires pynacl, which is not installed. ndg-httpsclient 0.5.1 requires PyOpenSSL, which is not installed. msrest 0.7.0 requires azure-core, which is not installed. msal 1.20.0 requires PyJWT, which is not installed. msal-extensions 1.0.0 requires portalocker, which is not installed. knack 0.10.1 requires tabulate, which is not installed. knack 0.10.1 requires pygments, which is not installed. Keras-Preprocessing 1.1.2 requires numpy, which is not installed. Jinja2 2.11.3 requires MarkupSafe, which is not installed. google-auth 2.15.0 requires rsa, which is not installed. databricks-cli 0.17.4 requires pyjwt, which is not installed. databricks-cli 0.17.4 requires tabulate, which is not installed. catboost 0.26.1 requires numpy, which is not installed. catboost 0.26.1 requires matplotlib, which is not installed. catboost 0.26.1 requires scipy, which is not installed. catboost 0.26.1 requires graphviz, which is not installed. catboost 0.26.1 requires plotly, which is not installed. catboost 0.26.1 requires pandas, which is not installed. adal 1.2.7 requires PyJWT, which is not installed.

Vulnerabilities that will be fixed

By pinning:

Severity | Priority Score (*) | Issue | Upgrade | Breaking Change | Exploit Maturity :-------------------------:|-------------------------|:-------------------------|:-------------------------|:-------------------------|:------------------------- | 551/1000
Why? Recently disclosed, Has a fix available, CVSS 5.3 | Regular Expression Denial of Service (ReDoS)
SNYK-PYTHON-SETUPTOOLS-3180412 | setuptools:
39.0.1 -> 65.5.1
| No | No Known Exploit

(*) Note that the real score may have changed since the PR was raised.

Some vulnerabilities couldn't be fully fixed and so Snyk will still find them when the project is tested again. This may be because the vulnerability existed within more than one direct dependency, but not all of the affected dependencies could be upgraded.

Check the changes in this PR to ensure they won't cause issues with your project.

Note: You are seeing this because you or someone else with access to this repository has authorized Snyk to open fix PRs.

For more information: 🧐 View latest project report

🛠 Adjust project settings

📚 Read more about Snyk's upgrade and patch logic

Learn how to fix vulnerabilities with free interactive lessons:

🦉 Regular Expression Denial of Service (ReDoS)
opened by hholi 0
[Snyk] Fix for 2 vulnerabilities
This PR was automatically created by Snyk using the credentials of a real user.

Snyk has created this PR to fix one or more vulnerable packages in the `pip` dependencies of this project.

Changes included in this PR

Changes to the following files to upgrade the vulnerable dependencies to a fixed version:

requirements/test_requirements.txt

⚠️ Warning

pytest-cov 2.12.1 requires coverage, which is not installed. Jinja2 2.11.3 requires MarkupSafe, which is not installed.

Vulnerabilities that will be fixed

By pinning:

Severity | Priority Score (*) | Issue | Upgrade | Breaking Change | Exploit Maturity :-------------------------:|-------------------------|:-------------------------|:-------------------------|:-------------------------|:------------------------- | 479/1000
Why? Has a fix available, CVSS 5.3 | Regular Expression Denial of Service (ReDoS)
SNYK-PYTHON-MISTUNE-2940625 | mistune:
0.8.4 -> 2.0.3
| No | No Known Exploit | 441/1000
Why? Recently disclosed, Has a fix available, CVSS 3.1 | Regular Expression Denial of Service (ReDoS)
SNYK-PYTHON-SETUPTOOLS-3113904 | setuptools:
39.0.1 -> 65.5.1
| No | No Known Exploit

(*) Note that the real score may have changed since the PR was raised.

Some vulnerabilities couldn't be fully fixed and so Snyk will still find them when the project is tested again. This may be because the vulnerability existed within more than one direct dependency, but not all of the affected dependencies could be upgraded.

Check the changes in this PR to ensure they won't cause issues with your project.

Note: You are seeing this because you or someone else with access to this repository has authorized Snyk to open fix PRs.

For more information: 🧐 View latest project report

🛠 Adjust project settings

📚 Read more about Snyk's upgrade and patch logic

Learn how to fix vulnerabilities with free interactive lessons:

🦉 Regular Expression Denial of Service (ReDoS) 🦉 Regular Expression Denial of Service (ReDoS)
opened by hholi 0
[Snyk] Security upgrade setuptools from 39.0.1 to 65.5.1
This PR was automatically created by Snyk using the credentials of a real user.

Snyk has created this PR to fix one or more vulnerable packages in the `pip` dependencies of this project.

Changes included in this PR

Changes to the following files to upgrade the vulnerable dependencies to a fixed version:

requirements/full_requirements.txt

⚠️ Warning

paramiko 2.11.0 requires pynacl, which is not installed. ndg-httpsclient 0.5.1 requires PyOpenSSL, which is not installed. msrest 0.7.0 requires azure-core, which is not installed. msal 1.20.0 requires PyJWT, which is not installed. msal-extensions 1.0.0 requires portalocker, which is not installed. knack 0.10.0 requires pygments, which is not installed. knack 0.10.0 requires tabulate, which is not installed. Keras-Preprocessing 1.1.2 requires numpy, which is not installed. Jinja2 2.11.3 requires MarkupSafe, which is not installed. google-auth 2.14.0 requires rsa, which is not installed. databricks-cli 0.17.3 requires pyjwt, which is not installed. databricks-cli 0.17.3 requires tabulate, which is not installed. azure-mgmt-authorization 2.0.0 requires azure-mgmt-core, which is not installed. adal 1.2.7 requires PyJWT, which is not installed.

Vulnerabilities that will be fixed

By pinning:

Severity | Priority Score (*) | Issue | Upgrade | Breaking Change | Exploit Maturity :-------------------------:|-------------------------|:-------------------------|:-------------------------|:-------------------------|:------------------------- | 441/1000
Why? Recently disclosed, Has a fix available, CVSS 3.1 | Regular Expression Denial of Service (ReDoS)
SNYK-PYTHON-SETUPTOOLS-3113904 | setuptools:
39.0.1 -> 65.5.1
| No | No Known Exploit

(*) Note that the real score may have changed since the PR was raised.

Some vulnerabilities couldn't be fully fixed and so Snyk will still find them when the project is tested again. This may be because the vulnerability existed within more than one direct dependency, but not all of the affected dependencies could be upgraded.

Check the changes in this PR to ensure they won't cause issues with your project.

Note: You are seeing this because you or someone else with access to this repository has authorized Snyk to open fix PRs.

For more information: 🧐 View latest project report

🛠 Adjust project settings

📚 Read more about Snyk's upgrade and patch logic

Learn how to fix vulnerabilities with free interactive lessons:

🦉 Regular Expression Denial of Service (ReDoS)
opened by o-fedorov 0

Releases(4.0.2)

4.0.2(Dec 9, 2022)
What's Changed

Update gordo-core and gordo-client by @koropets in https://github.com/equinor/gordo/pull/1293

Full Changelog: https://github.com/equinor/gordo/compare/4.0.1...4.0.2
Source code(tar.gz)
Source code(zip)
4.0.1(Nov 19, 2022)
What's Changed

Fix workflow lint error: cannot unmarshal number into Go struct field (#1290)

by @koropets Full Changelog: https://github.com/equinor/gordo/compare/4.0.0...4.0.1
Source code(tar.gz)
Source code(zip)
4.0.0(Nov 16, 2022)
What's Changed

Migrate to gordo-core (#1266)

Open source docker image (#1281)

Machine loader. Introduction of a new config schema (#1268,#1276)

Remove WORKFLOW_GENERATOR_CUSTOM_MODEL_BUILDER_ENVS environment variable (#1270)

WORKFLOW_GENERATOR_DOCKER_REGISTRY is ghcr.io by default (#1286)

Remove flask-restplus (#1271)

Update dependencies (#1272,#1283)

Migrate to PyPI (#1273)

Argo workflow v3.0.0 (#1275)

Support of argo 2/3 versions (#1279,#1282)

Do not push PR's docker images (#1277)

Refactoring github_docker.py script (#1285)

and other fixes

by @koropets Full Changelog: https://github.com/equinor/gordo/compare/3.0.0...4.0.0
Source code(tar.gz)
Source code(zip)
4.0.0rc3(Nov 3, 2022)

No description
Source code(tar.gz)
Source code(zip)
4.0.0rc2(Oct 4, 2022)

No description
Source code(tar.gz)
Source code(zip)
4.0.0rc1(Oct 4, 2022)

No description
Source code(tar.gz)
Source code(zip)
4.0.0rc0(Sep 21, 2022)

Source code(tar.gz)
Source code(zip)
3.1.0rc0(Aug 29, 2022)

Source code(tar.gz)
Source code(zip)
3.0.0(Aug 12, 2022)
What's Changed

gordo-dataset updated up to 5.0.0.

Machine arguments dataset, metadata should be specified only with actual object instances. Machine.from_dict should be used for passing them as dict.

WORKFLOW_GENERATOR_DEFAULT_DATA_PROVIDER environment variable for gordo workflow.

by @koropets Full Changelog: https://github.com/equinor/gordo/compare/2.2.3...3.0.0
Source code(tar.gz)
Source code(zip)
3.0.0rc2(Jul 29, 2022)

No detail.
Source code(tar.gz)
Source code(zip)
3.0.0rc1(Jul 27, 2022)

No details
Source code(tar.gz)
Source code(zip)
3.0.0rc0(Jun 30, 2022)

No details
Source code(tar.gz)
Source code(zip)
2.2.3(Jun 9, 2022)
What's Changed

gordo-dataset==4.2.3 (#1254) by @koropets

Full Changelog: https://github.com/equinor/gordo/compare/2.2.2...2.2.3
Source code(tar.gz)
Source code(zip)
2.2.2(May 20, 2022)
What's Changed

Fix for the issue with ValueError in create_model_builder() #1249 by @koropets

Full Changelog: https://github.com/equinor/gordo/compare/2.2.1...2.2.2
Source code(tar.gz)
Source code(zip)
2.2.1(May 19, 2022)
What's Changed

Custom environment variables for run_workflow_and_argo.sh in envs.sh #1247 by @koropets

Full Changelog: https://github.com/equinor/gordo/compare/2.2.0...2.2.1
Source code(tar.gz)
Source code(zip)
2.2.0(May 19, 2022)
What's Changed

Refactor ModelBuilderBase.calculate_cache_key(). securityContext for Workflow #1245 by @koropets

Refactoring run_workflow_and_argo.sh #1244 by @koropets

gordo-dataset==4.2.1 #1246 by @koropets

Full Changelog: https://github.com/equinor/gordo/compare/2.1.0...2.2.0
Source code(tar.gz)
Source code(zip)
2.1.0(Apr 28, 2022)
What's Changed

Python 3.9 by @koropets in #1240

Store "checksum" to the metadata by @koropets in #1195

Security dependencies update. USER 999 in Dockerfile by @koropets in #1243

Full Changelog: https://github.com/equinor/gordo/compare/2.0.0...2.1.0
Source code(tar.gz)
Source code(zip)
2.1.0rc0(Apr 21, 2022)

Source code(tar.gz)
Source code(zip)
2.0.0(Mar 28, 2022)
What's Changed

DELETE model revisions by @koropets in #1222

Providing a way to customize initialDelaySeconds for livenessProbe and readinessProbe by @koropets in #1221

user client-5.0.0 and dataset-4.0.0 by @devProdigy in #1233

Update base docker image by @koropets in #1238

Full Changelog: https://github.com/equinor/gordo/compare/1.12.0...2.0.0
Source code(tar.gz)
Source code(zip)
2.0.0dev(Mar 16, 2022)
What's Changed

DELETE model revisions by @koropets in https://github.com/equinor/gordo/pull/1222

Providing a way to customize initialDelaySeconds for livenessProbe and readinessProbe by @koropets in https://github.com/equinor/gordo/pull/1221

user client-5.0.0 and dataset-4.0.0 by @devProdigy in https://github.com/equinor/gordo/pull/1233

Full Changelog: https://github.com/equinor/gordo/compare/1.12.0...2.0.0dev
Source code(tar.gz)
Source code(zip)
2.0.0.dev0(Mar 17, 2022)
What's Changed

DELETE model revisions by @koropets in https://github.com/equinor/gordo/pull/1222

Providing a way to customize initialDelaySeconds for livenessProbe and readinessProbe by @koropets in https://github.com/equinor/gordo/pull/1221

user client-5.0.0 and dataset-4.0.0 by @devProdigy in https://github.com/equinor/gordo/pull/1233

Full Changelog: https://github.com/equinor/gordo/compare/1.12.0...2.0.0.dev0
Source code(tar.gz)
Source code(zip)
2.0.0.dev(Mar 17, 2022)
What's Changed

DELETE model revisions by @koropets in https://github.com/equinor/gordo/pull/1222

Providing a way to customize initialDelaySeconds for livenessProbe and readinessProbe by @koropets in https://github.com/equinor/gordo/pull/1221

user client-5.0.0 and dataset-4.0.0 by @devProdigy in https://github.com/equinor/gordo/pull/1233

Full Changelog: https://github.com/equinor/gordo/compare/1.12.0...2.0.0.dev
Source code(tar.gz)
Source code(zip)
1.12.1.dev0(Jan 26, 2022)

Source code(tar.gz)
Source code(zip)
1.12.0(Jan 13, 2022)
What's Changed

31230 update tensorflow and numpy versions (vulnerabilities) by @devProdigy in https://github.com/equinor/gordo/pull/1214

update dataset to 3.3.0 by @devProdigy in https://github.com/equinor/gordo/pull/1215

Full Changelog: https://github.com/equinor/gordo/compare/1.11.3...1.12.0
Source code(tar.gz)
Source code(zip)
1.12.0dev0(Jan 13, 2022)

Source code(tar.gz)
Source code(zip)
1.11.3(Jan 5, 2022)
gordo-dataset==3.2.2 (#1213)

Source code(tar.gz)
Source code(zip)
1.11.2(Dec 23, 2021)
Providing REN_DL2_SERVICE_AUTH_STR env var (#1210)

Source code(tar.gz)
Source code(zip)
1.11.1(Dec 20, 2021)
gordo-dataset==3.2.1 (#1208)

Source code(tar.gz)
Source code(zip)
1.11.0(Dec 20, 2021)
Using private PyPi (#1203,#1206)

Source code(tar.gz)
Source code(zip)
1.10.5(Nov 26, 2021)
gordo-dataset == 3.1.1 (#1202)

Make gordo_dataset.dataset._get_dataset public

Source code(tar.gz)
Source code(zip)