Aoc 2021 kedro playground with python

Related tags

Miscellaneous aoc-2021-kedro-playground
Overview

AOC 2021

Overview

This is your new Kedro project, which was generated using Kedro 0.17.5.

Take a look at the Kedro documentation to get started.

Rules and guidelines

In order to get the best out of the template:

  • Don't remove any lines from the .gitignore file we provide
  • Make sure your results can be reproduced by following a data engineering convention
  • Don't commit data to your repository
  • Don't commit any credentials or your local configuration to your repository. Keep all your credentials and local configuration in conf/local/

How to install dependencies

Declare any dependencies in src/requirements.txt for pip installation and src/environment.yml for conda installation.

To install them, run:

kedro install

How to run your Kedro pipeline

You can run your Kedro project with:

kedro run

How to test your Kedro project

Have a look at the file src/tests/test_run.py for instructions on how to write your tests. You can run your tests as follows:

kedro test

To configure the coverage threshold, go to the .coveragerc file.

Project dependencies

To generate or update the dependency requirements for your project:

kedro build-reqs

This will copy the contents of src/requirements.txt into a new file src/requirements.in which will be used as the source for pip-compile. You can see the output of the resolution by opening src/requirements.txt.

After this, if you'd like to update your project requirements, please update src/requirements.in and re-run kedro build-reqs.

Further information about project dependencies

How to work with Kedro and notebooks

Note: Using kedro jupyter or kedro ipython to run your notebook provides these variables in scope: context, catalog, and startup_error.

Jupyter, JupyterLab, and IPython are already included in the project requirements by default, so once you have run kedro install you will not need to take any extra steps before you use them.

Jupyter

To use Jupyter notebooks in your Kedro project, you need to install Jupyter:

pip install jupyter

After installing Jupyter, you can start a local notebook server:

kedro jupyter notebook

JupyterLab

To use JupyterLab, you need to install it:

pip install jupyterlab

You can also start JupyterLab:

kedro jupyter lab

IPython

And if you want to run an IPython session:

kedro ipython

How to convert notebook cells to nodes in a Kedro project

You can move notebook code over into a Kedro project structure using a mixture of cell tagging and Kedro CLI commands.

By adding the node tag to a cell and running the command below, the cell's source code will be copied over to a Python file within src/ /nodes/ :

kedro jupyter convert

Note: The name of the Python file matches the name of the original notebook.

Alternatively, you may want to transform all your notebooks in one go. Run the following command to convert all notebook files found in the project root directory and under any of its sub-folders:

kedro jupyter convert --all

How to ignore notebook output cells in git

To automatically strip out all output cell contents before committing to git, you can run kedro activate-nbstripout. This will add a hook in .git/config which will run nbstripout before anything is committed to git.

Note: Your output cells will be retained locally.

Package your Kedro project

Further information about building project documentation and packaging your project

Comments
  • [Snyk] Security upgrade mistune from 0.8.4 to 2.0.1

    [Snyk] Security upgrade mistune from 0.8.4 to 2.0.1

    Snyk has created this PR to fix one or more vulnerable packages in the `pip` dependencies of this project.

    Changes included in this PR

    • Changes to the following files to upgrade the vulnerable dependencies to a fixed version:
      • src/requirements.txt
    ⚠️ Warning 
    pytest-mock 1.13.0 requires pytest, which is not installed.
pytest-cov 2.12.1 requires pytest, which is not installed.
notebook 5.7.13 requires terminado, which is not installed.
nbformat 4.4.0 requires jsonschema, which is not installed.
nbconvert 5.6.1 has requirement mistune<2,>=0.8.1, but you have mistune 2.0.2.
jupyter 1.0.0 requires qtconsole, which is not installed.

    Vulnerabilities that will be fixed

    By pinning:

    Severity | Priority Score (*) | Issue | Upgrade | Breaking Change | Exploit Maturity :-------------------------:|-------------------------|:-------------------------|:-------------------------|:-------------------------|:------------------------- medium severity | 484/1000
    Why? Has a fix available, CVSS 5.4 | Cross-site Scripting (XSS)
    SNYK-PYTHON-MISTUNE-2328096 | mistune:
    0.8.4 -> 2.0.1
    | No | No Known Exploit

    (*) Note that the real score may have changed since the PR was raised.

    Some vulnerabilities couldn't be fully fixed and so Snyk will still find them when the project is tested again. This may be because the vulnerability existed within more than one direct dependency, but not all of the effected dependencies could be upgraded.

    Check the changes in this PR to ensure they won't cause issues with your project.

    Note: You are seeing this because you or someone else with access to this repository has authorized Snyk to open fix PRs.

    For more information: 🧐 View latest project report

    🛠 Adjust project settings

    📚 Read more about Snyk's upgrade and patch logic

    Learn how to fix vulnerabilities with free interactive lessons:

    🦉 Cross-site Scripting (XSS)

    opened by snyk-bot 0
  • [Snyk] Fix for 2 vulnerabilities

    [Snyk] Fix for 2 vulnerabilities

    Snyk has created this PR to fix one or more vulnerable packages in the `pip` dependencies of this project.

    Changes included in this PR

    • Changes to the following files to upgrade the vulnerable dependencies to a fixed version:
      • src/requirements.txt
    ⚠️ Warning 
    pytest-mock 1.13.0 requires pytest, which is not installed.
pytest-cov 2.12.1 requires pytest, which is not installed.
pytest-cov 2.12.1 requires coverage, which is not installed.
notebook 5.7.16 requires terminado, which is not installed.
nbformat 4.4.0 requires jsonschema, which is not installed.
nbconvert 5.6.1 has requirement mistune<2,>=0.8.1, but you have mistune 2.0.4.
jupyter 1.0.0 requires qtconsole, which is not installed.
ipython 5.10.0 requires simplegeneric, which is not installed.

    Vulnerabilities that will be fixed

    By pinning:

    Severity | Priority Score (*) | Issue | Upgrade | Breaking Change | Exploit Maturity :-------------------------:|-------------------------|:-------------------------|:-------------------------|:-------------------------|:------------------------- high severity | 726/1000
    Why? Proof of Concept exploit, Has a fix available, CVSS 8.1 | Command Injection
    SNYK-PYTHON-COOKIECUTTER-2414281 | cookiecutter:
    1.7.3 -> 2.1.1
    | No | Proof of Concept medium severity | 551/1000
    Why? Recently disclosed, Has a fix available, CVSS 5.3 | Regular Expression Denial of Service (ReDoS)
    SNYK-PYTHON-WHEEL-3092128 | wheel:
    0.36.2 -> 0.38.0
    | No | No Known Exploit

    (*) Note that the real score may have changed since the PR was raised.

    Some vulnerabilities couldn't be fully fixed and so Snyk will still find them when the project is tested again. This may be because the vulnerability existed within more than one direct dependency, but not all of the affected dependencies could be upgraded.

    Check the changes in this PR to ensure they won't cause issues with your project.

    Note: You are seeing this because you or someone else with access to this repository has authorized Snyk to open fix PRs.

    For more information: 🧐 View latest project report

    🛠 Adjust project settings

    📚 Read more about Snyk's upgrade and patch logic

    Learn how to fix vulnerabilities with free interactive lessons:

    🦉 Regular Expression Denial of Service (ReDoS)

    opened by snyk-bot 0
  • [Snyk] Security upgrade cookiecutter from 1.7.3 to 2.1.1

    [Snyk] Security upgrade cookiecutter from 1.7.3 to 2.1.1

    Snyk has created this PR to fix one or more vulnerable packages in the `pip` dependencies of this project.

    Changes included in this PR

    • Changes to the following files to upgrade the vulnerable dependencies to a fixed version:
      • src/requirements.txt
    ⚠️ Warning 
    pytest-mock 1.13.0 requires pytest, which is not installed.
pytest-cov 2.12.1 requires pytest, which is not installed.
notebook 5.7.16 requires terminado, which is not installed.
nbformat 4.4.0 requires jsonschema, which is not installed.
nbconvert 5.6.1 has requirement mistune<2,>=0.8.1, but you have mistune 2.0.4.
jupyter 1.0.0 requires qtconsole, which is not installed.
ipython 5.10.0 requires simplegeneric, which is not installed.

    Vulnerabilities that will be fixed

    By pinning:

    Severity | Priority Score (*) | Issue | Upgrade | Breaking Change | Exploit Maturity :-------------------------:|-------------------------|:-------------------------|:-------------------------|:-------------------------|:------------------------- high severity | 726/1000
    Why? Proof of Concept exploit, Has a fix available, CVSS 8.1 | Command Injection
    SNYK-PYTHON-COOKIECUTTER-2414281 | cookiecutter:
    1.7.3 -> 2.1.1
    | No | Proof of Concept

    (*) Note that the real score may have changed since the PR was raised.

    Some vulnerabilities couldn't be fully fixed and so Snyk will still find them when the project is tested again. This may be because the vulnerability existed within more than one direct dependency, but not all of the affected dependencies could be upgraded.

    Check the changes in this PR to ensure they won't cause issues with your project.

    Note: You are seeing this because you or someone else with access to this repository has authorized Snyk to open fix PRs.

    For more information: 🧐 View latest project report

    🛠 Adjust project settings

    📚 Read more about Snyk's upgrade and patch logic

    Learn how to fix vulnerabilities with free interactive lessons:

    🦉 Learn about vulnerability in an interactive lesson of Snyk Learn.

    opened by snyk-bot 0
  • [Snyk] Security upgrade cookiecutter from 1.7.3 to 2.1.1

    [Snyk] Security upgrade cookiecutter from 1.7.3 to 2.1.1

    Snyk has created this PR to fix one or more vulnerable packages in the `pip` dependencies of this project.

    Changes included in this PR

    • Changes to the following files to upgrade the vulnerable dependencies to a fixed version:
      • src/requirements.txt
    ⚠️ Warning 
    pytest-mock 1.13.0 requires pytest, which is not installed.
pytest-cov 2.12.1 requires pytest, which is not installed.
notebook 5.7.16 requires terminado, which is not installed.
nbformat 4.4.0 requires jsonschema, which is not installed.
nbconvert 5.6.1 has requirement mistune<2,>=0.8.1, but you have mistune 2.0.4.
jupyter 1.0.0 requires qtconsole, which is not installed.
ipython 5.10.0 requires simplegeneric, which is not installed.

    Vulnerabilities that will be fixed

    By pinning:

    Severity | Priority Score (*) | Issue | Upgrade | Breaking Change | Exploit Maturity :-------------------------:|-------------------------|:-------------------------|:-------------------------|:-------------------------|:------------------------- high severity | 726/1000
    Why? Proof of Concept exploit, Has a fix available, CVSS 8.1 | Command Injection
    SNYK-PYTHON-COOKIECUTTER-2414281 | cookiecutter:
    1.7.3 -> 2.1.1
    | No | Proof of Concept

    (*) Note that the real score may have changed since the PR was raised.

    Some vulnerabilities couldn't be fully fixed and so Snyk will still find them when the project is tested again. This may be because the vulnerability existed within more than one direct dependency, but not all of the affected dependencies could be upgraded.

    Check the changes in this PR to ensure they won't cause issues with your project.

    Note: You are seeing this because you or someone else with access to this repository has authorized Snyk to open fix PRs.

    For more information: 🧐 View latest project report

    🛠 Adjust project settings

    📚 Read more about Snyk's upgrade and patch logic

    Learn how to fix vulnerabilities with free interactive lessons:

    🦉 Learn about vulnerability in an interactive lesson of Snyk Learn.

    opened by snyk-bot 0
  • [Snyk] Security upgrade cookiecutter from 1.7.3 to 2.1.1

    [Snyk] Security upgrade cookiecutter from 1.7.3 to 2.1.1

    This PR was automatically created by Snyk using the credentials of a real user.


    Snyk has created this PR to fix one or more vulnerable packages in the `pip` dependencies of this project.

    Changes included in this PR

    • Changes to the following files to upgrade the vulnerable dependencies to a fixed version:
      • src/requirements.txt
    ⚠️ Warning 
    pytest-mock 1.13.0 requires pytest, which is not installed.
pytest-cov 2.12.1 requires pytest, which is not installed.
notebook 5.7.13 requires terminado, which is not installed.
nbformat 4.4.0 requires jsonschema, which is not installed.
nbconvert 5.6.1 has requirement mistune<2,>=0.8.1, but you have mistune 2.0.2.
jupyter 1.0.0 requires qtconsole, which is not installed.
ipython 5.10.0 requires simplegeneric, which is not installed.

    Vulnerabilities that will be fixed

    By pinning:

    Severity | Priority Score (*) | Issue | Upgrade | Breaking Change | Exploit Maturity :-------------------------:|-------------------------|:-------------------------|:-------------------------|:-------------------------|:------------------------- high severity | 798/1000
    Why? Proof of Concept exploit, Recently disclosed, Has a fix available, CVSS 8.1 | Command Injection
    SNYK-PYTHON-COOKIECUTTER-2414281 | cookiecutter:
    1.7.3 -> 2.1.1
    | No | Proof of Concept

    (*) Note that the real score may have changed since the PR was raised.

    Some vulnerabilities couldn't be fully fixed and so Snyk will still find them when the project is tested again. This may be because the vulnerability existed within more than one direct dependency, but not all of the effected dependencies could be upgraded.

    Check the changes in this PR to ensure they won't cause issues with your project.

    Note: You are seeing this because you or someone else with access to this repository has authorized Snyk to open fix PRs.

    For more information: 🧐 View latest project report

    🛠 Adjust project settings

    📚 Read more about Snyk's upgrade and patch logic

    Learn how to fix vulnerabilities with free interactive lessons:

    🦉 Learn about vulnerability in an interactive lesson of Snyk Learn.

    opened by pypeaday 0
Owner
Dedicated student, seeking to automate life with Python. I develop data and ML pipelines to make life easier for my co-workers.
null
GitHub
A demo of a data science project using Kedro

iris Overview This is your new Kedro project, which was generated using Kedro 0.17.4. Take a look at the Kedro documentation to get started. Rules and

Khuyen Tran 14 Oct 14, 2022
Connect Playground - easy way to fill in your account with production-like objects

Just set of scripts to initialise accpunt with production-like data: A - Basic Distributor Account Initialization INPUT Distributor Account Token ACTI

CloudBlue 5 Jun 25, 2021
JimShapedCoding Python Crash Course 2021

Python CRASH Course by JimShapedCoding - Click Here to Start! This Repository includes the code and MORE exercises on each section of the entire cours

Jim Erg 64 Dec 23, 2022
Assignment for python course, BUPT 2021.

pyFuujinrokuDestiny Assignment for python course, BUPT 2021. Notice username and password must be ASCII encoding. If username exists in database, syst

Ellias Kiri Stuart 3 Jun 18, 2021
TallerStereoVision Convencion Python Chile 2021

TallerStereoVision Convencion Python Chile 2021 Taller Stereo Vision & Python PyCon.cl 2021 Instalación Se recomienta utilizar Virtual Environment pyt

null 2 Oct 20, 2022
My solutions to Advent of Code 2021 (written in Python)

Advent of Code 2021 This repository contains my solutions for the 2021 edition of Advent of Code. Please do not expect perfectly polished solutions, m

Nils 2 May 29, 2022
Python solution of advent-of-code 2021

Advent of code 2021 Python solutions of Advent of Code 2021 written by Eric Bouteillon Requirements The solutions were developed and tested using Pyth

Eric Bouteillon 3 Oct 25, 2022
Python Interactive Graphical System made during Computer Graphics classes (INE5420-2021.1)

PY-IGS - The PYthon Interactive Graphical System The PY-IGS Installation To install this software you will need these dependencies (with their thevelo

Enzo Coelho Albornoz 4 Dec 3, 2021
Materials and information for my PyCascades 2021 Presentation

Materials and information for PyCascades 2021 Presentation: Sparking Creativity in LED Art with CircuitPython

GeekMomProjects 19 May 4, 2022
Visualize Data From Stray Scanner https://keke.dev/blog/2021/03/10/Stray-Scanner.html

StrayVisualizer A set of scripts to work with data collected using Stray Scanner. Usage Installing Dependencies Install dependencies with pip -r requi

Kenneth Blomqvist 45 Dec 30, 2022
The code for 2021 MGTV AI Challenge Anti Stealing Link, and the online result ranks 10th.

赛题介绍 芒果TV-第二届“马栏山杯”国际音视频算法大赛-防盗链 随着业务的发展，芒果的视频内容也深受网友的喜欢，不少视频网站和应用开始盗播芒果的视频内容，盗链网站不经过芒果TV的前端系统，跳过广告播放，且消耗大量的服务器、带宽资源，直接给公司带来了巨大的经济损失，因此防盗链在日常运营中显得尤为重要

tongji40 16 Jun 17, 2022
propuestas electorales de los candidatos a constituyentes, Chile 2021

textos-constituyentes propuestas electorales de los candidatos a constituyentes, Chile 2021 Programas descargados desde https://elecciones2021.servel.

Sergio Lucero 6 Nov 19, 2021
Яндекс тренировки по алгоритмам. Июнь 2021

Young&&Yandex Тренировки по алгоритмам Если вы хотите попасть на летнюю стажировку в Яндекс, но пока не уверены в своих силах, приходите на наши трени

Podlevskiy Viktor 6 Sep 3, 2021
resultados (data) de elecciones 2021 y código para extraer data de la ONPE

elecciones-peru-2021-ONPE Resultados (data) de elecciones 2021 y código para extraer data de la ONPE Data Licencia liberal, pero si vas a usarlo por f

Ragi Yaser Burhum 21 Jun 14, 2021
ASVspoof 2021 Baseline Systems

ASVspoof 2021 Baseline Systems Baseline systems are grouped by task: Speech Deepfake (DF) Logical Access (LA) Physical Access (PA) Please find more de

null 91 Dec 28, 2022
Palestra sobre desenvolvimento seguro de imagens e containers para a DockerCon 2021 sala Brasil

Segurança de imagens e containers direto na pipeline Palestra sobre desenvolvimento seguro de imagens e containers para a DockerCon 2021 sala Brasil.

Fernando Guisso 10 May 19, 2022
This repo is related to Google Coding Challenge, given to Bright Network Internship Experience 2021.

BrightNetworkUK-GCC-2021 This repo is related to Google Coding Challenge, given to Bright Network Internship Experience 2021. Language used here is py

Dareer Ahmad Mufti 28 May 23, 2022
TeamFleming is a multicultural group of 20 young bioinformatics enthusiasts participating in the 2021 HackBio Virtual Summer Internship

?? Welcome to Team Fleming's Repo! #TeamFleming is a multicultural group of 20 young bioinformatics enthusiasts participating in the 2021 HackBio Virt

null 3 Aug 8, 2021
XHacks 2021 Startup Track Winner: Be Heard. Educate, Enact, Empower. No voice left behind. (backend)

Be Heard: X Hacks 2021 Submission Educate, Enact, Empower. No voice left behind. Inspiration To say 2020 was an eventful year would be an understateme

null 3 Jul 14, 2022