Spline is a tool that is capable of running locally as well as part of well known pipelines like Jenkins (Jenkinsfile), Travis CI (.travis.yml) or similar ones.

Overview

Welcome to spline - the pipeline tool

PyPI version PyPI format PyPI versions PyPI license Read The Docs Coverage Status BCH compliance Known Vulnerabilities


Important note:

Since change in my job I didn't had the chance to continue on this project. My main new project is here https://github.com/thomas-lehmann-private/hyperion-task-processor which is planned to be a improved task processing tool; step by step those one will be capable to have more functionality across all platforms (powershell, batch, bash, JShell, Groovy, Kotlin, Docker, ... features depend on platform). It's fully written in Java. Stay tuned.


Table Of Content:
Motivation
Quickstart
Example
Matrix
The Model
The Pipeline
Pipeline Stages
Tasks
The Shell
Environment Variables
Docker Container
Hooks
Include
Event Logging
Tool: spline-loc
How to contact?
Links

Features:

  • Python support for 2.7.x, 3.4.x, 3.5.x, 3.6.x, PyPy and PyPy3
  • automatic schema validation for yaml file
  • matrix based pipeline with tags
  • ordered and parallel pipelines (matrix)
  • ordered and parallel tasks execution
  • pipeline stages (named groups)
  • shell script execution: inline and file
  • environment variables merged across each level: matrix, pipeline, stage, and tasks
  • support for model data (a dictionary of anything you need)
  • cleanup hook
  • filtered execution via tags
  • supporting Jinja templating in Bash scripts (also nested inside model)
  • support for Docker containers and Docker images
  • support for the Packer tool
  • execution time on each level: pipeline, stage, tasks and shell (event logging)
  • documentation here and also at read the docs
  • usable by Jenkinsfile as well as by a .travis.yml (or other pipelines).
  • dry run and debug support
  • support for Python scripts
  • support for task variables
  • support for conditional tasks
  • enabled for code reuse: !include statement

How to contact?

Interesting Links

Comments
  • Recursive render fails on raw/endraw

    Recursive render fails on raw/endraw

    Following problem:

    $ PYTHONPATH=$PWD scripts/spline --definition=examples/docker.yaml --tags=remove-docker-container
    2018-08-16 05:50:04,315 - spline.application - Running with Python 2.7.13 (default, Nov 24 2017, 17:33:09) [GCC 6.3.0 20170516]
    2018-08-16 05:50:04,321 - spline.application - Running on platform Linux-4.9.0-6-amd64-x86_64-with-debian-9.4
    2018-08-16 05:50:04,321 - spline.application - Current cpu count is 4
    2018-08-16 05:50:04,321 - spline.application - Processing pipeline definition 'examples/docker.yaml'
    2018-08-16 05:50:04,339 - spline.application - Schema validation for 'examples/docker.yaml' succeeded
    2018-08-16 05:50:04,382 - spline.tools.version - Using tool 'Bash', Version(4.4.12)
    2018-08-16 05:50:04,382 - spline.tools.version - Using tool 'Docker', Version(17.12.1)
    2018-08-16 05:50:04,383 - spline.tools.version - Using tool 'Spline', Version(1.11)
    2018-08-16 05:50:04,383 - spline.components.stage - Processing pipeline stage 'example'
    2018-08-16 05:50:04,384 - spline.components.tasks - Processing group of tasks (parallel=no)
    2018-08-16 05:50:04,386 - spline.components.tasks - Processing Bash code: start
    2018-08-16 05:50:04,397 - spline.tools.filters - render(syntax error): unexpected '.'
    2018-08-16 05:50:04,398 - spline.components.tasks - Pipeline has failed: leaving as soon as possible!
    

    The yaml contains {% raw %} and {% endraw %} which seems to be rendered twice and second time the remaining string contains {{.ID}} which cannot be resolved.

    When I remove the recursive replace all is fine.

    bug 
    opened by Nachtfeuer 7
  • Allow renderable `with` content.

    Allow renderable `with` content.

    This patch permits the with option, of tasks, to contain Jinja2 renderable content. This is useful when the models section contains data to be reused across a number of tasks.

    opened by jbenden 5
  • spline-stash: stash functionality (spline-server, part one)

    spline-stash: stash functionality (spline-server, part one)

    On the path to #63 (brainstorming: spline-server) my idea is to start as soon as possible using written functionality which is currently #64 (InMemoryFiles). With this I would like to introduce a new tool spline-stash that can store a path (recursively) at the stash-server and of course I would like to get back the content (usually on another location).

    I still have to make my thoughts about the command line syntax but I'm a bit inspired by git stash of course. The server can run either locally or at a server; the only thing you would have to do is to install spline as usual and to run the server. For the moment no configuration is involved.

    For the server I feel pretty comfortable (the moment) using Falcon (https://falconframework.org). I never used it before except running the given demo.

    enhancement 
    opened by Nachtfeuer 3
  • Conditional tasks

    Conditional tasks

    Considering that the pipeline uploads/deploys something this usually will happen on concrete branches only. As an example in some projects using Git the branch master is used as main line. Each time a pull request gets merged there a successful build should provide artifacts somewhere. The next task snippet could be the way to do it:

    - shell:
        script: echo "upload"
        when: "{{ env.BRANCH_NAME }}" == "master"
    
    enhancement 
    opened by Nachtfeuer 3
  • Asynchronous Bash execution

    Asynchronous Bash execution

    When executing a Bash script the output of it is passed back the Python tool after completion which has two effects:

    • for the time of execution of one Bash you see no output
    • the logging of the lines of the output get wrong timestamps

    It's not really a bug since the code does work.

    enhancement 
    opened by Nachtfeuer 3
  • Recursively expand Jinja2 templates

    Recursively expand Jinja2 templates

    This patch introduces automatic expansion of Jinja2 templates, except the Docker set-up related templates.

    A direct benefit of this patch is one's pipeline configuration becomes less complex; with many Jinja2 render filters used everywhere becoming unnecessary.

    opened by jbenden 2
  • Require usage of single quotes for docker -e parameter (otherwise subsitutions do happen)

    Require usage of single quotes for docker -e parameter (otherwise subsitutions do happen)

    Following problem:

    FOO="\${}" spline
    ...
    2018-06-13 15:15:08,595 - spline.components.bash - Running script /tmp/pipeline-script-Kj8mAR.sh
    2018-06-13 15:15:08,612 - spline.components.tasks -  | /tmp/pipeline-script-Kj8mAR.sh: line 24: FOO=${}: bad substitution
    2018-06-13 15:15:08,612 - spline.components.bash - Exit code has been 1
    ...
    

    Simple pipeline demo for that issue (as sufficient):

    pipeline:
      - stage(demo):
          - tasks(ordered):
            - docker(container):
                script: |
                  echo "hello"
    
    bug 
    opened by Nachtfeuer 2
  • Put curly braces and quotes on variables for Docker mounts

    Put curly braces and quotes on variables for Docker mounts

    As I were told it seems that docker run --rm -v $PWD:/mnt/host -it centos:7 bash -c "ls /mnt/host" has failed on OSX while using ${PWD} should have been working. The docker container template is required to be updated for it.

    bug 
    opened by Nachtfeuer 2
  • docker(image) task: failed to add files to image

    docker(image) task: failed to add files to image

    Because final scripts runs on temporary path Dockerfile commands like ADD are not able to fetch content from the source path.

    It turns out that providing the source path wouldn't help because Docker doesn't allow access to path outside the context (relative to the path where the build process is running)

    bug 
    opened by Nachtfeuer 2
  • New task: ansible(simple)

    New task: ansible(simple)

    • embedded solution
    • generator for one environment (inventory, group_vars, playbook)
    • dev, qa, or prod via model
    • script section is playbook
    • no roles
    • updating the presentation
    enhancement 
    opened by Nachtfeuer 2
  • Adding ssh agent support for Docker container

    Adding ssh agent support for Docker container

    • Automatically organize mount and environment variable when ssh agent is active only.
    • Basic procedure: -v $SSH_AUTH_SOCK:/ssh-agent --env SSH_AUTH_SOCK=/ssh-agent
    enhancement 
    opened by Nachtfeuer 2
  • Bump wheel from 0.29.0 to 0.38.1 in /examples/python/primes

    Bump wheel from 0.29.0 to 0.38.1 in /examples/python/primes

    Bumps wheel from 0.29.0 to 0.38.1.

    Changelog

    Sourced from wheel's changelog.

    Release Notes

    UNRELEASED

    • Updated vendored packaging to 22.0

    0.38.4 (2022-11-09)

    • Fixed PKG-INFO conversion in bdist_wheel mangling UTF-8 header values in METADATA (PR by Anderson Bravalheri)

    0.38.3 (2022-11-08)

    • Fixed install failure when used with --no-binary, reported on Ubuntu 20.04, by removing setup_requires from setup.cfg

    0.38.2 (2022-11-05)

    • Fixed regression introduced in v0.38.1 which broke parsing of wheel file names with multiple platform tags

    0.38.1 (2022-11-04)

    • Removed install dependency on setuptools
    • The future-proof fix in 0.36.0 for converting PyPy's SOABI into a abi tag was faulty. Fixed so that future changes in the SOABI will not change the tag.

    0.38.0 (2022-10-21)

    • Dropped support for Python < 3.7
    • Updated vendored packaging to 21.3
    • Replaced all uses of distutils with setuptools
    • The handling of license_files (including glob patterns and default values) is now delegated to setuptools>=57.0.0 (#466). The package dependencies were updated to reflect this change.
    • Fixed potential DoS attack via the WHEEL_INFO_RE regular expression
    • Fixed ValueError: ZIP does not support timestamps before 1980 when using SOURCE_DATE_EPOCH=0 or when on-disk timestamps are earlier than 1980-01-01. Such timestamps are now changed to the minimum value before packaging.

    0.37.1 (2021-12-22)

    • Fixed wheel pack duplicating the WHEEL contents when the build number has changed (#415)
    • Fixed parsing of file names containing commas in RECORD (PR by Hood Chatham)

    0.37.0 (2021-08-09)

    • Added official Python 3.10 support
    • Updated vendored packaging library to v20.9

    ... (truncated)

    Commits
    • 6f1608d Created a new release
    • cf8f5ef Moved news item from PR #484 to its proper place
    • 9ec2016 Removed install dependency on setuptools (#483)
    • 747e1f6 Fixed PyPy SOABI parsing (#484)
    • 7627548 [pre-commit.ci] pre-commit autoupdate (#480)
    • 7b9e8e1 Test on Python 3.11 final
    • a04dfef Updated the pypi-publish action
    • 94bb62c Fixed docs not building due to code style changes
    • d635664 Updated the codecov action to the latest version
    • fcb94cd Updated version to match the release
    • Additional commits viewable in compare view

    Dependabot compatibility score

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
    • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    • @dependabot use these labels will set the current labels as the default for future PRs for this repo and language
    • @dependabot use these reviewers will set the current reviewers as the default for future PRs for this repo and language
    • @dependabot use these assignees will set the current assignees as the default for future PRs for this repo and language
    • @dependabot use this milestone will set the current milestone as the default for future PRs for this repo and language

    You can disable automated security fix PRs for this repo from the Security Alerts page.

    dependencies 
    opened by dependabot[bot] 0
  • Bump wheel from 0.29.0 to 0.38.1

    Bump wheel from 0.29.0 to 0.38.1

    Bumps wheel from 0.29.0 to 0.38.1.

    Changelog

    Sourced from wheel's changelog.

    Release Notes

    UNRELEASED

    • Updated vendored packaging to 22.0

    0.38.4 (2022-11-09)

    • Fixed PKG-INFO conversion in bdist_wheel mangling UTF-8 header values in METADATA (PR by Anderson Bravalheri)

    0.38.3 (2022-11-08)

    • Fixed install failure when used with --no-binary, reported on Ubuntu 20.04, by removing setup_requires from setup.cfg

    0.38.2 (2022-11-05)

    • Fixed regression introduced in v0.38.1 which broke parsing of wheel file names with multiple platform tags

    0.38.1 (2022-11-04)

    • Removed install dependency on setuptools
    • The future-proof fix in 0.36.0 for converting PyPy's SOABI into a abi tag was faulty. Fixed so that future changes in the SOABI will not change the tag.

    0.38.0 (2022-10-21)

    • Dropped support for Python < 3.7
    • Updated vendored packaging to 21.3
    • Replaced all uses of distutils with setuptools
    • The handling of license_files (including glob patterns and default values) is now delegated to setuptools>=57.0.0 (#466). The package dependencies were updated to reflect this change.
    • Fixed potential DoS attack via the WHEEL_INFO_RE regular expression
    • Fixed ValueError: ZIP does not support timestamps before 1980 when using SOURCE_DATE_EPOCH=0 or when on-disk timestamps are earlier than 1980-01-01. Such timestamps are now changed to the minimum value before packaging.

    0.37.1 (2021-12-22)

    • Fixed wheel pack duplicating the WHEEL contents when the build number has changed (#415)
    • Fixed parsing of file names containing commas in RECORD (PR by Hood Chatham)

    0.37.0 (2021-08-09)

    • Added official Python 3.10 support
    • Updated vendored packaging library to v20.9

    ... (truncated)

    Commits
    • 6f1608d Created a new release
    • cf8f5ef Moved news item from PR #484 to its proper place
    • 9ec2016 Removed install dependency on setuptools (#483)
    • 747e1f6 Fixed PyPy SOABI parsing (#484)
    • 7627548 [pre-commit.ci] pre-commit autoupdate (#480)
    • 7b9e8e1 Test on Python 3.11 final
    • a04dfef Updated the pypi-publish action
    • 94bb62c Fixed docs not building due to code style changes
    • d635664 Updated the codecov action to the latest version
    • fcb94cd Updated version to match the release
    • Additional commits viewable in compare view

    Dependabot compatibility score

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
    • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    • @dependabot use these labels will set the current labels as the default for future PRs for this repo and language
    • @dependabot use these reviewers will set the current reviewers as the default for future PRs for this repo and language
    • @dependabot use these assignees will set the current assignees as the default for future PRs for this repo and language
    • @dependabot use this milestone will set the current milestone as the default for future PRs for this repo and language

    You can disable automated security fix PRs for this repo from the Security Alerts page.

    dependencies 
    opened by dependabot[bot] 0
  • [Snyk] Security upgrade setuptools from 39.0.1 to 65.5.1

    [Snyk] Security upgrade setuptools from 39.0.1 to 65.5.1

    This PR was automatically created by Snyk using the credentials of a real user.


    Snyk has created this PR to fix one or more vulnerable packages in the `pip` dependencies of this project.

    Changes included in this PR

    • Changes to the following files to upgrade the vulnerable dependencies to a fixed version:
      • requirements.txt
    โš ๏ธ Warning
    sphinxcontrib-inheritance 0.9.0 has requirement Sphinx<=1.4.9, but you have Sphinx 1.6.5.
    Sphinx 1.6.5 requires Pygments, which is not installed.
    astroid 1.6.6 requires lazy-object-proxy, which is not installed.
    
    

    Vulnerabilities that will be fixed

    By pinning:

    Severity | Priority Score (*) | Issue | Upgrade | Breaking Change | Exploit Maturity :-------------------------:|-------------------------|:-------------------------|:-------------------------|:-------------------------|:------------------------- low severity | 441/1000
    Why? Recently disclosed, Has a fix available, CVSS 3.1 | Regular Expression Denial of Service (ReDoS)
    SNYK-PYTHON-SETUPTOOLS-3113904 | setuptools:
    39.0.1 -> 65.5.1
    | No | No Known Exploit

    (*) Note that the real score may have changed since the PR was raised.

    Some vulnerabilities couldn't be fully fixed and so Snyk will still find them when the project is tested again. This may be because the vulnerability existed within more than one direct dependency, but not all of the affected dependencies could be upgraded.

    Check the changes in this PR to ensure they won't cause issues with your project.


    Note: You are seeing this because you or someone else with access to this repository has authorized Snyk to open fix PRs.

    For more information: ๐Ÿง View latest project report

    ๐Ÿ›  Adjust project settings

    ๐Ÿ“š Read more about Snyk's upgrade and patch logic


    Learn how to fix vulnerabilities with free interactive lessons:

    ๐Ÿฆ‰ Regular Expression Denial of Service (ReDoS)

    opened by Nachtfeuer 0
  • [Snyk] Security upgrade Pillow from 6.2.2 to 8.2.0

    [Snyk] Security upgrade Pillow from 6.2.2 to 8.2.0

    Snyk has created this PR to fix one or more vulnerable packages in the `pip` dependencies of this project.

    Changes included in this PR

    • Changes to the following files to upgrade the vulnerable dependencies to a fixed version:
      • requirements.txt
    โš ๏ธ Warning
    sphinxcontrib-inheritance 0.9.0 has requirement Sphinx<=1.4.9, but you have Sphinx 1.6.5.
    
    

    Vulnerabilities that will be fixed

    By pinning:

    Severity | Priority Score (*) | Issue | Upgrade | Breaking Change | Exploit Maturity :-------------------------:|-------------------------|:-------------------------|:-------------------------|:-------------------------|:------------------------- high severity | 661/1000
    Why? Recently disclosed, Has a fix available, CVSS 7.5 | Out-of-bounds Read
    SNYK-PYTHON-PILLOW-1292150 | Pillow:
    6.2.2 -> 8.2.0
    | No | No Known Exploit high severity | 661/1000
    Why? Recently disclosed, Has a fix available, CVSS 7.5 | Out-of-bounds Read
    SNYK-PYTHON-PILLOW-1292151 | Pillow:
    6.2.2 -> 8.2.0
    | No | No Known Exploit

    (*) Note that the real score may have changed since the PR was raised.

    Some vulnerabilities couldn't be fully fixed and so Snyk will still find them when the project is tested again. This may be because the vulnerability existed within more than one direct dependency, but not all of the effected dependencies could be upgraded.

    Check the changes in this PR to ensure they won't cause issues with your project.


    Note: You are seeing this because you or someone else with access to this repository has authorized Snyk to open fix PRs.

    For more information: ๐Ÿง View latest project report

    ๐Ÿ›  Adjust project settings

    ๐Ÿ“š Read more about Snyk's upgrade and patch logic

    opened by snyk-bot 0
  • [Snyk] Security upgrade Pillow from 6.2.2 to 8.2.0

    [Snyk] Security upgrade Pillow from 6.2.2 to 8.2.0

    Snyk has created this PR to fix one or more vulnerable packages in the `pip` dependencies of this project.

    Changes included in this PR

    • Changes to the following files to upgrade the vulnerable dependencies to a fixed version:
      • requirements.txt
    โš ๏ธ Warning
    sphinxcontrib-inheritance 0.9.0 has requirement Sphinx<=1.4.9, but you have Sphinx 1.6.5.
    
    

    Vulnerabilities that will be fixed

    By pinning:

    Severity | Priority Score (*) | Issue | Upgrade | Breaking Change | Exploit Maturity :-------------------------:|-------------------------|:-------------------------|:-------------------------|:-------------------------|:------------------------- high severity | 661/1000
    Why? Recently disclosed, Has a fix available, CVSS 7.5 | Denial of Service (DoS)
    SNYK-PYTHON-PILLOW-1090584 | Pillow:
    6.2.2 -> 8.2.0
    | No | No Known Exploit high severity | 661/1000
    Why? Recently disclosed, Has a fix available, CVSS 7.5 | Denial of Service (DoS)
    SNYK-PYTHON-PILLOW-1090586 | Pillow:
    6.2.2 -> 8.2.0
    | No | No Known Exploit high severity | 661/1000
    Why? Recently disclosed, Has a fix available, CVSS 7.5 | Denial of Service (DoS)
    SNYK-PYTHON-PILLOW-1090587 | Pillow:
    6.2.2 -> 8.2.0
    | No | No Known Exploit high severity | 661/1000
    Why? Recently disclosed, Has a fix available, CVSS 7.5 | Denial of Service (DoS)
    SNYK-PYTHON-PILLOW-1090588 | Pillow:
    6.2.2 -> 8.2.0
    | No | No Known Exploit

    (*) Note that the real score may have changed since the PR was raised.

    Some vulnerabilities couldn't be fully fixed and so Snyk will still find them when the project is tested again. This may be because the vulnerability existed within more than one direct dependency, but not all of the effected dependencies could be upgraded.

    Check the changes in this PR to ensure they won't cause issues with your project.


    Note: You are seeing this because you or someone else with access to this repository has authorized Snyk to open fix PRs.

    For more information: ๐Ÿง View latest project report

    ๐Ÿ›  Adjust project settings

    ๐Ÿ“š Read more about Snyk's upgrade and patch logic

    opened by snyk-bot 0
  • [Snyk] Security upgrade Pygments from 2.5.2 to 2.7.4

    [Snyk] Security upgrade Pygments from 2.5.2 to 2.7.4

    Snyk has created this PR to fix one or more vulnerable packages in the `pip` dependencies of this project.

    Changes included in this PR

    • Changes to the following files to upgrade the vulnerable dependencies to a fixed version:
      • requirements.txt
    โš ๏ธ Warning
    sphinxcontrib-inheritance 0.9.0 has requirement Sphinx<=1.4.9, but you have Sphinx 1.6.5.
    Sphinx 1.6.5 requires Pygments, which is not installed.
    
    

    Vulnerabilities that will be fixed

    By pinning:

    Severity | Priority Score (*) | Issue | Upgrade | Breaking Change | Exploit Maturity :-------------------------:|-------------------------|:-------------------------|:-------------------------|:-------------------------|:------------------------- high severity | 661/1000
    Why? Recently disclosed, Has a fix available, CVSS 7.5 | Denial of Service (DoS)
    SNYK-PYTHON-PYGMENTS-1088505 | Pygments:
    2.5.2 -> 2.7.4
    | No | No Known Exploit

    (*) Note that the real score may have changed since the PR was raised.

    Some vulnerabilities couldn't be fully fixed and so Snyk will still find them when the project is tested again. This may be because the vulnerability existed within more than one direct dependency, but not all of the effected dependencies could be upgraded.

    Check the changes in this PR to ensure they won't cause issues with your project.


    Note: You are seeing this because you or someone else with access to this repository has authorized Snyk to open fix PRs.

    For more information: ๐Ÿง View latest project report

    ๐Ÿ›  Adjust project settings

    ๐Ÿ“š Read more about Snyk's upgrade and patch logic

    opened by snyk-bot 0
Releases(1.12)
  • 1.12(Oct 26, 2018)

    You can install it with following command (for upgrade please add --upgrade):

    pip install spline
    

    One feature only: labels for docker(container) task.

    Source code(tar.gz)
    Source code(zip)
  • 1.11(Sep 3, 2018)

    You can install it with following command (for upgrade please add --upgrade):

    pip install spline
    
    • spline-loc tool (https://spline.readthedocs.io/en/latest/spline_loc.html)
    • bugfix for tool versions using spline on OSX
    • bugfix for escaping values of Bash variables
    • bugfix for closing of file handles (thanks to @jbenden)
    • bugfix for exec permission for user only (thanks to @jbenden)
    • extension to with field being renderable (thanks to @jbenden)
    Source code(tar.gz)
    Source code(zip)
  • 1.10.2(May 7, 2018)

    You can install it with following command (for upgrade please add --upgrade):

    pip install spline
    
    • break pipeline when required tools are missing
    • code reuse: !include support for main yaml
    • exclude copying some path variables (HOME, PYTHONPATH, JAVAHOME, PATH, ...)
    • rendering support for task title
    • some minor bugfixes (including quotes and curly braces for $PWD)
    • documentation and presentation extended
    Source code(tar.gz)
    Source code(zip)
  • 1.9(Apr 8, 2018)

    You can install it with following command (for upgrade please add --upgrade):

    pip install spline
    
    • new task: ansible(simple)
    • new task: packer
    • bugfix: variables field has not been available in templating (Jinja2) for Docker images
    • bugfix: path adjustment for Dockerfile to be in $PWD so that you can use ADD in Dockerfile for current source path
    • improvement: for ordered task execution enable immediate variable evaluation on next task
    • improvement: new optional field network for Docker container task
    • documentation as presentation updated (as usual)
    Source code(tar.gz)
    Source code(zip)
  • 1.8(Mar 11, 2018)

    You can install it with following command (for upgrade please add --upgrade):

    pip install spline
    
    • Bash strict mode (--strict)
    • Temporary scripts path adjustable (--temporary-scripts-path)
    • Forwarding SSH agent into Docker container
    • Fixed exit code (variable) for Python tasks
    • Documentation extended for Python development: http://spline.readthedocs.io/en/latest/development.html
    • Internal: Testing wheel file installation after build
    Source code(tar.gz)
    Source code(zip)
  • 1.7.2(Feb 1, 2018)

    You can install it with following command (for upgrade please add --upgrade):

    pip install spline
    

    Following content:

    • HTML report generation (with auto refresh) Details: http://spline.readthedocs.io/en/latest/report.html
    • tasks variables (writing output of a task into a variable being known py whole pipeline) Details: http://spline.readthedocs.io/en/latest/tasks.html#variables-on-tasks
    • condition tasks (allowing jinja templating and some concrete simple conditions) Details: http://spline.readthedocs.io/en/latest/conditions.html
    • You now can leave out --definition when you use a pipeline.yaml (it's now default)
    • Contribution guide added
    Source code(tar.gz)
    Source code(zip)
  • 1.6.1(Jan 6, 2018)

    You can install/upgrade it with following command:

    pip install spline [--upgrade]
    

    Following content:

    • New option --dry-run which does show all Bash script content instead of executing it
    • New option --debug which adds set -x to Bash script (verbose logging of Bash commands)
    • The tools now does log used cpu count
    • To some extend improved handling for unicode
    • Handling of Python 3.7 added to project (pipeline.yaml and examples/docker-image.yaml). Adding Python 3.7 to .travis.yml probably won't work since I detected issues with pylint and again with unicode. I tested following exact version: https://www.python.org/downloads/release/python-370a2/. Since it is called a "early developer preview" nothing to worry about yet.
    Source code(tar.gz)
    Source code(zip)
  • 1.5(Dec 27, 2017)

    You can install/upgrade it with following command:

    pip install spline [--upgrade]
    

    Following content:

    • New feature: Python task; read here: http://spline.readthedocs.io/en/latest/python.html
    • Better Code 10/10
    • Support for Pypy (2.7.x compatible) and Pypy3 (3.5.x compatible)
    • Bandit analyzer added
    • README (with badges) now also on PyPI
    • Code Of Conduct added
    Source code(tar.gz)
    Source code(zip)
  • 1.4(Dec 22, 2017)

    You can install/upgrade it with following command:

    pip install spline [--upgrade]
    

    Following content:

    • code complexity of spline reduced (radon and flake8)
    • coverage now clearly above 90% (roughly 94%)
    • pipeline definition for spline itself added (docker tests will be skipped there)
    • new feature docker(image)
    Source code(tar.gz)
    Source code(zip)
  • 1.3(Dec 20, 2017)

    You can install it with following command:

    pip install spline --upgrade
    
    • added with field for shell and docker container allowing to run same task with different data
    • fixed asynchronous bash execution
    • improved schema by replacing quite nice pykwalify with a bit more Pythonic schema solution being more independent from file format.
    Source code(tar.gz)
    Source code(zip)
  • 1.2(Dec 10, 2017)

    You can install it with following command:

    pip install spline --upgrade
    

    One main task was to have unitests in place. The project is currently at 86% code coverage. Another main task has been to provide a realistic demo on how to build a project with spline + Python + tox.

    Please check https://github.com/Nachtfeuer/pipeline/milestone/3?closed=1 for details. In addition one problem with PIPELINE_BASH_FILE and availability of model and env inside Docker container.

    The documentation of the first example can be seen here:

    • https://github.com/Nachtfeuer/pipeline/blob/master/docs/example.rst
    • or here: http://spline.readthedocs.io/en/latest/example.html
    Source code(tar.gz)
    Source code(zip)
Owner
Thomas Lehmann
Highly interested in development, programming languages, infrastructure, processes and design. Favor C++, Java and Python.
Thomas Lehmann
Just playing with getting CLIP Guided Diffusion running locally, rather than having to use colab.

CLIP-Guided-Diffusion Just playing with getting CLIP Guided Diffusion running locally, rather than having to use colab. Original colab notebooks by Ka

Nerdy Rodent 336 Dec 9, 2022
A modified version of DeepMind's Alphafold2 to divide CPU part (MSA and template searching) and GPU part (prediction model)

ParallelFold Author: Bozitao Zhong This is a modified version of DeepMind's Alphafold2 to divide CPU part (MSA and template searching) and GPU part (p

Bozitao Zhong 77 Dec 22, 2022
A (PyTorch) imbalanced dataset sampler for oversampling low frequent classes and undersampling high frequent ones.

Imbalanced Dataset Sampler Introduction In many machine learning applications, we often come across datasets where some types of data may be seen more

Ming 2k Jan 8, 2023
(Arxiv 2021) NeRF--: Neural Radiance Fields Without Known Camera Parameters

NeRF--: Neural Radiance Fields Without Known Camera Parameters Project Page | Arxiv | Colab Notebook | Data Zirui Wangยน, Shangzhe Wuยฒ, Weidi Xieยฒ, Min

Active Vision Laboratory 411 Dec 26, 2022
Unofficial & improved implementation of NeRF--: Neural Radiance Fields Without Known Camera Parameters

[Unofficial code-base] NeRF--: Neural Radiance Fields Without Known Camera Parameters [ Project | Paper | Official code base ] โฌ…๏ธ Thanks the original

Jianfei Guo 239 Dec 22, 2022
A list of papers about point cloud based place recognition, also known as loop closure detection in SLAM (processing)

A list of papers about point cloud based place recognition, also known as loop closure detection in SLAM (processing)

Xin Kong 17 May 16, 2021
Updated for TTS(CE) = Also Known as TTN V3. The code requires the first server to be 'ttn' protocol.

Updated Updated for TTS(CE) = Also Known as TTN V3. The code requires the first server to be 'ttn' protocol. Introduction This balenaCloud (previously

Remko 1 Oct 17, 2021
A Python Automated Machine Learning tool that optimizes machine learning pipelines using genetic programming.

Master status: Development status: Package information: TPOT stands for Tree-based Pipeline Optimization Tool. Consider TPOT your Data Science Assista

Epistasis Lab at UPenn 8.9k Dec 30, 2022
A simple image/video to Desmos graph converter run locally

Desmos Bezier Renderer A simple image/video to Desmos graph converter run locally Sample Result Setup Install dependencies apt update apt install git

Kevin JY Cui 339 Dec 23, 2022
Code for the CVPR2021 paper "Patch-NetVLAD: Multi-Scale Fusion of Locally-Global Descriptors for Place Recognition"

Patch-NetVLAD: Multi-Scale Fusion of Locally-Global Descriptors for Place Recognition This repository contains code for the CVPR2021 paper "Patch-NetV

QVPR 368 Jan 6, 2023
[CVPR'21] Locally Aware Piecewise Transformation Fields for 3D Human Mesh Registration

Locally Aware Piecewise Transformation Fields for 3D Human Mesh Registration This repository contains the implementation of our paper Locally Aware Pi

sfwang 70 Dec 19, 2022
Locally Enhanced Self-Attention: Rethinking Self-Attention as Local and Context Terms

LESA Introduction This repository contains the official implementation of Locally Enhanced Self-Attention: Rethinking Self-Attention as Local and Cont

Chenglin Yang 20 Dec 31, 2021
Locally Constrained Self-Attentive Sequential Recommendation

LOCKER This is the pytorch implementation of this paper: Locally Constrained Self-Attentive Sequential Recommendation. Zhankui He, Handong Zhao, Zhe L

Zhankui (Aaron) He 8 Jul 30, 2022
Locally cache assets that are normally streamed in POPULATION: ONE

Population One Localizer This is no longer needed as of the build shipped on 03/03/22, thank you bigbox :) Locally cache assets that are normally stre

Ahman Woods 2 Mar 4, 2022
A python script to dump all the challenges locally of a CTFd-based Capture the Flag.

A python script to dump all the challenges locally of a CTFd-based Capture the Flag. Features Connects and logins to a remote CTFd instance. Dumps all

Podalirius 77 Dec 7, 2022
Official implementation of the paper 'Details or Artifacts: A Locally Discriminative Learning Approach to Realistic Image Super-Resolution' in CVPR 2022

LDL Paper | Supplementary Material Details or Artifacts: A Locally Discriminative Learning Approach to Realistic Image Super-Resolution Jie Liang*, Hu

null 150 Dec 26, 2022
FLAVR is a fast, flow-free frame interpolation method capable of single shot multi-frame prediction

FLAVR is a fast, flow-free frame interpolation method capable of single shot multi-frame prediction. It uses a customized encoder decoder architecture with spatio-temporal convolutions and channel gating to capture and interpolate complex motion trajectories between frames to generate realistic high frame rate videos. This repository contains original source code for the paper accepted to CVPR 2021.

Tarun K 280 Dec 23, 2022
TorchGeo is a PyTorch domain library, similar to torchvision, that provides datasets, transforms, samplers, and pre-trained models specific to geospatial data.

TorchGeo is a PyTorch domain library, similar to torchvision, that provides datasets, transforms, samplers, and pre-trained models specific to geospatial data.

Microsoft 1.3k Dec 30, 2022
Fuzzification helps developers protect the released, binary-only software from attackers who are capable of applying state-of-the-art fuzzing techniques

About Fuzzification Fuzzification helps developers protect the released, binary-only software from attackers who are capable of applying state-of-the-

gts3.org (SSLab@Gatech) 55 Oct 25, 2022