Allows including an action inside another action (by preprocessing the Yaml file). This is how composite actions should have worked.

Overview

actions-includes

Allows including an action inside another action (by preprocessing the Yaml file).

Instead of using uses or run in your action step, use the keyword includes.

Once you are using the includes argument, the workflows can be expanded using the tool like follows;

# python -m actions_include <input-workflow-with-includes> <output-workflow-flattened>
python -m actions_includes ./.github/workflows-src/workflow-a.yml ./.github/workflows/workflow-a.yml

includes: step

steps:
- name: Other step
  run: |
    command

- includes: {action-name}
  with:
    {inputs}

- name: Other step
  run: |
    command

The {action-name} follows the same syntax as the standard GitHub action uses and the action referenced should look exactly like a GitHub "composite action" except runs.using should be includes.

For example;

  • {owner}/{repo}@{ref} - Public action in github.com/{owner}/{repo}
  • {owner}/{repo}/{path}@{ref} - Public action under {path} in github.com/{owner}/{repo}.
  • ./{path} - Local action under local {path}, IE ./.github/actions/my-action`.

As it only makes sense to reference composite actions, the docker:// form isn't supported.

As you frequently want to include local actions, actions-includes extends the {action-name} syntax to also support;

  • /{name} - Local action under ./.github/actions/{name}.

This is how composite actions should have worked.

includes-script: step

File: script.py

print('Hello world')

File: workflow.yml

steps:
- name: Other step
  run: |
    command

- name: Hello
  includes-script: script.py

- name: Other step
  run: |
    command

python -m actions_includes.py workflow.in.yml workflow.out.yml

File: oworkflow.out.yml

steps:
- name: Other step
  run: |
    command

- name: Hello
  shell: python
  run: |
    print('Hello world')

- name: Other step
  run: |
    command
Comments
  • Convert action to use docker

    Convert action to use docker

    The tool needs;

    • Latest Python version to get order preserving dicts.
    • python3-yaml installed.

    These dependencies mean it would work well as a docker action.

    opened by mithro 4
  • includes-script in included actions

    includes-script in included actions

    It would be nice to be able to use includes-script in actions included with includes.

    For example:

    .github/workflows-src/workflow.yml

    jobs:
      job1:
        steps:
        - includes: /an_action
    

    .github/actions/an_action/action.yml (or maybe .github/actions-src/an_action/action.yml)

    runs:
      using: "includes"
      steps:
        - name: An action
          includes-script: script.py
    

    .github/actions/an_action/script.py (or maybe .github/actions-src/an_action/script.py)

        print('Hello world')
    

    would result in this workflow:

    .github/workflows-src/workflow.yml

    jobs:
      job1:
        steps:
        - name: An action
          shell: python
          run: |
             print('Hello world')
    
    opened by drdanz 3
  • Latest version is broken

    Latest version is broken

    non existing include_action referenced: https://github.com/mithro/actions-includes/blob/bb6134d1033d42e7d591645a756a3c49442be480/actions_includes/init.py#L963

    line 954: set only if running in this repo line 963: used

    opened by awoimbee 2
  • Add tests for the GitHub expression evaluator / simplifier code

    Add tests for the GitHub expression evaluator / simplifier code

    The tool does some nice stuff around if an input makes a step's if statement a static value, then it removes the step (if: false) or if: condition when if: true.

    opened by mithro 1
  • comment_handling attribute gone in ruamel?

    comment_handling attribute gone in ruamel?

      File "/usr/local/lib/python3.9/site-packages/ruamel/yaml/parser.py", line 637, in parse_block_mapping_value
        return self.parse_block_node_or_indentless_sequence()
      File "/usr/local/lib/python3.9/site-packages/ruamel/yaml/parser.py", line 351, in parse_block_node_or_indentless_sequence
        return self.parse_node(block=True, indentless_sequence=True)
      File "/usr/local/lib/python3.9/site-packages/ruamel/yaml/parser.py", line 414, in parse_node
        if self.loader and self.loader.comment_handling is None:
    AttributeError: 'RoundTripLoaderWithExp' object has no attribute 'comment_handling'
    Source of .github/workflows/test.actions-ifexpands.yml is ../../tests/workflows/actions-ifexpands.yml found at mithro/actions-includes/tests/workflows/actions-ifexpands.yml@d3712aa248266971c5aa89b44ff4e5044f259e8d
    
    opened by mithro 0
  • update README

    update README

    The README previously stated incorrectly that /{action-name} would be mapped to ./.github/actions/{action-name}, when it is actually mapped to ./.github/includes/actions/{action-name}.

    Since ./github/includes/actions/... seems to be used extensively in the actions-includes code, it seems best/easiest to update the README to remove this claim. Users can just use the normal filepath syntax: ./.github/actions/{action-name}

    opened by iacobfred 0
  • Large rework of how include processing works.

    Large rework of how include processing works.

    • Change to using ruamel's RoundTrip tools (RoundTripConstructor, RoundTripRepresenter, RoundTripDumper).

    • Use the CommentedXXXX structures from ruamel and enable them to be more easily pretty printed.

    • Allow Yaml merge keys (<< : {}) to be used with the inputs substitution.

    • Use path of .github/includes/(actions|workflows) as the default location for included actions / workflows.

    opened by mithro 0
  • Do not overwrite shell and support a few more extensions

    Do not overwrite shell and support a few more extensions

    • Support a few more file extensions:

      • .ps1 -> pwsh [1]
      • .cmd -> cmd [1]
      • .pl -> perl {0} [2]
      • .cmake -> cmake -P {0} [3]
    • Do not overwrite shell if explicitly set by the user

    opened by drdanz 0
  • Adding full expression simplifier.

    Adding full expression simplifier.

    Now expressions will be simplified if their values are already known.

    Produces much nicer output workflows with steps that would never execute removed.

    Fixes #5. Fixes #8.

    opened by mithro 0
  • Auto replace full expanded expressions in strings

    Auto replace full expanded expressions in strings

    Current if you have

    - name: 📤 Publish ${{ inputs.type }} to Test PyPI
    

    and you set inputs.type: wheels you end up with

    - name: 📤 Publish ${{ 'wheels' }} to Test PyPI
    

    It would be nicer if you ended up with;

    - name: 📤 Publish wheels to Test PyPI
    
    opened by mithro 0
  • Support reusable workflows

    Support reusable workflows

    Reusable workflows are a new feature [1] that allow a workflow to be dispatched from another workflow.

    Using reusable workflows involves creating a job with a uses key and no steps key [2].

    This change allows these jobs to pass silently through actions-includes instead of causing a failure.

    [1] https://github.blog/2021-11-29-github-actions-reusable-workflows-is-generally-available/ [2] https://docs.github.com/en/actions/learn-github-actions/reusing-workflows#calling-a-reusable-workflow

    opened by tdsmith 1
  • UnicodeEncodeError: 'charmap' codec can't encode character '\u23f0' in position 328: character maps to <undefined>_

    UnicodeEncodeError: 'charmap' codec can't encode character '\u23f0' in position 328: character maps to _

    I have a very simple test here: https://github.com/devkeydet/actions-includes-test/blob/main/.github/workflows-src/convertme.yml

    I installed actions-includes using pip. I run the following command:

    python -m actions_includes ./.github/workflows-src/convertme.yml ./.github/workflows/converted.yml

    I get the following error: _Expanding ./.github/workflows-src/convertme.yml into ./.github/workflows/converted.yml Expanding workflow file from: C:\github\actions-includes-test.github\workflows-src\convertme.yml to: .github\workflows\converted.yml Loading yaml file C:\github\actions-includes-test.github\workflows-src\convertme.yml with contents md5 of 7ce0775b6ab05a87deb53ae9d298df80 get_action_data: C:\github\actions-includes-test.github\workflows-src\convertme.yml ./actions/basic C:\github\actions-includes-test\actions\basic Including: C:\github\actions-includes-test\actions\basic/action.yml Loading yaml file C:\github\actions-includes-test\actions\basic/action.yml with contents md5 of bb475f262af8e4df34241a188a4f451f Traceback (most recent call last): File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.9_3.9.1776.0_x64__qbz5n2kfra8p0\lib\runpy.py", line 197, in run_module_as_main return run_code(code, main_globals, None, File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.9_3.9.1776.0_x64__qbz5n2kfra8p0\lib\runpy.py", line 87, in run_code exec(code, run_globals) File "C:\Users\Marc\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.9_qbz5n2kfra8p0\LocalCache\local-packages\Python39\site-packages\actions_includes_main.py", line 25, in sys.exit(main()) File "C:\Users\Marc\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.9_qbz5n2kfra8p0\LocalCache\local-packages\Python39\site-packages\actions_includes_init.py", line 1035, in main f.write(out_data) File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.9_3.9.1776.0_x64__qbz5n2kfra8p0\lib\encodings\cp1252.py", line 19, in encode return codecs.charmap_encode(input,self.errors,encoding_table)[0] UnicodeEncodeError: 'charmap' codec can't encode character '\u23f0' in position 328: character maps to

    opened by devkeydet 0
  • Doesn't handle

    Doesn't handle "toJSON" function

    I have this in my main src action (not in the include file) and it causes an error:

            env:
              GITHUB_CONTEXT: ${{ toJSON(github) }}
    

    Error dump is:

    Traceback (most recent call last):
      File "/usr/local/lib/python3.9/runpy.py", line 197, in _run_module_as_main
        return _run_code(code, main_globals, None,
      File "/usr/local/lib/python3.9/runpy.py", line 87, in _run_code
        exec(code, run_globals)
      File "/usr/local/lib/python3.9/site-packages/actions_includes/__main__.py", line 25, in <module>
        sys.exit(main())
      File "/usr/local/lib/python3.9/site-packages/actions_includes/__init__.py", line 1032, in main
        out_data = expand_workflow(current_action, to_path, insert_check)
      File "/usr/local/lib/python3.9/site-packages/actions_includes/__init__.py", line 917, in expand_workflow
        data = yaml_load(current_workflow, '\n'.join(workflow_data))
      File "/usr/local/lib/python3.9/site-packages/actions_includes/__init__.py", line 818, in yaml_load
        return yaml.load(yaml_data, Loader=RoundTripLoaderWithExp)
      File "/usr/local/lib/python3.9/site-packages/ruamel/yaml/main.py", line 1071, in load
        return loader._constructor.get_single_data()
      File "/usr/local/lib/python3.9/site-packages/ruamel/yaml/constructor.py", line 122, in get_single_data
        return self.construct_document(node)
      File "/usr/local/lib/python3.9/site-packages/ruamel/yaml/constructor.py", line 132, in construct_document
        for _dummy in generator:
      File "/usr/local/lib/python3.9/site-packages/ruamel/yaml/constructor.py", line 1617, in construct_yaml_map
        self.construct_mapping(node, data, deep=True)
      File "/usr/local/lib/python3.9/site-packages/actions_includes/__init__.py", line 730, in construct_mapping
        return RoundTripConstructor.construct_mapping(self, node, maptyp, deep)
      File "/usr/local/lib/python3.9/site-packages/ruamel/yaml/constructor.py", line 1500, in construct_mapping
        value = self.construct_object(value_node, deep=deep)
      File "/usr/local/lib/python3.9/site-packages/ruamel/yaml/constructor.py", line 155, in construct_object
        data = self.construct_non_recursive_object(node)
      File "/usr/local/lib/python3.9/site-packages/ruamel/yaml/constructor.py", line 197, in construct_non_recursive_object
        for _dummy in generator:
      File "/usr/local/lib/python3.9/site-packages/ruamel/yaml/constructor.py", line 1617, in construct_yaml_map
        self.construct_mapping(node, data, deep=True)
      File "/usr/local/lib/python3.9/site-packages/actions_includes/__init__.py", line 730, in construct_mapping
        return RoundTripConstructor.construct_mapping(self, node, maptyp, deep)
      File "/usr/local/lib/python3.9/site-packages/ruamel/yaml/constructor.py", line 1500, in construct_mapping
        value = self.construct_object(value_node, deep=deep)
      File "/usr/local/lib/python3.9/site-packages/ruamel/yaml/constructor.py", line 155, in construct_object
        data = self.construct_non_recursive_object(node)
      File "/usr/local/lib/python3.9/site-packages/ruamel/yaml/constructor.py", line 197, in construct_non_recursive_object
        for _dummy in generator:
      File "/usr/local/lib/python3.9/site-packages/ruamel/yaml/constructor.py", line 1617, in construct_yaml_map
        self.construct_mapping(node, data, deep=True)
      File "/usr/local/lib/python3.9/site-packages/actions_includes/__init__.py", line 730, in construct_mapping
        return RoundTripConstructor.construct_mapping(self, node, maptyp, deep)
      File "/usr/local/lib/python3.9/site-packages/ruamel/yaml/constructor.py", line 1500, in construct_mapping
        value = self.construct_object(value_node, deep=deep)
      File "/usr/local/lib/python3.9/site-packages/ruamel/yaml/constructor.py", line 155, in construct_object
        data = self.construct_non_recursive_object(node)
      File "/usr/local/lib/python3.9/site-packages/ruamel/yaml/constructor.py", line 197, in construct_non_recursive_object
        for _dummy in generator:
      File "/usr/local/lib/python3.9/site-packages/ruamel/yaml/constructor.py", line 1609, in construct_yaml_seq
        data.extend(self.construct_rt_sequence(node, data))
      File "/usr/local/lib/python3.9/site-packages/ruamel/yaml/constructor.py", line 1341, in construct_rt_sequence
        ret_val.append(self.construct_object(child, deep=deep))
      File "/usr/local/lib/python3.9/site-packages/ruamel/yaml/constructor.py", line 155, in construct_object
        data = self.construct_non_recursive_object(node)
      File "/usr/local/lib/python3.9/site-packages/ruamel/yaml/constructor.py", line 197, in construct_non_recursive_object
        for _dummy in generator:
      File "/usr/local/lib/python3.9/site-packages/ruamel/yaml/constructor.py", line 1617, in construct_yaml_map
        self.construct_mapping(node, data, deep=True)
      File "/usr/local/lib/python3.9/site-packages/actions_includes/__init__.py", line 730, in construct_mapping
        return RoundTripConstructor.construct_mapping(self, node, maptyp, deep)
      File "/usr/local/lib/python3.9/site-packages/ruamel/yaml/constructor.py", line 1500, in construct_mapping
        value = self.construct_object(value_node, deep=deep)
      File "/usr/local/lib/python3.9/site-packages/ruamel/yaml/constructor.py", line 155, in construct_object
        data = self.construct_non_recursive_object(node)
      File "/usr/local/lib/python3.9/site-packages/ruamel/yaml/constructor.py", line 197, in construct_non_recursive_object
        for _dummy in generator:
      File "/usr/local/lib/python3.9/site-packages/ruamel/yaml/constructor.py", line 1617, in construct_yaml_map
        self.construct_mapping(node, data, deep=True)
      File "/usr/local/lib/python3.9/site-packages/actions_includes/__init__.py", line 730, in construct_mapping
        return RoundTripConstructor.construct_mapping(self, node, maptyp, deep)
      File "/usr/local/lib/python3.9/site-packages/ruamel/yaml/constructor.py", line 1500, in construct_mapping
        value = self.construct_object(value_node, deep=deep)
      File "/usr/local/lib/python3.9/site-packages/ruamel/yaml/constructor.py", line 155, in construct_object
        data = self.construct_non_recursive_object(node)
      File "/usr/local/lib/python3.9/site-packages/ruamel/yaml/constructor.py", line 190, in construct_non_recursive_object
        data = constructor(self, node)
      File "/usr/local/lib/python3.9/site-packages/actions_includes/__init__.py", line 687, in construct_expression
        v = exp.parse(v)
      File "/usr/local/lib/python3.9/site-packages/actions_includes/expressions.py", line 1346, in parse
        return simplify(exp[3:-2].strip())
      File "/usr/local/lib/python3.9/site-packages/actions_includes/expressions.py", line 1301, in simplify
        o = tokens_eval(tokenizer(exp), context)
      File "/usr/local/lib/python3.9/site-packages/actions_includes/expressions.py", line 373, in tokens_eval
        assert not isinstance(t, list), t
    AssertionError: [<class 'exp.ToJSONF'>, Value(github)]
    
    opened by mike-potter 0
  • Support for processing entire folder or wildcard

    Support for processing entire folder or wildcard

    Rather than using

    python -m actions_includes ./.github/src/my-action.yml ./.github/workflows/my-action.yml
    

    individually for each action, it would be nice if it supported folders, such as:

    python -m actions_includes ./.github/src ./.github/workflows
    

    or wildcards:

    python -m actions_includes ./.github/src/*.yml ./.github/workflows/*.yml
    
    opened by mike-potter 0
  • Requires running from a top-level git repository

    Requires running from a top-level git repository

    When using the docker container run command to execute, it gives this error:

    fatal: not a git repository (or any parent up to mount point /github)
    Stopping at filesystem boundary (GIT_DISCOVERY_ACROSS_FILESYSTEM not set).
    Traceback (most recent call last):
      File "/usr/local/lib/python3.9/runpy.py", line 197, in _run_module_as_main
        return _run_code(code, main_globals, None,
      File "/usr/local/lib/python3.9/runpy.py", line 87, in _run_code
        exec(code, run_globals)
      File "/usr/local/lib/python3.9/site-packages/actions_includes/__main__.py", line 25, in <module>
        sys.exit(main())
      File "/usr/local/lib/python3.9/site-packages/actions_includes/__init__.py", line 999, in main
        git_root_output = subprocess.check_output(
      File "/usr/local/lib/python3.9/subprocess.py", line 424, in check_output
        return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
      File "/usr/local/lib/python3.9/subprocess.py", line 528, in run
        raise CalledProcessError(retcode, process.args,
    subprocess.CalledProcessError: Command '['git', 'rev-parse', '--show-toplevel']' returned non-zero exit status 128.
    

    if you run it from a folder that is not a git repository. Would be nice to remove the git dependency here. As a yml processor it should just be talking the input src files and outputting the resulting workflow files regardless of whether I'm in a git checkout repo or not.

    opened by mike-potter 0
  • Why does the included action need to be action.yml?

    Why does the included action need to be action.yml?

    I have an include file within .github/includes/my-action.yml

    When using

        steps:
          - includes: my-action
    

    it tries to find .github/includes/actions/my-action/action.yml file, which doesn't exist.

    If I try to use:

        steps:
          - includes: .github/includes/my-action
    

    Then it tries to find .github/includes/my-action/action.yml file, which doesn't exist.

    If I try to use:

        steps:
          - includes: .github/includes/my-action.yml
    

    Then it tries to find .github/includes/my-action.yml/action.yml file, which doesn't exist.

    In all cases, it seems to have this action.yml filename hardcoded somewhere. Would be nice if I could just point to a specific file and not make assumptions about folder structure or file names.

    opened by mike-potter 0
Owner
Tim Ansell
Founder and Leader of @timvideos
Tim Ansell
I tried to apply the CAM algorithm to YOLOv4 and it worked.

YOLOV4:You Only Look Once目标检测模型在pytorch当中的实现 2021年2月7日更新: 加入letterbox_image的选项,关闭letterbox_image后网络的map得到大幅度提升。 目录 性能情况 Performance 实现的内容 Achievement

null 55 Dec 5, 2022
Automatic packaging of the open-composite libs for OvGME

OvGME Packager for OpenXR – OpenComposite for DCS Note This repository is currently unsupported and needs to be migrated to the upstream OpenComposite

null 12 Nov 3, 2022
An image base contains 490 images for learning (400 cars and 90 boats), and another 21 images for testingAn image base contains 490 images for learning (400 cars and 90 boats), and another 21 images for testing

SVM Données Une base d’images contient 490 images pour l’apprentissage (400 voitures et 90 bateaux), et encore 21 images pour fait des tests. Prétrait

Achraf Rahouti 3 Nov 30, 2021
Have you ever wondered how cool it would be to have your own A.I

Have you ever wondered how cool it would be to have your own A.I. assistant Imagine how easier it would be to send emails without typing a single word, doing Wikipedia searches without opening web browsers, and performing many other daily tasks like playing music with the help of a single voice command.

Harsh Gupta 1 Nov 9, 2021
A Runtime method overload decorator which should behave like a compiled language

strongtyping-pyoverload A Runtime method overload decorator which should behave like a compiled language there is a override decorator from typing whi

null 20 Oct 31, 2022
TorchIO is a Medical image preprocessing and augmentation toolkit for deep learning. Part of the PyTorch Ecosystem.

Medical image preprocessing and augmentation toolkit for deep learning. Part of the PyTorch Ecosystem.

Fernando Pérez-García 1.6k Jan 6, 2023
NVIDIA Merlin is an open source library providing end-to-end GPU-accelerated recommender systems, from feature engineering and preprocessing to training deep learning models and running inference in production.

NVIDIA Merlin NVIDIA Merlin is an open source library designed to accelerate recommender systems on NVIDIA’s GPUs. It enables data scientists, machine

null 419 Jan 3, 2023
DWIPrep is a robust and easy-to-use pipeline for preprocessing of diverse dMRI data.

DWIPrep: A Robust Preprocessing Pipeline for dMRI Data DWIPrep is a robust and easy-to-use pipeline for preprocessing of diverse dMRI data. The transp

Gal Ben-Zvi 1 Jan 9, 2023
Kinetics-Data-Preprocessing

Kinetics-Data-Preprocessing Kinetics-400 and Kinetics-600 are common video recognition datasets used by popular video understanding projects like Slow

Kaihua Tang 7 Oct 27, 2022
Aiming at the common training datsets split, spectrum preprocessing, wavelength select and calibration models algorithm involved in the spectral analysis process

Aiming at the common training datsets split, spectrum preprocessing, wavelength select and calibration models algorithm involved in the spectral analysis process, a complete algorithm library is established, which is named opensa (openspectrum analysis).

Fu Pengyou 50 Jan 7, 2023
Fast and customizable reconnaissance workflow tool based on simple YAML based DSL.

Fast and customizable reconnaissance workflow tool based on simple YAML based DSL, with support of notifications and distributed workload of that work

Américo Júnior 3 Mar 11, 2022
PyQt6 configuration in yaml format providing the most simple script.

PyamlQt(ぴゃむるきゅーと) PyQt6 configuration in yaml format providing the most simple script. Requirements yaml PyQt6, ( PyQt5 ) Installation pip install Pya

Ar-Ray 7 Aug 15, 2022
A way to store images in YAML.

YAMLImg A way to store images in YAML. I made this after seeing Roadcrosser's JSON-G because it was too inspiring to ignore this opportunity. Installa

null 5 Mar 14, 2022
Official implementation of ACTION-Net: Multipath Excitation for Action Recognition (CVPR'21).

ACTION-Net Official implementation of ACTION-Net: Multipath Excitation for Action Recognition (CVPR'21). Getting Started EgoGesture data folder struct

V-Sense 171 Dec 26, 2022
Official Pytorch Implementation of 'Learning Action Completeness from Points for Weakly-supervised Temporal Action Localization' (ICCV-21 Oral)

Learning-Action-Completeness-from-Points Official Pytorch Implementation of 'Learning Action Completeness from Points for Weakly-supervised Temporal A

Pilhyeon Lee 67 Jan 3, 2023
Human Action Controller - A human action controller running on different platforms.

Human Action Controller (HAC) Goal A human action controller running on different platforms. Fun Easy-to-use Accurate Anywhere Fun Examples Mouse Cont

null 27 Jul 20, 2022
The official TensorFlow implementation of the paper Action Transformer: A Self-Attention Model for Short-Time Pose-Based Human Action Recognition

Action Transformer A Self-Attention Model for Short-Time Human Action Recognition This repository contains the official TensorFlow implementation of t

PIC4SeRCentre 20 Jan 3, 2023