Software build automation tool for Python.

Overview

PyBuilder — an easy-to-use build automation tool for Python

Follow PyBuilder on Twitter Gitter Build Status Coverage Status

PyBuilder Version PyBuilder Python Versions PyBuilder Downloads Per Day PyBuilder Downloads Per Week PyBuilder Downloads Per Month

PyBuilder is a software build tool written in 100% pure Python, mainly targeting Python applications.

PyBuilder is based on the concept of dependency based programming, but it also comes with a powerful plugin mechanism, allowing the construction of build life cycles similar to those known from other famous (Java) build tools.

PyBuilder is running on the following versions of Python: 2.7, 3.5, 3.6, 3.7, 3.8, 3.9 and PyPy 2.7, 3.5 and 3.6.

See the Travis Build for version specific output.

Installing

PyBuilder is available using pip:

$ pip install pybuilder

For development builds use:

$ pip install --pre pybuilder

See the PyPI for more information.

Getting started

PyBuilder emphasizes simplicity. If you want to build a pure Python project and use the recommended directory layout, all you have to do is create a file build.py with the following content:

from pybuilder.core import use_plugin

use_plugin("python.core")
use_plugin("python.unittest")
use_plugin("python.coverage")
use_plugin("python.distutils")

default_task = "publish"

See the PyBuilder homepage for more details and a list of plugins.

Release Notes

The release notes can be found here. There will also be a git tag with each release. Please note that we do not currently promote tags to GitHub "releases".

Development

See Developing PyBuilder

Comments
  • Unable to specify version for custom plugin.

    Unable to specify version for custom plugin.

    I'm developing a custom plugin and can't get pybuilder to pull new versions when they're available. How do I specify min and/or locked version for a plugin pulled from a pypi repository?

    bug 
    opened by mrburrito 40
  • OSX + homebrew python@3.9: pybuilder/_vendor/virtualenv.py change_prefix does not consider symlinks, raises invalid AssertionError

    OSX + homebrew [email protected]: pybuilder/_vendor/virtualenv.py change_prefix does not consider symlinks, raises invalid AssertionError

    I have installed Python 3 on macOS 11.3 (Big Sur) with brew install [email protected]. Homebrew installs Python 3.9 under /usr/local/Cellar/[email protected], and symlinks /usr/local/opt/[email protected] -> ../Cellar/[email protected]/3.9.4.

    python3 is a symlink /usr/local/bin/python3 -> ../Cellar/[email protected]/3.9.4/bin/python3. pip3 is a symlink /usr/local/bin/pip3 -> ../Cellar/[email protected]/3.9.4/bin/pip3.

    $ head -1 $(which pip3)
    #!/usr/local/opt/[email protected]/bin/python3.9
    

    I used pip3 install --user pybuilder to install pybuilder:

    $ which pyb
    /Users/username/Library/Python/3.9/bin/pyb
    $ head -1 $(which pyb)
    #!/usr/local/opt/[email protected]/bin/python3.9
    
    $ pyb
    PyBuilder version 0.12.10
    Build started at 2021-04-28 10:36:15
    ------------------------------------------------------------
    ------------------------------------------------------------
    BUILD FAILED - AssertionError: Filename /usr/local/Cellar/[email protected]/3.9.4/Frameworks/Python.framework/Versions/3.9/lib/python3.9/os.py does not start with any of these prefixes: ['/usr/local/opt/[email protected]/Frameworks/Python.framework/Versions/3.9/Extras/lib/python', '/usr/local/opt/[email protected]/Frameworks/Python.framework/Versions/3.9', '/usr/local/opt/[email protected]/Frameworks/Python.framework/Versions/3.9', '/Users/username/Library/Python/3.9/lib/python/site-packages', '/Users/username/.local/lib/python/3.9/site-packages', '/Users/username/Library/Python/3.9/site-packages', '/Library/Python/3.9/site-packages'] (pybuilder/_vendor/virtualenv.py:1272)
    ------------------------------------------------------------
    Build finished at 2021-04-28 10:36:15
    Build took 0 seconds (64 ms)
    

    AssertionError is raised from change_prefix:

    https://github.com/pybuilder/pybuilder/blob/6ac1ef4e3f504fec7d4f86378fde1c5420d6a261/src/main/python/pybuilder/_vendor/virtualenv.py#L1272

    bug 
    opened by hoxu 39
  • Generate description from README, README.md, README.txt

    Generate description from README, README.md, README.txt

    As a python developer I want to create a nice description for PyPI. Nearly every GitHub project comes with a README.md (or something like that). It would be nice to have a simple way to generate the PyPI description from the GitHub flavored markdown / text file. Within build.py

    description = description_from_text_file("README")
    

    or

    description = description_from_github_markdown_file("README.md")
    
    enhancement 
    opened by aelgru 33
  • pybuilder 0.11.12 could not be installed in Python 2.7

    pybuilder 0.11.12 could not be installed in Python 2.7

    [INFO]  Writing MANIFEST.in as /build/python-pybuilder/src/pybuilder-0.11.12-py2/target/dist/pybuilder-0.11.12/MANIFEST.in
    [INFO]  Writing setup.py as /build/python-pybuilder/src/pybuilder-0.11.12-py2/target/dist/pybuilder-0.11.12/setup.py
    ------------------------------------------------------------
    BUILD FAILED - 'ascii' codec can't encode character u'\u201c' in position 5932: ordinal not in range(128)
    ------------------------------------------------------------
    Build finished at 2017-11-30 23:14:44
    Build took 3 seconds (3610 ms)
    

    Python 3 installation works fine, though. The installation command I used was:

    python2 setup.py install --root="$pkgdir" -O1
    
    opened by felixonmars 31
  • Automatically upload .dev* releases to pypi as part of the travis CI chain

    Automatically upload .dev* releases to pypi as part of the travis CI chain

    As suggested by @esc if our automated tests pass on travis then we can do a pre-release so users can pip install --pre pybuilder.

    Ideally there is a way to do it only if tests on all python versions passed... I'm not sure if there is though.

    enhancement infrastructure 
    opened by mriehl 29
  • 0.11.* coverage plugin causes issues during build deleting sys modules

    0.11.* coverage plugin causes issues during build deleting sys modules

    Hello,

    I think the new coverage plugin should be refactored, as it's a potential source of problems. Looks like it's deleting a bunch of global modules, sparing only the ones that it consider "essential". We have a custom python installation, the coverage plugin gets a bit confused and considers non essential things like datetime. We end up with errors like:

    Traceback (most recent call last): File "/tmp/test/env-infra/bin/pyb", line 25, in sys.exit(pybuilder.cli.main(*sys.argv[1:])) File "/tmp/test/env-infra/lib/python2.7/site-packages/pybuilder/cli.py", line 366, in main end = datetime.datetime.now() AttributeError: 'module' object has no attribute 'datetime'

    Because of this issue we are unable to update to 0.11.1 I know there should be a way to fix on our side by reverse-engineering your code and put modules in strategic locations, but honestly nothing in pybuilder (nothing in anything IMHO) should play with sys.modules, as it will cause issues that are a nightmare to debug.

    Downgrading to 0.10.63 fixes the issue obviously, but the install task would be nice to have for our purposes.

    bug 
    opened by MirkoRossini 23
  • Pathing issues on pypi upload on Windows

    Pathing issues on pypi upload on Windows

    BUILD FAILED - [Errno 22] invalid mode ('w') or filename: 'c:\Users\jcorson\pythonWorkspace\projectNameHere\target/reports\distutils\sdist__upload__-r__http:123.123.120.117:8080'

    I am on windows working in the Git Bash. pybuilder version 0.11.8. I saw another closed issue about pathing and you made him upgrade but this is very recent

    This issue didn't exist when it was the public PyPI server but I changed it to a local pypi that I need to store my project in. As a result, it seems to append the -r and the URL I provided

    project.set_property("distutils_upload_repository", 'http://123.123.120.117:8080')

    bug windows specific 
    opened by nexeh 22
  • Release 0.11.3

    Release 0.11.3

    This is a release prep for milestone 0.11.3.

    • [x] Bugs and features reviewed and signed off
    • [x] Do we want anything else for this release?

    @mriehl @esc @locolupo

    enhancement question code-review-required 
    opened by arcivanov 22
  • Release v0.11.0

    Release v0.11.0

    I think we should get a new release out - the improvements by @arcivanov are too sweet to pass on (xml reports are a boon for jenkins users) and there are a couple of bug fixes too.

    I also think we should go for 0.11.0 since there are some deprecated properties and coverage might be affected if some task was using library functions from the covered project.

    @locolupo @arcivanov what do you think? It would be great if you want to push out the new release, the release is process is very simple:

    • bump the version, create a new tag
    • build and upload to PyPI
    • add release notes to the documentation
    question 
    opened by mriehl 22
  • Feature: Cram plugin

    Feature: Cram plugin

    Cram is a functional testing framework for command line applications. This pull-requests presents a pybuilder plugin for it.

    Cram and the details can be found on: https://pypi.python.org/pypi/cram

    There are also two dog-fooding test files in src/cmdlinetest which is the default location for cram tests. The location can be configured, as can the glob used for discovering cram tests. The plugin respects the verbose setting.

    One thing I wasn't able to work out, was how to launch the plugin as part of publish, so remains as its own task for now.

    Another issue is that the unit test is not exactly DRY right now.

    Anyway, I wanted to submit this for review now, because it is already in a working state and just needs some more polishing.

    in progress 
    opened by esc 22
  • Using a python project as dependency in other python project

    Using a python project as dependency in other python project

    Hi,

    This is a question regarding using one python project as a dependency in other project.

    I have a project with test related utilities which is kept in separate repo. As of now, we were doing "git submodule" command to get this project in our main project and use its files in our tests. Now we are trying to create a separate package (tar.gz and .whl) of utilities project and upload to Artifactory. For our main project, we want to give utlities project as dependency in build.py so that when unit tests are executed during build, main project should be able to resolved those dependencies.

    Any thoughts?

    question 
    opened by farooqind 21
  • Should flake8 still be pinned?

    Should flake8 still be pinned?

    I had flake8 installed, but due to the dependency of the flake8-plugin, 4.0.1 was installed.

    My question is here: should this still be pinned? flake8 is at v6 already.

    The reason I noticed the failure was because I have flake8-pyproject installed and it started to fail all of a sudden...

    opened by svaningelgem 0
  • There is no option to make Pybuilder use pre-release versions of modules when resolving dependencies with pip

    There is no option to make Pybuilder use pre-release versions of modules when resolving dependencies with pip

    I am using Pybuilder to develop a project that depends on a library that is being developed concurrently. In the requirements.txt file I am imposing a condition like this:

    sample-library~=0.2.0
    

    Currently, the only 0.2.0 version available on our PyPI clone is 0.2.0.dev20220210180228.

    I can install the project's dependencies via pip using the --pre option, but I can't use this feature with Pybuilder: using the python.install_dependencies plugin and the pyb install_dependencies command, this is the output I get:

    [INFO]  Going to execute task install_dependencies
    [INFO]  Processing plugin packages 'flake8~=4.0' to be installed with {'upgrade': True}
    [INFO]  Processing plugin packages 'pypandoc~=1.4' to be installed with {'upgrade': True}
    [INFO]  Processing plugin packages 'setuptools>=38.6.0' to be installed with {'upgrade': True}
    [INFO]  Processing plugin packages 'twine>=1.15.0' to be installed with {'upgrade': True}
    [INFO]  Processing plugin packages 'unittest-xml-reporting~=3.0.4' to be installed with {'upgrade': True}
    [INFO]  Processing plugin packages 'wheel>=0.34.0' to be installed with {'upgrade': True}
    [INFO]  Installing all dependencies
    [INFO]  Processing dependency packages 'requirements.txt' to be installed with {}
    ------------------------------------------------------------
    BUILD FAILED - Unable to install dependency packages into <project-dir>\.venv. Please see '<project-dir>\target\logs\install_dependencies\install_batch' for full details:
            Looking in indexes: http://<our-pypi-instance>/repository/pypi-group/simple
            ERROR: Could not find a version that satisfies the requirement sample-library~=0.2.0 (from versions: 0.1.0, 0.2.0.dev20220210180228)
            ERROR: No matching distribution found for sample-library~=0.2.0
    

    Would it be possible to add an option to pyb that allows pip to be internally invoked with the --pre option?

    enhancement 
    opened by lormico 5
  • Relative import error related to the importlib_metadata/__init__.py file

    Relative import error related to the importlib_metadata/__init__.py file

    After upgrading from v0.12.10 to v0.13.3, we noticed a new error: `
    File "/ws/.venv/lib/python3.8/site-packages/pybuilder/_vendor/importlib_metadata/init.py", line 6, in from .. import zipp ValueError: attempted relative import beyond top-level package

    --------------------------------------------------------------------------------`

    This importlib_metadata seems to have been added in v0.13.0

    opened by atos-ghassen-bchir 7
  • Test error traceback suppressed by default

    Test error traceback suppressed by default

    Looks like without -v the default is to suppress the traceback of unit test errors:

    [INFO]  Executed 1 unit tests
    Error:  Test has error: unittest.loader._FailedTest.xxx
    ------------------------------------------------------------
    BUILD FAILED - There were 1 error(s) and 0 failure(s) in unit tests (pybuilder/plugins/python/unittest_plugin.py:109)
    ------------------------------------------------------------
    

    I think the default should be to show the traceback that caused the build to fail, regardless of whether -v is used.

    enhancement 
    opened by hoxu 0
  • Dependencies with constraints file

    Dependencies with constraints file

    This is pretty much an edge case but it seems that some projects need to be built with a pip constraints file, such as Airflow (detailed description as to why here).

    The equivalent on the command line with pip would be to run e.g.:

    pip install "apache-airflow[celery]==2.2.2" --constraint "https://raw.githubusercontent.com/apache/airflow/constraints-2.2.2/constraints-3.6.txt"
    

    I'm unsure which syntax / design would make most sense here. In our use-case, the dependency is required at build time as well in order to use the library in tests.

    opened by manuelbernhardt 0
Releases(v0.13.8)
  • v0.13.8(Nov 4, 2022)

    What's Changed

    • Prepare 0.13.8.dev [skip ci] by @arcivanov in https://github.com/pybuilder/pybuilder/pull/866
    • Added 'exception' placeholder to report_item. by @lyager in https://github.com/pybuilder/pybuilder/pull/867
    • Make sure to auto-close the file by @arcivanov in https://github.com/pybuilder/pybuilder/pull/869
    • Vendorize dependencies 2022-10-27 by @arcivanov in https://github.com/pybuilder/pybuilder/pull/873
    • Managed venvs will now distinguish between debug and release versions of Python by @arcivanov in https://github.com/pybuilder/pybuilder/pull/872
    • Implemented log format command line parameter. by @svaningelgem in https://github.com/pybuilder/pybuilder/pull/659
    • Release v0.13.8 by @arcivanov in https://github.com/pybuilder/pybuilder/pull/874

    New Contributors

    • @lyager made their first contribution in https://github.com/pybuilder/pybuilder/pull/867
    • @svaningelgem made their first contribution in https://github.com/pybuilder/pybuilder/pull/659

    Full Changelog: https://github.com/pybuilder/pybuilder/compare/v0.13.7...v0.13.8

    Source code(tar.gz)
    Source code(zip)
  • v0.13.7(Aug 11, 2022)

    What's Changed

    • Prepare version 0.13.7 by @arcivanov in https://github.com/pybuilder/pybuilder/pull/860
    • Python 3.11b3 by @arcivanov in https://github.com/pybuilder/pybuilder/pull/861
    • Vendorization 2022-07-30 by @arcivanov in https://github.com/pybuilder/pybuilder/pull/864
    • Fix improper class.filename and package.name values on XML coverage report by @lormico in https://github.com/pybuilder/pybuilder/pull/863
    • Release 0.13.7 by @arcivanov in https://github.com/pybuilder/pybuilder/pull/865

    New Contributors

    • @lormico made their first contribution in https://github.com/pybuilder/pybuilder/pull/863

    Full Changelog: https://github.com/pybuilder/pybuilder/compare/v0.13.6...v0.13.7

    Source code(tar.gz)
    Source code(zip)
  • v0.13.6(May 29, 2022)

    What's Changed

    • Prepare 0.13.6 by @arcivanov in https://github.com/pybuilder/pybuilder/pull/851
    • Make sure that project.description is always set by @arcivanov in https://github.com/pybuilder/pybuilder/pull/855
      1. 3.11+ requires mp.spawn.set_executable to be used by @arcivanov in https://github.com/pybuilder/pybuilder/pull/857
    • Release 0.13.6 by @arcivanov in https://github.com/pybuilder/pybuilder/pull/859

    Full Changelog: https://github.com/pybuilder/pybuilder/compare/v0.13.5...v0.13.6

    Source code(tar.gz)
    Source code(zip)
  • v0.13.5(Feb 23, 2022)

  • v0.13.4(Jan 12, 2022)

  • v0.13.3(Oct 14, 2021)

  • v0.13.2(Oct 6, 2021)

  • v0.13.1(Oct 3, 2021)

  • v0.13.0(Sep 28, 2021)

    Migration to GHA. Migration to VirtualEnv 20+. Removal of support for Python 2.7 & 3.5.

    Remove Frosted, PyChecker, PyTDDMon, Snakefood plugins as these utilities are no longer maintained. Multiple fixes.

    Source code(tar.gz)
    Source code(zip)
Buildout is a deployment automation tool written in and extended with Python

Buildout Buildout is a project designed to solve 2 problems: Application-centric assembly and deployment Assembly runs the gamut from stitching togeth

buildout 552 Nov 26, 2022
task management & automation tool

README doit - automation tool doit comes from the idea of bringing the power of build-tools to execute any kind of task Sample Code Define functions r

doit 1.5k Dec 30, 2022
A Star Trek Online build tool in Python

SETS - STO Equipment and Trait Selector A Star Trek Online build tool in Python Description Pre-alpha version of build tool for STO Getting Started De

Star Trek Online Community Developers 7 Nov 12, 2022
SCons - a software construction tool

SCons - a software construction tool Welcome to the SCons development tree. The real purpose of this tree is to package SCons for production distribut

SCons Project 1.6k Jan 3, 2023
A pynt of Python build.

A pynt of Python build. Raghunandan Rao Features Easy to learn. Build tasks are just python funtions. Manages dependencies between tasks. Automaticall

Raghunandan Rao 154 Jan 4, 2023
Clang-based cross platform build system written in Python

Clang-build Find the full documentation at https://clang-build.readthedocs.io First steps Customisations Multiple targets Multiple projects Defaults M

Trick 17 9 Jun 29, 2022
Official project repository for the Setuptools build system

See the Installation Instructions in the Python Packaging User's Guide for instructions on installing, upgrading, and uninstalling Setuptools. Questio

Python Packaging Authority 1.9k Jan 8, 2023
The Meson Build System

Meson® is a project to create the best possible next-generation build system. Status Dependencies Python (version 3.6 or newer) Ninja (version 1.8.2 o

The Meson Build System 4.4k Jan 2, 2023
The Pants Build System

Pants Build System Pants is a scalable build system for monorepos: codebases containing multiple projects, often using multiple programming languages

Pants Build 2.5k Jan 7, 2023
bitbake tool

Bitbake ======= BitBake is a generic task execution engine that allows shell and Python tasks to be run efficiently and in parallel while working wit

openembedded 336 Dec 27, 2022
This is a simple tool for bootstrapping Chimera systems from binaries. For source builds, you want cports.

chimera-bootstrap This is a simple tool for bootstrapping Chimera systems from binaries. For source builds, you want cports. Simple usage: $ # run as

Chimera Linux 7 Feb 11, 2022
The official binary distribution format for Python

wheel This library is the reference implementation of the Python wheel packaging standard, as defined in PEP 427. It has two different roles: A setupt

Python Packaging Authority 368 Dec 23, 2022
Python-based continuous integration testing framework; your pull requests are more than welcome!

Buildbot The Continuous Integration Framework Buildbot is based on original work from Brian Warner, and currently maintained by the Botherders. Visit

Buildbot 5k Jan 5, 2023
Simplified packaging of Python modules

Flit is a simple way to put Python packages and modules on PyPI. It tries to require less thought about packaging and help you avoid common mistakes.

Thomas Kluyver 1.9k Jan 5, 2023
Python-based project scripting.

Paver - Easy Scripting for Software Projects Web: https://pythonhosted.org/Paver/ Download: https://pypi.python.org/pypi/Paver/ Source: https://github

Paver community 452 Dec 9, 2022
Python program for installing many tools automatically

Tool Installer is a script made with python which help user in installing tools automatically

Spider Anongreyhat 6 Mar 13, 2022
PyPacker: a dumb little script for turning Python apps into standalone executable packages on Windows

PyPacker: a dumb little script for turning Python apps into standalone executable packages on Windows PyPacker is my attempt at creating a way to make

Serdar Yegulalp 9 Mar 15, 2022
This is a python helper package for Telebirr H5 Web payment integration helper.

Telebirr This is a python helper package for Telebirr H5 Web payment integration helper. The use case for this interface is payment via web. Third par

null 24 Dec 13, 2022
Python package used on Hardfight projects to make building, testing and deploying easy.

Hardfight Devtools Build, test and deploy Hardfight projects easly ?? What is it Devtools is a Python tool to make building, testing and deploying int

Hardfight 1 Dec 5, 2021