Smaller, easier, more powerful, and more reliable than make. An implementation of djb's redo.

Overview

redo - a recursive build system

Smaller, easier, more powerful, and more reliable than make.

This is an implementation of Daniel J. Bernstein's redo build system. He never released his version, so other people have implemented different variants based on his published specification.

This version, sometimes called apenwarr/redo, is probably the most advanced one, including parallel builds, improved logging, extensive automated tests, and helpful debugging features.

To build and test redo, run:

	./do -j10 test

To install it, run something like this:

	DESTDIR= PREFIX=/usr/local ./do -j10 install

Issues
  • Avoid bashism >&file

    Avoid bashism >&file

    The >& form is only for file descriptors, passing a file name there is a bash extension.

    $ /bin/dash -c 'echo foo >&/dev/null'
    /bin/dash: 1: Syntax error: Bad fd number
    
    opened by tv42 8
  • Add compatibility to Python 3 (and retain Python 2)

    Add compatibility to Python 3 (and retain Python 2)

    This PR tries to make redo also available for Python 3. I came across this as I make data analyses using redo and Python 3 and using two Pythons struck me as odd.

    I tested this version under Python 2.7 and Python 3.7 and all tests pass.

    opened by mlell 8
  • Replaced all instances of 'python' with 'python2'

    Replaced all instances of 'python' with 'python2'

    On systems where 'python' refers to python3, redo failed to launch. All invocations of python have been made explicitly python2 invocations. All tests pass on an Arch Linux system as of this commit.

    opened by SeamusConnor 7
  • redo is no longer available on homebrew

    redo is no longer available on homebrew

    Update the docs to reflect this

    opened by llimllib 6
  • Add redo-static

    Add redo-static

    redo-static marks each argument as static (i.e an input sources), even if they were previously marked as generated.

    It's not exactly elegant, but I've encountered two cases where it seems necessary:

    1. foo.do generates foo, but then the build is restructured and foo becomes a source (non-generated) file. redo warns that you've modified a generated file.
    2. redo foo triggers default.do to build foo, but default.do doesn't know how to build foo (it's supposed to be a source file). This is probably a bug, discussed here: https://groups.google.com/forum/#!topic/redo-list/6_XvB6ZKlZ8
    opened by timbertson 3
  • Add installation instructions

    Add installation instructions

    I was unable to find any installation instructions. I documented the method I used to install in the README. It may be helpful to others.

    opened by rfdonnelly 2
  • Failing to restore SIGPIPE leads to surprising and dangerous behavior

    Failing to restore SIGPIPE leads to surprising and dangerous behavior

    Python chooses to ignore SIGPIPE, however most unix processes expect to terminate on the signal. Therefore failing to restore the default action results in surprising behavior. For example, we expect dd if=/dev/zero | head -c1 to return immediately. However, prior to this commit, that pipeline would hang forever. Insidious forms of data corruption or loss were also possible.

    See:

    http://www.chiark.greenend.org.uk/ucgi/~cjwatson/blosxom/2009-07-02-python-sigpipe.html http://blog.nelhage.com/2010/02/a-very-subtle-bug/

    opened by traviscross 2
  • Performance improvement to state.files() on simple branch

    Performance improvement to state.files() on simple branch

    In a slightly-odd real-world case this sped up redo-targets by an order of magnitude (if you are curious it was a generated website with lots and lots of directories like 2001/11/05/title/index.html)

    opened by jasom 2
  • Hierarchical dodir support

    Hierarchical dodir support

    Update the definition of possible do files to include "do" directories, and update docs.

    opened by nwf 2
  • Suggestion: tidy up the project root by allowing .do files within do/ directories

    Suggestion: tidy up the project root by allowing .do files within do/ directories

    Hi,

    It could be just my eye, but I think a growing project using redo will soon have its root directory filled up with different .do files, when the number of different build targets grows. I spent a while studying the problem, and ended up in this suggestion where .do files are allowed to reside in a specific do/ directory on any level of .do file lookup.

    On the downside, the number of looked-up paths for a matching .do file doubles, but on the upside, there's now one separate place where you'll find all the build related stuff. I think it's for the better.

    But I'll leave it to you to decide.

    /Pyry Jahkola

    opened by pyrtsa 2
  • minimal/do: allow for relative shebangs

    minimal/do: allow for relative shebangs

    The current version of do expects #! to be followed by a /. A variety of tested systems treat the first word which follows #! as a path (which could be absolute or relative). Additionally, #! can be followed by spaces before the first path character. This patch addresses this potential incompatibility as a side effect.

    An example usecase of relative paths in shebang in a redo based build system can be seen here:

    https://git.sr.ht/~tk/pack/commit/01438f00d44adf33f59da729b347eca60cb461a8

    This patch was tested to work in a variety of situations and systems.

    Possibly mandatory legalese: I disclaim any copyright to this patch thereby placing it in the public domain, in line with the public domain status of minimal/do.

    opened by EliteTK 0
  • Don't assume a

    Don't assume a "python" executable exists.

    On Debian Linux at least, "python" is Python 2.x, and doesn't exist if only Python 3.x is installed.

    opened by Screwtapello 0
  • FIX default.required.rc.od to save changes

    FIX default.required.rc.od to save changes

    The default.required.rc.od modifier seems to be broken in the current build. It does not allow variables which have been modified in targets using appendln or replaceln to actually propagate upward to the file which calls include mytarget.required.rc. I added an rc_save call after the successful evaluation of base target and it appears to work as expected now.

    opened by gotnone 0
  • Re-implement rc_splitwords to accommodate arbitrarily many words

    Re-implement rc_splitwords to accommodate arbitrarily many words

    Tested with zsh, bash, dash, ash (busybox) and /bin/sh pointing to bash. Test patterns:

    rc_splitwords '"a b"' "'c d'" 'e\ f' "g\ h" i\\\ j
    
    rc_splitwords $(seq 10000)
    rc_splitwords "$(seq 10000)"
    

    For the long sequence tests, performance was still a sub-second response with bash being considerably slower than the other shells.

    As a side note: I object to the suppression of empty arguments, as these might be valid input, but kept it for compatibility.

    opened by spacefrogg 1
  • Make location of redoconf scripts definable

    Make location of redoconf scripts definable

    Introduce REDOCONF variable that can be set in 'configure' just as S.

    Rationale: Some people might want to hide redoconf in .redoconf and, generally, just not to fix things that can stay variable.

    Also fixes a bug in configure.sh, which unconditionally overwrites S.

    opened by spacefrogg 0
  • Remove absolute pathnames to binaries and rely on PATH

    Remove absolute pathnames to binaries and rely on PATH

    Fixes issues with distributions that use non-standard binary paths (e.g., in isolated build environments). Scripts rely on PATH for other programs, anyway.

    opened by spacefrogg 7
  • Fix directory nonexistent issue

    Fix directory nonexistent issue

    I found the minimal do fails with directory nonexistent in some case when the full redo works fine. This PR fixes the issue.

    A minimal repro with the instructions are available at https://github.com/maoe/minimal-do-repro.

    Although the issue goes away with this fix, I'm not really sure if this is the right way. Please let me know if there's a better way to fix the issue.

    opened by maoe 0
Owner
Making WireGuard 2FA easy @Tailscale, and generally fighting off the demons of complexity.
null
🔨🐍Make-like build automation tool for Python projects with extensive DSL features.

Pyke (WIP, Beta Release) Make-like build automation tool for Python projects with extensive DSL features. Features: Users can specify tasks, subtasks,

Ire 12 Sep 12, 2021
PyPacker: a dumb little script for turning Python apps into standalone executable packages on Windows

PyPacker: a dumb little script for turning Python apps into standalone executable packages on Windows PyPacker is my attempt at creating a way to make

Serdar Yegulalp 2 Oct 19, 2021
Pythonic task management & command execution.

Welcome to Invoke! Invoke is a Python (2.7 and 3.4+) library for managing shell-oriented subprocesses and organizing executable Python code into CLI-i

null 3.4k Oct 16, 2021
Software build automation tool for Python.

PyBuilder — an easy-to-use build automation tool for Python PyBuilder is a software build tool written in 100% pure Python, mainly targeting Python ap

PyBuilder 1.3k Oct 24, 2021
This is a simple tool for bootstrapping Chimera systems from binaries. For source builds, you want cports.

chimera-bootstrap This is a simple tool for bootstrapping Chimera systems from binaries. For source builds, you want cports. Simple usage: $ # run as

Chimera Linux 2 Oct 23, 2021
the blessed package to manage your versions by scm tags

setuptools_scm setuptools_scm handles managing your Python package versions in SCM metadata instead of declaring them as the version argument or in a

Python Packaging Authority 507 Oct 19, 2021
A pynt of Python build.

A pynt of Python build. Raghunandan Rao Features Easy to learn. Build tasks are just python funtions. Manages dependencies between tasks. Automaticall

Raghunandan Rao 152 Oct 6, 2021
Python-based continuous integration testing framework; your pull requests are more than welcome!

Buildbot The Continuous Integration Framework Buildbot is based on original work from Brian Warner, and currently maintained by the Botherders. Visit

Buildbot 4.7k Oct 20, 2021
Buildout is a deployment automation tool written in and extended with Python

Buildout Buildout is a project designed to solve 2 problems: Application-centric assembly and deployment Assembly runs the gamut from stitching togeth

buildout 521 Oct 18, 2021
PlatformIO is a professional collaborative platform for embedded development :alien: A place where Developers and Teams have true Freedom! No more vendor lock-in!

PlatformIO Quick Links: Web | PlatformIO IDE | Project Examples | Docs | Donate | Contact Us Social: LinkedIn | Twitter | Facebook | Community Forums

PlatformIO 5.4k Oct 23, 2021
The official binary distribution format for Python

wheel This library is the reference implementation of the Python wheel packaging standard, as defined in PEP 427. It has two different roles: A setupt

Python Packaging Authority 269 Oct 23, 2021
Python-based project scripting.

Paver - Easy Scripting for Software Projects Web: https://pythonhosted.org/Paver/ Download: https://pypi.python.org/pypi/Paver/ Source: https://github

Paver community 445 Sep 24, 2021
Simplified packaging of Python modules

Flit is a simple way to put Python packages and modules on PyPI. It tries to require less thought about packaging and help you avoid common mistakes.

Thomas Kluyver 1.5k Oct 21, 2021
task management & automation tool

README doit - automation tool doit comes from the idea of bringing the power of build-tools to execute any kind of task Sample Code Define functions r

doit 1.1k Oct 24, 2021
Package, distribute, and update any app for Linux and IoT.

Snapcraft Package, distribute, and update any app for Linux and IoT. Snaps are containerised software packages that are simple to create and install.

null 1k Oct 15, 2021
Program for convert py & js file to exe

Converter JS & PY to Exe Converter Coded by Lamp Requirements : Node.js Python How to Use : Install latest python Dont forget to add path Install node

null 5 Oct 4, 2021
Official project repository for the Setuptools build system

See the Installation Instructions in the Python Packaging User's Guide for instructions on installing, upgrading, and uninstalling Setuptools. Questio

Python Packaging Authority 1.5k Oct 20, 2021
The Meson Build System

Meson® is a project to create the best possible next-generation build system. Status Dependencies Python (version 3.6 or newer) Ninja (version 1.8.2 o

The Meson Build System 3.6k Oct 24, 2021
Utilities for interacting with PyPI

twine Twine is a utility for publishing Python packages on PyPI. It provides build system independent uploads of source and binary distribution artifa

Python Packaging Authority 1.2k Oct 24, 2021