This PR tries to make redo also available for Python 3. I came across this as I make data analyses using redo and Python 3 and using two Pythons struck me as odd.

I tested this version under Python 2.7 and Python 3.7 and all tests pass.

The >& form is only for file descriptors, passing a file name there is a bash extension.

$ /bin/dash -c 'echo foo >&/dev/null'
/bin/dash: 1: Syntax error: Bad fd number

On systems where 'python' refers to python3, redo failed to launch. All invocations of python have been made explicitly python2 invocations. All tests pass on an Arch Linux system as of this commit.

redo-static marks each argument as static (i.e an input sources), even if they were previously marked as generated.

It's not exactly elegant, but I've encountered two cases where it seems necessary:

1. foo.do generates foo, but then the build is restructured and foo becomes a source (non-generated) file. redo warns that you've modified a generated file.
2. redo foo triggers default.do to build foo, but default.do doesn't know how to build foo (it's supposed to be a source file). This is probably a bug, discussed here: https://groups.google.com/forum/#!topic/redo-list/6_XvB6ZKlZ8

Sudo does not always preserve the environment so when doing the install with sudo and a DESTDIR= it will not be set in the sudo session. This makes install fail. This can be fixed by adding the -E option to sudo.

Python chooses to ignore SIGPIPE, however most unix processes expect to terminate on the signal. Therefore failing to restore the default action results in surprising behavior. For example, we expect dd if=/dev/zero | head -c1 to return immediately. However, prior to this commit, that pipeline would hang forever. Insidious forms of data corruption or loss were also possible.

See:

http://www.chiark.greenend.org.uk/ucgi/~cjwatson/blosxom/2009-07-02-python-sigpipe.html http://blog.nelhage.com/2010/02/a-very-subtle-bug/

In a slightly-odd real-world case this sped up redo-targets by an order of magnitude (if you are curious it was a generated website with lots and lots of directories like 2001/11/05/title/index.html)

Hi,

It could be just my eye, but I think a growing project using redo will soon have its root directory filled up with different .do files, when the number of different build targets grows. I spent a while studying the problem, and ended up in this suggestion where .do files are allowed to reside in a specific do/ directory on any level of .do file lookup.

On the downside, the number of looked-up paths for a matching .do file doubles, but on the upside, there's now one separate place where you'll find all the build related stuff. I think it's for the better.

But I'll leave it to you to decide.

/Pyry Jahkola

Installing redo on Alpine (which uses busybox for most everything incl /bin/true) yields an error,

redo-always: applet not found

because minimal/do symlinks /bin/true to redo-always, and /bin/true is in turn a link to busybox, which uses $0 to figure out what to do.

This PR changes minimal/do, making it create a +x file containing just ":" instead of symlinking to /bin/true.

Reproduction steps

PREFIX=/opt/redo ./redo install
ln -sf /opt/redo/bin/* /usr/local/bin/.

Expected outcome

Running redo should work

Actual outcome

Running redo complains about an unavailable module

Remedy

Set path by expanding symlinks to find the canonical path to the invoked executable.

Hi,

Attached two commits for the simple branch.

The first one add a test for a problem I found when I had a target depending on two redo-stamp targets that were also marked with redo-always. Basically, the use case for this is:

cflags.do:

redo-always
echo $CFLAGS >$3
redo-stamp <$3

config.do:

redo-always
echo $CONFIG >$3
redo-stamp <$3

compile.do

redo-ifchange cflags config
. ./cflags
. ./config
cc-$CONFIG $CFLAGS -o $3
...

This was a very useful technique I found to be able to depend on environment variables.

The problem was that compile.do was always rebuilt even if neither cflags or config changed. I should have fixed that.

The other problem is that the test 130 failed because ls -l shown -rw-r--r--. instead of just -rw-r--r-- making the test fail. I don't know if the correction is ideal but it works on my computer. You might want to drop this find a workaround (such as tr -d .).

Mildred

Except when necessary, I prefer not to install into /usr/local as root. My setup is as follows:

$ ls -ld /usr/local/bin /usr/local/share/man/man1
drwxrwsr-x 3 root staff 4096 Jun  8 11:29 /usr/local/bin
drwxrwsr-x 2 root staff 4096 Oct 12  2021 /usr/local/share/man/man1

Since I'm a member of the staff group it all works out. But running do on install.do failed with an error message:

Installing to: /usr/local
install: cannot change permissions of ‘/usr/local/share/man/man1’: Operation not permitted
install: cannot change permissions of ‘/usr/local/bin’: Operation not permitted

This change calls "$INSTALL" only when the directory to be installed isn't already there.

The current version of do expects #! to be followed by a /. A variety of tested systems treat the first word which follows #! as a path (which could be absolute or relative). Additionally, #! can be followed by spaces before the first path character. This patch addresses this potential incompatibility as a side effect.

An example usecase of relative paths in shebang in a redo based build system can be seen here:

https://git.sr.ht/~tk/pack/commit/01438f00d44adf33f59da729b347eca60cb461a8

This patch was tested to work in a variety of situations and systems.

Possibly mandatory legalese: I disclaim any copyright to this patch thereby placing it in the public domain, in line with the public domain status of minimal/do.

The default.required.rc.od modifier seems to be broken in the current build. It does not allow variables which have been modified in targets using appendln or replaceln to actually propagate upward to the file which calls include mytarget.required.rc. I added an rc_save call after the successful evaluation of base target and it appears to work as expected now.

Tested with zsh, bash, dash, ash (busybox) and /bin/sh pointing to bash. Test patterns:

rc_splitwords '"a b"' "'c d'" 'e\ f' "g\ h" i\\\ j

rc_splitwords $(seq 10000)
rc_splitwords "$(seq 10000)"

For the long sequence tests, performance was still a sub-second response with bash being considerably slower than the other shells.

As a side note: I object to the suppression of empty arguments, as these might be valid input, but kept it for compatibility.