I have no idea how hard this will be, but MacroPy does not have many dependencies on the underlying runtime except for PEP302 and the ast library.

Some of the ASTs look slightly different (e.g. functions can't have nested parameter lists, no print statement) and we may have to remove any print statements from the implementation code, but I don't imagine it will be very difficult or require big changes.

Pattern matching seems to be broken since 774461c76f8db57cad072ea6a9d39b82c4a6447e, where also most of the tests got removed.

For example, the following code prints None on current revision (from readme):

from macropy.macros.pattern import macros, patterns
from macropy.macros.adt import macros, case

@case
class Rect(p1, p2): pass

@case
class Line(p1, p2): pass

@case
class Point(x, y): pass

def area(rect):
    with patterns:
        Rect(Point(x1, y1), Point(x2, y2)) << rect
        return (x2 - x1) * (y2 - y1)

print area(Rect(Point(1, 1), Point(3, 3))) # 4

The python code created (by unparse_ast) is following. Note the area function.

import inspect
from ast import *
from macropy.core import util
from macropy.core.macros import *
from macropy.core.lift import *
from macropy.core.lift import *
macros = Macros()

class PatternMatchException(Exception):
    '\n    Thrown when a nonrefutable pattern match fails\n    '
    pass

# snip, edited for brevity

@link_children
class Point(CaseClass):

    def __init__(self, *args, **kwargs):
        CaseClass.__init__(self, *args, **kwargs)
        pass
    _children = []
    _fields = ['x', 'y']

def area(rect):
    None
print area(Rect(Point(1, 1), Point(3, 3)))

The original decision to use macro%... was due to:

• Lesser chance of collisions (people use fewer % operations than function calls)
• Shorter syntax for macros such as String Interpolation

Although the macro%... syntax still benefits macros such as string interpolation (s%"..."), it seems the vast bulk of our macros would look much better using the macro(...) or macro[...] syntax instead. This is partially because of the high precedence of the % operator, turning basically many macro%... calls into macro%(...) calls anyway:

sql%(x.size for x in db.countries)
require%(value == 10)
trace%(1 + 2 + 3)
f%(_ + _)
q%(1 + 2)
with q as code:
    x = 10
code[0].targets[0].id = n
with match:
    Point(x, y) << my_point
s%"i a {cow}" 
p%"<h1>Hello World</h1>
with peg:
    op = "+" | "-" | "*" | "/"

vs

sql(x.size for x in db.countries)
require(value == 10)
trace(1 + 2 + 3)
f(_ + _)
q(1 + 2)
with q as code:
    x = 10
code[0].targets[0].id = n
with match:
    Point(x, y) << my_point
s("i am {cow}")
p("<h1>Hello World</h1>")
with peg:
    op = "+" | "-" | "*" | "/"

vs

sql[(x.size for x in db.countries)]
require[value == 10]
trace[1 + 2 + 3]
f[_ + _]
q[1 + 2]
with q as code:
    name[n] = 10
match[Point(x, y)] = my_point
s["i am {cow}"]
p["<h1>Hello World</h1>"]
peg[op] = "+" | "-" | "*" | "/"

In addition, using macro[...] means you can place macros on the left hand side of an assignment, which allows for a nice looking shorthand match and peg syntax, rather than having to use with ...: block macros when you really only need a single-statement.

Due to the fact that you can now rename macros when importing them, the downside of a name collision is much less severe: just rename the macro using ...import macros, sql as macro_sql or similar.

We could:

• Choose one and make everyone stick to it (for consistency)
• Allow all and let the macro-writer decide at registration point (i.e. have three separate macros.pct_expr macros.paren_expr macros.square_expr decorators) to allow macros to more closely follow the syntax of similar operations (e.g. functions)
• Allow all and let the macro user pick one of them to use at use site.

I installed macropy from git (git clone, python setup.py install).

I go to a python shell and try the case classes example.

Python 2.7.4 (default, Apr 19 2013, 18:32:33) 
[GCC 4.7.3] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> from macropy.macros.adt import macros, case
0=[]=====> MacroPy Enabled <=====[]=0
>>> @case
... class Point(x, y): pass
Traceback (most recent call last):
  File "<console>", line 1, in <module>
NameError: name 'case' is not defined

I'm probably missing something obvious :-)

from macropy.tracing import macros, trace
trace[[len(x)*3 for x in ["omg", "wtf", "b" * 2 + "q", "lo" * 3 + "l"]]]

gives:

Traceback (most recent call last):
  File "C:\Users\User\Desktop\Tfach\testmacropy.py", line 2, in <module>
    trace[[len(x)*3 for x in ["omg", "wtf", "b" * 2 + "q", "lo" * 3 + "l"]]]
  File "C:\Users\User\AppData\Local\Programs\Python\Python37\lib\site-packages\macropy\core\macros.py", line 32, in __getitem__
    raise TypeError(self.msg.replace("%s", self.func.__name__))
TypeError: Macro `trace` illegally invoked at runtime; did you import it properly using `from ... import macros, trace`?

from macropy.tracing import macros, trace
with trace:
    sum = 0
    for i in range(0, 5):
        sum = sum + 5

gives:

Traceback (most recent call last):
  File "C:\Users\User\Desktop\Tfach\testmacropy.py", line 2, in <module>
    with trace:
AttributeError: __enter__

from macropy.string_interp import macros, s
A = 10
B = 5
print(s["{A} + {B} = {A + B}"])
# 10 + 5 = 15

gives

Traceback (most recent call last):
  File "C:\Users\User\Desktop\Tfach\testmacropy.py", line 4, in <module>
    print(s["{A} + {B} = {A + B}"])
  File "C:\Users\User\AppData\Local\Programs\Python\Python37\lib\site-packages\macropy\core\macros.py", line 32, in __getitem__
    raise TypeError(self.msg.replace("%s", self.func.__name__))
TypeError: Macro `s` illegally invoked at runtime; did you import it properly using `from ... import macros, s`?

basically nothing works, am i missing something here ?

• macropy: v1.1.0b2
• python: 3.7
• OS: Windows

with fork as x:
    ... do some expensive operations ...
    result
with fork as y:
    ... do more expensive operations ...
    result

res_x, res_y = join(x, y)
do_stuff(res_x, res_y)

Although it is possible to grab the bytecode of a function and send it to a separate process to be run in parallel, macros would let us parallelize tasks on a sub-function level. Exactly how they are run (process pool, thread pool, remote machines) is completely abstracted away. Another syntax that looks nicer but only works for expressions may be:

# these two get run in parallel
x, y = fork_join%(
    ...some expensive computation...,
    ...another expensive computation...
)
do_stuff(x, y)

Macros could also give us a concise way of emulating "lexical scoping", by delimiting which parts of the forked code are to be evaluated locally:

with forked as x:
    temp1 = do_expensive_op_with(u%value1)
    temp2 = do_expensive_op_with(u%value2)
    ...
    result

In this case, the u% syntax indicates that value1 and value2 are to be evaluated in the local scope and their value sent over to the forked task, rather than sending over the names value1 or value2 which may not exist in the scope where the forked task is executing. There are probably a bunch of other clever things you could do with macros, along with a bunch of pitfalls to do with concurrency/parallelism, neither of which has been thought through.

There's a bug with block quotations and ast_literal.

Create following macro:

@macros.block
def test(tree, **kw):
	with q as new_tree:
		ast_literal[tree]

	print('Original: ' + real_repr(tree))
	print('Block:    ' + real_repr(new_tree))
	new_tree = q[ast_literal[tree]]
	print('Expr:     ' + real_repr(new_tree))
	return new_tree

Now, for example, use it with just pass as body:

with test():
	pass

Output is:

Original: [Pass()]
Block:    [Expr([Pass()])]
Expr:     [Pass()]

As you can see when using block quotation it changes tree to [Expr(tree)] which results in an error getting thrown.

This doesn't work:

@macros.block
def test(tree, **kw):
	with q as new_tree:
		ast_literal[tree]
	return new_tree

This does work:

@macros.block
def test(tree, **kw):
	new_tree = q[ast_literal[tree]]
	return new_tree

from macropy.experimental.tco import macros, tco

@tco def fact(n, acc=0): if n == 0: return acc else: return fact(n-1, n * acc)

print(fact(10000))

gives:

Traceback (most recent call last): File "", line 2, in File "C:\Program Files\Python36\lib\site-packages\macropy-1.0.3-py3.6.egg\macropy\core\macros.py", line 31, in call return self.func(*args, **kwargs) File "C:\Program Files\Python36\lib\site-packages\macropy-1.0.3-py3.6.egg\macropy\experimental\tco.py", line 131, in tco tree.decorator_list = ([hq[trampoline_decorator]] + File "C:\Program Files\Python36\lib\site-packages\macropy-1.0.3-py3.6.egg\macropy\core\macros.py", line 34, in getitem raise TypeError(self.msg.replace("%s", self.func.name)) TypeError: Macro hq illegally invoked at runtime; did you import it properly using from ... import macros, hq?

from macropy.experimental.tco import macros, tco

@tco ... def fact(n, acc=0): ... if n == 0: ... return acc ... else: ... return fact(n-1, n * acc) ... Traceback (most recent call last): File "", line 2, in File "C:\Python27\lib\site-packages\macropy\core\macros.py", line 28, in call return self.func(*args, **kwargs) File "C:\Python27\lib\site-packages\macropy\experimental\tco.py", line 131, in tco tree.decorator_list = ([hq[trampoline_decorator]] + File "C:\Python27\lib\site-packages\macropy\core\macros.py", line 31, in getitem raise TypeError(self.msg.replace("%s", self.func.name)) TypeError: Macro hq illegally invoked at runtime; did you import it properly using from ... import macros, hq?

Macros should have access to the name of the module they were imported as. That way, if a macro is well-written, all the macro caller needs to do to ensure that everything works correctly is make sure they don't shadow the macro module.

For example, if a macro module was imported as foo, and a macro in foo needed to use a temporary variable, it could use foo.tmp instead of just tmp.

Some form of preprocessor script that could be run on files with macros to make them directly runnable by replacing all macro code with the actual generated code.

This would allow:

1. Packaging of scripts as directly runnable
2. Removal of macropy as a dependency for distributed code
3. Allows for macros to be used in projects that don't want another dependency as code-generation tools for individual developers

To take an example from the README:

point.py:

from macropy.case_classes import macros, case

@case
class Point(x, y): pass

CLI:

$ python expand_macros.py point.py

exp_point.py:

class Point(object):
    __slots__ = ['x', 'y']
    def __init__(self, x, y):
        self.x = x
        self.y = y

    def __str__(self):
        return "Point(" + self.x + ", " + self.y + ")"

    def __repr__(self):
        return self.__str__()

    def __eq__(self, other):
        return self.x == other.x and self.y == other.y

    def __ne__(self, other):
        return not self.__eq__(other)

    def __iter__(self, other):
        yield self.x
        yield self.y

I for one would love to be able to use macros like this to generate boilerplate code but still maintain my scripts as directly runnable and easily distributable.

@finiteloop mentioned in #43 that macropy slows stuff down a bunch. It would be nice if we had a nice benchmark suite that we could use to see how fast MacroPy was in a few cases:

• Large file with one or two macros inside (e.g. logs)
• Large file full of macros inside (e.g. a test suite with lots of requires)
• Large file with one or two macros which contain a whole pile of non-macro code (e.g. a few case classes with a pile of methods inside)

Only then would we be able to properly measure what effect (if any) our changes are having on performance, and therefore make systematic improvements in that direction.

I am unable to use macropy.activate from version 1.1.0b2 in Python 3.10.2 on Arch Linux. When running import macropy.activate I expected Python to cleanly import the package, but instead it gives an error related to lineno.

$ python
Python 3.10.2 (main, Jan 15 2022, 19:56:27) [GCC 11.1.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import macropy.activate
Error while compiling file /usr/lib/python3.10/site-packages/macropy/core/hquotes.py
Traceback (most recent call last):
  File "/usr/lib/python3.10/site-packages/macropy/core/import_hooks.py", line 113, in expand_macros
    return compile(tree, filename, "exec"), new_tree
TypeError: required field "lineno" missing from alias
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/python3.10/site-packages/macropy/activate.py", line 4, in <module>
    macropy.activate()
  File "/usr/lib/python3.10/site-packages/macropy/__init__.py", line 18, in activate
    from .core import hquotes  # noqa
  File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1002, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 945, in _find_spec
  File "/usr/lib/python3.10/site-packages/macropy/core/import_hooks.py", line 147, in find_spec
    code, tree = self.expand_macros(source, origin, spec)
  File "/usr/lib/python3.10/site-packages/macropy/core/import_hooks.py", line 113, in expand_macros
    return compile(tree, filename, "exec"), new_tree
TypeError: required field "lineno" missing from alias

/local/lib/python2.7/site-packages/macropy/core/import_hooks.py", line 43, in find_module
    tree = ast.parse(txt)
  File "/usr/lib/python2.7/ast.py", line 37, in parse
    return compile(source, filename, mode, PyCF_ONLY_AST)
TypeError: compile() expected string without null bytes

Is this package still maintained? It appears to be broken on python3.8 and python3.9 with errors like this.

Traceback (most recent call last):
  File "run_tests.py", line 5, in <module>
    import macropy.test
  File "/build/source/macropy/test/__init__.py", line 14, in <module>
    from . import case_classes
  File "<frozen importlib._bootstrap>", line 991, in _find_and_load
  File "<frozen importlib._bootstrap>", line 971, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 914, in _find_spec
  File "/build/source/macropy/core/import_hooks.py", line 147, in find_spec
    code, tree = self.expand_macros(source, origin, spec)
  File "/build/source/macropy/core/import_hooks.py", line 113, in expand_macros
    return compile(tree, filename, "exec"), new_tree
TypeError: required field "posonlyargs" missing from arguments

FYI. A different report of the same bug, which has a possible fix, is here: https://github.com/xonsh/xonsh/issues/3271

There is a small typo in docs/reference.rst.

Should read stitch rather than stich.

Semi-automated pull request generated by https://github.com/timgates42/meticulous/blob/master/docs/NOTE.md

When using @tco on a simple recursive function with python 3.8

import macropy.activate
from macropy.experimental.tco import macros, tco

@tco
def fact(n):
    if n == 0:
        return 1
    else:
        return n * fact(n-1)