Parsing python2 code without print functions turned on seems to be broken? The README seems to imply that this should still work (Bowler supports modifications to code from any version of Python 2 or 3)

Dockerfile:

FROM python:3.7

RUN pip3 install git+https://github.com/facebookincubator/Bowler.git
RUN echo "print 'something'" > test.py
RUN bowler dump test.py

Output:

$ bowler dump test.py
ERROR:bowler.tool:Skipping test.py: failed to parse

As a developer in charge of CI and automation I want this project distributed to pypi as a wheel because wheels are faster to deploy, and they don't require the runtime version of python to be available when running pip install. The vast majority of projects are using wheels now, and it's really not much more work for the maintainer. Another thing that should be fixed is that the project should be marked as python3-only on PyPI.

thanks!

Hello,

It looks like this method never works and there were no tests for it. I have repaired the method to generate the correct indentation and added tests to prevent future regression.

Best regards,

I wrote a thing to convert self.assertFalse(x) to assert not x, etc. Trouble is, it falls over easily when the expression in the parentheses isn't valid without the parentheses:

self.assertFalse(
    'a'
    'b'
)

My code naively converts this to

assert not 'a'
    'b'

• I think a parenthesize_if_necessary helper would be really useful for this kind of situation. I imagine this bug is widespread. I tried to write one, but I'm a bit out of my depth.
• It'd be useful to have access to a function which asserts a list of nodes has no syntax errors. I realise that would vary depending on targeted python version and __future__ imports, but perhaps something can be done?

Thanks

After installation of pybowler using pip install on windows 10, running bowler in powershell returns TB. Steps:

1. in Python folder (typically something like C:\Python36_32), go to Scripts folder, and run ".\pip.exe install bowler.
2. run .\bowler.exe in same folder (Python36_32/Scripts)
3. TB as follows:

Traceback (most recent call last):

File "C:\Python36\Scripts\bowler-script.py", line 11, in <module>
  load_entry_point('bowler==0.5.1', 'console_scripts', 'bowler')()

File "c:\python36\lib\site-packages\pkg_resources\__init__.py", line 565, in load_entry_point
  return get_distribution(dist).load_entry_point(group, name)

File "c:\python36\lib\site-packages\pkg_resources\__init__.py", line 2631, in load_entry_point
  return ep.load()

File "c:\python36\lib\site-packages\pkg_resources\__init__.py", line 2291, in load
  return self.resolve()

File "c:\python36\lib\site-packages\pkg_resources\__init__.py", line 2297, in resolve
  module = __import__(self.module_name, fromlist=['__name__'], level=0)

File "c:\python36\lib\site-packages\bowler\__init__.py", line 12, in <module>
  from .tool import BowlerTool

File "c:\python36\lib\site-packages\bowler\tool.py", line 12, in <module>
  import sh

File "c:\python36\lib\site-packages\sh.py", line 36, in <module>
  support." % __version__)

ImportError: sh 1.12.14 is currently only supported on linux and osx. please install pbs 0.110 (http://pypi.python.org/pypi/pbs) for windows support.

https://pybowler.io/docs/basics-intro

lib2to3 provides a concrete syntax tree (CST) implementation that recognizes and supports the grammar of all Python versions back to 2.6.

By building on lib2to3, Bowler is capable of reading and modifying source files written for both Python 2 and 3. That said, Bowler requires Python 3.6 or newer to run

Everything was working fine until I tried running on a (Py 2.7) source file which had a print statement in it

Then I get:

ERROR:RefactoringTool:Can't parse tests/fixtures/py27.py: ParseError: bad input: type=3, value='"Python 2 syntax"', context=(' ', (34, 10))

After putting some breakpoints in and digging through the source it seems the problem is here: https://github.com/facebookincubator/Bowler/blob/195fbae4550ea82d72b9c59f3dca372b20d1edf5/bowler/tool.py#L98

BowlerTool hard-codes it so that the underlying fissix RefactorTool will use the no-print-statements grammar, which is therefore not Python 2 compatible.

Commenting out that line fixes it for me... I wonder why it was added?

The readme appears to say you can make a query that has a select followed by filters and modifiers, then chain a subsequent select onto it: https://github.com/facebookincubator/Bowler/tree/master/bowler#query-reference

When I try to do this I get an error:

  File "/.virtualenvs/waterloo-sHXra0II/lib/python3.7/site-packages/bowler/tool.py", line 209, in refactor_queue
    hunks = self.refactor_file(filename)
  File "/.virtualenvs/waterloo-sHXra0II/lib/python3.7/site-packages/bowler/tool.py", line 171, in refactor_file
    tree = self.refactor_string(input, filename)
  File "/.virtualenvs/waterloo-sHXra0II/lib/python3.7/site-packages/fissix/refactor.py", line 370, in refactor_string
    self.refactor_tree(tree, name)
  File "/.virtualenvs/waterloo-sHXra0II/lib/python3.7/site-packages/fissix/refactor.py", line 410, in refactor_tree
    self.traverse_by(self.bmi_post_order_heads, tree.post_order())
  File "/.virtualenvs/waterloo-sHXra0II/lib/python3.7/site-packages/fissix/refactor.py", line 488, in traverse_by
    node.replace(new)
  File "/.virtualenvs/waterloo-sHXra0II/lib/python3.7/site-packages/fissix/pytree.py", line 108, in replace
    assert self.parent is not None, str(self)
AssertionError

I am guessing that this is because the subsequent selector has to select some sub-element of what was already selected by the initial select? You can't chain an unrelated select query on?

(I wanted to do that because I am naïvely trying to pass metadata about the current file between my modifiers)

Unrelated to the sh problem in windows I've caught another one:

Traceback (most recent call last):
  File "E:\packages\try_bowler\test.py", line 7, in <module>
    rename.dump()
  File "C:\Users\Levitanus\AppData\Local\Programs\Python\Python37-32\lib\site-packages\bowler\query.py", line 967, in dump
    return self.execute(write=False)
  File "C:\Users\Levitanus\AppData\Local\Programs\Python\Python37-32\lib\site-packages\bowler\query.py", line 957, in execute
    self.retcode = BowlerTool(fixers, **kwargs).run(self.paths)
  File "C:\Users\Levitanus\AppData\Local\Programs\Python\Python37-32\lib\site-packages\bowler\tool.py", line 342, in run
    self.refactor(paths)
  File "C:\Users\Levitanus\AppData\Local\Programs\Python\Python37-32\lib\site-packages\bowler\tool.py", line 223, in refactor
    child.start()
  File "C:\Users\Levitanus\AppData\Local\Programs\Python\Python37-32\lib\multiprocessing\process.py", line 112, in start
    self._popen = self._Popen(self)
  File "C:\Users\Levitanus\AppData\Local\Programs\Python\Python37-32\lib\multiprocessing\context.py", line 223, in _Popen
    return _default_context.get_context().Process._Popen(process_obj)
  File "C:\Users\Levitanus\AppData\Local\Programs\Python\Python37-32\lib\multiprocessing\context.py", line 322, in _Popen
    return Popen(process_obj)
  File "C:\Users\Levitanus\AppData\Local\Programs\Python\Python37-32\lib\multiprocessing\popen_spawn_win32.py", line 65, in __init__
    reduction.dump(process_obj, to_child)
  File "C:\Users\Levitanus\AppData\Local\Programs\Python\Python37-32\lib\multiprocessing\reduction.py", line 60, in dump
    ForkingPickler(file, protocol).dump(obj)
AttributeError: Can't pickle local object 'Query.create_fixer.<locals>.Fixer'
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "C:\Users\Levitanus\AppData\Local\Programs\Python\Python37-32\lib\multiprocessing\spawn.py", line 99, in spawn_main
    new_handle = reduction.steal_handle(parent_pid, pipe_handle)
  File "C:\Users\Levitanus\AppData\Local\Programs\Python\Python37-32\lib\multiprocessing\reduction.py", line 87, in steal_handle
    _winapi.DUPLICATE_SAME_ACCESS | _winapi.DUPLICATE_CLOSE_SOURCE)
PermissionError: [WinError 5] Отказано в доступе
[Finished in 0.3s with exit code 1]

Original import in tool.py was replaced to import pbs as sh, but. as I understant, flow hasn't gone to the patching part of refactor

original source:

import bowler as bl


rename = bl.Query()
rename.select_var('value')
rename.rename('new_val')
rename.dump()

Unfortunately facebook bowler does not return this project, nor is it easy to do a search for bowler + problem or facebook bowler + problem to see if anyone else is having this issue.

Cutesy names are cool and all, but there is so much noise looking for bowler that it makes it hard to use the project.

I've built a selector that finds common class attributes and moves them to a superclass definition - but rather than removing the node from the matched class - I can only make the line blank, by setting by calling node.children.clear()

This also only makes an empty line not a full removal attrib_assignment = capture.get("attr_assignment") attrib_assignment.parent.children.remove(attrib_assignment)

Thanks in advance

Use a more specific return type at https://github.com/facebookincubator/Bowler/blob/master/bowler/types.py#L47

Right now it's Any, and seems like it should be Optional[LN]

1. Prepare a change to do that (see getting started docs)
2. Verify that mypy is happy with it using make lint
3. Send a PR!

Bumps wheel from 0.35.1 to 0.38.1.

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

bowler/tests/lib.py: switch from volatile back to tempfile

The volatile project is unmaintained. Use tempfile module, which provides similar functionality.

See also: 3664c3f145a819ddd1f5d5e39df86a4f34c4f3f2

Hello, I am trying to apply changes also on some notebooks from databricks stored as .py files, from time to time, there is %pip install or %magic command, could you please tell me how to ignore it easily, do you support such a case?

Thank you! Best regards, Andrej

Not at all sure where something like this belongs, so I'm creating an issue in the hope that the team can give some guidance. When first starting with bowler, I struggled to wrap my head around how the lib2to3 patterns worked. The problem was compounded by the general lack of easily googleable information. To help myself understand (and after some reading though the source code), I came up with a basic way to play around:

from pprint import pprint

from fissix import pytree, pygram
from fissix.patcomp import PatternCompiler
from fissix.pgen2 import driver

d = driver.Driver(pygram.python_grammar.copy(), pytree.convert)
pc = PatternCompiler()

def test(pattern, code):
    pat = pc.compile_pattern(pattern)
    for node in d.parse_string(code).pre_order():
        for match in pat.generate_matches([node]):
            pprint(match)

# Anything calling .foo() with any args:
pattern="""
call=power<
  any*
  trailer<"." "foo">
  trailer<"(" args=any* ")">
>
"""
# Prints out details about both the f.foo and bar.foo calls
test(pattern,'a = f.foo(123) + bar.foo("adf")\n')

This test function made it much easier for me to start writing + understanding patterns because I could quickly iterate tests inside of a REPL.

Should something like this function be added to the website/documentation/somewhere else? Even without extensive documentation for the lib2to3 patterns (which would be a lot of work), it would be nice for new users to have a good starting point like this. (It also might be that this function already exists somewhere, and I just reinvented the wheel - whoops!).

Hi,

bowler/README.md documents rename_function however this functionality does not appear to exist. 'hope to see it added soon!

Happy New Year, Nicholas

Hi,

I'm packaging Bowler for Debian, and while working on the man page I wasn't able to find documentation for bowler test. Is it:

test [path | module] [-- options]

or something else? https://pybowler.io/docs/api-commands is missing this info too.

Other than that, I also found what appears to be either obsolete or yet-to-be-written functionality documented here: https://github.com/facebookincubator/Bowler/blame/master/bowler/README.md#L46

In Debian we have a policy of providing as complete documentation as possible so that a user can put the software on a disk, disappear to a desert island, install it there, and have the all information at hand that is necessary to learn to use the software proficiently without internet access. At this time, the documentation appears to require Github-specific tooling. Do you know where I can find that tooling? If for some reason that tooling is not permissively licensed enough to meet the Debian Free Software Guidelines, or flexible enough, would you be willing to accept a PR demonstrates full and careful conversion to sphinxdoc? I understand this would require changes to the website's config, which is why I'm offering to help out (probably in January). Why sphinxdoc? It's a great format for conversion to HTML, PDF, man, info, and even EPUB, and it has some nice automatic features. I think you'd like the tooling :-)

Here's a practical and to-the-point intro article: https://medium.com/@andreas.007/automate-documentation-in-python-925c38eae69f And some of the available themes: https://sphinx-themes.org

The HTML output is also supports fancy JS features, but by January I'll be limited to the ones that are already in Debian (we're going into a feature freeze then, so no new packages, and all packages and docs must be buildable without internet access).

Thank you for your work on Bowler! Nicholas