@jorgenschaefer, @aligrudi , @JulianEberius I would like to renew my effort to make rope acceptable for both Emacs and SublimeText world. I guess I have already lost Emacs people, but still I would like to make rope behave better.

Patch originally from #44 (authored by @JulianEberius) which unfortunately breaks on the current rope. For purposes of bisecting, I have split the patch into four commits, but git bisect still shows that both 7506026cae and b5eaf343c5 break tests. Any idea what's wrong?

An alternative could be to use jorgenschaefer/elpy@e75c963d7b ? @JulianEberius, would it solve problems rope had with SublimeText (or vice versa) which were supposed to be fixed by your patch?

Also, wouldn't it be better to use functools.lru_cache (aka repoze.lru.lru_cache) instead of doing our own utils.memoize?

Given https://github.com/Microsoft/vscode-python/issues/3867 I would like to consider relicensing whole rope project under another license more accessible to editors/IDE desiring to use rope.

It is not only attempt to butter VS Code developers up, but truly plain GPL doesn't make sense much for the project, which is essentially a library. Despite what some others say, I don't think tricking somebody into free software by spreading infection is not The Right Thing to do, and so I would prefer to relicense under an non-infectious license.

The question remains which license to use. In order to lessen friction (and because I like it), I would suggest LGPL3+ because it is the up-to-date version closest to the current license, and thus hopefully least controversial.

I would like to ask all you listed below add a comment with your agreement (just +1 is enough, but please make a separate comment so we have record of your name) if you agree with this change to LGPL3+. If you don't agree or if you really want a different license, please, make a comment as well. Of course, I give my +1 by filing this issue now.

This list should be hopefully list of all people who have more than two commits in the tree. If you are a committer and you are not listed below, please, let us know as well.

Thank you.

• [ ] @abingham
• [x] @aligrudi
• [x] @brendanator
• [x] @digenis
• [x] @dsyzling
• [x] @emacsway
• [x] @enomado
• [x] @ferada
• [x] @gwelymernans
• [x] @hayd
• [x] @hpapaxen
• [x] @jorgenschaefer
• [x] @MarekLew
• [x] @mcepl
• [x] @orestis
• [x] @piranha
• [x] @sergeyglazyrindev
• [x] @soupytwist
• [x] @stpierre
• [x] @tzing

Rope, by default, rewrites import statements by removing all the top-level import lines from the file, then sorting them, then reinserting them at the top. This behavior of pulling everything to the top of the file can be undesirable. As an alternative, we can rewrite the imports in place, which won't sort them, but also won't impose this extra constraint that all top-level imports must be at the top.

Modernize rope to install and run on Python 3 versions 3.3 and newer. See python-future (prefered) or six for a single-codebase route to support Python 2 and Python 3.

That exception occurs when it try to rename any class in :

https://github.com/vyapp/vy/blob/master/vyapp/areavi.py

The plugin for vy is:

https://github.com/vyapp/vy/blob/master/vyapp/plugins/rope.py

The way of how it works consists of passing one of the offsets of the object name thats gonna be renamed altogether with the new name as it is explained in the docs.

However it seem to work for smaller projects but failing with bigger ones.

I'm using python3.8 on an arch system.

The full excepltion below:

[tau@archlinux untwisted-code]$ git checkout *
Updated 1 path from the index
[tau@archlinux untwisted-code]$ cd ..
[tau@archlinux projects]$ vy vy-code/vyapp/areavi.py -v
Loading vyrc...
Project - Setting sentinels =  ('.git', '.svn', '.hg', '._')
Quick Search - Setting confs =  {'background': 'yellow', 'foreground': 'black'}
Find - Setting confs =  {'background': 'green', 'foreground': 'white'}
Sniper - Setting dirs = /home/tau/projects
Sniper - Setting ag path =  ag
Fstmt - Setting ag path =  ag
Snakerr - Setting Pyflakes path =  pyflakes
Snakerr - Setting Mypy path =  mypy
Deadcode - Setting Vulture path =  vulture
Project - Setting project path =  /home/tau/projects/vy-code
Exception in Tkinter callback
Traceback (most recent call last):
  File "/usr/lib/python3.8/tkinter/__init__.py", line 1883, in __call__
    return self.func(*args)
  File "/usr/lib/python3.8/site-packages/vyapp/tools.py", line 48, in shell
    return handle(*args, **kwargs)
  File "/usr/lib/python3.8/site-packages/vyapp/plugins/rope.py", line 119, in get_rename_data
    self.rename(ask.data)
  File "/usr/lib/python3.8/site-packages/vyapp/plugins/rope.py", line 125, in rename
    project = Project(path)
  File "/usr/lib/python3.8/site-packages/rope/base/project.py", line 223, in __init__
    self._init_prefs(prefs)
  File "/usr/lib/python3.8/site-packages/rope/base/project.py", line 273, in _init_prefs
    self._init_ropefolder()
  File "/usr/lib/python3.8/site-packages/rope/base/project.py", line 249, in _init_ropefolder
    config.write(self._default_config())
  File "/usr/lib/python3.8/site-packages/rope/base/resources.py", line 128, in write
    self._perform_change(change.ChangeContents(self, contents),
  File "/usr/lib/python3.8/site-packages/rope/base/resources.py", line 99, in _perform_change
    self.project.do(changes)
  File "/usr/lib/python3.8/site-packages/rope/base/project.py", line 113, in do
    self.history.do(changes, task_handle=task_handle)
  File "/usr/lib/python3.8/site-packages/rope/base/history.py", line 36, in do
    changes.do(change.create_job_set(task_handle, changes))
  File "/usr/lib/python3.8/site-packages/rope/base/change.py", line 65, in do
    change.do(job_set)
  File "/usr/lib/python3.8/site-packages/rope/base/change.py", line 126, in call
    function(self)
  File "/usr/lib/python3.8/site-packages/rope/base/change.py", line 150, in do
    self._operations.write_file(self.resource, self.new_contents)
  File "/usr/lib/python3.8/site-packages/rope/base/change.py", line 335, in write_file
    observer.resource_changed(resource)
  File "/usr/lib/python3.8/site-packages/rope/base/resourceobserver.py", line 30, in resource_changed
    self.changed(resource)
  File "/usr/lib/python3.8/site-packages/rope/base/pycore.py", line 65, in _file_changed_for_soa
    perform_soa_on_changed_scopes(self.project, resource, old_contents)
  File "/usr/lib/python3.8/site-packages/rope/base/pycore.py", line 302, in perform_soa_on_changed_scopes
    pycore.analyze_module(resource, should_analyze, search_subscopes)
  File "/usr/lib/python3.8/site-packages/rope/base/pycore.py", line 209, in analyze_module
    rope.base.oi.soa.analyze_module(
  File "/usr/lib/python3.8/site-packages/rope/base/oi/soa.py", line 15, in analyze_module
    _analyze_node(pycore, pymodule, should_analyze,
  File "/usr/lib/python3.8/site-packages/rope/base/oi/soa.py", line 23, in _analyze_node
    _analyze_node(pycore, scope.pyobject, should_analyze,
  File "/usr/lib/python3.8/site-packages/rope/base/oi/soa.py", line 38, in _analyze_node
    rope.base.ast.walk(child, visitor)
  File "/usr/lib/python3.8/site-packages/rope/base/ast.py", line 39, in walk
    return method(node)
  File "/usr/lib/python3.8/site-packages/rope/base/oi/soa.py", line 122, in _Assign
    pyobject = instance.get_object()
  File "/usr/lib/python3.8/site-packages/rope/base/pynamesdef.py", line 42, in get_object
    result = self.pyfunction.get_parameter(self.index)
  File "/usr/lib/python3.8/site-packages/rope/base/pyobjectsdef.py", line 69, in get_parameter
    if index < len(self.parameter_pyobjects.get()):
  File "/usr/lib/python3.8/site-packages/rope/base/utils/__init__.py", line 29, in newfunc
    return func(self, *args, **kwds)
  File "/usr/lib/python3.8/site-packages/rope/base/pynames.py", line 188, in get
    self.set(self.get_inferred(*args, **kwds))
  File "/usr/lib/python3.8/site-packages/rope/base/pyobjectsdef.py", line 40, in _infer_parameters
    pyobjects = rope.base.oi.soi.infer_parameter_objects(self)
  File "/usr/lib/python3.8/site-packages/rope/base/utils/__init__.py", line 41, in newfunc
    return func(*args, **kwds)
  File "/usr/lib/python3.8/site-packages/rope/base/oi/soi.py", line 47, in infer_parameter_objects
    result = _parameter_objects(pyfunction)
  File "/usr/lib/python3.8/site-packages/rope/base/oi/soi.py", line 130, in _parameter_objects
    type_ = hint_param(pyobject, name)
  File "/usr/lib/python3.8/site-packages/rope/base/oi/type_hinting/providers/inheritance.py", line 21, in __call__
    result = self._delegate(superfunc, param_name)
  File "/usr/lib/python3.8/site-packages/rope/base/oi/type_hinting/providers/composite.py", line 19, in __call__
    result = delegate(pyfunc, param_name)
  File "/usr/lib/python3.8/site-packages/rope/base/oi/type_hinting/providers/docstrings.py", line 46, in __call__
    type_strs = self._parse_docstring(pyfunc.get_doc(), param_name)
  File "/usr/lib/python3.8/site-packages/rope/base/pyobjects.py", line 224, in get_doc
    isinstance(expr.value, ast.Str):
AttributeError: module 'rope.base.ast' has no attribute 'Str'

@mcepl, I'm in the process of trying to resolve the issues in rope library when running under Python 3.5. So far I've made two PRs. Over the next few weeks/months, more issues will be fixed (that's the intent), hence more PRs will be submitted.

I'm going to assume that most of the future fixes will rely on older fixes that I may have made. for instance the PR #169 is crucial to getting #170 working. However if I keep adding all the changes to the same PR it could get quite complicated for you to review the changes.

Let me know what you prefer.

• Keep updating the original PR #169 with new fixes (feels like the best solution in my opinion). However when you start reviewing you could have a who lot of files, and it would be best if you inform me when you start reviewing, so I can pause submitting more changes to the same PR)
• Create news PRs for each fix (however fixes from previous PRs will have to be added into them to ensure unit tests pass). This approach feels too complicated, but you'll have fewer changes to review.
• Or any other

A preliminary PR for PR #632.

Background

rope.base.ast.call_for_nodes looks fishy to me:

• It defines neither a depth first nor a breadth first traversal, but a strange partial mix. I know of no simple description of what it does.
• It is used in two search-related contexts, but in each case only a single node is returned.
• Almost any change to call_for_nodes causes multiple unit tests to fail.

@lieryan Is there a simple explanation for what call_for_nodes does?

Summary of changes

• [x] Replace calls to call_for_nodes where the recursive kwarg is (implicitly) false. See patched_ast and _PatchingASTWalker._handle.
• [x] Remove the recursive kwarg from call_for_nodes.
• [x] Remove the call to call_for_nodes in _ASTMatcher.find_matches.
• [x] Remove the call to call_for_nodes in _ResultChecker._find_node.
• [x] Remove call_for_nodes itself.

Why call_for_nodes should not be a utility

I understand neither what _ASTMatcher.find_matches and _ResultChecker._find_node are supposed to do nor how they do it. Imo, these strange traversals should be presented adjacent to the search code. Duplicating these traversals is better than creating a strange helper.

See Issue #570 for previous discussions.

This PR disambiguates rope.base.ast and stdlib.ast with small, easily understood, diffs.

• [x] Rename rope.base.ast to rope.base.rast.
• [x] Remove from ast import * from rast.py.
• [x] Use rast only to access rast.parse, rast.call_for_nodes, and class rast.RopeNodeVisitor(ast.NodeVisitor)
• [x] Add import ast statements as needed.
• [x] In numpydocstrings.py, add ast qualifier to literal_eval.
• [x] In module_imports, temporarily mark a dubious comment with ###.

Advantages

• The code makes clear from immediate context alone which names come from stdlib.ast and which names come from rope.base.rast.
• The diffs show that there is no cost to this new clarity.
• This PR removes misdirections (lies) based on rope.base.ast. For example, rope.base.ast.iter_child_nodes actually means ast.iter_child_nodes! This mistake was another legacy of from ast import *.
• This PR distinguishes rast.parse from ast.parse.
• This PR makes it easier to wrap stdlib.ast should that ever become desirable.
• Eliminating from ast import * helps all checker tools.

Note: Several of these advantages became clear only after looking at the diffs!

Note: It took longer to write this comment than it did to change the code.

Description

Fixes #401 Use pyproject.toml based configuration via pytoolconfig.

Behavior is now

1. Check if [tool.rope] is in pyproject.toml. If so, use that configuration
2. Check if config.py exists. If so, use that configuration. (Supports project_opened and callable functions)
3. Use default config specified in pydantic model

Rope will no longer create a config.py, but instead document each configuration key. All the keys have been placed in rope/base/prefs.py. Future keys must be placed and documented here.

Documentation

The project now uses sphinx to generate documentation (includeing configuration keys). It should work with read the docs out of the box.

Pyproject.toml

I moved setuptools metadata to pyproject.toml https://setuptools.pypa.io/en/latest/userguide/quickstart.html - use config.py stub

Python 3.6

Python 3.6 is unsupported

• setuptools doesn't work properly with pyproject.toml on python 3.6. (It may have been my fault, I can check)
• tomli doesn't support python3.6 with their latest versions
• python doesn't support python 3.6
• my library, pytoolconfig doesn't support python 3.6 (but this can be fixed if needed)

I will implement the ability for rope to target old python versions automatically in a future PR, but this PR drops 3.6 support.

Pathlib

I added a pathlib method to the resource object to ease use of pathlib. I'm not sure if I used the correct name for it or should remove it.

Checklist (delete if not relevant):

• [x] I have added tests that prove my fix is effective or that my feature works (One test)
• [x] I will resolve the config.py generation issue - doesn't generate it.
• [x] I will add minimum_py_version. This will allow rope to detect the lowest python version used by the project, then target it. This way, rope can be run in up-to-date versions of python (3.7+) and be used on (3.4) - preference added but not used.
• [x] I will add a list of dependencies. This will allow autoimport to only cache modules that are used by the project. Will do after #469
• [x] I have updated CHANGELOG.md
• [x] I have made corresponding changes to user documentation for new features
• [x] I have made corresponding changes to library documentation for API changes - should be api compatible

This builds upon the async def PR to add support for all of the new async-related nodes.

A few notes:

• It didn't seem necessary to do anything with the Await node, so I didn't add it to the AST walker.
• I'm not entirely sure if aliasing _With for _AsyncWith is correct. rope.base.pyobjectsdef._ScopeVisitor._With is doing something with __enter__, but the equivalent for async with would be __aenter__. Everything seems to work, but a more expert eye would certainly be welcome.
• Do we need to do anything with rope.base.pyobjectsdef._ClassInitVisitor? What about _SuiteWalker and _PatchingASTWalker?

I'm having trouble making rope work with code that uses async and await. For example, with code like this

def some_name():
    return 42

some_name()

I can run a rename factoring from some_name to some_other_name.

But if I do something like this like this:

import asyncio

def some_name():
    return 42

loop = asyncio.get_event_loop()
loop.run_until_complete(some_name())

and try the same refactoring, I get a RefactoringError:

  File "/Users/abingham/.virtualenvs/traad/lib/python3.6/site-packages/rope/refactor/multiproject.py", line 26, in __call__
    self.addpath, project, *args, **kwds)
  File "/Users/abingham/.virtualenvs/traad/lib/python3.6/site-packages/rope/refactor/multiproject.py", line 42, in __init__
    self.refactoring(other, *args, **kwds))
  File "/Users/abingham/.virtualenvs/traad/lib/python3.6/site-packages/rope/refactor/rename.py", line 28, in __init__
    'Rename refactoring should be performed'
rope.base.exceptions.RefactoringError: Rename refactoring should be performed on resolvable python identifiers.

Should rope work with this code? Am I missing something?

(FWIW, I'm doing the refactoring through traad, but I don't see how that would interfere with rope's support for the language.)

Description

These config settings tells rope whether to cache the projectdb and retain undo history respectively. We should just always save these files.

Rope should just always have sensible default behaviors, there should be no reason for users to disable these config. In order to minimize the number of unnecessary settings and keeping the config file simple, these settings are being deprecated.

Is your feature request related to a problem? Please describe.

#456 adds pattern matching to rope, but it seems like support is incomplete as patchedast currently doesn't handle these nodes:

'MatchOr', 'MatchSequence', 'MatchSingleton', 'MatchStar',

This will likely cause extract refactoring to have issues when these syntaxes are being used.

Describe the solution you'd like

Implement these nodes.

mypy issues 6 errors in master:

rope\base\pyobjects.py:304: error: "_PyModule" has no attribute "get_name"
rope\base\pynamesdef.py:6: error: Name "AssignedName" already defined (possibly by an import)
rope\base\pynamesdef.py:38: error: Name "ParameterName" already defined (possibly by an import)
rope\base\project.py:14: error: Module "rope.base.prefs" has no attribute "Prefs"
rope\base\project.py:14: error: Module "rope.base.prefs" has no attribute "get_config"
rope\refactor\importutils\module_imports.py:50: error: "AssignedName" has no attribute "assignments"

The following errors disappear with PR #613:

rope\base\pynamesdef.py:6: error: Name "AssignedName" already defined (possibly by an import)
rope\base\pynamesdef.py:38: error: Name "ParameterName" already defined (possibly by an import)
rope\refactor\importutils\module_imports.py:50: error: "AssignedName" has no attribute "assignments"

Subtracting these three errors, the following errors are worth a closer look.

rope\base\pyobjects.py:304: error: "_PyModule" has no attribute "get_name"
rope\base\project.py:14: error: Module "rope.base.prefs" has no attribute "Prefs"
rope\base\project.py:14: error: Module "rope.base.prefs" has no attribute "get_config"

• The warning about _PyModule is valid. It probably should be fixed in a new PR.
• rope.base.prefs does have Prefs and get_config attributes. I am investigating.

To do:

• [x] Suppress complaints re optional imports such as cPickle/pickle, pysvn, and mercurial.*.
• [x] Suppress mypy checks entirely in rope.base.prefs and rope/base/oi/type_hinting/evaluate.py.
• [x] Annotate lists and dicts.
• [x] Add more difficult (experimental) annotations.

pickle is insecure and it's not very efficient for larger, more complex datasets.

This will be a multiple steps plan towards moving into sqlite-backed storage:

1. [x] Remove rarely used features
   1. [x] Deprecate objectdb and history compression: This was never the default and disk space nowadays makes this unlikely to be an issue nowadays, maybe we can re-introduce compression when sqlite migration is complete (#606)
   2. [x] Deprecate importing of old format (objectdb.pickle and history.pickle): rope hasn't been generating this for decades (#607)
2. [ ] Make ScopeInfo JSON-serializable: there are some data here that are python class instances, but they really doesn't need to be, we can just __getstate__()/__setstate__() them instead when de/serializing. #611
3. [ ] Implement sqlite database schema versioning #565
4. [ ] These files contains a bunch of nested dictionaries, these can be translated to sqlite tables in a straightforward manner, we'll likely will still store JSON documents inside the sqlite tables instead of normalizing everything
5. [ ] Make sure that the tables have appropriate indexes
6. [ ] Remove write support for legacy objectdb and history files
7. [ ] Remove read support for legacy objectdb and history files

@lieryan said that few if any Rope uses seem to be using dynamic inference.

It so, removing the doi code might suggest other simplifications.

In turn, those simplifications might help the proposed aoi inference, #568.

@lieryan Your comments, please.