• Python 3.6.6 on Ubuntu as a WSL on Windows 10
• wily==1.8.0 (btw would be nice to see version number when using "wily" or "wily help"
• fairly big project
• after like 10-15 minutes I receive the following output

Found 50 revisions from 'git' archiver in '/mnt/d/Projects/bliss_deployment/work/zope/checkout/bliss.git'.
Running operators - raw,cyclomatic,maintainability


Failed to build cache: ''
multiprocessing.pool.RemoteTraceback:
"""
Traceback (most recent call last):
  File "/usr/lib/python3.6/multiprocessing/pool.py", line 119, in worker
    result = (True, func(*args, **kwds))
  File "/usr/lib/python3.6/multiprocessing/pool.py", line 47, in starmapstar
    return list(itertools.starmap(args[0], args[1]))
  File "/mnt/d/Projects/envs/wily/lib/python3.6/site-packages/wily/commands/build.py", line 24, in run_operator
    return operator.name, instance.run(revision, config)
  File "/mnt/d/Projects/envs/wily/lib/python3.6/site-packages/wily/operators/cyclomatic.py", line 81, in run
    raise TypeError
TypeError
"""

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/mnt/d/Projects/envs/wily/bin/wily", line 11, in <module>
    sys.exit(cli())
  File "/mnt/d/Projects/envs/wily/lib/python3.6/site-packages/click/core.py", line 764, in __call__
    return self.main(*args, **kwargs)
  File "/mnt/d/Projects/envs/wily/lib/python3.6/site-packages/click/core.py", line 717, in main
    rv = self.invoke(ctx)
  File "/mnt/d/Projects/envs/wily/lib/python3.6/site-packages/click/core.py", line 1137, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/mnt/d/Projects/envs/wily/lib/python3.6/site-packages/click/core.py", line 956, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/mnt/d/Projects/envs/wily/lib/python3.6/site-packages/click/core.py", line 555, in invoke
    return callback(*args, **kwargs)
  File "/mnt/d/Projects/envs/wily/lib/python3.6/site-packages/click/decorators.py", line 17, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "/mnt/d/Projects/envs/wily/lib/python3.6/site-packages/wily/__main__.py", line 123, in build
    operators=resolve_operators(config.operators),
  File "/mnt/d/Projects/envs/wily/lib/python3.6/site-packages/wily/commands/build.py", line 125, in build
    raise e
  File "/mnt/d/Projects/envs/wily/lib/python3.6/site-packages/wily/commands/build.py", line 84, in build
    [(operator, revision, config) for operator in operators],
  File "/usr/lib/python3.6/multiprocessing/pool.py", line 296, in starmap
    return self._map_async(func, iterable, starmapstar, chunksize).get()
  File "/usr/lib/python3.6/multiprocessing/pool.py", line 670, in get
    raise self._value
TypeError```

The rank command utilized the report command as the initial template.

There are some outstanding items that could use some improvement or are lacking at the moment:

1. informational metrics are not considered at the moment
2. sub-folders within a path are ignored
3. the referencing of the revision number feels a little too brittle/sensitive
4. the click / CLI aspect of the command still needs to be implemented

This PR is quite big and changes several things:

• fix main deps declaration https://github.com/tonybaloney/wily/commit/52c86c5083addcb71371cdc6efc901ecf5711326
• declare dev and test deps in pyproject.toml using flit extras
• add dev/test tools configuration to proper config files
• add linting and testing stages to Makefile
• removes tox support (flit and Makefile should be used for local development)
• merges main and lint_python workflows

Things to do (but probably in separate PRs):

• [x] add better coverage configuration - @skarzi
• [x] investigate why tests fail and fix them
• [x] add .editorconfig and clean whitespaces in the codebase - check #158
• [ ] update .gitignore (e.g. with wily_report and foo)
• [ ] move codecov token to secrets - @tonybaloney
• [x] set proper black configuration, apply it on the codebase and require it in Makefile
• [x] set proper isort configuration, apply it on the codebase and require it in Makefile
• [ ] resolve flake8 issues and clean ignore in its config and set its proper configuration
• [ ] set proper codespell configuration, apply it on the codebase and require it in Makefile
• [ ] set proper mypy configuration, resolve its issues and require it in Makefile
• [ ] add some useful pytest plugins, like pytest-timeout or pytest-randomly

Describe the bug I have added a specific job to my Github Workflow without installing wily locally and Wily fails with a git error:

Failed to setup archiver: '<class 'git.exc.GitCommandError'> - Cmd('git') failed due to: exit code(128)
  cmdline: git diff --numstat 4b2122317b4775a75d30403fc158ea89bac3d204 ce16aee57f19e7e948a4794779d5aa5135b64e8f --
  stderr: 'fatal: bad object 4b2122317b4775a75d30403fc158ea89bac3d204''

To Reproduce Add a job to a Github pipeline to your code like so:

  evaluate-complexity:
    name: Evaluate code complexity
    runs-on: ubuntu-latest
    steps:
    - name: Checkout repository
      uses: actions/checkout@v2
    - name: Set up Python
      uses: actions/setup-python@v2
      with:
        python-version: 3.10.0
    - name: Install Wily and compile cache
      run: |
            pip install wily
            wily build my_proj
    - name: Compute code complexity diff with main
      run: "wily diff my_proj/ -r main"
      id: run_wily

Expected behavior Build would succeed

Additional context It's interesting that the git diff command copy-pasted on my machine works correctly

Setting exit status via predefined threshold will allow using wily via CI. Relating to issue #66.

Changes

• Added ability to pass thresholds to diff command per operator

Usage:

$ wily diff [FILES] --thresholds halstead.h1=1,raw.loc=50

TODO

• [x] Check for metric type when determining if threshold was passed or not
• [ ] Handle diffs for scenarios where files/modules were renamed, resulting in - as the base value
• [x] Add ability to pass threshold via config file
• [ ] Add more tests
• [ ] Have a nicer output for errors
• [ ] Add logging

I attempted to run wily build nengo/ on the master branch of https://github.com/nengo/nengo and I get the following error:

~/C/nengo (nengo) wily build nengo/                                           (master) 
Found 50 revisions from 'git' archiver in '/home/tbekolay/Code/nengo'.
Running operators - maintainability,halstead,cyclomatic,raw
Processing |                                | 1/200Failed to build cache: 'unsupported operand type(s) for +: 'int' and 'dict''
Traceback (most recent call last):
  File "/home/tbekolay/.virtualenvs/nengo/bin/wily", line 10, in <module>
    sys.exit(cli())
  File "/home/tbekolay/.virtualenvs/nengo/lib/python3.7/site-packages/click/core.py", line 764, in __call__
    return self.main(*args, **kwargs)
  File "/home/tbekolay/.virtualenvs/nengo/lib/python3.7/site-packages/click/core.py", line 717, in main
    rv = self.invoke(ctx)
  File "/home/tbekolay/.virtualenvs/nengo/lib/python3.7/site-packages/click/core.py", line 1137, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/home/tbekolay/.virtualenvs/nengo/lib/python3.7/site-packages/click/core.py", line 956, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/home/tbekolay/.virtualenvs/nengo/lib/python3.7/site-packages/click/core.py", line 555, in invoke
    return callback(*args, **kwargs)
  File "/home/tbekolay/.virtualenvs/nengo/lib/python3.7/site-packages/click/decorators.py", line 17, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "/home/tbekolay/.virtualenvs/nengo/lib/python3.7/site-packages/wily/__main__.py", line 132, in build
    operators=resolve_operators(config.operators),
  File "/home/tbekolay/.virtualenvs/nengo/lib/python3.7/site-packages/wily/commands/build.py", line 127, in build
    raise e
  File "/home/tbekolay/.virtualenvs/nengo/lib/python3.7/site-packages/wily/commands/build.py", line 116, in build
    result[str(root)][metric.name] = func(values)
TypeError: unsupported operand type(s) for +: 'int' and 'dict'

This is a little puzzling since the line that it's pointing to doesn't contain an addition statement, as far as I can tell.

This was run in a Python 3.7 virtual environment with wily version 1.12.2. Let me know if I can provide any more information to help with debugging.

This project uses black, but that's not a requirement to execute it. It also uses f-strings, but those could be replaced with str.format easily

Please 👍 if you want to see this back-ported to 3.5

I discovered that the aggregated statistics were not properly computed.

In particular in case of file movements, file deletions and deeply nested directories.

This PR should resolve the issue. Please validate if this does not break other functionality (as I am relatively new to the code base.)

Hello,

first of all, thanks for developing wily!

Is your feature request related to a problem? Please describe.

Adding cache_path option to wily.cfg will make using custom cache path much, much easier, because developers won't be force to add --cache CLI option in every command call.

Describe the solution you'd like

Add cache_path to wily.cfg.

Maintainability Index scale from 0 to 100 where the lower the value, the worst the Maintainability Index. The diff command set red colors for increasing values of all metrics, but an increasing Maintainability Index index is actually good.

Signed-off-by: Willian Rampazzo [email protected]

wily version 1.12.1, installed via pip in a clean venv with python 3.6.7 on ubuntu.

I am trying to run wily build on a project of mine. Unfortunately I cannot make it work. It finds an "invalid character" but there is none. See screenshot of relevant code piece below. Even deleting the file from the repo does not help because of course it is still in the git history (and I am not going to soft-reset through 30 commits to get wily to work).

Even if there were an invalid character, couldn't there be a way to more gracefully handle this and simply exclude that file from the index/reporting?

Key error: Failed to run CC harvester on dashtry.py : invalid character in identifier (<unknown>, line 29) Following stack:

multiprocessing.pool.RemoteTraceback: 
"""
Traceback (most recent call last):
  File "/usr/lib/python3.6/multiprocessing/pool.py", line 119, in worker
    result = (True, func(*args, **kwds))
  File "/usr/lib/python3.6/multiprocessing/pool.py", line 47, in starmapstar
    return list(itertools.starmap(args[0], args[1]))
  File "/Data/Code/projects/finance_data/.venv/lib/python3.6/site-packages/wily/commands/build.py", line 24, in run_operator
    return operator.name, instance.run(revision, config)
  File "/Data/Code/projects/finance_data/.venv/lib/python3.6/site-packages/wily/operators/halstead.py", line 81, in run
    results[filename] = self._report_to_dict(instance)
  File "/Data/Code/projects/finance_data/.venv/lib/python3.6/site-packages/wily/operators/halstead.py", line 86, in _report_to_dict
    "h1": report.h1,
AttributeError: 'str' object has no attribute 'h1'
"""

Running wily subsequently on a subdirectory leads to this, so apparently wily leaves "corrupted" files behind when exiting in the above fashion:

output of pip freeze: attrs==19.1.0 certifi==2019.3.9 chardet==3.0.4 Click==7.0 colorama==0.4.1 colorlog==4.0.2 dataclasses==0.6 decorator==4.3.2 entrypoints==0.3 flake8==3.7.7 flake8-polyfill==1.0.2 gitdb2==2.0.5 GitPython==2.1.11 idna==2.8 ipython-genutils==0.2.0 jsonschema==3.0.1 jupyter-core==4.4.0 mando==0.6.4 mccabe==0.6.1 nbformat==4.4.0 pkg-resources==0.0.0 plotly==3.7.0 progress==1.5 pycodestyle==2.5.0 pyflakes==2.1.1 pyrsistent==0.14.11 pytz==2018.9 radon==3.0.1 requests==2.21.0 retrying==1.3.3 six==1.12.0 smmap2==2.0.5 tabulate==0.8.3 traitlets==4.3.2 urllib3==1.24.1 wily==1.12.1

Regardless of where the build command is executed (whether it is filesystem for a git folder). It is always says No data collected. I have tested it on host and also from inside the container. But the result is the same. I have the analysis of module level. But I don't have the analysis for function level inside the module.

wily index is OK there no problem in output of it:

Is your feature request related to a problem? Please describe. Our use case assumes using wily in the unit tests to validate different metrics thresholds. It is not the best practice to parse cli string to get the metrics, instead, it would be very useful to work with python packages, but without accessing output results in the objects for diff and rank methods it's impossible

Describe the solution you'd like If we add return results to the diff method and return data to the rank method, we can work with wily in unit tests and operate output in better way

Describe alternatives you've considered Parsing cli

While using Wily, I noted that a class with an explicit constructor and a single method has a complexity of three or four. This is the same whatever change I make to the class or its method. I would like to know if this is expected behavior. @tonybaloney

Is your feature request related to a problem? Please describe. I would like to identify specific candidates for refactoring. wily rank reports metrics by file which is too coarse grained for this

Describe the solution you'd like something like wily rank --functions ./src which woul return a ranke list of functions rather than files

Describe alternatives you've considered Radon reports cyclomatic complexity (only) by function but bins the scores into letter grades. I would like to get the actual score so I can view say the top 20 functions in a package

Is your feature request related to a problem? Please describe. There are other Complexity metrics not included in this repo, that can be included.

Additional context

• Koopman Spaghetti Factor https://github.com/ElectronVector/ravioli
• "Churn" (variable changes) https://github.com/simonrenoult/code-complexity
• Cognitive Complexity https://pypi.org/project/cognitive-complexity/
• Indentation Complexity https://github.com/thoughtbot/complexity

https://github.com/priv-kweihmann/multimetric

• Difficulty according to Halstead
• Effort according to Halstead
• Fan-Out
• Maintainability index
• Metric according to pylint
• Metric according to TIOBE
• Number of delivered bugs according to Halstead
• Time required to program according to Halstead
• Volume according to Halstead

With https://github.com/mxmssh/IDAmetrics

• Basic blocks count (function/module)
• Functions count (module)
• Conditions count (function/module)
• Assignments count (function/module)
• Jilb's metric (function/module)
• ABC metric (function/module)
• Pivovarsky metric (function/module)
• Harrison metric (function/module)
• Boundary value metric (function/module)
• Span metric (function/module)
• Global variables access count (function/module)
• Oviedo metric (function/module)
• Chepin metric (function/module)
• Card & Glass metric (function/module)
• Henry & Cafura metric (function/module)
• Cocol metric (function/module)