flake8 plugin that integrates isort

Hey there, I have these import statements at the top of my module:

import os

from flask import Blueprint, abort, current_app, jsonify
from flask.views import MethodView
from webargs import fields, missing
from webargs.flaskparser import use_kwargs
from werkzeug.utils import secure_filename

from tcst_api.models.data import DataContract, DataFile
from tcst_api.schemas.data import DataContractSchema, DataFileSchema

When I run flake8-isort on the file I get the following output:

tcst_api/views/data.py:2:1: I001 isort found an import in the wrong position
tcst_api/views/data.py:4:1: I003 isort expected 1 blank line in imports, found 0
tcst_api/views/data.py:4:1: I003 isort expected 1 blank line in imports, found 0
tcst_api/views/data.py:4:1: I003 isort expected 1 blank line in imports, found 0
tcst_api/views/data.py:4:1: I003 isort expected 1 blank line in imports, found 0
tcst_api/views/data.py:4:1: I003 isort expected 1 blank line in imports, found 0
tcst_api/views/data.py:4:1: I003 isort expected 1 blank line in imports, found 0
tcst_api/views/data.py:4:1: I003 isort expected 1 blank line in imports, found 0
tcst_api/views/data.py:4:1: I003 isort expected 1 blank line in imports, found 0
tcst_api/views/data.py:4:1: I003 isort expected 1 blank line in imports, found 0
tcst_api/views/data.py:4:1: I003 isort expected 1 blank line in imports, found 0
tcst_api/views/data.py:4:1: I003 isort expected 1 blank line in imports, found 0
tcst_api/views/data.py:4:1: I003 isort expected 1 blank line in imports, found 0
tcst_api/views/data.py:4:1: I003 isort expected 1 blank line in imports, found 0
tcst_api/views/data.py:4:1: I003 isort expected 1 blank line in imports, found 0
tcst_api/views/data.py:4:1: I003 isort expected 1 blank line in imports, found 0
tcst_api/views/data.py:4:1: I003 isort expected 1 blank line in imports, found 0
tcst_api/views/data.py:4:1: I003 isort expected 1 blank line in imports, found 0
tcst_api/views/data.py:4:1: I003 isort expected 1 blank line in imports, found 0
tcst_api/views/data.py:4:1: I003 isort expected 1 blank line in imports, found 0
tcst_api/views/data.py:4:1: I003 isort expected 1 blank line in imports, found 0

Which is kinda meaningless and seems to be broken. However if I run isort on the same content I get no errors as expected.

I'm using the following flake8 and isort versions:

flake8 - 3.5.0
flake8-isort - 2.3
isort - 4.3.4

Can you please help me with this?

Hi,

This PR adds support for isort >=5 while retaining the code for isort 4.

The new code parses the unified diff output produced by isort.api.check_file and isort.api.check_code_string.

isort5 only reads one config file and uses a specific order. So the tests need their own temporary dir each. So I started to rewrite the tests and because the self.assertEqual() calls in the unit tests made it hard to debug, I moved everything to pytest fixtures.

Pytest is used by the CI anyway.

I started to work on a PR to support isort 5 before I saw https://github.com/gforcada/flake8-isort/issues/88#issuecomment-657172965, but I think it is the same approach. Please review.

Fixes #88

I just installed this extension but there is no error output for any of the files that fail isort. I added a print statement before the yield so the config is fine and it is getting the error from isort but seems to not be passed along or displayed by flake8.

$ flake8 --version
3.2.0 (pyflakes: 1.3.0, flake8_isort: 2.0, mccabe: 0.5.2, pycodestyle: 2.2.0) CPython 2.7.12 on Linux

Hi!

I was profiling flake8 and found that ~90% of the runtime was just isort waiting for git subprocesses in order to implement skip_gitignore = True. (isort doesn't use libgit2 or anything, it just starts short-lived subprocesses targeting the git binary.)

skip_gitignore = True is not very useful when running flake8-isort since flake8 uses its own ignore list already. This PR adds an option --isort-no-skip-gitignore that allows users to force skip_gitignore = False when running flake8-isort without having to remove skip_gitignore = True from their isort config.

On the library I was profiling, --isort-no-skip-gitignore brought the flake8 runtime from ~50 s to ~5 s. Note that this measurement is on a Linux box. On Windows the speedup provided by this option appears to be much larger (apparently subprocess is extremely slow on Windows?). I actually did not finish the measurement on a Windows box because without --isort-no-skip-gitignore, flake8 was taking > 5 min.

Note that skip_gitignore was added in isort 5.2, so I added the option to Flake8Isort5 rather than Flake8IsortBase.

Hi,

I'm using flake8-isort plugin in my projects and since isort released the new version, v5.0.0, the plugin stopped working. Here is the log with traceback:

Traceback (most recent call last):
  File "/app/.tox/lint/lib/python3.6/site-packages/flake8/plugins/manager.py", line 157, in load_plugin
    self._load()
  File "/app/.tox/lint/lib/python3.6/site-packages/flake8/plugins/manager.py", line 134, in _load
    self._plugin = self.entry_point.load()
  File "/app/.tox/lint/lib/python3.6/site-packages/importlib_metadata/__init__.py", line 105, in load
    module = import_module(match.group('module'))
  File "/usr/local/lib/python3.6/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 994, in _gcd_import
  File "<frozen importlib._bootstrap>", line 971, in _find_and_load
  File "<frozen importlib._bootstrap>", line 955, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 665, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 678, in exec_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "/app/.tox/lint/lib/python3.6/site-packages/flake8_isort.py", line 3, in <module>
    from isort import SortImports
ImportError: cannot import name 'SortImports'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/app/.tox/lint/bin/flake8", line 8, in <module>
    sys.exit(main())
  File "/app/.tox/lint/lib/python3.6/site-packages/flake8/main/cli.py", line 22, in main
    app.run(argv)
  File "/app/.tox/lint/lib/python3.6/site-packages/flake8/main/application.py", line 360, in run
    self._run(argv)
  File "/app/.tox/lint/lib/python3.6/site-packages/flake8/main/application.py", line 347, in _run
    self.initialize(argv)
  File "/app/.tox/lint/lib/python3.6/site-packages/flake8/main/application.py", line 328, in initialize
    self.find_plugins(config_finder)
  File "/app/.tox/lint/lib/python3.6/site-packages/flake8/main/application.py", line 159, in find_plugins
    self.check_plugins.load_plugins()
  File "/app/.tox/lint/lib/python3.6/site-packages/flake8/plugins/manager.py", line 415, in load_plugins
    plugins = list(self.manager.map(load_plugin))
  File "/app/.tox/lint/lib/python3.6/site-packages/flake8/plugins/manager.py", line 302, in map
    yield func(self.plugins[name], *args, **kwargs)
  File "/app/.tox/lint/lib/python3.6/site-packages/flake8/plugins/manager.py", line 413, in load_plugin
    return plugin.load_plugin()
  File "/app/.tox/lint/lib/python3.6/site-packages/flake8/plugins/manager.py", line 164, in load_plugin
    raise failed_to_load
flake8.exceptions.FailedToLoadPlugin: Flake8 failed to load plugin "I00" due to cannot import name 'SortImports'.

This PR introduces new warning output style. It now includes isort diff with how to fix your warning.

Works the same way as pytest-isort.

Problem (multiple spaces):

import   os

Old warning:

flake8_isort.py:7:1: I001 isort found an import in the wrong position
flake8_isort.py:7:7: E271 multiple spaces after keyword

New warning:

flake8_isort.py:7:1: I001 isort found an import in the wrong position

 from os.path import expanduser
 from testfixtures import OutputCapture
 
-import   os
+import os
 
 
 try:

flake8_isort.py:7:7: E271 multiple spaces after keyword

Closes #60

Just encountered a weird issue:

> pip freeze | grep flake8-isort
flake8-isort==5.0.0

but flakeheaven tells me something different:

> flakeheaven plugins

NAME                         | VERSION  | CODES               | RULES
...
flake8-isort                 | 4.2.1    | I00                 | +*
...

I think this is because 4.2.1 is set here as version string: https://github.com/gforcada/flake8-isort/blob/master/flake8_isort.py#L12 Even though there is no such version released on pypi, could you maybe double check?

Files with changes are moved to a temporal directory when flake8 git hook is used.

$ flake8 --install-hook git

flake8-isort should look for configuration files in the current directory while checking the files in the temporary directory.

Before fix, no configuration is found.

$ git commit -m "Commit message" script.py:0:1: I002 no configuration found (.isort.cfg or [isort] on setup.cfg)

After fix, project configuration is found.

$ git commit -m "Commit message" script.py:1:1: D200 One-line docstring should fit on one line with quotes

On version 2.1.0 and 2.1.1, empty __init__.py files triggers an exception.

$ > __init__.py
$ flake8 __init__.py
Process Process-1:
Traceback (most recent call last):
  File "/Users/mnencia/.pyenv/versions/2.7.12/Python.framework/Versions/2.7/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap
    self.run()
  File "/Users/mnencia/.pyenv/versions/2.7.12/Python.framework/Versions/2.7/lib/python2.7/multiprocessing/process.py", line 114, in run
    self._target(*self._args, **self._kwargs)
  File "/Users/mnencia/.pyenv/versions/2.7.12/envs/test/lib/python2.7/site-packages/flake8/checker.py", line 666, in _run_checks_from_queue
    checker.run_checks(results_queue, statistics_queue)
  File "/Users/mnencia/.pyenv/versions/2.7.12/envs/test/lib/python2.7/site-packages/flake8/checker.py", line 606, in run_checks
    self.run_ast_checks()
  File "/Users/mnencia/.pyenv/versions/2.7.12/envs/test/lib/python2.7/site-packages/flake8/checker.py", line 517, in run_ast_checks
    for (line_number, offset, text, check) in runner:
  File "/Users/mnencia/.pyenv/versions/2.7.12/envs/test/lib/python2.7/site-packages/flake8_isort.py", line 70, in run
    for line_num, message in self.sortimports_linenum_msg(sort_result):
  File "/Users/mnencia/.pyenv/versions/2.7.12/envs/test/lib/python2.7/site-packages/flake8_isort.py", line 116, in sortimports_linenum_msg
    self._fixup_sortimports_wrapped(sort_result)
  File "/Users/mnencia/.pyenv/versions/2.7.12/envs/test/lib/python2.7/site-packages/flake8_isort.py", line 172, in _fixup_sortimports_wrapped
    for idx, line in enumerate(sort_imports.out_lines):
AttributeError: 'SortImports' object has no attribute 'out_lines'

On 2.0.3 it works as expected.

Empty files with a different name don't trigger the error.

I am not sure where or how to fix it, but I found out that .isort.cfg is not looked for recursively. It gets looked up in cwd and in the home directory. Right now I need to put an .isort.cfg in my project directory, because there I start vim and it looks at cwd. Then I need to symlink the .isort.cfg to src/package_name, because this is the cwd when this plugin gets called

Is there any special reason that GPL version 2.0 has been chosen for flake8-isort?

isort and flake8 are both released under MIT. Using flake8-isort some users might have licensing issues.

I'd like to use flake8-isort in my project, but I can't because it is extremely slow. Here is a script to reproduce the issue.

import os
import shutil
from pathlib import Path
from time import time

def prepare_files():
    shutil.rmtree("files", ignore_errors=True)
    Path("files").mkdir(exist_ok=True)
    content = "\n".join(10000 * ["def a(): pass"])
    for i in range(100):
        Path(f"files/file{i:03d}.py").write_text(content)

def execute(cmd):
    start = time()
    os.system(cmd)
    duration = time() - start
    return duration

if __name__ == "__main__":
    prepare_files()
    isort_duration = execute("isort --check-only files")
    print(f"isort: {isort_duration:.3f} sec")
    flake8_duration = execute("flake8 --select I001,I002,I003,I004,I005 files")
    print(f"flake8: {flake8_duration:.3f} sec")
    print(f"ratio: {flake8_duration / isort_duration:.1f}")

I see following numbers on my machine (the same is on github actions).

isort: 2.511 sec
flake8: 394.109 sec
ratio: 157.0

Is it possible to do something with it?

Versions:

Python: 3.9.7
flake8: 3.9.2
flake8-isort: 4.0.0
isort: 5.9.3

This PR is an experiment in ergonomics of flake8 output with the --isort-show-traceback option.

At the moment the full traceback is printed with each message emitted. When I'm running flake8 on the command line this leads to very verbose output.

This PR changes that behaviour so that the traceback is printed only once with the last message of the file.

Feel free to throw this away if you prefer the existing behaviour; while this change is nicer for me I can imagine there are situations where repeating the traceback with every message might be desirable.

In a project I'm working on (can provide link if you need it) we have a pipeline setup to run flake8, but flake8-isort doesn't seem to run correctly as it doesn't detect isort failures. Running isort only works fine, so I guess its the plugin :smile:

Isort config (.isort.cfg)

[settings]
profile = black
force_single_line = true

Flake8 config (.flake8)

[flake8]
count = true

ignore =
    A002,      # Argument is shadowing a python builtin.
    A003,      # Class attribute is shadowing a python builtin.
    CFQ002,    # Function has too many arguments.
    D102,      # Missing docstring in public method.
    D105,      # Magic methods not having a docstring.
    D412,      # No blank lines allowed between a section header and its content
    E402,      # Module level import not at top of file (isn't compatible with our import style).
    IFSTMT001  # "use a oneliner here".
    T101,      # TO-DO comment detection (T102 is FIX-ME and T103 is XXX).
    W503,      # line break before binary operator.
    W504,      # line break before binary operator (again, I guess).
    S101,      # Use of assert detected. The enclosed code will be removed when compiling to optimised byte code.

# F401: unused import.
# F403: cannot detect unused vars if we use starred import
# FS003: f-string missing prefix.
# R102: unnecessary parenthesis on raised exception (raises false positives in places)
# S106: posible hardcoded password (we dont care about this in the tests)
# D106, D104, D103, D101, D100: Missing docstring in public function/class/module (ignore in tests)
per-file-ignores =
    hikari/__init__.py:              F401,F403
    hikari/events/__init__.py:       F401,F403
    hikari/utilities/routes.py:      FS003
    hikari/utilities/date.py:        FS003
    hikari/impl/stateless_cache.py:  R102
    tests/hikari/*:                  FS003,S106,D106,D104,D103,D101,D100

max-complexity = 20
# TODO reset to 100 again if possible.
max-function-length = 130
# Technically this is 120, but black has a policy of "1 or 2 over is fine if it is tidier", so we have to raise this.
max-line-length = 130
show_source = False
statistics = False

accept-encodings = utf-8
docstring-convention = numpy

Flake8 plugins

# Android support
#flake8==3.8.3
git+https://gitlab.com/PyCQA/flake8@master

# Plugins
# Ref: https://github.com/DmytroLitvinov/awesome-flake8-extensions

flake8-bandit~=2.1.2                    # runs bandit
flake8-black==0.2.1                     # runs black
flake8-broken-line==0.2.1               # forbey "\" linebreaks
flake8-builtins==1.5.3                  # builtin shadowing checks
flake8-coding==1.3.2                    # coding magic-comment detectiong
flake8-comprehensions==3.2.3            # comprehension checks
flake8-deprecated==1.3                  # deprecated call checks
flake8-docstrings==1.5.0                # pydocstyle support
flake8-executable==2.0.4                # shebangs
flake8-fixme==1.1.1                     # "fix me" counter
flake8-functions==0.0.4                 # function linting
flake8-html==0.4.1                      # html output
flake8-if-statements==0.1.0             # condition linting
flake8-isort==4.0.0                     # runs isort
isort==5.5.2
flake8_formatter_junit_xml==0.0.6       # junit
flake8-mutable==1.2.0                   # mutable default argument detection
flake8-pep3101==1.3.0                   # new-style format strings only
flake8-print==3.1.4                     # complain about print statements in code
flake8-printf-formatting==1.1.0         # forbey printf-style python2 string formatting
flake8-pytest-style==1.3.0              # pytest checks
flake8-raise==0.0.5                     # exception raising linting
flake8-use-fstring==1.1                 # format string checking

In one of my files I have an import line that with a recent commit exceeded the length of 79 (the default line length in isort).

For some reason, in my local environment (with virtualenv) flake8 did not report any errors. However, my CI pipeline failed during the linting step/check reporting an error on the above mentioned line. After lots of digging I figured out that this is due to exceeding the max line length (e.g., running isort in the Docker container that is used by the CI pipeline wrapped the imported classes in parentheses over multiple lines).

Is there a reason why inconsistent results could be produced in this case?

Adding line_length = n to my config file fixed this problem. Ideally, I feel that flake8-isort should use the max-line-length setting from flake8 and pass it to isort.

In reference to #42

Within parse_options, options.config should be the command-line parameter received by flake8 itself. Once you have that, it's just matter of giving it proper priority.

Note that this quick implementation does not check if the provided file actually contains a valid configuration. While thinking about it, I'm not certain whether that should be an immediate error, or whether it should keep looking for a valid configuration elsewhere...

There's an awesome setting for the isort that I use quite frequently in my setup: settings-path. It allows to specify the desired configuration path.

Could I please add a support for this option to this library? I really miss it, and it won't interfere with nay other options.

That's how I think it should work:

0. If --no-isort-config is passed, then we ignore all other steps and use the default options 0.1 We can also raise an exception if both --no-isort-config and --isort-settings-path are passed 0.2 We can just ignore the --isort-settings-path
1. If this option is passed - then try to read this file. I have two options in case this file does not exist or can not be red: 1.1 We can raise an exception. All flake8 check will fail with this exception. This will help to indicate about the problem early 1.2 Just fallback to the config finding behavior in case this file is missing
2. If no options are passed we should fallback to the config finding logic

@gforcada What do you think?

I personally prefer explicit exceptions. As they indicate about the problem. And not hiding it.