In GitLab by @sigmavirus24 on Mar 15, 2016, 15:15

Merges proposed/3.0 -> master

Description of changes

• Rewrite flake8's internals to be independent of pep8
• Rework flake8's handling of options parsed from config files in a unified way that's better tested than pep8's handling was
• Add proper logging to flake8 to allow more informative verbose information to be shown to the user
• Internalize some of pep8's check handling and token generation

Related to:

• #117
• #111
• #106
• #105
• #100
• #90
• Option 2 from #89
• #84
• #74
• #69
• #66
• #54
• #21
• #14

And more that I'll update this with later

Note that this is a merge in from https://gitlab.com/pycqa/flake8-engine which still has some open issues which will be fixed in this branch.

In GitLab by @tuxlife on Jul 8, 2016, 04:55

We use flake8 on travic-ci for your builds and currently all build return an error.

See here the travis-ci output

• Version of Python which has Flake8 installed (e.g., python -V): 3.4.2
• How you installed Flake8 (e.g., pip, apt, yum, etc.): pip
  • If you used pip, the version of pip installed (e.g., pip --version): 6.0.7
• Version of setuptools installed (e.g., python -c 'import setuptools; print(setuptools.__version__)'): i don't know
• Version of Flake8 installed (i.e., copy and paste all of the output from flake8 --version): 3.0.0b2

thank you

The original issue

Id: 164
Title: Version 3.0.x hangs with PicklingError on travis-ci.org

could not be created. This is a dummy issue, replacing the original one. It contains everything but the original issue description. In case the gitlab repository is still existing, visit the following link to show the original issue:

TODO

In GitLab by @asottile on Nov 22, 2016, 15:45

Merges faster -> master

This is a bit of a WIP, I moved away from Queue (since it seems to be the bottleneck)

From #265 the same test finishes (still slower) but in reasonable time:

$ time flake8 -j8 bar

real	0m17.583s
user	0m26.312s
sys	0m2.288s

In GitLab by @bittner on Sep 8, 2015, 09:28

At the moment it's only possible to ignore

• either a complete file (# flake8: noqa on a single line),
• a single line (# noqa at the end of a line),
• or specific errors in all files being checked (ignore = F403,E501 in a [flake8] section of setup.cfg or tox.ini)

See also the Configuration chapter in the docs.

Enhancement Proposal

1.) Allow ignoring only specific errors in a single file:

# flake8: ignore=F403,E501

While I think this only would be a valuable, almost necessary addition to flake8 already the very same concept could be taken further in addition.

Apply the Same Idea to Lines

2.) Allow ignoring only specific errors on a single line:

# noqa: ignore=F403,E501

Taking the Proposal Further

3.) Allow configuration within certain scopes of code:

Following this syntax it could be used to apply certain (flake8) rules to a code block (e.g. a function). For example, this would allow complexity 12 within the scope of the function my_func:

def my_func():
    # noqa: max-complexity=12
    for i in range(99):
        for j in range(99):
            for k in range(99):
                for m in range(99):
                    pass

The original issue

Id: 89
Title: Allow ignoring specific errors in files

could not be created. This is a dummy issue, replacing the original one. It contains everything but the original issue description. In case the gitlab repository is still existing, visit the following link to show the original issue:

TODO

In GitLab by @nekokatt on Aug 27, 2020, 05:18

Merges bugfix/cpython-3770-semopen-missing -> master

Platforms such as Termux on Android, and other exotic devices do not provide a sem_open implementation on the OS level. This is problematic, as the error resulting from this occurs when calling multiprocessing.Pool, throwing an unhandled ImportError.

The issue itself is outlined in https://bugs.python.org/issue3770.

This change allows devices missing this system call to respond to the missing feature by falling back to synchronous execution, which appears to be the default behaviour if the multiprocessing module is not found.

This change also adds a potential fix for developers working on platforms where multiprocessing itself cannot be imported. The existing code would set the name referencing the import to None, but there are no clear checks to ensure this does not result in an AttributeError later when multiprocessing.Pool has accession attempts.

Existing users should see no difference in functionality, as they will assumably already be able to use flake8, so will not be missing this sem_open call.

Users on devices without the sem_open call will now be able to use flake8 where they would be unable to before due to unhandled ImportErrors.

In GitLab by @tyrelsouza on Apr 29, 2015, 20:43

Merges fix_includes -> master

Fix parsing ignore #40 at https://gitlab.com/pycqa/flake8/issues/40

Adding an ignore option in [flake8] wasn't working because pep8.StyleGuide turned the string sent in into a tuple, which the option parser needs to receive as an iterable that isn't a string. Split on spaces, commas, or semicolons using re.findall in order to get a list of error/warnings to pass to StyleGuide properly.

Also fixes a typo I found.

In GitLab by @tommilligan on Dec 4, 2018, 13:46

Merges physical-line-multiple -> master

flake8 is currently inconsistent in how it handles the return results of checks. This PR amends run_physical_checks in a backwards compatible way, so that multiple results can be returned by a single physical check.

Currently in run_logical_checks, multiple results can be returned.

Currently in run_physical_checks, only a single result can be returned.

My current use case is writing a plugin that runs one complex regex over each physical line. Depending on what matches, I'd like to return different error codes and descriptions. There may be more than one match per line; at the moment, I have to split this out into multiple plugin entrypoints, one for each possible return type.

In GitLab by @demosdemon on Nov 6, 2018, 14:41

Merges per-file-ignores -> master

Add a configuration option to Option for colon_prefixed_map. The accepted value is a YAML-style mapping but with only one depth. When True, the normalize method will return a Mapping.

In GitLab by @christianmlong on Feb 14, 2015, 24:55

Merges issue8 -> master

I added warnings for cases where the --jobs option is given but is invalid.

• running on Windows
• the multiprocessing module is not available
• the user passed the --diff option
• the user wants to pull from stdin

I got the tests working on linux and windows, with two exceptions.

Look at test_windows_disables_jobs and test_stdin_disables_jobs in test_engine.py. They mock is_windows, and stdin, but they don't seem to mock --jobs. So, they are not issuing the warnings I expect them to issue.

I'm not sure how to add the --jobs option to those tests. Ideas?

Once I get those two tests working, I will add a --verbose flag, and hide the warnings behind it.

how did you install flake8?

$ cat requirements_hash.txt
GitPython==3.1.29 --hash=sha256:41eea0deec2deea139b459ac03656f0dd28fc4a3387240ec1d3c259a2c47850f2
PyYAML==6.0 --hash=sha256:68fb519c14306fec9720a2a5b45bc9f0c8d1b9c72adf45c37baedfcd949c35a23
bandit==1.7.4 --hash=sha256:412d3f259dab4077d0e7f0c11f50f650cc7d10db905d98f6520a95a18049658a4
coverage==7.0.0 --hash=sha256:b8f7cd942dda3795fc9eadf303cc53a422ac057e3b70c2ad6d4276ec6a83a5415
entrypoints==0.4 --hash=sha256:f174b5ff827504fd3cd97cc3f8649f3693f51538c7e4bdf3ef002c8429d42f9f6
flake8-bandit==4.1.1 --hash=sha256:4c8a53eb48f23d4ef1e59293657181a3c989d0077c9952717e98a0eace43e06d7
flake8-junit-report==2.1.0 --hash=sha256:f8890c1ebe0acb516fefacddec4b802bca9f89bb07db933e4ee3cd11ceaa1e8b8
flake8-polyfill==1.0.2 --hash=sha256:12be6a34ee3ab795b19ca73505e7b55826d5f6ad7230d31b18e106400169b9e99
flake8==6.0.0 --hash=sha256:3833794e27ff64ea4e9cf5d410082a8b97ff1a06c16aa3d2027339cd0f1195c710
gitdb==4.0.10 --hash=sha256:c286cf298426064079ed96a9e4a9d39e7f3e9bf15ba60701e95f5492f28415c711
mccabe==0.7.0 --hash=sha256:6c2d30ab6be0e4a46919781807b4f0d834ebdd6c6e3dca0bda5a15f863427b6e12
pbr==5.11.0 --hash=sha256:db2317ff07c84c4c63648c9064a79fe9d9f5c7ce85a9099d4b6258b3db83225a13
pycodestyle==2.10.0 --hash=sha256:8a4eaf0d0495c7395bdab3589ac2db602797d76207242c17d47018681570661014
pyflakes==3.0.1 --hash=sha256:ec55bf7fe21fff7f1ad2f7da62363d749e2a470500eab1b555334b67aa1ef8cf15
six==1.16.0 --hash=sha256:8abb2f1d86890a2dfb989f9a77cfcfd3e47c2a354b01111771326f8aa26e025416
smmap==5.0.0 --hash=sha256:2aba19d6a040e78d8b09de5c57e96207b09ed71d8e55ce0959eeee6c8e190d9417
stevedore==4.1.1 --hash=sha256:aa6436565c069b2946fe4ebff07f5041e0c8bf18c7376dd29edf80cf7d524e4e18
typing-extensions==4.4.0 --hash=sha256:16fa4864408f655d35ec496218b85f79b3437c829e93320c7c9215ccfd92489e
$ PYTHONDONTWRITEBYTECODE=1 PYTHONHASHSEED=0 SOURCE_DATE_EPOCH=946702800 pip install --no-build-isolation --no-cache-dir --no-compile --require-hashes --requirement requirements_hash.txt

# Note: identical behavior with `pip install flake8` as the installation method.

unmodified output of flake8 --bug-report

{
  "platform": {
    "python_implementation": "CPython",
    "python_version": "3.11.1",
    "system": "Linux"
  },
  "plugins": [
    {
      "plugin": "flake8-bandit",
      "version": "4.1.1"
    },
    {
      "plugin": "mccabe",
      "version": "0.7.0"
    },
    {
      "plugin": "pycodestyle",
      "version": "2.10.0"
    },
    {
      "plugin": "pyflakes",
      "version": "3.0.1"
    }
  ],
  "version": "6.0.0"
}

describe the problem

I've been using flake8 in a CI environment called with something like:

find . -name '*.py' -print0 | xargs -0 flake --output-file output.txt --tee --

When upgrading to v6.0.0 I got errors from the above command. It appears to be some interaction between --output-file and --tee.

This sequence sums up the issue pretty well. If you use the -- which indicates no further arguments it fails to operate properly. I expected the last two commands to behave the same.

$ touch a
$ flake8 ./a
$ flake8 --output-file output.txt ./a
$ flake8 --output-file output.txt -- ./a
Unable to find qualified name for module: output.txt
--output-file:0:1: E902 FileNotFoundError: [Errno 2] No such file or directory: '--output-file'
$

how did you install flake8?

pip install flake8

also when cloned from github master

unmodified output of flake8 --bug-report

{
  "platform": {
    "python_implementation": "CPython",
    "python_version": "3.10.8",
    "system": "Linux"
  },
  "plugins": [
    {
      "plugin": "mccabe",
      "version": "0.7.0"
    },
    {
      "plugin": "pycodestyle",
      "version": "2.10.0"
    },
    {
      "plugin": "pyflakes",
      "version": "3.0.1"
    }
  ],
  "version": "6.0.0"
}

describe the problem

what I expected to happen

my argparse.Action to be called regardless of if the parameter is specified on the command line or in the config

sample code

I implemented a test that reproduces the error. The first one passes, the second one doesn't.

from collections.abc import Sequence
from flake8.options import config

# always sets the value to bar regardless of what `values` is
class MyAction(argparse.Action):
    def __call__(
        self,
        parser: argparse.ArgumentParser,
        namespace: argparse.Namespace,
        values: Sequence[str] | None,
        option_string: str | None = None,
    ) -> None:
        setattr(namespace, self.dest, "bar")


def test_action_cmdline(optmanager):
    optmanager.add_option(
        "--my-option",
        parse_from_config=True,
        required=False,
        action=MyAction,
    )
    options = optmanager.parse_args(["--my-option", "foo"])
    assert options.my_option == 'bar'

def test_action_config(tmpdir):
    tmpdir.join("setup.cfg").write("[flake8]\nmy-option=foo\n")

    with tmpdir.as_cwd():
        cfg, cfg_dir = config.load_config(None, [], isolated=False)

    assert cfg.get("flake8", "my-option") == "bar"

commands ran

$ pytest -k test_action_ 
===================================== test session starts =====================================
platform linux -- Python 3.10.8, pytest-7.2.0, pluggy-1.0.0
rootdir: /home/h/Git/flake8, configfile: pytest.ini
collected 465 items / 463 deselected / 2 selected                                             

tests/unit/test_option_manager.py .F                                                    [100%]

========================================== FAILURES ===========================================
_____________________________________ test_action_config ______________________________________

tmpdir = local('/tmp/pytest-of-h/pytest-62/test_action_config0')

    def test_action_config(tmpdir):
        tmpdir.join("setup.cfg").write("[flake8]\nmy-option=foo\n")
    
        with tmpdir.as_cwd():
            cfg, cfg_dir = config.load_config(None, [], isolated=False)
    
>       assert cfg.get("flake8", "my-option") == "bar"
E       AssertionError: assert 'foo' == 'bar'
E         - bar
E         + foo

tests/unit/test_option_manager.py:243: AssertionError
========================= 1 failed, 1 passed, 463 deselected in 0.26s =========================

The value is foo, which it would be if there's no action specified, but it completely sidesteps calling the action completely.

What is covered in this PR

I have recently attempted to write a plugin for Flake8. I found the video tutorial to be a great source of info, but understand it is desirable to move more and more of its contents to written form.

This PR is my attempt at moving some of that information to the docs.

Note this is based on my understanding after watching the video tutorial and having read the code to understand the different types of plugins, how they are loaded, how they are called, what parameters they get and what Flake8 expects them to return.

One notable omission

When describing the physical line plugins, I have deliberately omitted the fact that it supports receiving a single tuple from the plugin. I am basing this omission on my intuition that this is a special case to deal with legacy, one that Flake8 might not want to promote.

describe the request

did some quick profiling, flake8 -j1 spends about 5% of its time collecting these (even if they're not selected).

I also feel like these are not that useful and could probably be derived in a separate formatter if desired? I don't think I've ever used them but I've definitely broken them a few times.

what say to removing these options? (and perhaps providing a third party formatter for them as a middle-ground?)

this would also "trivially" solve #1458

describe the request

there's some cargo cult copy paste of adding select / extend-select to "enable plugins" however this is unnecessary as any installed plugin is enabled by default