Tool for automatically reordering python imports. Similar to isort but uses static analysis more.

Anthony Sottile

Last update: Dec 26, 2022

Overview

reorder_python_imports

Tool for automatically reordering python imports. Similar to isort but uses static analysis more.

Installation

pip install reorder-python-imports

Console scripts

Consult reorder-python-imports --help for the full set of options.

reorder-python-imports takes filenames as positional arguments

Common options:

--py##-plus: see below.
--add-import / --remove-import: see below.
--replace-import: see below.
--application-directories: by default, reorder-python-imports assumes your project is rooted at .. If this isn't true, tell it where your import roots live. For example, when using the popular ./src layout you'd use --application-directories=.:src (note: multiple paths are separated using a :).
--unclassifiable-application-module: (may be specified multiple times) modules names that are considered application modules. this setting is intended to be used for things like C modules which may not always appear on the filesystem.

As a pre-commit hook

See pre-commit for instructions

Sample .pre-commit-config.yaml

-   repo: https://github.com/asottile/reorder_python_imports
    rev: v2.6.0
    hooks:
    -   id: reorder-python-imports

What does it do?

Separates imports into three sections

import sys
import pyramid
import reorder_python_imports

becomes (stdlib, third party, first party)

import sys

import pyramid

import reorder_python_imports

`import` imports before `from` imports

from os import path
import sys

becomes

import sys
from os import path

Splits `from` imports

from os.path import abspath, exists

becomes

from os.path import abspath
from os.path import exists

Removes duplicate imports

import os
import os.path
import sys
import sys

becomes

import os.path
import sys

Using `# noreorder`

Lines containing and after lines which contain a # noreorder comment will be ignored. Additionally any imports that appear after non-whitespace non-comment lines will be ignored.

For instance, these will not be changed:

import sys

try:  # not import, not whitespace
    import foo
except ImportError:
    pass

import sys

import reorder_python_imports

import matplotlib  # noreorder
matplotlib.use('Agg')
import matplotlib.pyplot as plt

# noreorder
import sys
import pyramid
import reorder_python_imports

why this style?

The style chosen by reorder-python-imports has a single aim: reduce merge conflicts.

By having a single import per line, multiple contributors can add / remove imports from a single module without resulting in a conflict.

Consider the following example which causes a merge conflict:

# developer 1
-from typing import Dict, List
+from typing import Any, Dict, List

# developer 2
-from typing import Dict, List
+from typing import Dict, List, Tuple

no conflict with the style enforced by reorder-python-imports:

+from typing import Any
 from typing import Dict
 from typing import List
+from typing import Tuple

Adding / Removing Imports

Let's say I want to enforce absolute_import across my codebase. I can use: --add-import 'from __future__ import absolute_import'.

$ cat test.py
print('Hello world')
$ reorder-python-imports --add-import 'from __future__ import absolute_import' test.py
Reordering imports in test.py
$ cat test.py
from __future__ import absolute_import
print('Hello world')

Let's say I no longer care about supporting Python 2.5, I can remove from __future__ import with_statement with --remove-import 'from __future__ import with_statement'

$ cat test.py
from __future__ import with_statement
with open('foo.txt', 'w') as foo_f:
    foo_f.write('hello world')
$ reorder-python-imports --remove-import 'from __future__ import with_statement' test.py
Reordering imports in test.py
$ cat test.py
with open('foo.txt', 'w') as foo_f:
    foo_f.write('hello world')

Replacing imports

Imports can be replaced with others automatically (if they provide the same names). This can be useful for factoring out compatibility libraries such as six (see below for automated six rewriting).

This rewrite avoids NameErrors as such it only occurs when:

the imported symbol is the same before and after
the import is a from import

The argument is specified as orig.mod=new.mod or with an optional checked attribute orig.mod=new.mod:attr. The checked attribute is useful for renaming some imports from a module instead of a full module.

For example:

# full module move
--replace-import six.moves.queue=queue
# specific attribute move
--replace-import six.moves=io:StringIO

Removing obsolete `future` imports

The cli provides a few options to help "burn the bridges" with old python versions by removing __future__ imports automatically. Each option implies all older versions.

--py22-plus: nested_scopes
--py23-plus: generators
--py26-plus: with_statement
--py3-plus: division, absolute_import, print_function, unicode_literals
--py37-plus: generator_stop

Removing / rewriting obsolete `six` imports

With --py3-plus, reorder-python-imports will also remove / rewrite imports from six. Rewrites follow the same rules as replacing imports above.

For example:

+import queue
+from io import StringIO
+from urllib.parse import quote_plus
+
 import six.moves.urllib.parse
-from six.moves import queue
-from six.moves import range
-from six.moves import StringIO
-from six.moves.urllib.parse import quote_plus

Rewriting mock imports

With --py3-plus, reorder-python-imports will also rewrite various mock imports:

-from mock import patch
+from unittest.mock import patch

Rewriting `mypy_extensions` and `typing_extension` imports

With --py36-plus and higher, reorder-python-imports will also rewrite mypy_extensions and typing_extensions imports ported to typing.

-from mypy_extensions import TypedDict
+from typing import TypedDict

Rewriting pep 585 typing imports

With --py39-plus and higher, reorder-python-imports will replace imports which were moved out of the typing module in pep 585.

-from typing import Sequence
+from collections.abc import Sequence

Comments

Import order loop error

Hi ! I've been using reorder-python-imports for at least a year and I love it. I usually set it up in a pre-commit AND in CI, but this time I'm stuck in an infinite loop:

https://asciinema.org/a/1yEjs3DNNki3KeLQWntUbmEsH
question

opened by Seluj78 37
Issues when package is installed in virtualenv

I'm having issues when the package I'm editing is present in my current virtualenv.

I run pip install -e . for my packages, since they're cli apps, and that's the easiest way to test them.

When the package I'm editing is installed in the current virtualenv, it's treated as third party instead of first party.

Any hints on how to work around this? Or have I come across a bug?
question

opened by WhyNotHugo 21

Different separation of the imports during pre-commit hook and cli command

I get different results of sorting imports during pre-commit hook and CLI command:

1. pre-commit:

- repo: https://github.com/asottile/reorder_python_imports
  rev: v2.5.0
  hooks:
    - id: reorder-python-imports
      stages: [push]

The result is:

import os
from typing import Text
from typing import TypedDict

from chaoshub_aws.ec2 import AwsEc2
from chaoshub_aws.ec2.decorator import ec2_injector

2. reorder-python-imports test_file.py The result is:

import os
from chaoshub_aws.ec2 import AwsEc2
from chaoshub_aws.ec2.decorator import ec2_injector
from typing import Text
from typing import TypedDict

I tried to path the same arguments in both CLI command and pre-commit hook, but the results are still different. I guess I'm just missing something ..?

opened by dimaka-wix 13

Different output in vagrant box vs host

Hi, I am at a loss for what to do next to troubleshoot this. Same version of Ubuntu, same version of python. And same version of the library with same args and I get different output. I even diffed the library file to make sure it's the same. one is in virtualbox the other on host machine. Is there something else from the environment that can cause this? I triple checked the list of files I passed in and it's the same.

Only option I'm passing is --separate-from-import.

opened by curioussavage 11
Support STDIN/STDOUT

If there was a flag like --stdout to output the changed text to STDOUT, that would be nice.

Being able to read from STDIN would make it perfect for autoformatters like neoformat.

opened by alok 11
Allow multiple one-line imports
I usually import modules in a single import statement like so:

from __future__ import absolute_imports, unicode_literals

reorder-python-imports demands each import to be placed on seperate lines. Isort has this as a config option: force_single_line
opened by dinoshauer 11
Relative import ordering

I'm looking at moving over from isort, my only point of friction is relative imports.

I know they are a topic where a lot of people have strong opinions. Our house style uses flake8-tidy-imports to limit them to same package only, we find that for same package relative is lower cognitive load than absolute, while cross package it's more cognitive load.

I see from comments and issues that you don't personally use relative imports and that having them in their own section like isort does caused a bunch of issues.

It looks like changing the sort order to place relatives last was maybe also a feature at some point although without release notes I'm not sure if/when/why, it's quite hard to tell from the git log of classify-imports.

How do you feel about changing the sort key so that relatives get placed after absolute imports rather than before?

As an unconditional behaviour it looks like it'd be maybe a half dozen lines + tests. Obviously conditional would be a lot more work, however it seems unlikely that anyone using relative imports particularly expects them to be first.

opened by tolomea 10

why this style?

Consider the following example which causes a merge conflict:
# developer 1
-from typing import Dict, List
+from typing import Any, Dict, List
# developer 2
-from typing import Dict, List
+from typing import Dict, List, Tuple
no conflict with the style enforced by reorder-python-imports:
+from typing import Any
 from typing import Dict
 from typing import List
+from typing import Tuple

Why not

from typing import (
+    Any,
     Dict,
     List,
+    Tuple,
)

It also doesn't conflicts and you have shorter, maybe even more readable lines.

opened by karolpawlowski 10

Hook removes blank lines not related to imports at all

Fantastic tool, however below is probably a bug.

Hook removes blank lines, that are not related to imports at all.

Example python script (notice blank lines around docstring):

$ cat test.py 
#!/usr/bin/env python3

"""
Here goes synopsis
"""

import os
import random
import time

print("asdf")

Here is my pre-commit config:

$ cat .pre-commit-config.yaml 
repos:
-   repo: https://github.com/asottile/reorder_python_imports
    rev: v1.3.5
    hooks:
    -   id: reorder-python-imports
        language_version: python3

Here is the run of the hook:

$ pre-commit run reorder-python-imports
Reorder python imports...................................................Failed
hookid: reorder-python-imports

Files were modified by this hook. Additional output:

Reordering imports in test.py

And the result:

$ cat test.py 
#!/usr/bin/env python3
"""
Here goes synopsis
"""
import os
import random
import time

print("asdf")

question

opened by IvanBoyko 10

issue due to setuptools

Just want to make it documented in case somebody faces the same issue due to the bug in virtualenv

reorder-python-imports fails with

 Traceback (most recent call last):
  File "/home/runner/.cache/pre-commit/repoizu8jol0/py_env-python3.10/bin/reorder-python-imports", line 8, in <module>
    sys.exit(main())
  File "/home/runner/.cache/pre-commit/repoizu8jol0/py_env-python3.10/lib/python3.10/site-packages/reorder_python_imports.py", line 907, in main
    retv |= _fix_file(filename, args)
  File "/home/runner/.cache/pre-commit/repoizu8jol0/py_env-python3.10/lib/python3.10/site-packages/reorder_python_imports.py", line 475, in _fix_file
    new_contents = fix_file_contents(
  File "/home/runner/.cache/pre-commit/repoizu8jol0/py_env-python3.10/lib/python3.10/site-packages/reorder_python_imports.py", line 456, in fix_file_contents
    partitioned = step(partitioned)
  File "/home/runner/.cache/pre-commit/repoizu8jol0/py_env-python3.10/lib/python3.10/site-packages/reorder_python_imports.py", line 366, in apply_import_sorting
    sorted_blocks = sort(import_obj_to_partition.keys(), **sort_kwargs)
  File "/home/runner/.cache/pre-commit/repoizu8jol0/py_env-python3.10/lib/python3.10/site-packages/aspy/refactor_imports/sort.py", line 96, in sort
    imports_partitioned[classify_func(import_obj)].append(import_obj)
  File "/home/runner/.cache/pre-commit/repoizu8jol0/py_env-python3.10/lib/python3.10/site-packages/aspy/refactor_imports/sort.py", line 73, in classify_func
    return classify_import(obj.import_statement.module, **kwargs)
  File "/home/runner/.cache/pre-commit/repoizu8jol0/py_env-python3.10/lib/python3.10/site-packages/aspy/refactor_imports/classify.py", line 138, in classify_import
    found, module_path, is_builtin = _get_module_info(
  File "/home/runner/.cache/pre-commit/repoizu8jol0/py_env-python3.10/lib/python3.10/site-packages/aspy/refactor_imports/classify.py", line 103, in _get_module_info
    assert spec.submodule_search_locations is not None
AssertionError

solution will be to update CI to force downgrade virtualenv:

name: pre-commit

on:
  push:
    branches:
      - master
  pull_request:

jobs:
  pre-commit:
    runs-on: ubuntu-latest
    steps:
    - uses: actions/checkout@v2
    - uses: actions/setup-python@v2

# Install venv to avoid issues https://github.com/pre-commit/action/issues/135
    - name: Install virtualenv
      run: |
        python -m pip install -U virtualenv==20.10.0
    - uses: pre-commit/[email protected]

opened by beliaev-maksim 9

Add --quiet option
Added a --quiet args option. When used, user messages on success (Reordering imports in …) are suppressed. Similar to Black’s --quiet option.

Steps to reproduce:

Create a file that would be fixed by reorder_python_imports.

import ast import argparse

Run reorder_python_imports with the --quiet option. reorder_python_imports --quiet demo.py

See that the contents of the Python file changed, but no output was printed.

Together with --exit-zero-even-if-changed, this makes reorder_python_imports a more well-behaved UNIX tool.
opened by Glutexo 9
Support for pyi files now types_or is implemented

Further to #103 which was closed at the time.

Would it be possible to update the pre-commit hooks for this repo to allow it to run by default on pyi files?

opened by Greedquest 3
Documentation on how to use with Visual Studio Code
I'm trying to use reorder_python_imports with Visual Studio Code. Unfortunately, built-in support in the Python extension was already rejected due to lack of demand, but I'd still like to use it.

I've tried setting "python.sortImports.path": "reorder-python-imports", but that results in the error:

> reorder-python-imports ~/path/to/file.py --diff Error: $PYTHONPATH set, import order may be unexpected

Fortunately (since it can't be changed by specifying python.sortImports.args), it looks like y'all already support --diff as an alias for --diff-only, but I don't see any arguments I can give to reorder_python_imports to make it work within the Python extension, which evidently sets $PYTHONPATH.
opened by kbd 9