The python source code sorter

Overview

SSort

Build Status

The python source code sorter.

Sorts the contents of python modules so that statements are placed after the things they depend on, but leaves grouping to the programmer. Groups class members by type and enforces topological sorting of methods.

Makes old fashioned code navigation easier, you can always scroll up to see where something is defined, and reduces bikeshedding.

Compatible with and intended to complement isort and black.

Before:

from module import BaseClass

def function():
    return _dependency()

def _decorator(fn):
    return fn

@_decorator
def _dependency():
    return Class()

class Class(BaseClass):
    def public_method(self):
        return self

    def __init__(self):
        pass

After:

from module import BaseClass

class Class(BaseClass):
    def __init__(self):
        pass

    def public_method(self):
        return self

def _decorator(fn):
    return fn

@_decorator
def _dependency():
    return Class()

def function():
    return _dependency()

Installation

SSort can be installed manually using pip.

$ pip install ssort

Usage

To check that a file is correctly sorted use the --check flag. --diff can be passed to see what changes ssort would make.

$ ssort --check --diff path/to/python_module.py

To allow ssort to rearrange your file, simply invoke with no extra flags. If ssort needs to make changes to a black conformant file, the result will not necessarily be black conformant. The result of running black on an ssort conformant file will always be ssort conformant. We recommend that you reformat using isort and black immediately _after_ running ssort.

$ ssort src/ tests/ setup.py; isort src/ tests/ setup.py; black src/ tests/ setup.py

Output

ssort will sort top level statements and statements in classes.

When sorting top level statements, ssort follows three simple rules:
  • Statements must always be moved after the statements that they depend on, unless there is a cycle.
  • If there is a cycle, the order of statements within the cycle must not be changed.
  • If there is no dependency between statements then, to the greatest extent possible, the original order should be kept.
ssort is more opinionated about the order of statements in classes:
  • Class attributes should be moved to the top of the class and must always be kept in their original order.
  • Lifecycle (__init__, __new__, etc) methods, and the methods they depend on, should go next.
  • Regular methods follow, dependencies always ahead of the methods that depend on them.
  • Other d'under methods should go at the end in a fixed order.

Links

License

The project is made available under the terms of the MIT license. See LICENSE for details.

Comments
  • pre-commit config

    pre-commit config

    Proposed changes

    resolves https://github.com/bwhmather/ssort/issues/24

    solution is based from this url

    https://pre-commit.com/#new-hooks

    this also include following change/feature

    • fix typo on readme
    • precommit config
    • example to use pre commit config
    • github workflow to run pre commit
    • tox env for pre-commit

    added yaml ini file is formatted using

    https://github.com/macisamuele/language-formatters-pre-commit-hooks

    rst is checked using

    https://github.com/Lucas-C/pre-commit-hooks-markup

    github pre commit workflow is based from

    https://github.com/pre-commit/action

    isort and black pre commit config is added and modified from following urls

    • https://pycqa.github.io/isort/docs/configuration/pre-commit.html
    • https://black.readthedocs.io/en/stable/integrations/source_version_control.html?highlight=pre%20commit#version-control-integration

    after merging this pr, the default rev value have to be changed to next version (0.10.1 or 0.11.0)

    to test the pr add following hook to your pre commit config

    - repo: https://github.com/rachmadaniHaryono/ssort
      rev: 545dc67985605706f8c26d5b596e7373d4ce9cf1
      hooks:
      - id: ssort
    

    adjust rev value pr status

    Types of changes

    Put an x in the boxes that apply

    • Bugfix (non-breaking change which fixes an issue)
    • [x] New feature (non-breaking change which adds functionality)
    • Breaking change (fix or feature that would cause existing functionality to not work as expected)
    • [x] Documentation Update (if none of the other choices apply)

    Checklist

    • I have read the CONTRIBUTING doc
    • Lint and unit tests pass locally with my changes
    • I have added tests that prove my fix is effective or that my feature works
    • [x] I have added necessary documentation (if appropriate)
    • Any dependent changes have been merged and published in downstream modules

    Further comments

    this pr is modified from

    https://github.com/appium/appium/blob/master/.github/PULL_REQUEST_TEMPLATE.md

    opened by rachmadaniHaryono 20
  • Python 3.10 support

    Python 3.10 support

    Python 3.10 introduces pattern matching, which looks just as much of a pain to add support for as the walrus operator. Simply ignoring pattern matching is not an option as patterns introduce bindings which may be relied on later.

    help wanted good first issue 
    opened by bwhmather 8
  • Generic AST traversal

    Generic AST traversal

    There are quite a few changes here so I'll try to break them down as best I can:

    Generally simplified the _bindings and _requirements modules

    The _bindings and _requirements modules both perform AST traversal so they share a lot of behavior. By extracting the common functionality of both modules to a new module, named _visitor, I was able to significantly reduce the size of both modules.

    Improved overall performance

    My initial intention was to utilize existing functionality from the standard library ast module to simplify the AST traversal done in both the _bindings and _requirements modules. However, after running some primitive benchmarks and realizing an over 20% reduction in performance I decided to scrap the idea. Instead, I put together the _visitors module and implemented a high speed alternative to the ast.NodeVisitor class. Swapping out the previously used functools.singledispatch AST traversal approach for the _visitor.NodeVisitor resulted in performance improvements in the 10%-15% range.

    Fixes

    While making the above changes I ran into a few existing issues which I fixed:

    1. Fixed bindings not being discovered in return statements (test case).
    2. Fixed bindings not being discovered in exception handler type expressions (test case).
    3. Fixed dict not resolving requirements from its keys (test case)
    4. Fixed augmented assignment not resolving requirements from its target (test case)
    5. Fixed annotated assignment not resolving requirements from its annotations (test case)
    6. Fixed functions and lambdas not resolving requirements from their arguments and annotations (test case)
    7. Enabled walrus tests for Python 3.8 (test case)
    8. Fixed with statement not resolving requirements from its optional_vars (test case)
    9. Fixed for statement not resolving requirements from its target (test case)

    Type annotations

    Additionally, I added type annotations to the _bindings, _requirements, and _visitor modules.

    opened by jgberry 7
  • Discover bindings made in format spec of format string

    Discover bindings made in format spec of format string

    Discover bindings made in the format spec of a format string.

    Simple example illustrating how a variable can be bound inside a format spec:

    >>> x
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
    NameError: name 'x' is not defined
    >>>
    >>> f"{3.1415:{(x := 0.3)}}"
    '3.14'
    >>>
    >>> x
    0.3
    
    opened by jgberry 7
  • pre_commit

    pre_commit

    Proposed changes

    related https://github.com/bwhmather/ssort/pull/25

    • pre commit and its config
    • pre commit github action ci config
    • tox env rst_linter

    goal of this pr

    1. put all check on tox
    2. put all formatter on pre commit
    3. avoid cycles

    hooks used

    name | description | comment ---- | --- | --- trailing-whitespace | trims trailing whitespace | default end-of-file-fixer | ensures that a file is either empty, or ends with one newline | default check-yaml | checks yaml files for parseable syntax | recommended, to use with pretty-format-yaml check-added-large-files | prevents giant files from being committed | default pretty-format-yaml | hook sets a standard for formatting YAML files | recommended isort | python isort | recommended black | python black | recommended tox | run local tox | - tox-ini-fmt | Formats tox ini | recommended

    check-yaml hook violate goal 1 but putting it on tox will violate goal 3

    tox rst_linter only check for README.rst

    pre commit github action skip black, isort and tox hooks to avoid duplicate with other github action

    this pr wil not pass commit check because repo is not formatted by pre commit

    after pr merge run pre-commit run --all-files and commit on single commit

    to ignore that commit follow guide here https://github.com/ipython/ipython/blob/master/.git-blame-ignore-revs

    related to this pr i found the comment from pre-commit

    https://github.com/pre-commit/pre-commit/issues/2069

    Types of changes

    Put an x in the boxes that apply

    • Bugfix (non-breaking change which fixes an issue)
    • [x] New feature (non-breaking change which adds functionality)
    • Breaking change (fix or feature that would cause existing functionality to not work as expected)
    • Documentation Update (if none of the other choices apply)

    Checklist

    Put an x in the boxes that apply. You can also fill these out after creating the PR. If you're unsure about any of them, don't hesitate to ask. We're here to help! This is simply a reminder of what we are going to look for before merging your code.

    • [x] I have read the CONTRIBUTING.md
    • [x] Lint and unit tests pass locally with my changes
    • I have added tests that prove my fix is effective or that my feature works
    • I have added necessary documentation (if appropriate)
    • Any dependent changes have been merged and published in downstream modules

    Further comments

    opened by rachmadaniHaryono 6
  • Skip files listed in .gitignore

    Skip files listed in .gitignore

    I ran ssort at the root of my project and it started sorting the files in my venv directory. (incidentally, this threw on venv/lib/python3.10/site-packages/blib2to3/pgen2/parse.py)

    enhancement 
    opened by kevlened 5
  • Add --reverse option

    Add --reverse option

    A proof of concept for a --reverse option as mentioned in #18. There is still some more work to be done here, but I wanted to open a draft to foster discussion on the approach I'm using here. Surprisingly, it seems this is a fairly straightforward addition.

    opened by jgberry 4
  • Line ending problem

    Line ending problem

    Just tried ssort on a Windows 10 machine. I copied your example and saved it with CRLF line endings then ran ssort. It rewrote the line endings without removing the old ones (added a CR for each CRLF), here are some screenshots from Notepad++:

    Initial file: image

    After: image

    bug 
    opened by EricE 3
  • Wildcard import callback fix

    Wildcard import callback fix

    Fixed signature of _on_wildcard_import_raise and _on_wildcard_import_ignore. Existing behavior causes a TypeError to be raised if a wildcard import is encountered and either on_wildcard_import="ignore" or on_wildcard_import="raise" is specified.

    opened by jgberry 3
  • Replace SyntaxError with custom ParseError and fix corresponding hooks

    Replace SyntaxError with custom ParseError and fix corresponding hooks

    This gives better control over error reporting, and is more consistent with how other errors are handled. It makes it practical to fix some stupid bugs in the hook where attributes are being set the wrong way.

    opened by bwhmather 3
  • Add mypy checks

    Add mypy checks

    Added mypy checks and some general clean up to make the checks pass.

    _utils.memoize_weak was giving mypy a hard time and, as far as I can tell, this function isn't actually used anywhere so I deleted it. @bwhmather, is this function needed?

    opened by jgberry 3
  • Support Jupyter notebooks

    Support Jupyter notebooks

    Would it be feasible and desirable to extend ssort to support Jupyter notebooks? Jupyter supports running cells out of order, which makes it easy to have a notebook that won't run top to bottom. Using ssort to arrange the cells (and content within cells?) in an order that respects dependencies could be a helpful tool for resolving those issues.

    enhancement help wanted 
    opened by gsganden 1
  • Add pyproject.toml / setup.py configuration

    Add pyproject.toml / setup.py configuration

    Hi! Though I use ssort in pre-commit and CI I think it will be nice to have a possibility to set up ssort configuration in default pyproject.toml / setup.py files. There one can explicitly exclude some folders or files from ssorting, set the ordering and all other other possible cli-configs. Is it possible?

    enhancement help wanted good first issue 
    opened by SerGeRybakov 1
  • ssort messes with class attribute docstrings

    ssort messes with class attribute docstrings

    Before:

    from dataclasses import dataclass
    
    
    @dataclass
    class User:
        id: int
        """User identifier."""
    
        email: str
        """E-mail address."""
    
        phone: str
        """Phone number in the international format."""
    
        age: int
        """Number of full years."""
    

    After:

    from dataclasses import dataclass
    
    
    @dataclass
    class User:
        id: int
    
        email: str
    
        phone: str
    
        age: int
        """User identifier."""
        """E-mail address."""
        """Phone number in the international format."""
        """Number of full years."""
    

    There are two PEPs that describe such docstrings. Both of them were rejected.

    • https://peps.python.org/pep-0224/
    • https://peps.python.org/pep-0258/

    I know that this is now a "real" docstring syntax but people still use it to document fields in their classes. Some tools support it as well (like sphinx).

    It could be useful to recognise such docstrings and preserve their positions.

    enhancement good first issue 
    opened by and-semakin 1
  • Fix path escaping on windows

    Fix path escaping on windows

    Escaping of paths for printing to the console is currently disabled on windows. At some point, we will need to untangle the byzantine rules for preparing strings to paste into the console and get this implemented.

    The problem function: https://github.com/bwhmather/ssort/blob/f179281bfe7f56e203ba5a2ac55630855398f1e1/src/ssort/_utils.py#L52-L61

    bug good first issue 
    opened by bwhmather 1
  • Explicitly choose to run on all files

    Explicitly choose to run on all files

    I just learned how to use git reset --hard (hardware guy, just learning Git) after fat fingering ssort without the file name... How about ssort . to explicitly say to process all files in the current file tree or maybe have a way to say all files in the current directory or all files in the current tree?

    enhancement 
    opened by EricE 1
  • Optional lexicographic ordering

    Optional lexicographic ordering

    Topological ordering is good for understanding code but becomes less effective as the module becomes larger and dependency becomes more complex.

    In such cases (and even in small modules), I find lexicographic (cos identifiers might also include digits) ordering much more fitting.

    enhancement help wanted open challenge 
    opened by AnonymouX47 12
Releases(v0.11.6)
  • v0.11.6(Jul 26, 2022)

  • v0.11.5(Mar 28, 2022)

  • v0.11.4(Mar 24, 2022)

  • v0.10.6(Mar 28, 2022)

  • v0.11.3(Mar 17, 2022)

    Main feature in this release is that ssort will now skip files listed in .gitignore, unless they are explicitly specified. This release also contains a cleanup of the error hooks in the python API.

    Source code(tar.gz)
    Source code(zip)
  • v0.11.2(Mar 14, 2022)

  • v0.11.1(Mar 4, 2022)

    Contains a massive refactoring of requirement and binding extraction by jgberry, which fixes a number of bugs. In particular:

    • Fixes bindings not being discovered in return statements.
    • Fixes bindings not being discovered in exception handler type expressions.
    • Fixes dict not resolving requirements from its keys.
    • Fixes augmented assignment not resolving requirements from its target.
    • Fixes annotated assignment not resolving requirements from its annotations.
    • Fixes functions and lambdas not resolving requirements from their arguments and annotations.
    • Fixes with statement not resolving requirements from its optional_vars.

    Fixes for statement not resolving requirements from its target.

    In addition, enables walrus tests for Python 3.8 and introduces a new API for hooking in to error reporting.

    Source code(tar.gz)
    Source code(zip)
  • v0.11.0(Feb 26, 2022)

    Breaking changes:

    • Moves inner classes to top of class body instead of sorting them somewhere in the middle.

    Other changes:

    • Relaxes topological sorting of class members to only apply to private members.
    Source code(tar.gz)
    Source code(zip)
  • v0.10.5(Feb 25, 2022)

  • v0.10.4(Feb 25, 2022)

  • v0.9.0(Aug 9, 2021)

  • v0.8.0(Aug 8, 2021)

  • v0.7.0(Aug 7, 2021)

  • v0.6.1(May 20, 2021)

  • v0.6.0(May 20, 2021)

    Change logging to print out a warning for all unresolved requirements, not just the first.

    Add support for list arguments to del statement.

    Recognize class scoped builtins (__module__ and qualname`).

    Source code(tar.gz)
    Source code(zip)
  • v0.5.0(Apr 29, 2021)

  • v0.4.0(Apr 27, 2021)

  • v0.3.1(Apr 25, 2021)

  • v0.3.0(Apr 25, 2021)

  • v0.2.0(Apr 19, 2021)

    First usable release. Still no stability guarantee. Supports basic topological sorting of top level statements. Supports --check and --diff modes, and finding python files recursively in directories.

    Source code(tar.gz)
    Source code(zip)
  • v0.1.0(Mar 28, 2021)

Owner
Ben Mather
Ben Mather
Simple, hassle-free, dependency-free, AST based source code refactoring toolkit.

refactor is an end-to-end refactoring framework that is built on top of the 'simple but effective refactorings' assumption. It is much easier to write a simple script with it rather than trying to figure out what sort of a regex you need in order to replace a pattern (if it is even matchable with regexes).

Batuhan Taskaya 385 Jan 6, 2023
Code generation and code search for Python and Javascript.

Codeon Code generation and code search for Python and Javascript. Similar to GitHub Copilot with one major difference: Code search is leveraged to mak

null 51 Dec 8, 2022
IDE allow you to refactor code, Baron allows you to write refactoring code.

Introduction Baron is a Full Syntax Tree (FST) library for Python. By opposition to an AST which drops some syntax information in the process of its c

Python Code Quality Authority 278 Dec 29, 2022
Find dead Python code

Vulture - Find dead code Vulture finds unused code in Python programs. This is useful for cleaning up and finding errors in large code bases. If you r

Jendrik Seipp 2.4k Dec 27, 2022
Safe code refactoring for modern Python.

Safe code refactoring for modern Python projects. Overview Bowler is a refactoring tool for manipulating Python at the syntax tree level. It enables s

Facebook Incubator 1.4k Jan 4, 2023
Removes commented-out code from Python files

eradicate eradicate removes commented-out code from Python files. Introduction With modern revision control available, there is no reason to save comm

Steven Myint 146 Dec 13, 2022
Turn your C++/Java code into a Python-like format for extra style points and to make everyone hates you

Turn your C++/Java code into a Python-like format for extra style points and to make everyone hates you

Tô Đức (Watson) 4 Feb 7, 2022
A simple Python bytecode framework in pure Python

A simple Python bytecode framework in pure Python

null 3 Jan 23, 2022
Awesome autocompletion, static analysis and refactoring library for python

Jedi - an awesome autocompletion, static analysis and refactoring library for Python Jedi is a static analysis tool for Python that is typically used

Dave Halter 5.3k Dec 29, 2022
a python refactoring library

rope, a python refactoring library ... Overview Rope is a python refactoring library. Notes Nick Smith <[email protected]> takes over maintaining rope

null 1.5k Dec 30, 2022
A system for Python that generates static type annotations by collecting runtime types

MonkeyType MonkeyType collects runtime types of function arguments and return values, and can automatically generate stub files or even add draft type

Instagram 4.1k Dec 28, 2022
Tool for translation type comments to type annotations in Python

com2ann Tool for translation of type comments to type annotations in Python. The tool requires Python 3.8 to run. But the supported target code versio

Ivan Levkivskyi 123 Nov 12, 2022
Bottom-up approach to refactoring in python

Introduction RedBaron is a python library and tool powerful enough to be used into IPython solely that intent to make the process of writing code that

Python Code Quality Authority 653 Dec 30, 2022
AST based refactoring tool for Python.

breakfast AST based refactoring tool. (Very early days, not usable yet.) Why 'breakfast'? I don't know about the most important, but it's a good meal.

eric casteleijn 0 Feb 22, 2022
Refactoring Python Applications for Simplicity

Python Refactoring Refactoring Python Applications for Simplicity. You can open and read project files or use this summary ?? Concatenate String my_st

Mohammad Dori 3 Jul 15, 2022
Leap is an experimental package written to enable the utilization of C-like goto statements in Python functions

Leap is an experimental package written to enable the utilization of C-like goto statements in Python functions

null 6 Dec 26, 2022
SCodeScanner stands for Source Code scanner where the user can scans the source code for finding the Critical Vulnerabilities.

The SCodeScanner stands for Source Code Scanner, where you can scan your source code files like PHP and get identify the vulnerabilities inside it. The tool can use by Pentester, Developer to quickly identify the weakness.

null 136 Dec 13, 2022
Empirical Study of Transformers for Source Code & A Simple Approach for Handling Out-of-Vocabulary Identifiers in Deep Learning for Source Code

Transformers for variable misuse, function naming and code completion tasks The official PyTorch implementation of: Empirical Study of Transformers fo

Bayesian Methods Research Group 56 Nov 15, 2022
This is the official source code for SLATE. We provide the code for the model, the training code, and a dataset loader for the 3D Shapes dataset. This code is implemented in Pytorch.

SLATE This is the official source code for SLATE. We provide the code for the model, the training code and a dataset loader for the 3D Shapes dataset.

Gautam Singh 66 Dec 26, 2022
Code for our paper "Transfer Learning for Sequence Generation: from Single-source to Multi-source" in ACL 2021.

TRICE: a task-agnostic transferring framework for multi-source sequence generation This is the source code of our work Transfer Learning for Sequence

THUNLP-MT 9 Jun 27, 2022