A tool to automatically convert old string literal formatting to f-strings

Overview

flynt - string formatting converter

Build Status Coverage PyPI version Downloads Code style: black

flynt is a command line tool to automatically convert a project's Python code from old "%-formatted" and .format(...) strings into Python 3.6+'s "f-strings".

F-Strings:

Not only are they more readable, more concise, and less prone to error than other ways of formatting, but they are also faster!

Installation

pip install flynt. It requires Python version 3.6+.

Usage

Flynt will modify the files it runs on. Add your project to version control system before using flynt.

To run: flynt {source_file_or_directory}

  • Given a single file, it will 'f-stringify' it: replace all applicable string formatting in this file (file will be modified).
  • Given a folder, it will search the folder recursively and f-stringify all the .py files it finds. It skips some hard-coded folder names: blacklist = {'.tox', 'venv', 'site-packages', '.eggs'}.

It turns the code it runs on into Python 3.6+, since 3.6 is when "f-strings" were introduced.

Command line options

From the output of flynt -h:

positional arguments:
  src                   source file(s) or directory

optional arguments:
  -h, --help            show this help message and exit
  -v, --verbose         run with verbose output
  -q, --quiet           run without output
  --no-multiline        convert only single line expressions
  -ll LINE_LENGTH, --line-length LINE_LENGTH
                        for expressions spanning multiple lines, convert only if the resulting single line will fit into the line length limit. Default value is 88 characters.
  -d, --dry-run         Do not change the files in-place and print the diff instead. Note that this must be used in conjunction with '--fail-on-change' when used for linting purposes.
  -s, --string          Interpret the input as a Python code snippet and print the converted version. The snippet must use single quotes or escaped double quotes.
  -tc, --transform-concats
                        Replace string concatenations (defined as + operations involving string literals) with f-strings. Available only if flynt is installed with 3.8+ interpreter.
  -f, --fail-on-change  Fail when changing files (for linting purposes)
  -a, --aggressive      Include conversions with potentially changed behavior.
  -e EXCLUDE [EXCLUDE ...], --exclude EXCLUDE [EXCLUDE ...]
                        ignore files with given strings in it's absolute path.
  --version             Print the current version number and exit.

Sample output of a successful run:

38f9d3a65222:~ ikkamens$ git clone https://github.com/pallets/flask.git
Cloning into 'flask'...
...
Resolving deltas: 100% (12203/12203), done.

38f9d3a65222:open_source ikkamens$ flynt flask
Running flynt v.0.40

Flynt run has finished. Stats:

Execution time:                            0.789s
Files modified:                            21
Character count reduction:                 299 (0.06%)

Per expression type:
Old style (`%`) expressions attempted:     40/42 (95.2%)
`.format(...)` calls attempted:            26/33 (78.8%)
F-string expressions created:              48
Out of all attempted transforms, 7 resulted in errors.
To find out specific error messages, use --verbose flag.

_-_._-_._-_._-_._-_._-_._-_._-_._-_._-_._-_._-_._-_._-_._-_._-_._-_._-_._-_._-_._-_._-_._-_._-_._-_.
Please run your tests before commiting. Did flynt get a perfect conversion? give it a star at:
~ https://github.com/ikamensh/flynt ~
Thank you for using flynt. Upgrade more projects and recommend it to your colleagues!

38f9d3a65222:~ ikkamens$

Pre-commit hook

To make sure all formatted strings are always converted to f-strings, you can add flynt to your pre-commit hooks.

Add a new section to .pre-commit-config.yaml:

-   repo: https://github.com/ikamensh/flynt/
    rev: ''
    hooks:
    -   id: flynt

This will run flynt on all modified files before commiting.

You can skip conversion of certain lines by adding # noqa [: anything else] flynt [anything else]

About

Read up on f-strings here:

After obsessively refactoring a project at work, and not even covering 50% of f-string candidates, I realized there was some place for automation. Also it was very interesting to work with ast module.

Dangers of conversion

It is not guaranteed that formatted strings will be exactly the same as before conversion.

'%s' % var is converted to f'{var}'. There is a case when this will behave different from the original - if var is a tuple of one element. In this case, %s displays the element, and f-string displays the tuple. Example:

foo = (1,)
print('%s' % foo) # prints '1'
print(f'{foo}')   # prints '(1,)'

Furthermore, some arguments cause formatting of strings to throw exceptions. One example where f-strings are inconsistent with previous formatting is %d vs {:d} - new format no longer accepts floats. While most cases are covered by taking the formatting specifiers to the f-strings format, the precise exception behaviour might differ as well. Make sure you have sufficient test coverage.

Other Credits / Dependencies / Links

  • astor is used to turn the transformed AST back into code.
  • Thanks to folks from pyddf for their support, advice and participation during spring hackathon 2019, in particular Holger Hass, Farid Muradov, Charlie Clark.
Comments
  • Remove direct wrapping of pyupgrade

    Remove direct wrapping of pyupgrade

    As discussed in this thread.

    • pyupgrade (intentionally) has no programmatic api
    • as agreed, I've implemented a run-on-folder version of pyupgrade:
      • pip install pyupgrade-directory
      • pyupgrade-directory [options] [folder / file] ...
      • for example: pyupgrade-directory /tmp/flynt/{src,test} --py37-plus
    opened by asottile 14
  • add a --version option

    add a --version option

    This adds a --version option to flynt. To avoid collisions with the required positional argument src, this required a small workaround. Maybe there's a cleaner solution I'm not seeing.

    opened by neutrinoceros 9
  • Changes behaviour of `'%s' % tuple`

    Changes behaviour of `'%s' % tuple`

    $ cat t.py
    foo = (1,)
    print('%s' % foo)
    $ python t.py
    1
    $ flynt t.py
    fstringifying /tmp/flynt/t.py...yes
    
    Flynt run has finished. Stats:
    
    Execution time: 0.003s
    Files modified: 1
    Expressions transformed: 1
    Character count reduction: 2 (6.90%)
    
    _-_._-_._-_._-_._-_._-_._-_._-_._-_._-_._-_._-_._-_._-_._-_._-_._-_._-_._-_._-_._-_._-_._-_._-_._-_.
    
    Please run your tests before commiting. Report bugs as github issues at: https://github.com/ikamensh/flynt
    Thank you for using flynt! Fstringify more projects and recommend it to your colleagues!
    
    _-_._-_._-_._-_._-_._-_._-_._-_._-_._-_._-_._-_._-_._-_._-_._-_._-_._-_._-_._-_._-_._-_._-_._-_._-_.
    $ python t.py
    (1,)
    

    (taken from the pyupgrade testsuite)

    pyupgrade avoids rewriting 1-ary formats with right-hand-side being variable

    opened by asottile 9
  • auto detect line endings

    auto detect line endings

    This PR adds a new function for detecting the newline character so that the files written back after changes retain the line endings the original file had.

    closes #87

    opened by Inveracity 8
  • add a pre-commit-hook configuration file

    add a pre-commit-hook configuration file

    In continuation of @cphyc 's contribution of a dry run, I'd like to propose this configuration file for a flynt pre-commit hook that we'd like to use for https://github.com/yt-project/yt/

    I was able to test this by merging it into my fork's master branch (pre-commits can not be configured to target specific branches apparently) and it works wonderfully well !

    In order to install the hook in a project, all that was need is this

    -   repo: https://github.com/neutrinoceros/flynt/
        rev: ''
        hooks:
        -   id: flynt
    

    see for instance what happens with a fake commit where I tried to introduce a non-fstring formatted string

    Screenshot 2020-08-10 at 21 28 11

    and here's what's happening to a badly formatted string here before

    from math import e
    print("big bad non-fstring with a float %.3f" % e)
    

    after

    from math import e
    print(f"big bad non-fstring with a float {e:.3f}")
    
    opened by neutrinoceros 8
  • pre-commit: Add bandit

    pre-commit: Add bandit

    https://github.com/PyCQA/bandit -- Would it be worthwhile to avoid Bandit B307?

    >> Issue: [B307:blacklist] Use of possibly insecure function - consider using safer ast.literal_eval.
       Severity: Medium   Confidence: High
       CWE: CWE-78 (https://cwe.mitre.org/data/definitions/78.html)
       Location: ./flynt/test/test_process.py:446:11
       More Info: https://bandit.readthedocs.io/en/1.7.4/blacklists/blacklist_calls.html#b307-eval
    
    opened by cclauss 7
  • fix KeyError on % {...} with reused key

    fix KeyError on % {...} with reused key

    Hi! I hit a KeyError on code like "%(var)s %(var)s" % {'var': 'x'}, which didn't get converted because it reuses a key, and flynt pop()s keys on their first use. This fixes that. Example exception below.

    One drawback here is that flynt no longer detects keys in the dict that aren't used in the literal. Let me know if that's a deal breaker, and I can probably add it back along with this fix.

    Thanks in advance!

    Exception 'type' during conversion of code '"foo %(var)s %(var)s" % {'var': 'x'}'
    Traceback (most recent call last):
      File "/Users/ryan/src/oauth-dropins/local/lib/python3.9/site-packages/flynt/transform/transform.py", line 29, in transform_chunk
        converted, changed, str_in_str = fstringify_node(copy.deepcopy(tree))
      File "/Users/ryan/src/oauth-dropins/local/lib/python3.9/site-packages/flynt/transform/FstringifyTransformer.py", line 91, in fstringify_node
        result = ft.visit(node)
      File "/opt/homebrew/Cellar/[email protected]/3.9.9/Frameworks/Python.framework/Versions/3.9/lib/python3.9/ast.py", line 407, in visit
        return visitor(node)
      File "/opt/homebrew/Cellar/[email protected]/3.9.9/Frameworks/Python.framework/Versions/3.9/lib/python3.9/ast.py", line 483, in generic_visit
        value = self.visit(value)
      File "/opt/homebrew/Cellar/[email protected]/3.9.9/Frameworks/Python.framework/Versions/3.9/lib/python3.9/ast.py", line 407, in visit
        return visitor(node)
      File "/opt/homebrew/Cellar/[email protected]/3.9.9/Frameworks/Python.framework/Versions/3.9/lib/python3.9/ast.py", line 492, in generic_visit
        new_node = self.visit(old_value)
      File "/opt/homebrew/Cellar/[email protected]/3.9.9/Frameworks/Python.framework/Versions/3.9/lib/python3.9/ast.py", line 407, in visit
        return visitor(node)
      File "/Users/ryan/src/oauth-dropins/local/lib/python3.9/site-packages/flynt/transform/FstringifyTransformer.py", line 80, in visit_BinOp
        result_node, str_in_str = transform_binop(node)
      File "/Users/ryan/src/oauth-dropins/local/lib/python3.9/site-packages/flynt/transform/percent_transformer.py", line 212, in transform_binop
        return transform_dict(node), False
      File "/Users/ryan/src/oauth-dropins/local/lib/python3.9/site-packages/flynt/transform/percent_transformer.py", line 110, in transform_dict
        fv = formatted_value(prefix, fmt_str, make_fv(var_key))
      File "/Users/ryan/src/oauth-dropins/local/lib/python3.9/site-packages/flynt/transform/percent_transformer.py", line 97, in make_fv
        return mapping.pop(key)
    KeyError: 'var'
    
    opened by snarfed 7
  • Improve formatting of .format() with string literals inside it

    Improve formatting of .format() with string literals inside it

    Great job writing flynt! It works really well :smiley:

    One thing that was a bit annoying is that we had lines like this one in our codebase:

    x = "{}, {}".format(some_expression, "literal string")
    

    Although this is quite bad, when it was rewritten it became more annoying:

    x = f"{some_expression}, {'literal string'}"
    

    This change will make the original expression to be rewritten like this instead:

    x = f"{some expression}, literal_string"
    

    Maybe it's something very specific to our codebase. But I'll open a PR anyway in case you're interested in putting this change into the upstream version.

    opened by cript0nauta 7
  • Linting and Coverage

    Linting and Coverage

    @ikamensh Do you have an opinion on coverage reporting and automated linting checks? If you want I can spend some time the coming days to implement:

    • pre-commit hooks
    • black enforcement
    • pylint/ flake8
    • pytest-cov reports at the end of test runs
    opened by neunkasulle 6
  • Multiline format statements not converted

    Multiline format statements not converted

    Hi, another one for you. This format statement is not processed (ignored):

    print("Finished: min time: {}, max_time: {}, ave time: {}".format(
        min_time,
        max_time,
        ave_time
    ))
    

    It would be an obvious candidate for f-strification.

    opened by mikeckennedy 6
  • Digit grouping may break conversion

    Digit grouping may break conversion

    Hi, great little utility! I have lots of perfect changes added to my project with it. But this one seems to be off:

    Take this line:

    log.notice("Search: finished in {0:,} ms.".format(vm.search_time_elapsed_ms))
    

    It was transformed to this incorrect line:

    log.notice(f"Search: finished in {{0:,}} ms.")
    
    opened by mikeckennedy 6
  • Transform concatenation does not work with

    Transform concatenation does not work with "--string"

    Version/Dist information: Python 3.10.6 Bash v5.1.16(1)-release Ubuntu 22.04 x86_64

    Expected behavior is achieved when reading from a python file. cat test.py:

    #!/bin/env python
    test_string = 'This' + '  ' + 'may ' + ' ' +  'not' + ' ' + 'work'
    

    flynt -tc t.py:

    test_string = f"This may not work"
    

    flynt -tc --string "test_string = 'This' + ' ' + 'may' + ' ' + 'not' + ' ' + 'work'"

    Expected result:

    test_string = f"This may not work"
    

    Actual result:

    test_string = 'This' + ' ' + 'may' + ' ' + 'not' + ' ' + 'work'
    

    Flynt does not report any errors or missed opportunities. It is simply not recognized. Passing it in from an array, using < <(...) , with escaped double quotes, etc., still does not produce the expected behavior.

    opened by EmmaJaneBonestell 0
  • Support for stdin

    Support for stdin

    To be able to use Flynt as an ALE Fixer (Vim plugin), Flynt needs to be able to

    • accept the data on standard input.
    • output data to standard out.

    Unless there's a work-around?

    opened by gerases 0
  • feature request: print function?

    feature request: print function?

    hello, thanks for great library. could it be possible for flynt to handle situations such as these in the future?

    print("***** Time elapsed for Python: ", str(t1 - t0), " Counts: ", str(count))
    
    opened by scarf005 0
  • String alignment is broken

    String alignment is broken

    In python "%Ns" can be used to align strings, e.g.

    >>> print("hello %30s" % "world")
    'hello                          world'
    

    flynt --aggressive erroneously convert the above statement to:

    print(f"hello {'world':30}")
    

    Instead it should be converted to:

    print(f"hello {'world':>30}")
    
    bug 
    opened by giampaolo 2
  • [req] Add support for removing spurious f-strings

    [req] Add support for removing spurious f-strings

    I checked out the project because I was hoping it would help from a common issue I run into

    F541 f-string is missing placeholders
    

    Would it be reasonable to have flynt strip out the f prefix for these cases?

    enhancement PR welcome 
    opened by ADraginda 1
Owner
Elijah K
Software Engineer, Machine Learning.
Elijah K
A startpage configured aesthetically with terminal-esque link formatting

Terminal-y Startpage Setup Clone the repository, then make an unformatted.txt file following the specifications in example.txt. Run format.py Open ind

belkarx 13 May 1, 2022
Ralph is a command-line tool to fetch, extract, convert and push your tracking logs from various storage backends to your LRS or any other compatible storage or database backend.

Ralph is a command-line tool to fetch, extract, convert and push your tracking logs (aka learning events) from various storage backends to your

France Université Numérique 18 Jan 5, 2023
QueraToCSV is a simple python CLI project to convert the Quera results file into CSV files.

Quera is an Iranian Learning management system (LMS) that has an online judge for programming languages. Some Iranian universities use it to automate the evaluation of programming assignments.

Amirmahdi Namjoo 16 Nov 11, 2022
Joji convert a text to corresponding emoji if emoji is available

Joji Joji convert a text to corresponding emoji if emoji is available How it Works ? 1. There is a json file with emoji names as keys and correspondin

Gopikrishnan Sasikumar 28 Nov 26, 2022
Convert ACSM files to DRM-free EPUB files with one command on Linux

Knock Convert ACSM files to DRM-free EPUB files using one command. This software does not utilize Adobe Digital Editions nor Wine. It is completely fr

Benton Edmondson 622 Dec 9, 2022
Convert markdown to HTML using the GitHub API and some additional tweaks with Python.

Convert markdown to HTML using the GitHub API and some additional tweaks with Python. Comes with full formula support and image compression.

phseiff 70 Dec 23, 2022
A simple CLI to convert snapshots into EAVT log, and EAVT log into SCD.

EAVT helper CLI Simple CLI to convert snapshots into eavt log, and eavt log into slowly changing dimensions Usage Installation Snapshot to EAVT log EA

null 2 Apr 7, 2022
Convert shellcode into :sparkles: different :sparkles: formats!

Bluffy Convert shellcode into ✨ different ✨ formats! Bluffy is a utility which was used in experiments to bypass Anti-Virus products (statically) by f

pre.empt.dev 305 Dec 17, 2022
A Bot Which Send Automatically Commands To Karuta Hub to Gain it's Currency

A Bot Which Send Automatically Commands To Karuta Hub to Gain it's Currency

HarshalWaykole 1 Feb 9, 2022
Shortcut-Maker - It is a tool that can be set to run any tool with a single command

Shortcut-Maker It is a tool that can be set to run any tool with a single command Coded by Dave Smith(Owner of Sl Cyber Warriors) Command list ?? pkg

Dave Smith 10 Sep 14, 2022
A simple CLI based any Download Tool, that find files and let you stream or download thorugh WebTorrent CLI or Aria or any command tool

Privateer A simple CLI based any Download Tool, that find files and let you stream or download thorugh WebTorrent CLI or Aria or any command tool How

Shreyash Chavan 2 Apr 4, 2022
Dead simple CLI tool to try Python packages - It's never been easier! :package:

try - It's never been easier to try Python packages try is an easy-to-use cli tool to try out Python packages. Features Install specific package versi

Timo Furrer 659 Dec 28, 2022
Python commandline tool for remembering linux/terminal commands

ehh Remember linux commands Commandline tool for remembering linux/terminal commands. It stores your favorite commands in ~/ehh.json in your homedir a

null 56 Nov 10, 2022
Command-line tool for looking up colors and palettes.

Colorpedia Colorpedia is a command-line tool for looking up colors, shades and palettes. Supported color models: HEX, RGB, HSL, HSV, CMYK. Requirement

Joohwan Oh 282 Dec 27, 2022
Simple command line tool for text to image generation using OpenAI's CLIP and Siren (Implicit neural representation network)

Simple command line tool for text to image generation using OpenAI's CLIP and Siren (Implicit neural representation network)

Phil Wang 4.4k Jan 9, 2023
Free and Open-Source Command Line tool for Text Replacement

Sniplet Free and Open Source Text Replacement Tool Description: Sniplet is a work in progress CLI tool which can do text replacement globally in Linux

Veeraraghavan Narasimhan 13 Nov 28, 2022
Simple CLI tool to track your cryptocurrency portfolio in real time.

Simple tool to track your crypto portfolio in realtime. It can be used to track any coin on the BNB network, even obscure coins that are not listed or trackable by major portfolio tracking applications.

Trevor White 69 Oct 24, 2022
Chameleon is yet another PowerShell obfuscation tool designed to bypass AMSI and commercial antivirus solutions.

Chameleon is yet another PowerShell obfuscation tool designed to bypass AMSI and commercial antivirus solutions. The tool has been developed as a Python port of the Chimera project, by tokioneon_.

null 332 Dec 26, 2022