A versatile token stream for handwritten parsers.

Overview

tokenstream

GitHub Actions PyPI PyPI - Python Version Code style: black

A versatile token stream for handwritten parsers.

from tokenstream import TokenStream

def parse_sexp(stream: TokenStream):
    """A basic S-expression parser."""
    with stream.syntax(brace=r"\(|\)", number=r"\d+", name=r"\w+"):
        brace, number, name = stream.expect(("brace", "("), "number", "name")
        if brace:
            return [parse_sexp(stream) for _ in stream.peek_until(("brace", ")"))]
        elif number:
            return int(number.value)
        elif name:
            return name.value

print(parse_sexp(TokenStream("(hello (world 42))")))  # ['hello', ['world', 42]]

Introduction

Writing recursive-descent parsers by hand can be quite elegant but it's often a bit more verbose than expected, especially when it comes to handling indentation and reporting proper syntax errors. This package provides a powerful general-purpose token stream that addresses these issues and more.

Features

  • Define the set of recognizable tokens dynamically with regular expressions
  • Transparently skip over irrelevant tokens
  • Expressive API for matching, collecting, peeking, and expecting tokens
  • Clean error reporting with line numbers and column numbers
  • Contextual support for indentation-based syntax
  • Checkpoints for backtracking parsers
  • Works well with Python 3.10+ match statements

Check out the examples directory for practical examples.

Installation

The package can be installed with pip.

pip install tokenstream

Getting started

You can define tokens with the syntax() method. The keyword arguments associate regular expression patterns to token types. The method returns a context manager during which the specified tokens will be recognized.

stream = TokenStream("hello world")

with stream.syntax(word=r"\w+"):
    print([token.value for token in stream])  # ['hello', 'world']

Check out the full API reference for more details.

Expecting tokens

The token stream is iterable and will yield all the extracted tokens one after the other. You can also retrieve tokens from the token stream one at a time by using the expect() method.

stream = TokenStream("hello world")

with stream.syntax(word=r"\w+"):
    print(stream.expect().value)  # "hello"
    print(stream.expect().value)  # "world"

The expect() method lets you ensure that the extracted token matches a specified type and will raise an exception otherwise.

stream = TokenStream("hello world")

with stream.syntax(number=r"\d+", word=r"\w+"):
    print(stream.expect("word").value)  # "hello"
    print(stream.expect("number").value)  # UnexpectedToken: Expected number but got word 'world'

Filtering the stream

Newlines and whitespace are ignored by default. You can reject interspersed whitespace by intercepting the built-in newline and whitespace tokens.

stream = TokenStream("hello world")

with stream.syntax(word=r"\w+"), stream.intercept("newline", "whitespace"):
    print(stream.expect("word").value)  # "hello"
    print(stream.expect("word").value)  # UnexpectedToken: Expected word but got whitespace ' '

The opposite of the intercept() method is ignore(). It allows you to ignore tokens and handle comments pretty easily.

stream = TokenStream(
    """
    # this is a comment
    hello # also a comment
    world
    """
)

with stream.syntax(word=r"\w+", comment=r"#.+$"), stream.ignore("comment"):
    print([token.value for token in stream])  # ['hello', 'world']

Indentation

To enable indentation you can use the indent() method. The stream will now yield balanced pairs of indent and dedent tokens when the indentation changes.

source = """
hello
    world
"""
stream = TokenStream(source)

with stream.syntax(word=r"\w+"), stream.indent():
    stream.expect("word")
    stream.expect("indent")
    stream.expect("word")
    stream.expect("dedent")

To prevent some tokens from triggering unwanted indentation changes you can use the skip argument.

source = """
hello
        # some comment
    world
"""
stream = TokenStream(source)

with stream.syntax(word=r"\w+", comment=r"#.+$"), stream.indent(skip=["comment"]):
    stream.expect("word")
    stream.expect("comment")
    stream.expect("indent")
    stream.expect("word")
    stream.expect("dedent")

Checkpoints

The checkpoint() method returns a context manager that resets the stream to the current token at the end of the with statement. You can use the returned commit() function to keep the state of the stream at the end of the with statement.

stream = TokenStream("hello world")

with stream.syntax(word=r"\w+"):
    with stream.checkpoint():
        print([token.value for token in stream])  # ['hello', 'world']
    with stream.checkpoint() as commit:
        print([token.value for token in stream])  # ['hello', 'world']
        commit()
    print([token.value for token in stream])  # []

Match statements

Match statements make it very intuitive to process tokens extracted from the token stream. If you're using Python 3.10+ give it a try and see if you like it.

from tokenstream import TokenStream, Token

def parse_sexp(stream: TokenStream):
    """A basic S-expression parser that uses Python 3.10+ match statements."""
    with stream.syntax(brace=r"\(|\)", number=r"\d+", name=r"\w+"):
        match stream.expect_any(("brace", "("), "number", "name"):
            case Token(type="brace"):
                return [parse_sexp(stream) for _ in stream.peek_until(("brace", ")"))]
            case Token(type="number") as number :
                return int(number.value)
            case Token(type="name") as name:
                return name.value

Contributing

Contributions are welcome. Make sure to first open an issue discussing the problem or the new feature before creating a pull request. The project uses poetry.

$ poetry install

You can run the tests with poetry run pytest.

$ poetry run pytest

The project must type-check with pyright. If you're using VSCode the pylance extension should report diagnostics automatically. You can also install the type-checker locally with npm install and run it from the command-line.

$ npm run watch
$ npm run check
$ npm run verifytypes

The code follows the black code style. Import statements are sorted with isort.

$ poetry run isort tokenstream examples tests
$ poetry run black tokenstream examples tests
$ poetry run black --check tokenstream examples tests

License - MIT

Comments
  • chore(deps-dev): bump pyright from 1.1.276 to 1.1.285

    chore(deps-dev): bump pyright from 1.1.276 to 1.1.285

    Bumps pyright from 1.1.276 to 1.1.285.

    Release notes

    Sourced from pyright's releases.

    Published 1.1.285

    Enhancement: Implemented a new --level command-line option that allows filtering of 'information' and 'warning' diagnostics.

    Enhancement: Updated TOML parser to one that is compliant with the TOML 1.0 spec.

    Enhancement: Added logic to detect uses of PEP 604 | syntax that generate exceptions due to runtime limitations. In particular, if one of the operands is a string (i.e. a forward reference) and the other is also a string or a class that is not explicitly specialized, this will result in an exception.

    Bug Fix: Fixed recent regression in completion provider that resulted in garbled type information for a symbol that is declared as a function (using a def statement) but transformed into a non-function type using a decorator.

    Bug Fix: Fixed a bug that resulted in a false positive error when an index expression with a numeric literal subscript was used in a loop that included a del statement targeting the same index expression.

    Behavior Change: Modified the overload matching algorithm to match the behavior of mypy when the overload match is ambiguous because an argument evaluates to Any or Unknown. In this case, the call expression evaluates to Unknown. Previously, pyright used the first of the matching overloads in this case.

    Bug Fix: Fixed a bug that led to extremely long type analysis times when determining type compatibility between an recursive type alias and a recursive protocol.

    Bug Fix (contribution from @​parched): Fixed recent regression that caused reportImportCycles diagnostic reporting to no longer work.

    Bug Fix: Fixed a bug that resulted in a false positive error when a property setter or deleter contained function-scoped type variables.

    Published 1.1.284

    Bug Fix: Fixed a bug that resulted in an incorrect type evaluation when using a literal integer index into a tuple that includes an unpacked TypeVarTuple element.

    Behavior Change: Removed diagnostic check that detects a non-ellipsis default value in a stub file. This check was based on a now-outdated best practice. We now recommend that stubs include default values for better usability in language servers.

    Bug Fix: Fixed recent regression that caused the hover text for a function symbol to be improperly formatted if its type was evaluated to something other than a function.

    Enhancement: Exposed new configuration setting analyzeUnannotatedFunctions which corresponds to the --skipunannotated command-line option. Added an information diagnostic for skipped functions.

    Bug Fix: Fixed bug that resulted in an incorrect target range when resolving the declaration of an import. There were cases where "Go To Declaration" would take you to a location within the target file that corresponded to the offset of the import statement within the importing file.

    Bug Fix: Fixed bugs that resulted in false positive errors when assigning a type to a Callable and when accessing members from a type[T] or a T that is bound to type.

    Enhancement: Improved bidirectional type inference for list, set and dictionary expressions when the "expected type" is a union that contains multiple potentially-compatible subtypes.

    Bug Fix: Fixed a bug that led to a false positive error when evaluating a recursive type alias definition that uses a generic type that is parameterized using a bound or constrained type variable.

    Bug Fix: Added missing diagnostic rule in diagnostics reported as part of the reportImportCycles check. Thanks for @​parched for this fix.

    Bug Fix: Fixed false positive error in multi-inheritance override check in cases where the override symbol was provided by one of the base classes.

    Enhancement: Updated typeshed stubs to the latest version.

    Behavior Change: Added check for disallowed expression forms used with TypeAlias and with PEP 695 type statement.

    Bug Fix (from pylance): Fixed exclude/include to work with '.' prefixed directories.

    Bug Fix: Fixed a bug that led to a false positive when a NoReturn or Never type argument was used for a covariant type parameter.

    Bug Fix: Fixed regression that resulted in inconsistent behavior when useLibraryCodeForTypes is false and a symbol is imported from a non-py.typed library.

    ... (truncated)

    Commits

    Dependabot compatibility score

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
    • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    dependencies javascript 
    opened by dependabot[bot] 1
  • chore(deps-dev): bump isort from 5.10.1 to 5.11.3

    chore(deps-dev): bump isort from 5.10.1 to 5.11.3

    Bumps isort from 5.10.1 to 5.11.3.

    Release notes

    Sourced from isort's releases.

    5.11.3

    Changes

    :beetle: Fixes

    :construction_worker: Continuous Integration

    v5.11.3

    Changes

    :beetle: Fixes

    :construction_worker: Continuous Integration

    5.11.2

    Changes

    5.11.1

    Changes December 12 2022

    :beetle: Fixes

    5.11.0

    Changes December 12 2022

    ... (truncated)

    Changelog

    Sourced from isort's changelog.

    5.11.3 December 16 2022

    5.11.2 December 12 2022

    5.11.1 December 12 2022

    5.11.0 December 12 2022

    Commits

    Dependabot compatibility score

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
    • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    dependencies python 
    opened by dependabot[bot] 1
  • chore(deps-dev): bump isort from 5.10.1 to 5.11.2

    chore(deps-dev): bump isort from 5.10.1 to 5.11.2

    Bumps isort from 5.10.1 to 5.11.2.

    Release notes

    Sourced from isort's releases.

    5.11.2

    Changes

    5.11.1

    Changes December 12 2022

    :beetle: Fixes

    5.11.0

    Changes December 12 2022

    :construction_worker: Continuous Integration

    :package: Dependencies

    Changelog

    Sourced from isort's changelog.

    5.11.2 December 12 2022

    5.11.1 December 12 2022

    5.11.0 December 12 2022

    Commits
    • 7eaab8c Merge pull request #2036 from PyCQA/feature/5.11.2-version-bump
    • 112eb30 Update version to 5.11.2
    • e64306d Merge pull request #2035 from felixxm/version-bump
    • b418718 Bump version to 5.11.1.
    • f8146c5 Merge pull request #2033 from PyCQA/hotfix/5.11.1
    • dd01cfe Hotfix 5.11.1
    • 68f0a25 Merge pull request #2032 from tomaarsen/hotfix/colorama_nameerror
    • c752a6c Only call colorama.init if colorama is available
    • 6525008 Merge pull request #2030 from PyCQA/example/update-formatting-pluging-isort-5...
    • 6c5a36c Bump formatting plugin to 0.1.1
    • Additional commits viewable in compare view

    Dependabot compatibility score

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
    • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    dependencies python 
    opened by dependabot[bot] 1
  • chore(deps-dev): bump pyright from 1.1.276 to 1.1.284

    chore(deps-dev): bump pyright from 1.1.276 to 1.1.284

    Bumps pyright from 1.1.276 to 1.1.284.

    Release notes

    Sourced from pyright's releases.

    Published 1.1.284

    Bug Fix: Fixed a bug that resulted in an incorrect type evaluation when using a literal integer index into a tuple that includes an unpacked TypeVarTuple element.

    Behavior Change: Removed diagnostic check that detects a non-ellipsis default value in a stub file. This check was based on a now-outdated best practice. We now recommend that stubs include default values for better usability in language servers.

    Bug Fix: Fixed recent regression that caused the hover text for a function symbol to be improperly formatted if its type was evaluated to something other than a function.

    Enhancement: Exposed new configuration setting analyzeUnannotatedFunctions which corresponds to the --skipunannotated command-line option. Added an information diagnostic for skipped functions.

    Bug Fix: Fixed bug that resulted in an incorrect target range when resolving the declaration of an import. There were cases where "Go To Declaration" would take you to a location within the target file that corresponded to the offset of the import statement within the importing file.

    Bug Fix: Fixed bugs that resulted in false positive errors when assigning a type to a Callable and when accessing members from a type[T] or a T that is bound to type.

    Enhancement: Improved bidirectional type inference for list, set and dictionary expressions when the "expected type" is a union that contains multiple potentially-compatible subtypes.

    Bug Fix: Fixed a bug that led to a false positive error when evaluating a recursive type alias definition that uses a generic type that is parameterized using a bound or constrained type variable.

    Bug Fix: Added missing diagnostic rule in diagnostics reported as part of the reportImportCycles check. Thanks for @​parched for this fix.

    Bug Fix: Fixed false positive error in multi-inheritance override check in cases where the override symbol was provided by one of the base classes.

    Enhancement: Updated typeshed stubs to the latest version.

    Behavior Change: Added check for disallowed expression forms used with TypeAlias and with PEP 695 type statement.

    Bug Fix (from pylance): Fixed exclude/include to work with '.' prefixed directories.

    Bug Fix: Fixed a bug that led to a false positive when a NoReturn or Never type argument was used for a covariant type parameter.

    Bug Fix: Fixed regression that resulted in inconsistent behavior when useLibraryCodeForTypes is false and a symbol is imported from a non-py.typed library.

    Published 1.1.283

    Enhancement: Added support for # pyright: ignore and # type: ignore comments that are not at the start of a comment.

    Enhancement: Improved parse recovery for common indent/dedent conditions.

    Bug Fix: Fixed recent regression that resulted in a false positive when a type[T] was assigned to a Callable[..., T].

    Bug Fix: Fixed a regression related to a recent change in typeshed that caused imports from google.cloud namespace packages to fail.

    Bug Fix: Fixed a bug that resulted in incorrect type evaluation when assigning an unpacked TypeVarTuple to a regular (non-variadic) TypeVar during constraint solving. This should result in a union of the TypeVarTuple types.

    Bug Fix: Fixed a bug that resulted in a false positive diagnostic when importing a symbol from a "py.typed" library that was imported (and re-exported) from a non-"py.typed" library.

    Bug Fix: Changed ParamSpec capture logic to track the expression used for a default argument value.

    Bug Fix: Fixed a recent regression that led to a false negative when evaluating the type of an unannotated variable in a loop.

    Enhancement: Added support for frozen_default for dataclass_transform.

    ... (truncated)

    Commits

    Dependabot compatibility score

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
    • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    dependencies javascript 
    opened by dependabot[bot] 1
  • chore(deps-dev): bump isort from 5.10.1 to 5.11.1

    chore(deps-dev): bump isort from 5.10.1 to 5.11.1

    Bumps isort from 5.10.1 to 5.11.1.

    Release notes

    Sourced from isort's releases.

    5.11.1

    Changes December 12 2022

    :beetle: Fixes

    5.11.0

    Changes December 12 2022

    :construction_worker: Continuous Integration

    :package: Dependencies

    Changelog

    Sourced from isort's changelog.

    5.11.1 December 12 2022

    5.11.0 December 12 2022

    Commits
    • f8146c5 Merge pull request #2033 from PyCQA/hotfix/5.11.1
    • dd01cfe Hotfix 5.11.1
    • 68f0a25 Merge pull request #2032 from tomaarsen/hotfix/colorama_nameerror
    • c752a6c Only call colorama.init if colorama is available
    • 6525008 Merge pull request #2030 from PyCQA/example/update-formatting-pluging-isort-5...
    • 6c5a36c Bump formatting plugin to 0.1.1
    • 657ed81 Merge pull request #2029 from PyCQA/example/update-formatting-pluging-isort-5...
    • 84e3687 Update isort on formatting pluging to 5.11.0
    • 3e99c22 Merge pull request #2028 from PyCQA/prepare-release-5.11.0
    • 8af078c Upgrade poetry to 1.3.1
    • Additional commits viewable in compare view

    Dependabot compatibility score

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
    • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    dependencies python 
    opened by dependabot[bot] 1
  • chore(deps-dev): bump pyright from 1.1.276 to 1.1.283

    chore(deps-dev): bump pyright from 1.1.276 to 1.1.283

    Bumps pyright from 1.1.276 to 1.1.283.

    Release notes

    Sourced from pyright's releases.

    Published 1.1.283

    Enhancement: Added support for # pyright: ignore and # type: ignore comments that are not at the start of a comment.

    Enhancement: Improved parse recovery for common indent/dedent conditions.

    Bug Fix: Fixed recent regression that resulted in a false positive when a type[T] was assigned to a Callable[..., T].

    Bug Fix: Fixed a regression related to a recent change in typeshed that caused imports from google.cloud namespace packages to fail.

    Bug Fix: Fixed a bug that resulted in incorrect type evaluation when assigning an unpacked TypeVarTuple to a regular (non-variadic) TypeVar during constraint solving. This should result in a union of the TypeVarTuple types.

    Bug Fix: Fixed a bug that resulted in a false positive diagnostic when importing a symbol from a "py.typed" library that was imported (and re-exported) from a non-"py.typed" library.

    Bug Fix: Changed ParamSpec capture logic to track the expression used for a default argument value.

    Bug Fix: Fixed a recent regression that led to a false negative when evaluating the type of an unannotated variable in a loop.

    Enhancement: Added support for frozen_default for dataclass_transform.

    Bug Fix: Fixed recent regression that caused a false positive reportShadowedStdlibModules for relative imports and imports starting with _.

    Enhancement: Show doc strings on hover over module names within import statements.

    Enhancement: Updated typeshed stubs to the latest version.

    Enhancement: Added small perf optimization for determining type compatibility between two unions, especially in cases where the number of items in the union is large.

    Bug Fix: Added logic to limit recursion when handling type compatibility checks between two different recursive type aliases that have the same definition.

    Bug Fix: Fixed bug that resulted in a false positive when passing an unpacked dict or unpacked iterable argument to the constructor of a class that has no __init__ and therefore uses object.__init__.

    Bug Fix: Fixed bug that led to a false positive error when using an unpacked argument that has a declared type that is a tuple with an unpacked TypeVarTuple.

    Bug Fix: Fixed bug that resulted in a false positive reportPrivateImportUsage diagnostic when importing from a py.typed library under certain circumstances.

    Published 1.1.282

    Bug Fix: Fixed bug that resulted in false positive error when a recursive type alias involving a union was used in certain circumstances.

    Bug Fix: Fixed bug that resulted in incorrect type evaluation when assigning a tuple with an unpacked TypeVarTuple to a tuple with a normal TypeVar.

    Bug Fix: Fixed a bug that resulted in a false negative (a missing error) when assigning a union that contains a TypeVar to another union with the same TypeVar.

    Bug Fix: Fixed bug that resulted in incorrect type inference for an instance variable that is assigned different types in different places.

    Bug Fix: Fixed bug in tokenizer that resulted in false positive error when a floating point literal started with one or more leading zeros.

    Bug Fix: Added logic to deal with conflicting namespace packages when one has no __init__.py(i) file and the other does.

    Enhancement: Updated typeshed stubs to the latest version.

    ... (truncated)

    Commits

    Dependabot compatibility score

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
    • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    dependencies javascript 
    opened by dependabot[bot] 1
  • chore(deps-dev): bump pyright from 1.1.276 to 1.1.282

    chore(deps-dev): bump pyright from 1.1.276 to 1.1.282

    Bumps pyright from 1.1.276 to 1.1.282.

    Release notes

    Sourced from pyright's releases.

    Published 1.1.282

    Bug Fix: Fixed bug that resulted in false positive error when a recursive type alias involving a union was used in certain circumstances.

    Bug Fix: Fixed bug that resulted in incorrect type evaluation when assigning a tuple with an unpacked TypeVarTuple to a tuple with a normal TypeVar.

    Bug Fix: Fixed a bug that resulted in a false negative (a missing error) when assigning a union that contains a TypeVar to another union with the same TypeVar.

    Bug Fix: Fixed bug that resulted in incorrect type inference for an instance variable that is assigned different types in different places.

    Bug Fix: Fixed bug in tokenizer that resulted in false positive error when a floating point literal started with one or more leading zeros.

    Bug Fix: Added logic to deal with conflicting namespace packages when one has no __init__.py(i) file and the other does.

    Enhancement: Updated typeshed stubs to the latest version.

    Bug Fix: Fixed a bug that results in a false positive error when a dataclass field in a child class overrides a non-init dataclass field in a parent class.

    Enhancement: Added code to report an error if a from __future__ import x statement is not found at the beginning of a file. This results in a syntax error.

    Behavior Change: Changed reportUnnecessaryTypeIgnoreComment to to ignore # type: ignore comments if enableTypeIgnoreComments is set to false.

    Bug Fix: Fixed a bug that led to an incorrect type evaluation when an empty list or dict literal is included in a list expression with an "expected type" (i.e. bidirectional type inference is being attempted).

    Bug Fix (from Pylance): Fixed recent regression that caused false positive errors to appear when using a multi-root workspace.

    Published 1.1.281

    Enhancement: Improved parse recovery for ternary expressions that are missing an else or a post-else expression.

    Enhancement: Changed the implicit declared type of a module-scoped doc to be str instead of str | None when a docstring is present at the top of the module.

    Bug Fix: Fixed a bug that resulted in incorrect type evaluation when dealing with a union of an unpacked TypeVarTuple.

    Bug Fix: Fixed bug that resulted in incorrect type evaluation when dealing with a union of an unpacked TypeVarTuple.

    Bug Fix: Fixed a bug that resulted in incorrect type evaluation when applying a solved unpacked TypeVarTuple in a Union.

    Bug Fix: Fixed a bug that resulted in a false positive during protocol matching for a protocol that includes a property with a getter whose self parameter is annotated with a TypeVar.

    Bug Fix: Fixed a bug that resulted in sporadic type evaluation errors when a quoted (forward-declared) type was used in a statement with an explicit PEP-613 TypeAlias annotation.

    Bug Fix: Fixed bug that resulted in false positive errors when doing protocol matching for a recursive protocol definition. I needed to increase an internal recursion limit to support this.

    Enhancement: Updated typeshed stubs to the latest.

    Bug Fix: Fixed regression that resulted in errant reportMissingImports diagnostics within stub files.

    Enhancement: Improved parse recovery when a suite contains an unexpected indent followed by a dedent that restores the indentation to that of the suite's body.

    Bug Fix: Fixed a bug in the evaluation of the "with" statement that resulted in a false positive error when a class implements a context manager via its metaclass.

    ... (truncated)

    Commits

    Dependabot compatibility score

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
    • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    dependencies javascript 
    opened by dependabot[bot] 1
  • chore(deps-dev): bump pyright from 1.1.276 to 1.1.281

    chore(deps-dev): bump pyright from 1.1.276 to 1.1.281

    Bumps pyright from 1.1.276 to 1.1.281.

    Release notes

    Sourced from pyright's releases.

    Published 1.1.281

    Enhancement: Improved parse recovery for ternary expressions that are missing an else or a post-else expression.

    Enhancement: Changed the implicit declared type of a module-scoped doc to be str instead of str | None when a docstring is present at the top of the module.

    Bug Fix: Fixed a bug that resulted in incorrect type evaluation when dealing with a union of an unpacked TypeVarTuple.

    Bug Fix: Fixed bug that resulted in incorrect type evaluation when dealing with a union of an unpacked TypeVarTuple.

    Bug Fix: Fixed a bug that resulted in incorrect type evaluation when applying a solved unpacked TypeVarTuple in a Union.

    Bug Fix: Fixed a bug that resulted in a false positive during protocol matching for a protocol that includes a property with a getter whose self parameter is annotated with a TypeVar.

    Bug Fix: Fixed a bug that resulted in sporadic type evaluation errors when a quoted (forward-declared) type was used in a statement with an explicit PEP-613 TypeAlias annotation.

    Bug Fix: Fixed bug that resulted in false positive errors when doing protocol matching for a recursive protocol definition. I needed to increase an internal recursion limit to support this.

    Enhancement: Updated typeshed stubs to the latest.

    Bug Fix: Fixed regression that resulted in errant reportMissingImports diagnostics within stub files.

    Enhancement: Improved parse recovery when a suite contains an unexpected indent followed by a dedent that restores the indentation to that of the suite's body.

    Bug Fix: Fixed a bug in the evaluation of the "with" statement that resulted in a false positive error when a class implements a context manager via its metaclass.

    Bug Fix: Fixed a bug that led to a false positive error when validating the variance of type parameters for a protocol class when a covariant type parameter is used in a return type in the form type[T_co].

    Bug Fix: Fixed a bug that resulted in a false positive error when using a cast to a TypeVar.

    Bug Fix: Fixed a bug in the type evaluator that could lead to unsolved TypeVars if used in a Callable parameter.

    Bug Fix: Fixed recent regression that led to false positive reportUnnecessaryCast diagnostics.

    Published 1.1.280

    Bug Fix: Fixed bug that led to an incorrect type evaluation (and potential false negative) when an iterable iterates over an unpacked TypeVarTuple.

    Bug Fix: Enabled the Never type to be used as an explicit type argument for a contravariant parameter.

    Bug Fix: Fixed a bug in the type printing logic where it omitted a Union when an unpacked TypeVarTuple is used within a Union.

    Bug Fix: Fixed a bug that resulted in incorrect type evaluation when the type arguments for a tuple included an unpacked TypeVarTuple. In this case, the type argument should be treated as a Union[*Ts] rather than *Ts.

    Bug Fix: Fixed a bug that resulted in a false positive for the reportIncompatibleMethodOverride check when the base method used an unpacked tuple for the *args parameter and the override used specific parameters.

    Bug Fix: Fixed a bug that resulted in a false negative when a non-frozen dataclass subclasses from a frozen dataclass. This generates a runtime error, so it should be flagged by the type checker.

    Bug Fix: Fixed a bug that resulted in a false positive error related to type narrowing in a match statement when matching against an enum with only one value.

    Published 1.1.279

    Behavior Change: Changed the way pyright handles the case where a dataclass or dataclass-like class handles a synthesized __init__ if one or more base classes are unknown.

    ... (truncated)

    Commits

    Dependabot compatibility score

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
    • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    dependencies javascript 
    opened by dependabot[bot] 1
  • chore(deps-dev): bump pyright from 1.1.276 to 1.1.280

    chore(deps-dev): bump pyright from 1.1.276 to 1.1.280

    Bumps pyright from 1.1.276 to 1.1.280.

    Release notes

    Sourced from pyright's releases.

    Published 1.1.280

    Bug Fix: Fixed bug that led to an incorrect type evaluation (and potential false negative) when an iterable iterates over an unpacked TypeVarTuple.

    Bug Fix: Enabled the Never type to be used as an explicit type argument for a contravariant parameter.

    Bug Fix: Fixed a bug in the type printing logic where it omitted a Union when an unpacked TypeVarTuple is used within a Union.

    Bug Fix: Fixed a bug that resulted in incorrect type evaluation when the type arguments for a tuple included an unpacked TypeVarTuple. In this case, the type argument should be treated as a Union[*Ts] rather than *Ts.

    Bug Fix: Fixed a bug that resulted in a false positive for the reportIncompatibleMethodOverride check when the base method used an unpacked tuple for the *args parameter and the override used specific parameters.

    Bug Fix: Fixed a bug that resulted in a false negative when a non-frozen dataclass subclasses from a frozen dataclass. This generates a runtime error, so it should be flagged by the type checker.

    Bug Fix: Fixed a bug that resulted in a false positive error related to type narrowing in a match statement when matching against an enum with only one value.

    Published 1.1.279

    Behavior Change: Changed the way pyright handles the case where a dataclass or dataclass-like class handles a synthesized __init__ if one or more base classes are unknown.

    Bug Fix: Fixed bug in type printer logic that resulted in a triple * before an unpacked TypedDict when it was used with a **kwargs parameter and the TypedDict was generic.

    Bug Fix: Fixed a bug that resulted in a false positive when invoking the constructor for a generic class that uses an unpacked TypedDict with an **args parameter.

    Bug Fix: Fixed regression that caused non-JSON output to be emitted when --verifytypes is used in conjunction with --outputjson.

    Behavior Change: Changed the reportUninitializedInstanceVariable so it doesn't run on stub files.

    Enhancement: Added new diagnostic rule reportShadowedImport that emits a diagnostic if a local module shadows a stdlib module.

    Bug Fix: Fixed a bug that resulted in a false positive when a call expression includes an unpacked argument that is a tuple with a known length.

    Bug Fix: Reverted recent change to the reportUnnecessaryComparison check that introduced a regression.

    Bug Fix: Fixed a bug that resulted in a false negative when a *args parameter with an unpacked tuple is followed by additional keyword-only parameters that are not supplied by a caller.

    Bug Fix: Added code to clean up the temp directory that pyright creates when determining whether the file system is case sensitive.

    Bug Fix: Fixed bug that resulted in false positive error when evaluating an unpacked tuple or TypeVarTuple in an *args parameter annotation.

    Bug Fix: Improved argument/parameter matching for unpacked tuple arguments.

    Bug Fix: Fixed a bug in detection of variance inconsistency with base classes in a class definition.

    Enhancement: Added support for detecting unhashable types within set and dict expressions. These result in a TypeError at runtime.

    Bug Fix: Fixed bug that resulted in poor hover text message for keyword arguments passed to a TypedDict constructor.

    Bug Fix: Fixed a bug that resulted in incorrect type evaluation when a generic class implicitly referenced one of its own magic methods (e.g. __iter__).

    Enhancement: Added support for recursive type references when using the functional "alternative syntax" for TypedDict.

    ... (truncated)

    Commits

    Dependabot compatibility score

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
    • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    dependencies javascript 
    opened by dependabot[bot] 1
  • chore(deps-dev): bump pyright from 1.1.276 to 1.1.279

    chore(deps-dev): bump pyright from 1.1.276 to 1.1.279

    Bumps pyright from 1.1.276 to 1.1.279.

    Release notes

    Sourced from pyright's releases.

    Published 1.1.279

    Behavior Change: Changed the way pyright handles the case where a dataclass or dataclass-like class handles a synthesized __init__ if one or more base classes are unknown.

    Bug Fix: Fixed bug in type printer logic that resulted in a triple * before an unpacked TypedDict when it was used with a **kwargs parameter and the TypedDict was generic.

    Bug Fix: Fixed a bug that resulted in a false positive when invoking the constructor for a generic class that uses an unpacked TypedDict with an **args parameter.

    Bug Fix: Fixed regression that caused non-JSON output to be emitted when --verifytypes is used in conjunction with --outputjson.

    Behavior Change: Changed the reportUninitializedInstanceVariable so it doesn't run on stub files.

    Enhancement: Added new diagnostic rule reportShadowedImport that emits a diagnostic if a local module shadows a stdlib module.

    Bug Fix: Fixed a bug that resulted in a false positive when a call expression includes an unpacked argument that is a tuple with a known length.

    Bug Fix: Reverted recent change to the reportUnnecessaryComparison check that introduced a regression.

    Bug Fix: Fixed a bug that resulted in a false negative when a *args parameter with an unpacked tuple is followed by additional keyword-only parameters that are not supplied by a caller.

    Bug Fix: Added code to clean up the temp directory that pyright creates when determining whether the file system is case sensitive.

    Bug Fix: Fixed bug that resulted in false positive error when evaluating an unpacked tuple or TypeVarTuple in an *args parameter annotation.

    Bug Fix: Improved argument/parameter matching for unpacked tuple arguments.

    Bug Fix: Fixed a bug in detection of variance inconsistency with base classes in a class definition.

    Enhancement: Added support for detecting unhashable types within set and dict expressions. These result in a TypeError at runtime.

    Bug Fix: Fixed bug that resulted in poor hover text message for keyword arguments passed to a TypedDict constructor.

    Bug Fix: Fixed a bug that resulted in incorrect type evaluation when a generic class implicitly referenced one of its own magic methods (e.g. __iter__).

    Enhancement: Added support for recursive type references when using the functional "alternative syntax" for TypedDict.

    Behavior Change: Changed the synthesized __dataclass_fields__ variable in dataclass to be a ClassVar so it cannot be overridden by an instance variable.

    Behavior Change: Changed protocol matching logic to enforce that a protocol with a ClassVar requires that any compatible class also declare the corresponding field as a ClassVar. Previously, any class-scoped variable satisfied the protocol in this case. This change brings pyright and mypy into agreement.

    Bug Fix: Fixed a bug that resulted in incorrect type evaluation involving a recursive type alias that is defined as the union of other types, one of which introduces the recursion.

    Published 1.1.278

    Behavior Change: Changed the reportUnusedImport check to not report an error for "from y import x as x" since x is considered to be re-exported in this case. Previously, this case was exempted only for type stubs.

    Enhancement: Expand enums in negative narrowing of x in y.

    Bug Fix: Fixed bug that resulted in incorrect type evaluation when an f-string uses expressions that are unions of literal strings. The result should be LiteralString, not str.

    Enhancement: Improved the reportUnnecessaryComparison diagnostic check to detect (and properly report) cases where a comparison if literals is always true or always false.

    ... (truncated)

    Commits

    Dependabot compatibility score

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
    • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    dependencies javascript 
    opened by dependabot[bot] 1
  • chore(deps-dev): bump pyright from 1.1.276 to 1.1.278

    chore(deps-dev): bump pyright from 1.1.276 to 1.1.278

    Bumps pyright from 1.1.276 to 1.1.278.

    Release notes

    Sourced from pyright's releases.

    Published 1.1.278

    Behavior Change: Changed the reportUnusedImport check to not report an error for "from y import x as x" since x is considered to be re-exported in this case. Previously, this case was exempted only for type stubs.

    Enhancement: Expand enums in negative narrowing of x in y.

    Bug Fix: Fixed bug that resulted in incorrect type evaluation when an f-string uses expressions that are unions of literal strings. The result should be LiteralString, not str.

    Enhancement: Improved the reportUnnecessaryComparison diagnostic check to detect (and properly report) cases where a comparison if literals is always true or always false.

    Bug Fix: Fixed bug that resulted in incorrect type evaluation when creating a union from two generic functions that are parameterized by different ParamSpecs but are otherwise identical.

    Behavior Change: Updated the default Python version from 3.10 to 3.11 now that 3.11 has been released in its final form. Pyright assumes the default version if it is not otherwise told which version to use (explicitly as a setting or command-line argument or implicitly via the selected Python environment).

    Enhancement: Enhanced --verifytypes so it honors the --pythonversion and --pythonplatform command-line arguments and falls back to the selected Python environment to get these values if they are unspecified.

    Bug Fix: Fixed a bug in the type evaluator related to TypeVarTuple that caused it to incorrectly report a type incompatibility between *Ts and *tuple[*Ts]. These are the same type, so they should be compatible.

    Bug Fix: Fixed a bug that results in incorrect specialization of a function or method that uses a TypeVarTuple parameter. The internal flags were being lost including the tracking of class methods and static methods.

    Bug Fix: Added support for unpacked tuple arguments passed to an *args parameter declared with a TypeVarTuple.

    Published 1.1.277

    Enhancement: Added support for falsy type guard expressions when the evaluated type is a NamedTuples with at least one element. These always evaluate to True.

    Bug Fix: Fixed bug that resulted in a false positive "possibly unbound" error when a builtin symbol is overridden in the global scope within a conditional block.

    Enhancement: Improved error message for TypeVar constraints that contain generics. Fixed a bug that prevented an error message when keyword argument was duplicated in a call to TypeVar.

    Enhancement: Added support for infer_variance parameter to TypeVar as specified in draft PEP 695.

    Bug Fix: Improved consistency of assignment-based type narrowing when the type of the assigned value includes an Any or Unknown type argument that is nested more than one level deep and the expected (declared) value contains a type argument that is not Any.

    Enhancement: Updated typeshed stubs to the latest version.

    Bug Fix: Fixed bug that results in a false positive error when a protocol class implements an overloaded method.

    Bug Fix: Added a workaround for a recent change to the typeshed stub dataclasses.pyi relating to the InitVar class. For Python 3.8, the stub indicates that the InitVar class has a metaclass called _InitVarMeta that has a __getitem__ method that returns a value of type InitVar[Any], but this creates a circularity because the evaluation of the expression InitVar[Any] depends on the return type of _InitVarMeta.__getitem__.

    Commits

    Dependabot compatibility score

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
    • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    dependencies javascript 
    opened by dependabot[bot] 1
  • chore(deps-dev): bump pyright from 1.1.276 to 1.1.286

    chore(deps-dev): bump pyright from 1.1.276 to 1.1.286

    Bumps pyright from 1.1.276 to 1.1.286.

    Release notes

    Sourced from pyright's releases.

    Published 1.1.286

    Bug Fix: Reverted a recent update to the TOML parser that resulted in a regression. This reversion means that some TOML 1.0 features will not be handled correctly.

    Bug Fix: Fixed a bug that resulted in incorrect handling of literals in the TypeVar constraint solver. This involved a pretty significant change to the constraint solver logic — one that eliminated some heuristics and special cases.

    Bug Fix: Fixed a bug that caused target expressions within a chained assignment to be evaluated in the wrong order (right to left instead of left to right). This resulted in false positives and negatives in some cases where one target referred to another target. This change also makes it illegal to use a Python 2-style type comment on a line containing a chained assignment statement, reflecting the fact that Python 3-style variable type annotations are not legal here either.

    Enhancement: Improved handling of TypeVarTuple constraint solving. Previously, if a TypeVarTuple appeared more than once, the corresponding tuple types needed to be identical. The constraint solver now supports the same sort of narrowing/widening within the tuple entries to find the best solution.

    Bug Fix: Fixed a bug that led to a false negative during protocol matching if the protocol class refers to itself within an invariant type argument.

    Enhancement: Improved handling of generic functions passed as arguments to generic higher-order functions. Pyright is now able to solve the type variables for both the generic callback and the called function.

    Enhancement: Updated typeshed stubs to the latest version.

    Enhancement: Improved handling of generic functions passed as arguments to generic higher-order functions that use a ParamSpec. Pyright is now able to solve the type variables for both the generic callback and the called function.

    Bug Fix: Fixed a bug in the code flow engine that resulted in incorrect type evaluation in some cases involving double nested loops.

    Bug Fix: Improved the method override consistency checks to detect the case where an override uses an *args parameter that is not type compatible with the overridden method's parameter types. Thanks to @​mehdigmira for this contribution.

    Enhancement: Improved handling of TypeVars that appear only within a Callable within a return type annotation for a function. By a strict reading of PEP 484, these should be bound to the function's scope, but practically, they are bound to the Callable. This allows a function to return a generic callable type. When TypeVars are rescoped in this manner, the TypeVar cannot be referenced within the function body because it is no longer in scope in that context.

    Enhancement: Improved error handling for NewType calls

    Enhancement: Completed initial implementation of PEP 696. Added support for default TypeVar types that refer to other TypeVars.

    Published 1.1.285

    Enhancement: Implemented a new --level command-line option that allows filtering of 'information' and 'warning' diagnostics.

    Enhancement: Updated TOML parser to one that is compliant with the TOML 1.0 spec.

    Enhancement: Added logic to detect uses of PEP 604 | syntax that generate exceptions due to runtime limitations. In particular, if one of the operands is a string (i.e. a forward reference) and the other is also a string or a class that is not explicitly specialized, this will result in an exception.

    Bug Fix: Fixed recent regression in completion provider that resulted in garbled type information for a symbol that is declared as a function (using a def statement) but transformed into a non-function type using a decorator.

    Bug Fix: Fixed a bug that resulted in a false positive error when an index expression with a numeric literal subscript was used in a loop that included a del statement targeting the same index expression.

    Behavior Change: Modified the overload matching algorithm to match the behavior of mypy when the overload match is ambiguous because an argument evaluates to Any or Unknown. In this case, the call expression evaluates to Unknown. Previously, pyright used the first of the matching overloads in this case.

    Bug Fix: Fixed a bug that led to extremely long type analysis times when determining type compatibility between an recursive type alias and a recursive protocol.

    Bug Fix (contribution from @​parched): Fixed recent regression that caused reportImportCycles diagnostic reporting to no longer work.

    Bug Fix: Fixed a bug that resulted in a false positive error when a property setter or deleter contained function-scoped type variables.

    Published 1.1.284

    Bug Fix: Fixed a bug that resulted in an incorrect type evaluation when using a literal integer index into a tuple that includes an unpacked TypeVarTuple element.

    Behavior Change: Removed diagnostic check that detects a non-ellipsis default value in a stub file. This check was based on a now-outdated best practice. We now recommend that stubs include default values for better usability in language servers.

    ... (truncated)

    Commits

    Dependabot compatibility score

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
    • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    dependencies javascript 
    opened by dependabot[bot] 0
  • chore(deps-dev): bump isort from 5.10.1 to 5.11.4

    chore(deps-dev): bump isort from 5.10.1 to 5.11.4

    Bumps isort from 5.10.1 to 5.11.4.

    Release notes

    Sourced from isort's releases.

    5.11.4

    Changes

    :package: Dependencies

    5.11.3

    Changes

    :beetle: Fixes

    :construction_worker: Continuous Integration

    v5.11.3

    Changes

    :beetle: Fixes

    :construction_worker: Continuous Integration

    5.11.2

    Changes

    5.11.1

    Changes December 12 2022

    ... (truncated)

    Changelog

    Sourced from isort's changelog.

    5.11.4 December 21 2022

    5.11.3 December 16 2022

    5.11.2 December 12 2022

    5.11.1 December 12 2022

    5.11.0 December 12 2022

    Commits
    • 98390f5 Merge pull request #2059 from PyCQA/version/5.11.4
    • df69a05 Bump version 5.11.4
    • f9add58 Merge pull request #2058 from PyCQA/deps/poetry-1.3.1
    • 36caa91 Bump Poetry 1.3.1
    • 3c2e2d0 Merge pull request #1978 from mgorny/toml-test
    • 45d6abd Remove obsolete toml import from the test suite
    • 3020e0b Merge pull request #2057 from mgorny/poetry-install
    • a6fdbfd Stop installing documentation files to top-level site-packages
    • ff306f8 Fix tag template to match old standard
    • 227c4ae Merge pull request #2052 from hugovk/main
    • Additional commits viewable in compare view

    Dependabot compatibility score

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
    • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    dependencies python 
    opened by dependabot[bot] 0
  • chore(deps-dev): bump black from 22.10.0 to 22.12.0

    chore(deps-dev): bump black from 22.10.0 to 22.12.0

    Bumps black from 22.10.0 to 22.12.0.

    Release notes

    Sourced from black's releases.

    22.12.0

    Preview style

    • Enforce empty lines before classes and functions with sticky leading comments (#3302)
    • Reformat empty and whitespace-only files as either an empty file (if no newline is present) or as a single newline character (if a newline is present) (#3348)
    • Implicitly concatenated strings used as function args are now wrapped inside parentheses (#3307)
    • Correctly handle trailing commas that are inside a line's leading non-nested parens (#3370)

    Configuration

    • Fix incorrectly applied .gitignore rules by considering the .gitignore location and the relative path to the target file (#3338)
    • Fix incorrectly ignoring .gitignore presence when more than one source directory is specified (#3336)

    Parser

    • Parsing support has been added for walruses inside generator expression that are passed as function args (for example, any(match := my_re.match(text) for text in texts)) (#3327).

    Integrations

    • Vim plugin: Optionally allow using the system installation of Black via let g:black_use_virtualenv = 0(#3309)
    Changelog

    Sourced from black's changelog.

    22.12.0

    Preview style

    • Enforce empty lines before classes and functions with sticky leading comments (#3302)
    • Reformat empty and whitespace-only files as either an empty file (if no newline is present) or as a single newline character (if a newline is present) (#3348)
    • Implicitly concatenated strings used as function args are now wrapped inside parentheses (#3307)
    • Correctly handle trailing commas that are inside a line's leading non-nested parens (#3370)

    Configuration

    • Fix incorrectly applied .gitignore rules by considering the .gitignore location and the relative path to the target file (#3338)
    • Fix incorrectly ignoring .gitignore presence when more than one source directory is specified (#3336)

    Parser

    • Parsing support has been added for walruses inside generator expression that are passed as function args (for example, any(match := my_re.match(text) for text in texts)) (#3327).

    Integrations

    • Vim plugin: Optionally allow using the system installation of Black via let g:black_use_virtualenv = 0(#3309)
    Commits
    • 2ddea29 Prepare release 22.12.0 (#3413)
    • 5b1443a release: skip bad macos wheels for now (#3411)
    • 9ace064 Bump peter-evans/find-comment from 2.0.1 to 2.1.0 (#3404)
    • 19c5fe4 Fix CI with latest flake8-bugbear (#3412)
    • d4a8564 Bump sphinx-copybutton from 0.5.0 to 0.5.1 in /docs (#3390)
    • 2793249 Wordsmith current_style.md (#3383)
    • d97b789 Remove whitespaces of whitespace-only files (#3348)
    • c23a5c1 Clarify that Black runs with --safe by default (#3378)
    • 8091b25 Correctly handle trailing commas that are inside a line's leading non-nested ...
    • ffaaf48 Compare each .gitignore found with an appropiate relative path (#3338)
    • Additional commits viewable in compare view

    Dependabot compatibility score

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
    • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    dependencies python 
    opened by dependabot[bot] 0
  • expect_eof() when the eof token was already consumed fails with nonsense message

    expect_eof() when the eof token was already consumed fails with nonsense message

    Calling stream.expect_eof() when the eof token was already consumed raises the following exception:

    File ".../tokenstream/stream.py", line 858, in expect
        raise self.emit_error(UnexpectedEOF(patterns))
    tokenstream.error.UnexpectedEOF: Expected eof but reached end of file.
    

    I would expect that expect_eof() would succeed if there are no tokens remaining, or if the only remaining token is the eof token.

    This is my current workaround:

    token = stream.peek()
    if token and token.type != "eof":
        raise set_location(stream.emit_error(UnexpectedToken(token, ("eof",))), token)
    

    (The ("eof",) is a 1-tuple to make pylance happy, pylance doesn't like passing ["eof"] to a *args style parameter, and passing just "eof" ends up treating the string is invidual characters)

    opened by vdvman1 0
  • chore(deps-dev): bump python-semantic-release from 7.28.1 to 7.32.2

    chore(deps-dev): bump python-semantic-release from 7.28.1 to 7.32.2

    Bumps python-semantic-release from 7.28.1 to 7.32.2.

    Release notes

    Sourced from python-semantic-release's releases.

    v7.32.2

    Fix

    • Fix changelog generation in tag-mode (#171) (482a62e)

    Documentation

    v7.32.1

    Fix

    • Corrections for deprecation warnings (#505) (d47afb6)

    Documentation

    v7.32.0

    Feature

    • Add setting for enforcing textual changelog sections (#502) (988437d)

    Documentation

    • Correct documented default behaviour for commit_version_number (#497) (ffae2dc)

    v7.31.4

    Fix

    • Account for trailing newlines in commit messages (#495) (111b151)

    v7.31.3

    Fix

    • Use commit_subject when searching for release commits (#488) (3849ed9)

    v7.31.2

    Fix

    • Add better handling of missing changelog placeholder (e7a0e81)
    • Add repo=None when not in git repo (40be804)

    Documentation

    • Add example for pyproject.toml (2a4b8af)

    v7.31.1

    Fix

    • Update git email in action (0ece6f2)

    v7.31.0

    Feature

    • Override repository_url w REPOSITORY_URL env var (#439) (cb7578c)
    • Add prerelease-patch and no-prerelease-patch flags for whether to auto-bump prereleases (b4e5b62)

    Fix

    • :bug: fix get_current_release_version for tag_only version_source (cad09be)

    v7.30.2

    ... (truncated)

    Changelog

    Sourced from python-semantic-release's changelog.

    v7.32.2 (2022-10-22)

    Fix

    • Fix changelog generation in tag-mode (#171) (482a62e)

    Documentation

    v7.32.1 (2022-10-07)

    Fix

    • Corrections for deprecation warnings (#505) (d47afb6)

    Documentation

    v7.32.0 (2022-09-25)

    Feature

    • Add setting for enforcing textual changelog sections (#502) (988437d)

    Documentation

    • Correct documented default behaviour for commit_version_number (#497) (ffae2dc)

    v7.31.4 (2022-08-23)

    Fix

    • Account for trailing newlines in commit messages (#495) (111b151)

    v7.31.3 (2022-08-22)

    Fix

    • Use commit_subject when searching for release commits (#488) (3849ed9)

    v7.31.2 (2022-07-29)

    Fix

    • Add better handling of missing changelog placeholder (e7a0e81)
    • Add repo=None when not in git repo (40be804)

    Documentation

    • Add example for pyproject.toml (2a4b8af)

    v7.31.1 (2022-07-29)

    Fix

    • Update git email in action (0ece6f2)

    v7.31.0 (2022-07-29)

    Feature

    • Override repository_url w REPOSITORY_URL env var (#439) (cb7578c)
    • Add prerelease-patch and no-prerelease-patch flags for whether to auto-bump prereleases (b4e5b62)

    Fix

    • :bug: fix get_current_release_version for tag_only version_source (cad09be)

    v7.30.2 (2022-07-26)

    ... (truncated)

    Commits

    Dependabot compatibility score

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
    • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    dependencies python 
    opened by dependabot[bot] 0
  • chore(deps-dev): bump mudkip from 0.7.0 to 0.8.0

    chore(deps-dev): bump mudkip from 0.7.0 to 0.8.0

    Bumps mudkip from 0.7.0 to 0.8.0.

    Release notes

    Sourced from mudkip's releases.

    v0.8.0

    Feature

    Changelog

    Sourced from mudkip's changelog.

    v0.8.0 (2022-08-09)

    Feature

    Commits

    Dependabot compatibility score

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
    • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    dependencies python 
    opened by dependabot[bot] 0
Releases(v1.4.2)
Owner
Valentin Berlier
French guy studying computer science. I build things for fun, mostly to scratch my own itch.
Valentin Berlier
Implementation of Token Shift GPT - An autoregressive model that solely relies on shifting the sequence space for mixing

Token Shift GPT Implementation of Token Shift GPT - An autoregressive model that relies solely on shifting along the sequence dimension and feedforwar

Phil Wang 32 Oct 14, 2022
Connectionist Temporal Classification (CTC) decoding algorithms: best path, beam search, lexicon search, prefix search, and token passing. Implemented in Python.

CTC Decoding Algorithms Update 2021: installable Python package Python implementation of some common Connectionist Temporal Classification (CTC) decod

Harald Scheidl 736 Jan 3, 2023
The ibet-Prime security token management system for ibet network.

ibet-Prime The ibet-Prime security token management system for ibet network. Features ibet-Prime is an API service that enables the issuance and manag

BOOSTRY 8 Dec 22, 2022
TaCL: Improve BERT Pre-training with Token-aware Contrastive Learning

TaCL: Improve BERT Pre-training with Token-aware Contrastive Learning

Yixuan Su 26 Oct 17, 2022
Long text token classification using LongFormer

Long text token classification using LongFormer

abhishek thakur 161 Aug 7, 2022
leaking paid token generator that was a shit lmao for 100$ haha

Discord-Token-Generator-Leaked leaking paid token generator that was a shit lmao for 100$ he selling it for 100$ wth here the code enjoy don't forget

Keevo 5 Apr 15, 2022
Python code for ICLR 2022 spotlight paper EViT: Expediting Vision Transformers via Token Reorganizations

Expediting Vision Transformers via Token Reorganizations This repository contain

Youwei Liang 101 Dec 26, 2022
Code for the Findings of NAACL 2022(Long Paper): AdapterBias: Parameter-efficient Token-dependent Representation Shift for Adapters in NLP Tasks

AdapterBias: Parameter-efficient Token-dependent Representation Shift for Adapters in NLP Tasks arXiv link: upcoming To be published in Findings of NA

Allen 16 Nov 12, 2022
Beyond Masking: Demystifying Token-Based Pre-Training for Vision Transformers

beyond masking Beyond Masking: Demystifying Token-Based Pre-Training for Vision Transformers The code is coming Figure 1: Pipeline of token-based pre-

Yunjie Tian 23 Sep 27, 2022
Discord-Token-Formatter - A simple script to convert discord tokens from email token to token only format

Discord-Token-Formatter A simple script to convert discord tokens from email:pas

null 2 Oct 23, 2022
Python library for creating PEG parsers

PyParsing -- A Python Parsing Module Introduction The pyparsing module is an alternative approach to creating and executing simple grammars, vs. the t

Pyparsing 1.7k Dec 27, 2022
Python library for creating PEG parsers

PyParsing -- A Python Parsing Module Introduction The pyparsing module is an alternative approach to creating and executing simple grammars, vs. the t

Pyparsing 1.7k Jan 3, 2023
A collection of convenient parsers for Advent of Code problems.

Advent of Code Parsers A collection of convenient Python parsers for Advent of Code problems. Installation pip install aocp Quickstart You can import

Miguel Blanco Marcos 3 Dec 13, 2021
Modeval (or Modular Eval) is a modular and secure string evaluation library that can be used to create custom parsers or interpreters.

modeval Modeval (or Modular Eval) is a modular and secure string evaluation library that can be used to create custom parsers or interpreters. Basic U

null 2 Jan 1, 2022
Modeval (or Modular Eval) is a modular and secure string evaluation library that can be used to create custom parsers or interpreters.

modeval Modeval (or Modular Eval) is a modular and secure string evaluation library that can be used to create custom parsers or interpreters. Basic U

null 2 Jan 1, 2022
Stream Framework is a Python library, which allows you to build news feed, activity streams and notification systems using Cassandra and/or Redis. The authors of Stream-Framework also provide a cloud service for feed technology:

Stream Framework Activity Streams & Newsfeeds Stream Framework is a Python library which allows you to build activity streams & newsfeeds using Cassan

Thierry Schellenbach 4.7k Jan 2, 2023
📢 Video Chat Stream Telegram Bot. Can ⏳ Stream Live Videos, Radios, YouTube Videos & Telegram Video Files On Your Video Chat Of Channels & Groups !

Telegram Video Chat Bot (Beta) ?? Video Chat Stream Telegram Bot ?? Can Stream Live Videos, Radios, YouTube Videos & Telegram Video Files On Your Vide

brut✘⁶⁹ // ユスフ 15 Dec 24, 2022
A Python 3 script for capturing and recording a SDR stream to a WAV file (or serving it to a HTTP audio stream).

rfsoapyfile A Python 3 script for capturing and recording a SDR stream to a WAV file (or serving it to a HTTP audio stream). The script is threaded fo

null 4 Dec 19, 2022
Nyon-stream - A python script that uses webtorrent to stream nyaa videos directly to mpv

nyon-stream A rather shitty script that uses webtorrent to stream nyaa videos di

null 18 Feb 8, 2022