Description

This PR will add support for the python 3.10 match statement to mypy.

Current progress:

• [x] parser and ast nodes
• [x] semantic analysis
• [x] type-checking
  • [x] as-pattern
  • [x] or-pattern
  • [x] literal-pattern
  • [x] value-pattern
  • [x] capture-pattern
  • [x] wildcard-pattern
  • [x] sequence-pattern
  • [x] mapping-pattern
  • [x] class-pattern
• [x] Lots of tests

Test Plan

Some tests have been added, but they don't pass yet. There will be a lot more tests in the future.

I'm currently developing and testing on cpython 3.10.0b1 and will make sure to update to newer versions every now and then.

PEP518 says that pyproject.toml is to support the minimum build requirements. I want to consider mypy as part of the minimum build on my package.

For example Pip uses a [tool.towncrier] section on pyproject.toml to generate a correctly updated NEWS.rst as part of its build. One can considers typing correctness as important to the distributed source pakcage as an well documented NEWS.rst. Other example is Black that also has it section on pyproject.toml.

Maybe you think that this is a silly feature request and that setup.cfg must be used instead, Pip or Flit or whatever can setup mypy after all. But an TOML module will be added to the standard library sooner or later, than could mypy accept this feature and antecipate TOML supporting pyproject.toml today?

I will be releasing mypy 0.920 soon. The release branch will be cut at a7d6e68541e1c6b52338dbe2e3f4a7055a033483. Please post here any PRs that need to be merged and cherry-picked.

This is an implementation of protocols proposed in PEP 544. This PR adds support for:

• generic protocols (including inference for protocols)
• recursive protocols
• special support for Type[] and isinstance()
• structural subtyping for Callable and Tuple
• other things

For example, now one can do this:

class Linked(Protocol[T]):
    val: T
    def next(self) -> 'Linked[T]': ...

class L:
    val: int
    def next(self) -> 'L': ...

def last(seq: Linked[T]) -> T:
    ...

reveal_type(last(L()))  # Revealed type is 'builtins.int*'

In general, the PR is practically feature complete, the main missing thing now is structural join/meet for partially overlapping protocols. For example, sometimes an explicit annotation is needed:

class A(Protocol):
    one: int
    def __len__(self) -> int: ...

class B(Protocol):
    another: int
    def __len__(self) -> int: ...

a: A
b: B
reveal_type([a, b])  # Revealed type is 'builtins.list[builtins.object*]'

y: List[Sized] = [a, b]  # This works OK

The reason for this is that I decided to re-use Instances for protocols, and real structural join/meet would require creating a "fake" TypeInfo to describe the resulting protocol. However, I have encountered problems with serialization/de-serialization of "fake" TypeInfos and postponed this feature (as proposed by Jukka). On the other hand, reusing Instances for protocols allowed to implement this in a very simple way. The PR is very small and 2/3 of it are extensive tests.

There is something I encountered while working on this PR. It looks like structural subtyping is significantly slower than nominal. This is why I also implement "subtype cache" for Instances, so that there is a net win in speed of few percent (even when I locally updated typeshed replacing all ABC in typing with protocols).

In terms of work-flow, I would be happy if people will try this PR and give also feedback on behaviour, not just a code review. I will soon submit a PR to typeshed with updates for typing stubs, and a PR with runtime implementation. As well, a PR to PEPs repo with an updated draft that takes into account comments that appeared so far.

@JukkaL @ambv @gvanrossum

Description

I was planning on adding a ParamSpec-based interface to something I'm making but... it turns out I need Concatenate and Z[[int]] syntax.

Closes #11833 Closes #12276 (needs test) Closes #12257 (needs test; not sure how to test cache issues) Refs #8645 External ref https://github.com/python/typeshed/issues/4827

This PR adds a new Parameters proper type to represent parameters (more later in this description), along with adding a Concatenate operator.

Test Plan

I copied all (seemingly relevant) tests from PEP 612. I plan on copying test cases from issues and thinking of a few of my own later.

Notes

(not a real section but seemed important enough)

• Concatenate support does not use a ProperType. That would be too much new code for an operator, in my opinion. Thus, it prepends to a ParamSpecType's .prefix.
• I would really really like CallableType to take in a parameters: ParamSpecType | Parameters -- this would simplify some code, and it fits better in your head. However, this would be a breaking change, so I did not do this (along with the fact that making basic changes in this direction led to 300 type errors when self-checking). ~~- Concatenate was super unspecified by the PEP in my opinion. I ended up making it work wherever ParamSpec could be (or so I think)... Which means Concatenate[int, Concatenate[str, P]] works!~~
• Error messages could be improved, but I'm not sure where I can get the context to improve those.

I tried mypy and was alarmed to see it unable to find imports that are clearly available. Reading the explanations in #1293 and #1339, I now understand why it doesn't merrily parse third-party code that will only add to a growing pile of unannotated code.

But this is a really bad first experience. I install something that claims to understand Python code, and it's confused by imports? Then it gives me two suggestions, neither of which is seems like the right advice:

note: (Perhaps setting MYPYPATH or using the "--ignore-missing-imports" flag would help)

I shouldn't try to set MYPYPATH, that's silly busy-work that if successful will only lead me into the unannotated third-party module trap. The imports aren't missing, so why would --ignore-missing-imports help? NOTE: using --ignore-missing-imports silences the warnings, but the option is misnamed: the imports are not missing.

Reading the help, I see --follow-imports, so I try those options. I try --follow-imports=silent, but that still complains about the imports. I try --follow-imports=skip, and that also still complains about the imports. Both of these sound like exactly what I need, and neither changes the behavior much:

$ mypy --version
mypy 0.560
$ mypy census.py
census.py:16: error: Cannot find module named 'aiohttp'
census.py:16: note: (Perhaps setting MYPYPATH or using the "--ignore-missing-imports" flag would help)
census.py:17: error: Cannot find module named 'async_timeout'
census.py:18: error: Cannot find module named 'attr'
census.py:20: error: Cannot find module named 'opaque_keys'
census.py:21: error: Cannot find module named 'opaque_keys.edx'
census.py:21: error: Cannot find module named 'opaque_keys.edx.keys'
helpers.py:3: error: No library stub file for module 'lxml'
helpers.py:3: note: (Stub files are from https://github.com/python/typeshed)
helpers.py:4: error: No library stub file for module 'lxml.html'
$ mypy --follow-imports=silent census.py
census.py:16: error: Cannot find module named 'aiohttp'
census.py:16: note: (Perhaps setting MYPYPATH or using the "--ignore-missing-imports" flag would help)
census.py:17: error: Cannot find module named 'async_timeout'
census.py:18: error: Cannot find module named 'attr'
census.py:20: error: Cannot find module named 'opaque_keys'
census.py:21: error: Cannot find module named 'opaque_keys.edx'
census.py:21: error: Cannot find module named 'opaque_keys.edx.keys'
$ mypy --follow-imports=skip census.py
census.py:16: error: Cannot find module named 'aiohttp'
census.py:16: note: (Perhaps setting MYPYPATH or using the "--ignore-missing-imports" flag would help)
census.py:17: error: Cannot find module named 'async_timeout'
census.py:18: error: Cannot find module named 'attr'
census.py:20: error: Cannot find module named 'opaque_keys'
census.py:21: error: Cannot find module named 'opaque_keys.edx'
census.py:21: error: Cannot find module named 'opaque_keys.edx.keys'
$ mypy --follow-imports=error census.py
census.py:16: error: Cannot find module named 'aiohttp'
census.py:16: note: (Perhaps setting MYPYPATH or using the "--ignore-missing-imports" flag would help)
census.py:17: error: Cannot find module named 'async_timeout'
census.py:18: error: Cannot find module named 'attr'
census.py:20: error: Cannot find module named 'opaque_keys'
census.py:21: error: Cannot find module named 'opaque_keys.edx'
census.py:21: error: Cannot find module named 'opaque_keys.edx.keys'
census.py:23: note: Import of 'html_writer' ignored
census.py:23: note: (Using --follow-imports=error, module not passed on command line)
census.py:24: note: Import of 'site_patterns' ignored
census.py:27: note: Import of 'sites' ignored
$ mypy --ignore-missing-imports census.py
$

Problem statement:

"Could there be some way to write a library like numpy so that a single codebase could simultaneously target CPython and the newer compilers, while achieving competitive speed in all cases? If so, what would it take to make that happen? If not, then what’s the next-best alternative?"**

Proposed Solution: a "PyIR" that can be consumed by various compilers. Details here: https://docs.google.com/document/d/1jGksgI96LdYQODa9Fca7EttFEGQfNODphVmbCX0DD1k/edit

Question for Mypy: A cython subset seems to be the recommended source format. Can a "strict mode" mypy be used instead to output this IR? Advantages include more expressive than cython (generics etc), bootstrap off mypy work and less fragmentation.

Excerpts from discussion on gitter:

@njsmith

at pycon this year Jukka and the mypy team were very interested in the idea of somehow using their static type stuff in (something like, this doesn't exist) "strict" mode to help with this

@kmod

I think the issue is that "programmer productivity types" fundamentally != "compiler-userful types"

For example, are subclasses subtypes?

If you pick either yes or no, then the type system becomes non-useful to one community or the other

bonus: Where would dynd's datashape play in? If so, can Dynd's datashape be used as a mypy plugin to annotate array types? @insertinterestingnamehere

[This is based on a discussion between me and @gvanrossum last week. I'm writing this down so that we won't forget about this.]

Several stdlib functions have signatures where the return type depends on the value of an argument. Examples:

• open returns an IO[str] or IO[bytes] depending on whether the mode argument has 'b' as a substring.
• subprocess.check_output returns a str or bytes depending on the value of the boolean universal_newlines argument.

A simple way to support these would be to introduce a few additional types that work with str and bool literals:

• A string pattern type that can describe the contents of a string literal, perhaps using a regular expression.
• Types for False and True.

For example, if we had FalseType and TrueType, we could write a function whose return value depends on the value of a boolean argument:

@overload
def f(x: FalseType) -> bytes: ...
@overload
def f(x: TrueType) -> str: ...

def f(x: Any) -> Any:
    if x:
        return 'x'
    else:
        return b'x'

reveal_type(f(False))  # bytes
reveal_type(f(True)  # str

Not sure about this one:

...
x = bool('')
reveal_type(f(x))  # Union[str, bytes] since we don't know the boolean value statically?

We could also have a generic class where a generic type argument depends on the value of an argument to __init__. Not sure how we should represent these -- here is one potential approach:

class C(Generic[AnyStr]):
    @overload
    @returntype('C[str]')
    def __init__(self, x: TrueType) -> None: ...
    @overload
    @returntype('C[bytes]')
    def __init__(self, x: FalseType) -> None: ...
    ...

reveal_type(C(True))  # C[str]

Alternatively, we could use __new__, but this would be problematic if there is no corresponding method defined at runtime.

Consider this example:

from typing import Callable

class A:
    f = None  # type: Callable[[int], int]

    def __init__(self, g: Callable[[int], int]) -> None:
        self.f = g

mypy reports:

trial.py: In member "__init__" of class "A":
trial.py, line 7: Cannot assign to a method
trial.py, line 7: Invalid method type
trial.py, line 7: Incompatible types in assignment (expression has type Callable[[int], int], variable has type Callable[[], int])

while I'd expect this to work.

Note that there is a second error in the reporting of the type of the variable.

Also, this passes type checking (correctly):

f = None  # type: Callable[[int], int]

def g(x: int) -> int: return 0

f = g

This is an attempt to implement the descriptor protocol (as per https://github.com/python/mypy/issues/244).

However, it currently is suffering from being being unable to resolve type variables during assignment, and I'm not sure what I would need to resolve that. For instance, using this implementation of properties:

from typing import Any, Text, TypeVar, Optional, Type, Generic, Callable, Union
from operator import attrgetter

T = TypeVar('T')
G = TypeVar('G')
S = TypeVar('S')

class MyProperty(Generic[T, G]):
    def __init__(
        self,
        fget: Callable[[T], G],
        fset: Optional[Callable[[T, G], None]] = None,
        fdel: Optional[Callable[[T], None]] = None,
        doc: Optional[Text] = None,
    ) -> None:
        self._getter = fget
        self._setter = fset
        self._deleter = fdel
        if doc:
            self.__doc__ = doc
        else:
            self.__doc__ = fget.__doc__

    def setter(self, fset: Callable[[T, G], None]):
        return MyProperty(self._getter, fset, self._deleter, self.__doc__)

    def deleter(self, fdel: Callable[[T], None]):
        return MyProperty(self._getter, self._setter, fdel, self.__doc__)

    def __get__(self, instance: Optional[T], owner: Type[T]) -> Union['MyProperty[T, G]', G]:
        if instance is None:
            return self

        return self._getter(instance)

    def __set__(self, instance: T, value: G) -> None:
        if self._setter:
            self._setter(instance, value)
        else:
            raise Exception()

    def __delete__(self, instance: T) -> None:
        if self._deleter:
            self._deleter(instance)
        else:
            raise Exception()

class NonGenericNonDataDescriptor(Generic[T]):
    def __init__(self, value: int) -> None:
        self._value = value  # type: int

    def __get__(self, instance: Optional[T], owner: Type[T]) -> int:
        return self._value

class NonGenericDataDescriptor(Generic[T]):
    def __init__(self, value: int) -> None:
        self._value = value  # type: int

    def __get__(self, instance: Optional[T], owner: Type[T]) -> Union['NonGenericDataDescriptor[T]', int]:
        if instance is None:
            return self

        return self._value

    def __set__(self, instance: T, value: int) -> None:
        self._value = value


class Test:
    my_unique_name = MyProperty(attrgetter('foo'))  # type: MyProperty[Test, int]
    ten = NonGenericNonDataDescriptor(10)  # type: NonGenericNonDataDescriptor[Test]
    some_int = NonGenericDataDescriptor(5)  # type: NonGenericDataDescriptor[Test]

    @MyProperty
    def my_other_name(self):
        return self.bar

    def __init__(self) -> None:
        self.foo = 0
        self.bar = "ten"

def f() -> None:
    t = Test()

    t.my_unique_name = 1
    t.my_unique_name + 1

    t.ten + 1

    t.some_int = 7
    t.some_int + 1

When checked with mypy, we get the following errors:

$ mypy --fast-parser test_descriptors.py --show-traceback

test_descriptors.py: note: In function "f":
test_descriptors.py:85: error: Incompatible types in assignment (expression has type "int", variable has type "G")
test_descriptors.py:86: error: Unsupported operand types for + ("Union[MyProperty[Test, int], int]" and "int")

The second error is because we don't have overloading outside of stub files (see https://github.com/python/mypy/issues/1136), I think, but the former is the one I'm not sure how to resolve.

There are some situations where a user wants to have more precisely typed **kwds. Current syntax only allows homogeneous **kwds:

def fun(x: int, *, **options: str) -> None:
    ...

However, in situations with many heterogeneous options listing all options in the signature could be verbose and will require rewriting some existing code. There is a vague idea to allow TypedDict for such situations. For example:

class Options(TypedDict):
    timeout: int
    alternative: str
    on_error: Callable[[int], None]
    on_timeout: Callable[[], None]
    ...

def fun(x: int, *, **options: Options) -> None:
    ...

Maybe for such cases the TypedDict used should be automatically understood as defined with total=False. Also it is worth mentioning that this feature will allow reusing the TypedDicts in modules where several functions have same (or similar) option sets.

Bug Report

I'm not sure what's going on, better check the example. Also mypy experts, please forgive me if this is intentional behavior and not a bug.

To Reproduce

import typing

class A: pass
class B: pass

T = typing.TypeVar('T', A, B)

class C(typing.Generic[T]):

    def method_A(self, argument: T) -> None: pass

    def method_C(self, argument: T) -> typing.Self:
        self.method_A(argument)
        return self

Actual Behavior

main.py:13: error: Argument 1 to "method_A" of "C" has incompatible type "A"; expected "T"  [arg-type]
main.py:13: error: Argument 1 to "method_A" of "C" has incompatible type "B"; expected "T"  [arg-type]
Found 2 errors in 1 file (checked 1 source file)

Your Environment

• Mypy version used: mypy 1.0.0+dev.9183b28401bd2928d921a068bbbc7e6565e77649 (compiled: no)
• Mypy command-line flags: none required to reproduce
• Mypy configuration options from mypy.ini (and other config files): none required to reproduce
• Python version used: 3.11

Bug Report

With pandas-stubs, we are using pyright and mypy. We have code that we want to test with pyright, but not mypy because of a mypy bug. We are using the always-true option, but the issue here can be shown using TYPE_CHECKING.

If you include the flag --warn-unused-ignores, then the warnings will occur on code that should not be checked if looking at constants like TYPE_CHECKING or those set by always-true.

To Reproduce

from typing import TYPE_CHECKING

if not TYPE_CHECKING:
    x: int = "abc" # type: ignore[assignment]                                                     

Using mypy --no-incremental --warn-unused-ignores ignoretst.py, you get the error:

ignoretst.py:4: error: Unused "type: ignore" comment

Expected Behavior

No warning reported.

Actual Behavior

See above - even though the code should NOT be type checked, you are getting a warning.

Your Environment

• Mypy version used: 0.991
• Mypy command-line flags: --no-incremental --warn-unused-ignores
• Mypy configuration options from mypy.ini (and other config files): None
• Python version used: 3.9

Comment by @huguesb in #14277:

The repeat checkout/compilation cost is a real problem for repeated measurements when iterating through possible approaches though. At the very least you'd want to keep the reference binary around for as long as possible instead of recompiling it every time.

We could cache build results. If we detect a previous build of the same target commit, reuse it instead of recompiling.

We automatically track changes in mypy performance over time (#14187). Currently we can detect changes of at least 1.5% pretty reliably, but smaller changes are hard to detect. #14187 has some relevant discussion, such as this comment: https://github.com/python/mypy/issues/14187#issuecomment-1362312576

I'd estimate that a cumulative performance regression of around 15% in 2022 was due to changes that were below the 1.5% noise floor. Getting the detection threshold down to 0.5% or below could be quite helpful in finding and fixing regressions.

I looked at individual measurements, and it seems possible that measurements slowly fluctuate over time. I'm not entirely sure what might be causing this. Just increasing the number of iterations we measure probably won't help much, since different batches of runs will cluster around different averages.

Here are some things that could help:

1. Interleave executions of current/previous builds and measure the delta. Instead of only collecting absolute performance values, interleave runs using the previous commit and the target commit and calculate the average delta. If performance gradually fluctuates over time, this should help.
2. Further tweak the configuration of the runner machine for stability. See https://github.com/scala/scala-dev/issues/338 as suggested by @A5rocks.
3. Collect samples over a long period of time (say, 1 sample every hour over 12 hours).
4. Collect detailed profiling data for each commit and also highlight differences in the time spent in different parts of the mypy implementation. If a single function gets 2x slower, it could be easy to detect this way, even if the change in overall performance is well below the noise floor. This could be quite noisy due to renaming/splitting functions, etc.

I'm going to start by investigating if the idea 1 seems feasible.

Crash Report

If .mypy_cache doesn't exists mypy . works, but every next run when .mypy_cache exists give me AttributeError. Master branch has with problem.

Traceback

mypy_reproduce/__init__.py:31: error: Argument 2 to "Use" has incompatible type "List[str]"; expected "SupportsLenAndGetItem[_T]"  [arg-type]
./mypy_reproduce/__init__.py:37: error: INTERNAL ERROR -- Please try using mypy master on GitHub:
https://mypy.readthedocs.io/en/stable/common_issues.html#using-a-development-mypy-build
Please report a bug at https://github.com/python/mypy/issues
version: 0.991
Traceback (most recent call last):
  File "mypy/checkexpr.py", line 4665, in accept
  File "mypy/nodes.py", line 1841, in accept
  File "mypy/checkexpr.py", line 410, in visit_call_expr
  File "mypy/checkexpr.py", line 530, in visit_call_expr_inner
  File "mypy/checkexpr.py", line 1181, in check_call_expr_with_callee_type
  File "mypy/checkexpr.py", line 1264, in check_call
  File "mypy/checkexpr.py", line 1406, in check_callable_call
  File "mypy/checkexpr.py", line 1731, in infer_function_type_arguments
  File "mypy/checkexpr.py", line 1788, in infer_function_type_arguments_pass2
  File "mypy/infer.py", line 54, in infer_function_type_arguments
  File "mypy/constraints.py", line 147, in infer_constraints_for_callable
  File "mypy/constraints.py", line 190, in infer_constraints
  File "mypy/constraints.py", line 269, in _infer_constraints
  File "mypy/types.py", line 1792, in accept
  File "mypy/constraints.py", line 982, in visit_callable_type
  File "mypy/constraints.py", line 1006, in infer_against_overloaded
  File "mypy/constraints.py", line 1140, in find_matching_overload_item
  File "mypy/subtypes.py", line 1376, in is_callable_compatible
  File "mypy/subtypes.py", line 1655, in unify_generic_callable
  File "mypy/constraints.py", line 187, in infer_constraints
  File "mypy/constraints.py", line 269, in _infer_constraints
  File "mypy/types.py", line 1283, in accept
  File "mypy/constraints.py", line 865, in visit_instance
  File "mypy/constraints.py", line 187, in infer_constraints
  File "mypy/constraints.py", line 269, in _infer_constraints
  File "mypy/types.py", line 1283, in accept
  File "mypy/constraints.py", line 702, in visit_instance
  File "mypy/nodes.py", line 2991, in has_base
AttributeError: attribute 'mro' of 'TypeInfo' undefined
./mypy_reproduce/__init__.py:37: : note: use --pdb to drop into pdb

mypy_reproduce/__init__.py:31: error: Argument 2 to "Use" has incompatible type "List[str]"; expected "SupportsLenAndGetItem[_T]"  [arg-type]
./mypy_reproduce/__init__.py:37: error: INTERNAL ERROR -- Please try using mypy master on GitHub:
https://mypy.readthedocs.io/en/stable/common_issues.html#using-a-development-mypy-build
Please report a bug at https://github.com/python/mypy/issues
version: 1.0.0+dev.632304f90923ba38483d94de3cf58d486966f4f0
Traceback (most recent call last):
  File "/var/cache/kron-cache/pypoetry/virtualenvs/mypy-reproduce-vVFCMVnz-py3.10/bin/mypy", line 8, in <module>
    sys.exit(console_entry())
  File "/var/cache/kron-cache/pypoetry/virtualenvs/mypy-reproduce-vVFCMVnz-py3.10/lib/python3.10/site-packages/mypy/__main__.py", line 15, in console_entry
    main()
  File "/var/cache/kron-cache/pypoetry/virtualenvs/mypy-reproduce-vVFCMVnz-py3.10/lib/python3.10/site-packages/mypy/main.py", line 95, in main
    res, messages, blockers = run_build(sources, options, fscache, t0, stdout, stderr)
  File "/var/cache/kron-cache/pypoetry/virtualenvs/mypy-reproduce-vVFCMVnz-py3.10/lib/python3.10/site-packages/mypy/main.py", line 174, in run_build
    res = build.build(sources, options, None, flush_errors, fscache, stdout, stderr)
  File "/var/cache/kron-cache/pypoetry/virtualenvs/mypy-reproduce-vVFCMVnz-py3.10/lib/python3.10/site-packages/mypy/build.py", line 194, in build
    result = _build(
  File "/var/cache/kron-cache/pypoetry/virtualenvs/mypy-reproduce-vVFCMVnz-py3.10/lib/python3.10/site-packages/mypy/build.py", line 277, in _build
    graph = dispatch(sources, manager, stdout)
  File "/var/cache/kron-cache/pypoetry/virtualenvs/mypy-reproduce-vVFCMVnz-py3.10/lib/python3.10/site-packages/mypy/build.py", line 2920, in dispatch
    process_graph(graph, manager)
  File "/var/cache/kron-cache/pypoetry/virtualenvs/mypy-reproduce-vVFCMVnz-py3.10/lib/python3.10/site-packages/mypy/build.py", line 3317, in process_graph
    process_stale_scc(graph, scc, manager)
  File "/var/cache/kron-cache/pypoetry/virtualenvs/mypy-reproduce-vVFCMVnz-py3.10/lib/python3.10/site-packages/mypy/build.py", line 3418, in process_stale_scc
    graph[id].type_check_first_pass()
  File "/var/cache/kron-cache/pypoetry/virtualenvs/mypy-reproduce-vVFCMVnz-py3.10/lib/python3.10/site-packages/mypy/build.py", line 2317, in type_check_first_pass
    self.type_checker().check_first_pass()
  File "/var/cache/kron-cache/pypoetry/virtualenvs/mypy-reproduce-vVFCMVnz-py3.10/lib/python3.10/site-packages/mypy/checker.py", line 472, in check_first_pass
    self.accept(d)
  File "/var/cache/kron-cache/pypoetry/virtualenvs/mypy-reproduce-vVFCMVnz-py3.10/lib/python3.10/site-packages/mypy/checker.py", line 580, in accept
    stmt.accept(self)
  File "/var/cache/kron-cache/pypoetry/virtualenvs/mypy-reproduce-vVFCMVnz-py3.10/lib/python3.10/site-packages/mypy/nodes.py", line 1121, in accept
    return visitor.visit_class_def(self)
  File "/var/cache/kron-cache/pypoetry/virtualenvs/mypy-reproduce-vVFCMVnz-py3.10/lib/python3.10/site-packages/mypy/checker.py", line 2154, in visit_class_def
    self.accept(defn.defs)
  File "/var/cache/kron-cache/pypoetry/virtualenvs/mypy-reproduce-vVFCMVnz-py3.10/lib/python3.10/site-packages/mypy/checker.py", line 580, in accept
    stmt.accept(self)
  File "/var/cache/kron-cache/pypoetry/virtualenvs/mypy-reproduce-vVFCMVnz-py3.10/lib/python3.10/site-packages/mypy/nodes.py", line 1202, in accept
    return visitor.visit_block(self)
  File "/var/cache/kron-cache/pypoetry/virtualenvs/mypy-reproduce-vVFCMVnz-py3.10/lib/python3.10/site-packages/mypy/checker.py", line 2593, in visit_block
    self.accept(s)
  File "/var/cache/kron-cache/pypoetry/virtualenvs/mypy-reproduce-vVFCMVnz-py3.10/lib/python3.10/site-packages/mypy/checker.py", line 580, in accept
    stmt.accept(self)
  File "/var/cache/kron-cache/pypoetry/virtualenvs/mypy-reproduce-vVFCMVnz-py3.10/lib/python3.10/site-packages/mypy/nodes.py", line 1289, in accept
    return visitor.visit_assignment_stmt(self)
  File "/var/cache/kron-cache/pypoetry/virtualenvs/mypy-reproduce-vVFCMVnz-py3.10/lib/python3.10/site-packages/mypy/checker.py", line 2641, in visit_assignment_stmt
    self.check_assignment(s.lvalues[-1], s.rvalue, s.type is None, s.new_syntax)
  File "/var/cache/kron-cache/pypoetry/virtualenvs/mypy-reproduce-vVFCMVnz-py3.10/lib/python3.10/site-packages/mypy/checker.py", line 2845, in check_assignment
    rvalue_type = self.expr_checker.accept(rvalue, type_context=type_context)
  File "/var/cache/kron-cache/pypoetry/virtualenvs/mypy-reproduce-vVFCMVnz-py3.10/lib/python3.10/site-packages/mypy/checkexpr.py", line 4777, in accept
    typ = node.accept(self)
  File "/var/cache/kron-cache/pypoetry/virtualenvs/mypy-reproduce-vVFCMVnz-py3.10/lib/python3.10/site-packages/mypy/nodes.py", line 1871, in accept
    return visitor.visit_call_expr(self)
  File "/var/cache/kron-cache/pypoetry/virtualenvs/mypy-reproduce-vVFCMVnz-py3.10/lib/python3.10/site-packages/mypy/checkexpr.py", line 424, in visit_call_expr
    return self.visit_call_expr_inner(e, allow_none_return=allow_none_return)
  File "/var/cache/kron-cache/pypoetry/virtualenvs/mypy-reproduce-vVFCMVnz-py3.10/lib/python3.10/site-packages/mypy/checkexpr.py", line 544, in visit_call_expr_inner
    ret_type = self.check_call_expr_with_callee_type(
  File "/var/cache/kron-cache/pypoetry/virtualenvs/mypy-reproduce-vVFCMVnz-py3.10/lib/python3.10/site-packages/mypy/checkexpr.py", line 1195, in check_call_expr_with_callee_type
    ret_type, callee_type = self.check_call(
  File "/var/cache/kron-cache/pypoetry/virtualenvs/mypy-reproduce-vVFCMVnz-py3.10/lib/python3.10/site-packages/mypy/checkexpr.py", line 1278, in check_call
    return self.check_callable_call(
  File "/var/cache/kron-cache/pypoetry/virtualenvs/mypy-reproduce-vVFCMVnz-py3.10/lib/python3.10/site-packages/mypy/checkexpr.py", line 1431, in check_callable_call
    callee = self.infer_function_type_arguments(
  File "/var/cache/kron-cache/pypoetry/virtualenvs/mypy-reproduce-vVFCMVnz-py3.10/lib/python3.10/site-packages/mypy/checkexpr.py", line 1761, in infer_function_type_arguments
    (callee_type, inferred_args) = self.infer_function_type_arguments_pass2(
  File "/var/cache/kron-cache/pypoetry/virtualenvs/mypy-reproduce-vVFCMVnz-py3.10/lib/python3.10/site-packages/mypy/checkexpr.py", line 1818, in infer_function_type_arguments_pass2
    inferred_args = infer_function_type_arguments(
  File "/var/cache/kron-cache/pypoetry/virtualenvs/mypy-reproduce-vVFCMVnz-py3.10/lib/python3.10/site-packages/mypy/infer.py", line 54, in infer_function_type_arguments
    constraints = infer_constraints_for_callable(
  File "/var/cache/kron-cache/pypoetry/virtualenvs/mypy-reproduce-vVFCMVnz-py3.10/lib/python3.10/site-packages/mypy/constraints.py", line 171, in infer_constraints_for_callable
    c = infer_constraints(callee.arg_types[i], actual_type, SUPERTYPE_OF)
  File "/var/cache/kron-cache/pypoetry/virtualenvs/mypy-reproduce-vVFCMVnz-py3.10/lib/python3.10/site-packages/mypy/constraints.py", line 214, in infer_constraints
    return _infer_constraints(template, actual, direction)
  File "/var/cache/kron-cache/pypoetry/virtualenvs/mypy-reproduce-vVFCMVnz-py3.10/lib/python3.10/site-packages/mypy/constraints.py", line 293, in _infer_constraints
    return template.accept(ConstraintBuilderVisitor(actual, direction))
  File "/var/cache/kron-cache/pypoetry/virtualenvs/mypy-reproduce-vVFCMVnz-py3.10/lib/python3.10/site-packages/mypy/types.py", line 1831, in accept
    return visitor.visit_callable_type(self)
  File "/var/cache/kron-cache/pypoetry/virtualenvs/mypy-reproduce-vVFCMVnz-py3.10/lib/python3.10/site-packages/mypy/constraints.py", line 1011, in visit_callable_type
    return self.infer_against_overloaded(self.actual, template)
  File "/var/cache/kron-cache/pypoetry/virtualenvs/mypy-reproduce-vVFCMVnz-py3.10/lib/python3.10/site-packages/mypy/constraints.py", line 1035, in infer_against_overloaded
    item = find_matching_overload_item(overloaded, template)
  File "/var/cache/kron-cache/pypoetry/virtualenvs/mypy-reproduce-vVFCMVnz-py3.10/lib/python3.10/site-packages/mypy/constraints.py", line 1169, in find_matching_overload_item
    if mypy.subtypes.is_callable_compatible(
  File "/var/cache/kron-cache/pypoetry/virtualenvs/mypy-reproduce-vVFCMVnz-py3.10/lib/python3.10/site-packages/mypy/subtypes.py", line 1386, in is_callable_compatible
    unified = unify_generic_callable(left, right, ignore_return=ignore_return)
  File "/var/cache/kron-cache/pypoetry/virtualenvs/mypy-reproduce-vVFCMVnz-py3.10/lib/python3.10/site-packages/mypy/subtypes.py", line 1670, in unify_generic_callable
    c = mypy.constraints.infer_constraints(
  File "/var/cache/kron-cache/pypoetry/virtualenvs/mypy-reproduce-vVFCMVnz-py3.10/lib/python3.10/site-packages/mypy/constraints.py", line 211, in infer_constraints
    res = _infer_constraints(template, actual, direction)
  File "/var/cache/kron-cache/pypoetry/virtualenvs/mypy-reproduce-vVFCMVnz-py3.10/lib/python3.10/site-packages/mypy/constraints.py", line 293, in _infer_constraints
    return template.accept(ConstraintBuilderVisitor(actual, direction))
  File "/var/cache/kron-cache/pypoetry/virtualenvs/mypy-reproduce-vVFCMVnz-py3.10/lib/python3.10/site-packages/mypy/types.py", line 1319, in accept
    return visitor.visit_instance(self)
  File "/var/cache/kron-cache/pypoetry/virtualenvs/mypy-reproduce-vVFCMVnz-py3.10/lib/python3.10/site-packages/mypy/constraints.py", line 892, in visit_instance
    return infer_constraints(template, actual.upper_bound, self.direction)
  File "/var/cache/kron-cache/pypoetry/virtualenvs/mypy-reproduce-vVFCMVnz-py3.10/lib/python3.10/site-packages/mypy/constraints.py", line 211, in infer_constraints
    res = _infer_constraints(template, actual, direction)
  File "/var/cache/kron-cache/pypoetry/virtualenvs/mypy-reproduce-vVFCMVnz-py3.10/lib/python3.10/site-packages/mypy/constraints.py", line 293, in _infer_constraints
    return template.accept(ConstraintBuilderVisitor(actual, direction))
  File "/var/cache/kron-cache/pypoetry/virtualenvs/mypy-reproduce-vVFCMVnz-py3.10/lib/python3.10/site-packages/mypy/types.py", line 1319, in accept
    return visitor.visit_instance(self)
  File "/var/cache/kron-cache/pypoetry/virtualenvs/mypy-reproduce-vVFCMVnz-py3.10/lib/python3.10/site-packages/mypy/constraints.py", line 727, in visit_instance
    elif self.direction == SUPERTYPE_OF and instance.type.has_base(template.type.fullname):
  File "/var/cache/kron-cache/pypoetry/virtualenvs/mypy-reproduce-vVFCMVnz-py3.10/lib/python3.10/site-packages/mypy/nodes.py", line 3313, in __getattribute__
    raise AssertionError(object.__getattribute__(self, "msg"))
AssertionError: De-serialization failure: TypeInfo not fixed
./mypy_reproduce/__init__.py:37: : note: use --pdb to drop into pdb

To Reproduce https://github.com/JMarkin/mypy_reproduce

Your Environment

• Mypy version used: 0.991 and 1.0.0+dev.632304f90923ba38483d94de3cf58d486966f4f0
• Mypy command-line flags: empty
• Mypy configuration options from mypy.ini (and other config files): empty
• Python version used: Python 3.10.9
• Operating system and version: Linux 6.0.6-xanmod1-1

Bug Report

Hello there,

I am witnessing (rare) cases where mypy emits fewer errors when running on multiple files at the same time than when running on each file individually.

More specifically, I noticed that mypy can fail to report an error when an import for a submodule is missing a one file but present in another file.

I failed to find anything relevant in the documentation or in the existing issues, so I thought I'd let you know. Below is my attempt at a minimal reproducible example.

To Reproduce

(Since multiple files are needed, here is an archive with the reproduction case: repro.zip)

Given a module mymodule with nested modules a and b, and this file foo.py:

import mymodule.a


def blah() -> None:
    mymodule.b.fn()


blah()

As expected, when called on foo.py, mypy emits an error:

$ mypy --cache-dir /dev/null foo.py
foo.py:5: error: Module has no attribute "b"
Found 1 error in 1 file (checked 1 source file)

yet if I add to mypy's command line a second file which does import mymodule.b, say bar.py:

import mymodule.b

then mypy does not emit the error any more, even though I would expect it to:

$ mypy --cache-dir /dev/null foo.py bar.py
Success: no issues found in 2 source files

Your Environment

I used mypy 0.991 on Ubuntu 22.04 with python 3.10.6; I can reproduce this behavior with master, as well as python 3.8, and older versions of mypy.