This is an alternative to #2440 (disallowing third-party stubs). The idea is that typeshed remains/becomes a central repository for third-party stubs that are not bundled with the parent package, similar to DefinitelyTyped. In the future I expect type checkers will not want to bundle all third-party stubs for a variety of reasons, so third-party stubs would be distributed as separate PEP 561 stub-only packages, one per upstream package.

(I tried to integrate points raised there into this issue, especially those by @JukkaL in this comment.)

Advantages

• Due to typeshed's tests, packages in typeshed will continue to work with the latests versions of mypy and pytype.
• Basic level of consistency and standards due to review by typeshed maintainers.
• Consistent naming scheme for third-party stubs packages, allowing users to just "pip install <guessed name>" and it will work when there are stubs.
• Tooling (like tests) is easier to manage as it can remain part of the typeshed package.
• Easier to contribute to stubs, since contributors don't need to learn the intricacies of multiple stubs projects.
• No need to start a separate project just to distribute stubs for a new package.

Issues

• Workload issues for typeshed maintainers.
• Typeshed maintainers will often not be familiar with the package for which pull requests are opened.
• Publishing stubs takes longer due to the necessary reviews.

Further Considerations

What should the generated packages be called? @ethanhs's PEP 561 actually requires stubs-only package to be named <package>-stubs. typeshed could squat these names and release them (and remove the stubs) on the request of upstream maintainers. Alternatively, typeshed could add a common prefix or suffix (ts, typeshed) or in addition to or instead of the -stubs suffix. This would be in violation of PEP 561, so we'd need to get broader consensus to amend the PEP. My personal favorite would be <package>-ts.

To guarantee a fairly quick turnaround on stubs, to minimize work for publishing stubs, and to prevent all third-party stub packages to be updated whenever a new typeshed version is released, stubs for a specific third-party package should be published automatically when it changes.

Possible Implementation

• Add a generic setup-tp.py to typeshed that takes its package name from the directory it's in and uses the current date and time as version number.
• Amend the CI process for master only so that after a successful test run, for every third-party package that was changed since the last successful run, the following is done automatically:
  1. Copy setup-tp.py into the third-party module directory as setup.py.
  2. Build the package in that directory.
  3. Upload to pypi.

Open problems:

• [ ] How to install deps on other platforms. brew works well on macos, but no idea how to use windows (I've never touched it since 2008)
• [x] Listing platform specific packages. Which ones should we add?
• [x] I still need to change daily.yml to do the same thing for all stubs
• [x] How to test this? Not quite clear for now. I will need to tweak some values

Refs https://github.com/python/typeshed/issues/8660

While tqdm is very popular, it is not only not typed, but strips the types of sequences it receives due to the way it works. This should resolve the issue for most users.

An issue to add type hints to tqdm has been around since 2016, but hasn't seen any progress.

I've filled in the types as best as I could. But I might need help with a few things.

1. ~~I am not sure how to deal with the placeholder _Pointer types yet. I think I may have to create new classes just to expose their methods when type-checking. Although I think this PR is probably acceptable keeping them as Incomplete for now.~~
2. ~~I created a placeholder _Frame type. Is it possible to type the D3DShot class based on which backend it uses? Otherwise I have to keep casting the _Frames when using it:~~ Edit, doesn't matter because of https://github.com/python/typeshed/issues/5768
3. ~~I don't know how to deal with import issues in the CI.~~ https://github.com/python/typeshed/issues/5768

from @JelleZijlstra in gitter:

I agree that getattr with a default should return the type of that default. Probably the three-argument version should return Union[Any, T] where T is the type of the default. Feel free to submit a PR or issue to typeshed to make that change.

The motivation for this comes from an example like the following, which I encountered while at work:

def foo(...) -> List[int]:
    ...
    o = ...
    return [
        getattr(o, "int_attribute", None),
        getattr(o, "another_int_attribute", None),
    ]

mypy accepts this because the analyzed return type is List[Any], but that's not actually safe.

However, we can actually know this is unsafe at analysis time, because the type of the default value to getattr should be available and incorporated into the analyzed type. That is, in this case, the analyzed type really ought to be List[Optional[Any]] or List[Union[Any, None]], and since I assume that Union[Any, None] != Any, this better type ought to result in a mypy error, as expected.

(as a secondary note, apparently invariance and Any don't play as one would expect if Any were supposed to be top-type, but that's not what matters for this issue.)

Added most of the missing sections present in the concurrent.futures module for the process.pyi and thread.pyi files, related to #4641.

I'm sure there are plenty of edits needed for formatting reasons (like using generic types or TypeVars) so please let me know what should be added/fixed!

I'd like to start using TypeGuard in typeshed. Here's what we need:

• [x] mypy
• [x] pyright
• [x] pytype (opened https://github.com/google/pytype/issues/916)
• [x] pyre (opened https://github.com/facebook/pyre-check/issues/425)
• [ ] PyCharm

This is required to address some re-exports in https://github.com/python/typeshed/pull/8439

The source code is available here: https://github.com/urllib3/urllib3/blob/main/src/urllib3/contrib/socks.py

Fix #8437

This merge request makes weakref.finalize generic:

1. _P is the ParamSpec of the function. This is used in the __init__ function and the detach / peek calls, as described here
2. _T1 is the captured object, which is used in the detach and peek returns
3. _T2 is the captured return value of the func callable, which is re-used in the detach and peek methods.
4. As _ is an unused bitbucket, change it from Any to object, as any object is accepted since it's an unused argument

This begins the processing of completing the requests package's stubs.

I attempted to use stubtest locally to do the whole thing, but it doesn't seem to load the custom typeshed directory using this command:

stubtest --custom-typeshed-dir /Users/kkirsche/git/python/typeshed requests

I'm assuming this may have to do with having types-requests installed?

1. Adds missing stubs for __version__.py: https://github.com/psf/requests/blob/main/requests/version.py
   • Removes the use of Any for these values
2. Add missing type alias names as seen here: https://github.com/psf/requests/blob/main/requests/compat.py#L74-L79
3. Adds missing atomic_open and resolve_proxies functions
   • atomic_open: https://github.com/psf/requests/blob/main/requests/utils.py#L295-L305
   • resolve_proxies: https://github.com/psf/requests/blob/main/requests/utils.py#L857-L881
4. Add missing get method to LookupDict:
   • get: https://github.com/psf/requests/blob/main/requests/structures.py#L98-L99
5. Adds missing help.py stubs
   • https://github.com/psf/requests/blob/main/requests/help.py

Many functions in stdlib that are documented to accept an "iterable" are annotated as Iterable[T]. This means the arguments must have __iter__. However, the actual functions don't require the arguments to have __iter__, they also work with arguments that have only __getitem__. That's because internally they use iter (in Python) or PySequence_Fast (in C).

As the docs for both iter and PySequence_Fast functions show, these functions support two protocols for the argument: the "iterable" protocol (__iter__) and the "sequence" protocol (__getitem__).

Example functions that have this bug right now:

enumerate
iter
str.join
bytes.__new__
bytes.join

For enumerate, a bug was actually filed on the mypy repo but in fact I believe this should be corrected in typeshed.

Aside: for loops also similarly accept "iterables" that have only __getitem__, but that is implemented inside type checkers themselves. For now mypy still doesn't support it properly, but Pyright does.

This PR provides a few minor fixes to the gdb stubs.

I've mainly referenced an unofficial mirror https://github.com/bminor/binutils-gdb/tree/gdb-12-branch in the commit notes, although when I checked these locally, I've used apt's distribution of gdb 12.1 (/usr/share/gdb/python/) for Ubuntu 22.04. The official git https://www.sourceware.org/gdb/current/ is less accessible when cross-checking.

Symbols available in the built-in module _gdb (accessible when running Python through gdb) can be double-checked by running stubgen inside gdb (although the function/method signatures suffers the same issues as https://github.com/python/mypy/issues/14094).

$ gdb --eval-command=python-interactive
GNU gdb (Ubuntu 12.1-0ubuntu1~22.04) 12.1
  ...
Type "apropos word" to search for commands related to "word".
>>> import sys, mypy.stubgen
>>> sys.argv = ["stubgen", "-m", "_gdb"]
>>> mypy.stubgen.main()
Processed 1 modules
Generated out/_gdb.pyi

Offer sensible default VSCode settings. These are the workspace settings, specific to typeshed, that I've been using and filling in over the past few months. Workspace settings override user settings, so we can offer sensible, all-ready configurations for the typeshed workspace, reducing a lot of the pains a new contributor can have if their favorite editor is VSCode.

These settings obviously won't configure any extension not specified in .vscode/extensions.json > recommendations, and will assume the user may not have all recommended extensions installed or set as the formatter. Hence I've also configure some basic VSCode settings.

The recommended extensions include:

• The full python extensions suite needed for typeshed
• .editorconfig support and syntax highlighter
• A yaml formatter and validator
• A TOML syntax highliter and validator (no formatting)
• Mardown pack for syntax highlighting, and for the Mardown preview to match Github's (no formatter, though I'd recommend davidanson.vscode-markdownlint) For those not familiar with VSCode: Note that recommendations are not forced onto the user, a popup appears once to mention the workspace has recommended extensions, then they'll only show up under the "recommended" tab in the extensions menu.

It can still make sense for a user to want to modify .vscode/settings.json if they want to configure one of their extension to work properly with typeshed, without having to outright disable it for the workspace. It's also up to the dev to decide if they wanna follow these specific editor settings.

Unfortunately, VSCode doesn't (yet) offer any way to have "workspace defaults" or "user-worspace settings", so offering defaults to copy is the best we can do at the moment. And is what I've seen done in other open-source projects.

Extracted from #9374 Work towards #5768

Support external (non-type) dependencies in pyright in the CI. This also adds a new reusable script that can be run from CLI to get a list of dependencies (similar to get_packages.py) and leverages the read_dependencies method added in #9382

Dependent on https://github.com/PyCQA/flake8-pyi/pull/313 , but won't need a new version (read that PR description for details). Flake8-pyi will no longer be dependent on typeshed pre-emptively ignoring new positives.

Just opening this PR now so that you can merge whenever it's ready.