mypy-protobuf: Generate mypy stub files from protobuf specs
We just released a new major release mypy-protobuf 2. on 02/02/2021! It includes some backward incompatible changes. See Changelog for recent changes.
Requirements
mypy >= v0.800 protoc 3.14.0 or greater python-protobuf >= 3.14.0 python >= 3.6 - for running mypy-protobuf plugin. Generated stubs will compatible back to python2.
Other configurations may work, but are not supported in testing currently. We would be open to expanding this list if a need arises - file an issue on the issue tracker.
Installation
The plugin can be installed with
pip install mypy-protobuf
To install unreleased
REV=master # or whichever unreleased git rev you'd like
pip install git+https://github.com/dropbox/mypy-protobuf.git@$REV
# Prior to directory structure flattening, you may need
pip install git+https://github.com/dropbox/mypy-protobuf.git@$REV#subdirectory=python
Implementation
The implementation of the plugin is in mypy_protobuf/main.py, which installs to a posix executable protoc-gen-mypy. On windows you will have to use protoc_gen_mypy.bat for the executable.
On posix, ensure that the protoc-gen-mypy script installed onto your $PATH. Then run.
protoc --python_out=output/location --mypy_out=output/location
Alternately, you can explicitly provide the path:
protoc --plugin=protoc-gen-mypy=path/to/protoc-gen-mypy --python_out=output/location --mypy_out=output/location
On windows, provide the bat file:
protoc --plugin=protoc-gen-mypy=path/to/protoc_gen_mypy.bat --python_out=output/location --mypy_out=output/location
Features
See Changelog for full listing
Types enum int values more strongly
Enum int values produce stubs which wrap the int values in NewType
enum MyEnum {
FOO = 0;
BAR = 1;
}
Will yield an enum type wrapper whose methods type to MyEnum.V rather than int. This allows mypy to catch bugs where the wrong enum value is being used.
mypy-protobuf autogenerates an instance of the EnumTypeWrapper as follows.
class _MyEnum(google.protobuf.internal.EnumTypeWrapper[MyEnum.V], builtins.type):
DESCRIPTOR: google___protobuf___descriptor___EnumDescriptor = ...
FOO = MyEnum.V(0)
BAR = MyEnum.V(1)
class MyEnum(metaclass=_OuterEnum):
V = typing___NewType('V', builtins.int)
FOO = MyEnum.V(0)
BAR = MyEnum.V(1)
Calling code may be typed as follows. Note that the type of x must be quoted until upstream protobuf supports V
def f(x: 'MyEnum.V'):
print(x)
f(MyEnum.Value("FOO"))
Supports generating type wrappers for fields and maps
M.proto
message M {
uint32 user_id = 1 [(mypy_protobuf.casttype)="mymod.UserId"
map<uint32, string> email_by_uid = 2 [
(mypy_protobuf.keytype)="path/to/mymod.UserId",
(mypy_protobuf.valuetype)="path/to/mymod.Email"
];
}
mymod.py
UserId = NewType("UserId", int)
Email = NewType("Email", Text)
py_generic_services
If py_generic_services is set in your proto file, then mypy-protobuf will generate service stubs. If you want GRPC stubs instead - use the GRPC instructions.
readable_stubs
If readable_stubs is set, mypy-protobuf will generate easier-to-read stubs. The downside to this approach - is that it's possible to generate stubs which do not pass mypy - particularly in the case of name collisions. mypy-protobuf defaults to generating stubs with fully qualified imports and mangled global-level identifiers to defend against name collisions between global identifiers and field names.
If you're ok with this risk, try it out!
protoc --python_out=output/location --mypy_out=readable_stubs:output/location
relax_strict_optional_primitives
If you are using proto3, then primitives cannot be represented as NULL on the wire - only as their zero value. By default mypy-protobuf types message constructors to have non-nullable primitives (eg int instead of Optional[int]). python-protobuf itself will internally convert None -> zero value, and if you intentionally want to use this behavior, set this! We recommend avoiding this, but it may be helpful when migrating existing proto2 code.
protoc --python_out=output/location --mypy_out=relax_strict_optional_primitives:output/location
Output suppression
To suppress output, you can run
protoc --python_out=output/location --mypy_out=quiet:output/location
GRPC
This plugin provides stubs generation for grpcio generated code.
protoc \
--python_out=output/location \
--mypy_out=output/location \
--grpc_out=output/location \
--mypy_grpc_out=output/location
Note that generated code for grpc will work only together with code for python and locations should be the same. If you need stubs for grpc internal code we suggest using this package https://github.com/shabbyrobe/grpc-stubs
Contributing
Contributions to the implementation are welcome. Please run tests using ./run_test.sh. Ensure code is formatted using black.
pip3 install black
black mypy_protobuf/main.py test/
Contributors
Dropboxers
Others
- @Ketouem
- @nmiculinic
- @onto
- @jcppkkk
- @drather19
- @smessmer
- @pcorpet
- @zozoens31
- @abhishekrb19
- @jaens
- @arussellsaw
- @shabbyrobe
- @reorx
- @zifter
- @juzna
- @mikolajz
- @chadrik
- @EPronovost
- @chrislawlor
- @henribru
- @Evgenus
Licence etc.
- License: Apache 2.0.
- Copyright attribution: Copyright (c) 2017 Dropbox, Inc.
- External contributions to the project should be subject to Dropbox's Contributor License Agreement (CLA): https://opensource.dropbox.com/cla/
139 Dec 30, 2022
19 Dec 27, 2022
103 Jun 23, 2022
238 Dec 27, 2022
7 Aug 7, 2022
1.2k Dec 30, 2022
194 Dec 19, 2022
3.3k Jan 2, 2023
1.1k Dec 30, 2022
24 Aug 28, 2022
0 Sep 5, 2021
2.2k Dec 22, 2022
218 Jan 3, 2023
28 Oct 20, 2022
18 Nov 26, 2022