Trio – a friendly Python library for async concurrency and I/O

Overview
Join chatroom Join forum Documentation Latest PyPi version Latest conda-forge version Test coverage

Trio – a friendly Python library for async concurrency and I/O

The Trio project aims to produce a production-quality, permissively licensed, async/await-native I/O library for Python. Like all async libraries, its main purpose is to help you write programs that do multiple things at the same time with parallelized I/O. A web spider that wants to fetch lots of pages in parallel, a web server that needs to juggle lots of downloads and websocket connections simultaneously, a process supervisor monitoring multiple subprocesses... that sort of thing. Compared to other libraries, Trio attempts to distinguish itself with an obsessive focus on usability and correctness. Concurrency is complicated; we try to make it easy to get things right.

Trio was built from the ground up to take advantage of the latest Python features, and draws inspiration from many sources, in particular Dave Beazley's Curio. The resulting design is radically simpler than older competitors like asyncio and Twisted, yet just as capable. Trio is the Python I/O library I always wanted; I find it makes building I/O-oriented programs easier, less error-prone, and just plain more fun. Perhaps you'll find the same.

This project is young and still somewhat experimental: the overall design is solid, and the existing features are fully tested and documented, but you may encounter missing functionality or rough edges. We do encourage you to use it, but you should read and subscribe to issue #1 to get a warning and a chance to give feedback about any compatibility-breaking changes.

Where to next?

I want to try it out! Awesome! We have a friendly tutorial to get you started; no prior experience with async coding is required.

Ugh, I don't want to read all that – show me some code! If you're impatient, then here's a simple concurrency example, an echo client, and an echo server.

How does Trio make programs easier to read and reason about than competing approaches? Trio is based on a new way of thinking that we call "structured concurrency". The best theoretical introduction is the article Notes on structured concurrency, or: Go statement considered harmful. Or, check out this talk at PyCon 2018 to see a demonstration of implementing the "Happy Eyeballs" algorithm in an older library versus Trio.

Cool, but will it work on my system? Probably! As long as you have some kind of Python 3.6-or-better (CPython or the latest PyPy3 are both fine), and are using Linux, macOS, Windows, or FreeBSD, then Trio will work. Other environments might work too, but those are the ones we test on. And all of our dependencies are pure Python, except for CFFI on Windows, which has wheels available, so installation should be easy (no C compiler needed).

I tried it, but it's not working. Sorry to hear that! You can try asking for help in our chat room or forum, filing a bug, or posting a question on StackOverflow, and we'll do our best to help you out.

Trio is awesome, and I want to help make it more awesome! You're the best! There's tons of work to do – filling in missing functionality, building up an ecosystem of Trio-using libraries, usability testing (e.g., maybe try teaching yourself or a friend to use Trio and make a list of every error message you hit and place where you got confused?), improving the docs, ... check out our guide for contributors!

I don't have any immediate plans to use it, but I love geeking out about I/O library design! That's a little weird? But let's be honest, you'll fit in great around here. We have a whole sub-forum for discussing structured concurrency (developers of other systems welcome!). Or check out our discussion of design choices, reading list, and issues tagged design-discussion.

I want to make sure my company's lawyers won't get angry at me! No worries, Trio is permissively licensed under your choice of MIT or Apache 2. See LICENSE for details.

Code of conduct

Contributors are requested to follow our code of conduct in all project spaces.

Issues
  • Make trio/_core/_run less magical and understandable for static code analysis issue(#542)

    Make trio/_core/_run less magical and understandable for static code analysis issue(#542)

    This is the first step on exec replacement. Manual code works (at least on OSX). Please take a look and add any comments. The code still has the old code as dead commented code in it as a reference.

    Addresses part of #542

    opened by jmfrank63 67
  • Simplify imports

    Simplify imports

    As advised I replaced all import * with explicit imports in __init__.py of the trio root.

    I tested against vscode code completion and it seems to work. I did not run any tests yet.

    Also pylint does not complain anymore about missing attributes.

    The issue was #542.

    opened by jmfrank63 44
  • Should we rename trio.hazmat?

    Should we rename trio.hazmat?

    On the one hand, the name hazmat is definitely effective at getting people to pay attention and be careful. OTOH, I'm not sure it's quite communicating what we want: people seem scared of it, and uncertain whether they can actually use it. A bit of that is fine, but... I'm not sure it's communicating what we want. The block of text at the beginning helps of course, but you can't really fix up a confusing name with good docs.

    Maybe trio.lowlevel? Though trio.socket is kind of low-level too. Does lowlevel have the kind of "if you see this during a code review, ask for a comment justifying its usage" implication that we're going for?

    trio.foundation? trio.core?

    design discussion potential API breaker 
    opened by njsmith 41
  • Raise at next Checkpoint if Non-awaited coroutine found.

    Raise at next Checkpoint if Non-awaited coroutine found.

    right now that works only if the task is not finisehd. Not sure what to do if the task is done as you have no occasions to throw in it. Should we still return the final_result as a Result , or make it an Error() ?

    Debugging is still a bit weird as you need to find the checkpoint in the middle of the stacktrace (I guess we can improve that). Othe question would be is there a way to get the previous checkpoint of the current task to narrow tings down.

    It's still hard to debug when the non-awaited coroutine is not at the same stacklevel than the schedule point. We may be able to do better by inspecting unawaited coro frames maybe ?

    The await_later and context managers to relax/enforce await are not exposed, and I'm unsure whether we want to have custom CoroProtectors (likely yes for testing). We may also want to list all the unawaited coroutines in the error message, and so far I have not tried with many tasks, but the internals of Trio are still unfamiliar.

    Docs and Tests are still missing.

    opened by Carreau 39
  • Add contextvars support.

    Add contextvars support.

    This adds PEP 567 contextvars support to the core of Trio.

    Todo:

    • [x] Tests
    • [x] Deprecate TaskLocal
    • [x] Wait for a backport for 3.5 and 3.6.

    Closes #420 and closes #417, see also: #178.

    opened by Fuyukai 38
  • Subprocess support

    Subprocess support

    Edit: if you're just coming to this issue, then this comment has a good overview of what there is to do: https://github.com/python-trio/trio/issues/4#issuecomment-398967572


    Original description

    Lots of annoying and fiddly details, but important!

    I think for waiting, the best plan is to just give up on SIGCHLD (seriously, SIGCHLD is the worst) and park a thread in waitpid for each child process. Threads are lighter weight than processes so one thread-per-process shouldn't be a big deal. At least on Linux - if we're feeling ambitious we can do better on kqueue platforms. On Windows, it depends on what the state of our WaitFor{Multiple,Single}Object-fu is.

    design discussion missing piece asyncio feature parity 
    opened by njsmith 37
  • Should we deprecate trio.Event.clear?

    Should we deprecate trio.Event.clear?

    It occurs to me that I've seen 4 different pieces of code try to use Event.clear lately, and they were all buggy:

    • My first suggestion for solving #591 used an Event that we called clear on, and this created a race condition. (This was using threading.Event, but the principle is the same.) Details: https://github.com/python-trio/trio/pull/596#issuecomment-415270221

    • #619 uses it correctly (I think), but it's only safe because we enforce that only one task can call SignalReceiver.__anext__, which is subtle enough that I originally forgot to enforce it

    • @belm0 tried to use Event to track a value that toggles between true and false, but then found it wasn't appropriate for what he needed after all. (Not exactly Event's fault, but if we didn't have the clear method then I'm sure he'd have realized more quickly that it wasn't what he was looking for.)

    • @HyperionGray's websocket library tries to use Event objects to pass control back and forth between calls to send_message (in the main task) and a background writer task. Here's the writer task:

      https://github.com/HyperionGray/trio-websocket/blob/b787bf1a8a026ef1d9ca995d044bc97d42e7f727/trio_websocket/init.py#L300-L305

      If another task calls send_message while the writer task is blocked in send_all, then the send_message call will set() the event again, and then when send_all completes, it gets unconditionally cleared, so we end up in the invalid state where there is data pending, but self._data_pending is not set.

    Now, maybe this isn't Event.clear's fault, but it makes me wonder :-). (And this is partly coming out of my general reassessment of the APIs we inherited from the stdlib threading module, see also #322 and #573.)

    The core use case for Event is tracking whether an event has happened, and broadcasting that to an arbitrary number of listeners. For this purpose, clear isn't meaningful: once an event has happened, it can't unhappen. And if you stick to this core use case, Event seems very robust and difficult to mis-use.

    All of the trouble above came when someone tried to use it for something outside of this core use case. Some of these patterns do make sense:

    • If you have a periodic event, you might want to have the semantics of "wait for the next event". That can be done with an Event, where waiters call await ev.wait() and wakers call ev.set(); ev.clear(). But it can also be done with a Condition or a ParkingLot, or we could have a PeriodicEvent type if it comes up enough... for a dedicated PeriodicEvent it might also make sense to have a close method of some kind to avoid race conditions at shutdown, where tasks call wait after the last event has happened and deadlock.

      • Another option in many cases is to model a periodic event by creating one Event object per period. This is nice because it allows you to have overlapping periods. For example, consider a batching API, where tasks submit requests, and then every once in a while they get gathered up and submitted together. The submitting tasks want to wait until their request has been submitted. One way to do it would be to have an Event for each submission period. When a batch is gathered up for submission, the Event gets replaced, but the old Event doesn't get set until after the submission finishes. Maybe this is a pattern we should be nudging people towards, because it's more general/powerful.
    • The websocket example above could be made correct by moving the clear so that it's right after the wait, and before the call that consumes the data (data = self._wsproto.bytes_to_send()). (It might be more complicated if the consuming call wasn't itself synchronous.) So ev.wait(); ev.clear() can make sense... IF we know there is exactly one task listening. Which is way outside Event's core use case. In this case, it's basically a way of "passing control" from one task to another, which is often a mistake anyway – Python already has a very easy way to express sequential control like this, just execute the two things in the same task :-). Here I think a Lock would be better in any case: https://github.com/HyperionGray/trio-websocket/issues/3

    Are there any other use cases where Event.clear is really the right choice? Or where maybe it's not the right choice, but it's currently the best available choice?

    design discussion potential API breaker 
    opened by njsmith 37
  • Windows event notification

    Windows event notification

    Problem

    Windows has 3 incompatible families of event notifications APIs: IOCP, select/WSAPoll, and WaitForMultipleEvents-and-variants. They each have unique capabilities. This means: if you want to be able to react to all the different possible events that Windows can signal, then you must use all 3 of these. Needless to say, this creates a challenge for event loop design. There are a number of potentially viable ways to arrange these pieces; the question is which one we should use.

    (Actually, all 3 together still isn't sufficient, b/c there are some things that still require threads – like console IO – and I'm ignoring GUI events entirely because Trio isn't a GUI library. But never mind. Just remember that when someone tells you that Windows' I/O subsystem is great, that their statement isn't wrong but does require taking a certain narrow perspective...)

    Considerations

    The WaitFor*Event family

    The Event-related APIs are necessary to, for example, wait for a notification that a child process has exited. (The job object API provides a way to request IOCP notifications about process death, but the docs warn that the notifications are lossy and therefore useless...) Otherwise though they're very limited – in particular they have both O(n) behavior and max 64 objects in an interest set – so you definitely don't want to use these as your primary blocking call. We're going to be calling these in a background thread of some kind. The two natural architectures are to use WaitForSingleObject(Ex) and allocate one-thread-per-event, or else use WaitForMultipleObjects(Ex) and try and coalesce up to 64 events into each thread (substantially more complicated to implement but with 64x less memory overhead for thread stacks, if it matters). This is orthogonal to the rest of this issue, so it gets its own thread: #233

    IOCP

    IOCP is the crown jewel of Windows the I/O subsystem, and what you generally hear recommended. It follows a natively asynchronous model where you just go ahead and issue a read or write or whatever, and it runs in the background until eventually the kernel tells you it's done. It provides an O(1) notification mechanism. It's pretty slick. But... it's not as obvious a choice as everyone makes it sound. (Did you know the Chrome team has mostly given up on trying to make it work?)

    Issues:

    • When doing a UDP send, the send is only notified as complete once the packet hits the wire; i.e., using IOCP for UDP totally removes in-kernel buffering/flow-control. So to get decent throughput you must implement your own buffering system allowing multiple UDP sends to be in flight at once (but not too many because you don't want to introduce arbitrary latency). Or you could just use the non-blocking API and the kernel worries about this for you. (This hit Chrome hard; they switched to using non-blocking IO for UDP on Windows. ref1, ref2.)

    • When doing a TCP receive with a large buffer, apparently the kernel does a Nagle-like thing where it tries to hang onto the data for a while before delivering it to the application, thus introducing pointless latency. (This also bit Chrome hard; they switched to using non-blocking IO for TCP receive on Windows. ref1, ref2)

    • Sometimes you really do want to check whether a socket is readable before issuing a read: in particular, apparently outstanding IOCP receive buffers get pinned into kernel memory or some such nonsense, so it's possible to exhaust system resources by trying to listen to a large number of mostly-idle sockets.

    • Sometimes you really do want to check whether a socket is writable before issuing a write: in particular, because it allows adaptive protocols to provide lower latency if they can delay deciding what bytes to write until the last moment.

    • Python provides a complete non-blocking API out-of-the-box, and we use this API on other platforms, so using non-blocking IO on Windows as well is much MUCH simpler for us to implement than IOCP, which requires us to pretty much build our own wrappers from scratch.

    On the other hand, IOCP is the only way to do a number of things like: non-blocking IO to the filesystem, or monitoring the filesystem for changes, or non-blocking IO on named pipes. (Named pipes are popular for talking to subprocesses – though it's also possible to use a socket if you set it up right.)

    select/WSAPoll

    You can also use select/WSAPoll. This is the only documented way to check if a socket is readable/writable. However:

    • As is well known, these are O(n) APIs, which sucks if you have lots of sockets. It's not clear how much it sucks exactly -- just copying the buffer into kernel-space probably isn't a big deal for realistic interest set sizes -- but clearly it's not as nice as O(1). On my laptop, select.select on 3 sets of 512 idle sockets takes <200 microseconds, so I don't think this will, like, immediately kill us. Especially since people mostly don't run big servers on Windows? OTOH an empty epoll on the same laptop returns in ~0.6 microseconds, so there is some difference...

    • select.select is limited to 512 sockets, but this is trivially overcome; the Windows fd_set structure is just a array of SOCKETs + a length field, which you can allocate in any size you like (#3). (This is a nice side-effect of Windows never having had a dense fd space. This also means WSAPoll doesn't have much reason to exist. Unlike other platforms where poll beats select because poll uses an array and select uses a bitmap, WSAPoll is not really any more efficient than select. Its only advantage is that it's similar to how poll works on other platforms... but it's gratuitously incompatible. The one other interesting feature is that you can do an alertable wait with it, which gives a way to cancel it from another thread without using an explicit wakeup socket, via QueueUserAPC.)

    • Non-blocking IO on windows is apparently a bit inefficient because it adds an extra copy. (I guess they don't have zero-copy enqueueing of data to receive buffers? And on send I guess it makes sense that you can do that legitimately zero-copy with IOCP but not with nonblocking, which is nice.) Again I'm not sure how much this matters given that we don't have zero-copy byte buffers in Python to start with, but it's a thing.

    • select only works for sockets; you still need IOCP etc. for responding to other kinds of notifications.

    Options

    Given all of the above, our current design is a hybrid that uses select and non-blocking IO for sockets, with IOCP available when needed. We run select in the main thread, and IOCP in a worker thread, with a wakeup socket to notify when IOCP events occur. This is vastly simpler than doing it the other way around, because you can trivially queue work to an IOCP from any thread, while if you want to modify select's interest set from another thread it's a mess. As an initial design, this makes a lot of sense, because it allows us to provide full features (including e.g. wait_writable for adaptive protocols), avoid the tricky issues that IOCP creates for sockets, and requires a minimum of special code.

    The other attractive option would be if we could solve the issues with IOCP and switch to using it alone – this would be simpler and get rid of the O(n) select. However, as we can see above, there are a whole list of challenges that would need to be overcome first.

    Working around IOCP's limitations

    UDP sends

    I'm not really sure what the best approach here is. One option is just to limit the number of outstanding UDP data to some fixed amount (maybe tunable through a "virtual" (i.e. implemented by us) sockopt), and drop packets or return errors if we exceed that. This is clearly solvable in principle, it's just a bit annoying to figure out the details.

    Spurious extra latency in TCP receives

    I think that using the MSG_PUSH_IMMEDIATE flag should solve this.

    Checking readability / writability

    It turns out that IOCP actually can check readability! It's not mentioned on MSDN at all, but there's a well-known bit of folklore about the "zero-byte read". If you issue a zero-byte read, it won't complete until there's data ready to read. ref1 (← official MS docs! also note this is ch. 6 of "NPfMW", referenced below), ref2, ref3.

    That's for SOCK_STREAM sockets. What about SOCK_DGRAM? libuv does zero-byte reads with MSG_PEEK set (to avoid consuming the packet, truncating it to zero bytes in the process). MSDN explicitly says that this doesn't work (MSG_PEEK and overlapped IO supposedly don't work together), but I guess I trust libuv more than MSDN? I don't 100% trust either – this would need to be verified.

    What about writability? Empirically, if you have a non-blocking socket on windows with a full send buffer and you do a zero-byte send, it returns EWOULDBLOCK. (This is weird; other platforms don't do this.) If this behavior also translates to IOCP sends, then this zero-byte send trick would give us a way to use IOCP to check writability on SOCK_STREAM sockets.

    For writability of SOCK_DGRAM I don't think there's any trick, but it's not clear how meaningful SOCK_DGRAM writability is anyway. If we do our own buffering than presumably we can implement it there.

    Alternatively, there is a remarkable piece of undocumented sorcery, where you reach down directly to make syscalls, bypassing the Winsock userland, and apparently can get OVERLAPPED notifications when a socket is readable/writable: ref1, ref2, ref3, ref4, ref5. I guess this is how select is implemented? The problem with this is that it only works if your sockets are implemented directly in the kernel, which is apparently not always the case (because of like... antivirus tools and other horrible things that can interpose themselves into your socket API). So I'm inclined to discount this as unreliable. [Edit: or maybe not, see below]

    Implementing all this junk

    I actually got a ways into this. Then I ripped it out when I realized how many nasty issues there were beyond just typing in long and annoying API calls. But it could easily be resurrected; see 7e7a809c51d05729011506bc9de38cd97a35be44 and its parent.

    TODO

    If we do want to switch to using IOCP in general, then the sequence would go something like:

    • [ ] ~~check whether zero-byte sends give a way to check TCP writability via IOCP – this is probably the biggest determinant of whether going to IOCP-only is even possible (might be worth checking what doing UDP sends with MSG_PARTIAL does too while we're at it)~~
    • [ ] ~~check whether you really can do zero-byte reads on UDP sockets like libuv claims~~
    • [ ] ~~figure out what kind of UDP send buffering strategy makes sense (or if we decide that UDP sends can just drop packets instead of blocking then I guess the non-blocking APIs remain viable even if we can't do wait_socket_writable on UDP sockets)~~

    ~~At this point we'd have the information to decide whether we can/should go ahead. If so, then the plan would look something like:~~

    • [ ] ~~migrate away from select for the cases that can't use IOCP readable/writable checking:~~ [Not necessary, AFD-based select should work for these too]
      • [ ] ~~connect~~
      • [ ] ~~accept~~
    • [ ] ~~implement wait_socket_readable and wait_socket_writable on top of IOCP and get rid of select (but at this point we're still doing non-blocking I/O on sockets, just using IOCP as a select replacement)~~
    • [ ] ~~(optional / someday) switch to using IOCP for everything instead of non-blocking I/O~~

    New plan:

    • [ ] Use the tricks from the thread below to reimplement wait_socket_{readable,writable} using AFD, and confirm it works
    • [ ] Add LSP testing to our Windows CI
    • [ ] Consider whether we want to switch to using IOCP in more cases, e.g. send/recv. Not sure it's worth bothering.
    design discussion todo soon Windows low-level 
    opened by njsmith 30
  • add asynchronous file io and path wrappers

    add asynchronous file io and path wrappers

    This is an attempt to implement #20.

    Todo:

    _file_io:

    • [x] make duck-file definition more restrictive https://github.com/python-trio/trio/pull/180#discussion_r121320844

    _path:

    • [x] improve test_path_wraps_path https://github.com/python-trio/trio/pull/180#discussion_r121321347
    • [x] add tests for non-rewrapping forwards https://github.com/python-trio/trio/pull/180#discussion_r121323894
    • [x] add __div__
    • [x] py35 https://github.com/python-trio/trio/pull/180#discussion_r121320712
    • [x] ~improve Path docstring~ I was going to write some glossary entry on asynchronous path object, then I realized there is no upstream concept of path object, only PathLike
    • [x] properties can return new pathlib.Path instances too https://github.com/python-trio/trio/pull/180#discussion_r121582161

    Currently supported usages:

    # files
    
    async with await trio.open_file(filename) as f:
        await f.read()
    
    async_string = trio.wrap_file(StringIO('test'))
    assert await async_string.read() == 'test'
    
    # paths
    
    path = trio.Path('foo')
    await path.resolve()
    
    opened by buhman 29
  • Migrate hazmat to faked explicit importing and reexporting, to aid static analysis (issue #542)

    Migrate hazmat to faked explicit importing and reexporting, to aid static analysis (issue #542)

    Continuing towards finalisation of #542 We do a try / except import of our symbols to aid static analysis tools to pick them up. Then we update the list dynamically to have python get the correct symbols.

    opened by jmfrank63 27
  • Add new awesome trio library

    Add new awesome trio library "triotp"

    Hello there :wave:

    First, thanks for the amazing work on trio!

    I'm a big fan of Erlang/Elixir, and I implemented a mini OTP framework on top of trio: triotp

    I'd be very happy to see it listed on your website :smile:

    opened by linkdd 1
  • trio's guest mode breaks sniffio.current_async_library in certain cases

    trio's guest mode breaks sniffio.current_async_library in certain cases

    I am opening this bug as requested on Gitter (see link below).

    Summary

    When running trio as a guest (in a tkinter event loop in this case), and using trio.Nursery::start_soon() from the host event loop to schedule work to be done in the trio event loop, sniffio.current_async_library() doesn't detect the trio loop and raises.

    Details

    When running trio in guest mode in the tkinter event loop, the docs advise to have both loop have the same lifetime. This is achieved by passing a done_callback to trio.start_guest_run() that terminates the host loop. The closing mechanism is then to terminate the trio loop (most likely by cancelling it), which in turn terminates the host loop.

    The doc does not provide examples on how to start async function inside the trio loop from the host loop. When reading it, it was unclear to me what functions or methods are safe to call from outside the trio loop. I considered trio.Nursery::start_soon, trio.SendChannel::put_nowait(), trio.lowlevel.TrioToken::run_sync_soon() and trio.lowlevel.spawn_system_task().

    As nurseries seemed like an excellent control flow mechanism and a central concept in trio, I tried the Nursery. A Nursery is created in the trio loop, immediately calls nursery.start_soon(trio.sleep_forever) to remain available, and the host loop gets a reference to this nursery to schedule async function in it.

    This works for trio code, but seem to breaks sniffio.current_async_library(), which raises an exception. I my particular use-case, sniffio was used under the hood by asks, which was used for http requests.

    I wrote a (relatively) small example code to showcase the issue, as well as the fix proposed on Gitter. See the link below.

    I am unsure if this is an actual bug in sniffio, or trio guest mode, or bad code on my part. Maybe the doc of trio.lowlevel.start_guest_run() could be improved to show how to do this correctly (e.g. using memory channels), so as to avoid future confusion or unsafe code.

    Additional resources

    • discussion on gitter https://gitter.im/python-trio/general?at=619f8ac5c6149e53494c05b7
    • reproducing code: https://gist.github.com/QuentinSoubeyran/138f0784f9d596f5727ca2bea830ffcd

    Versions:

    • OS: Ubuntu 20.04
    • python 3.10
    • trio 0.19.0
    • sniffio 1.2.0

    If any info is missing, I'll gladly improve this issue !

    Words of thank

    Thank you all for the trio lib, it looks awesome ! I was extremely glade to find about guest mode, it really feels like the correct way to use async I/O and a UI together.

    opened by QuentinSoubeyran 0
  • Fix test_close_at_bad_time_for_send_all on ppc64el

    Fix test_close_at_bad_time_for_send_all on ppc64el

    On ppc64el, PIPE_BUF is 8192 but make_clogged_pipe() ends up writing a total of 1048576 bytes before the pipe is full, and then a subsequent receive_some(10000) isn't sufficient for orig_wait_writable() to return for our subsequent aclose() call. It's necessary to empty the pipe further before this happens. So we must loop until the pipe is empty to make sure that the sender wakes up even in this case. Otherwise patched_wait_writable() never gets to the aclose(), so expect_closedresourceerror() never returns, the nursery never finishes all tasks and this test hangs.

    opened by basak 1
  • Bump typing-extensions from 3.10.0.2 to 4.0.0

    Bump typing-extensions from 3.10.0.2 to 4.0.0

    Bumps typing-extensions from 3.10.0.2 to 4.0.0.

    Changelog

    Sourced from typing-extensions's changelog.

    Release 4.0.0 (November 14, 2021)

    • Starting with version 4.0.0, typing_extensions uses Semantic Versioning. See the README for more information.
    • Dropped support for Python versions 3.5 and older, including Python 2.7.
    • Simplified backports for Python 3.6.0 and newer. Patch by Adam Turner (@​AA-Turner).

    Added in version 4.0.0

    • Runtime support for PEP 673 and typing_extensions.Self. Patch by James Hilton-Balfe (@​Gobot1234).
    • Runtime support for PEP 655 and typing_extensions.Required and NotRequired. Patch by David Foster (@​davidfstr).

    Removed in version 4.0.0

    The following non-exported but non-private names have been removed as they are unneeded for supporting Python 3.6 and newer.

    • TypingMeta
    • OLD_GENERICS
    • SUBS_TREE
    • HAVE_ANNOTATED
    • HAVE_PROTOCOLS
    • V_co
    • VT_co

    Previous releases

    Prior to release 4.0.0 we did not provide a changelog. Please check the Git history for details.

    Commits

    Dependabot compatibility score

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
    • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    dependencies 
    opened by dependabot[bot] 1
  • Bump typed-ast from 1.4.3 to 1.5.0

    Bump typed-ast from 1.4.3 to 1.5.0

    Bumps typed-ast from 1.4.3 to 1.5.0.

    Commits

    Dependabot compatibility score

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
    • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    dependencies 
    opened by dependabot[bot] 1
  • Bump sphinx from 3.3.1 to 4.3.0

    Bump sphinx from 3.3.1 to 4.3.0

    Bumps sphinx from 3.3.1 to 4.3.0.

    Changelog

    Sourced from sphinx's changelog.

    Release 4.3.0 (released Nov 11, 2021)

    Dependencies

    • Support Python 3.10

    Incompatible changes

    • #9649: searchindex.js: the embedded data has changed format to allow objects with the same name in different domains.
    • #9672: The rendering of Python domain declarations is implemented with more docutils nodes to allow better CSS styling. It may break existing styling.
    • #9672: the signature of :py:meth:domains.py.PyObject.get_signature_prefix has changed to return a list of nodes instead of a plain string.
    • #9695: domains.js.JSObject.display_prefix has been changed into a method get_display_prefix which now returns a list of nodes instead of a plain string.
    • #9695: The rendering of Javascript domain declarations is implemented with more docutils nodes to allow better CSS styling. It may break existing styling.
    • #9450: mathjax: Load MathJax via "defer" strategy

    Deprecated

    • sphinx.ext.autodoc.AttributeDocumenter._datadescriptor
    • sphinx.writers.html.HTMLTranslator._fieldlist_row_index
    • sphinx.writers.html.HTMLTranslator._table_row_index
    • sphinx.writers.html5.HTML5Translator._fieldlist_row_index
    • sphinx.writers.html5.HTML5Translator._table_row_index

    Features added

    • #9639: autodoc: Support asynchronous generator functions
    • #9664: autodoc: autodoc-process-bases supports to inject reST snippet as a base class
    • #9691: C, added new info-field retval for :rst:dir:c:function and :rst:dir:c:macro.
    • C++, added new info-field retval for :rst:dir:cpp:function.
    • #9618: i18n: Add :confval:gettext_allow_fuzzy_translations to allow "fuzzy" messages for translation
    • #9672: More CSS classes on Python domain descriptions
    • #9695: More CSS classes on Javascript domain descriptions
    • #9683: Revert the removal of add_stylesheet() API. It will be kept until

    ... (truncated)

    Commits
    • 3001cc9 Bump to 4.3.0 final
    • d321a97 Merge pull request #9837 from tk0miya/9833_RemovedInSphinx60Warning
    • 5339636 Fix the type of deprecation warning for get_signature_prefix (refs: #9833)
    • 5999cdb Merge pull request #9833 from jakobandersen/py_get_signature_prefix
    • 63bd4b8 Use warnings.warn and deprecation warning
    • 096e286 Merge pull request #9628 from tk0miya/9623_suppress_warnings_for_excluded_doc...
    • 1317eee Merge pull request #9828 from tk0miya/9618_gettext_allow_fuzzy_translations
    • 7357b17 Merge branch '4.x' into 9623_suppress_warnings_for_excluded_document_in_toctree
    • 22bec4f Fix #9623: Separate warning type 'toc.not_readable' to 'toc.excluded'
    • 563936b Merge pull request #9829 from tk0miya/9450_load_mathjax_as_defer
    • Additional commits viewable in compare view

    Dependabot compatibility score

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
    • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    dependencies 
    opened by dependabot[bot] 1
  • Bump regex from 2021.11.1 to 2021.11.10

    Bump regex from 2021.11.1 to 2021.11.10

    Bumps regex from 2021.11.1 to 2021.11.10.

    Commits

    Dependabot compatibility score

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
    • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    dependencies 
    opened by dependabot[bot] 1
  • Bump jinja2 from 3.0.2 to 3.0.3

    Bump jinja2 from 3.0.2 to 3.0.3

    Bumps jinja2 from 3.0.2 to 3.0.3.

    Release notes

    Sourced from jinja2's releases.

    3.0.3

    What's Changed

    New Contributors

    Full Changelog: https://github.com/pallets/jinja/compare/3.0.2...3.0.3

    Changelog

    Sourced from jinja2's changelog.

    Version 3.0.3

    Released 2021-11-09

    • Fix traceback rewriting internals for Python 3.10 and 3.11. :issue:1535
    • Fix how the native environment treats leading and trailing spaces when parsing values on Python 3.10. :pr:1537
    • Improve async performance by avoiding checks for common types. :issue:1514
    • Revert change to hash(Node) behavior. Nodes are hashed by id again :issue:1521
    • PackageLoader works when the package is a single module file. :issue:1512
    Commits
    • 2a48dd8 Merge pull request #1543 from pallets/release-3.0.3
    • ce1a539 release version 3.0.3
    • 199b6d5 Merge pull request #1542 from pallets/package-loader-file
    • eec0db8 PackageLoader works with single module file
    • 46f3a68 Merge pull request #1541 from pallets/docs-pow-order
    • 0d19990 document chained pow order
    • 99daa2a Merge pull request #1540 from pallets/docs-macro-defaults
    • a6162da rewrite docs about extending template objects
    • cab5545 Merge pull request #1539 from pallets/docs-macro-defaults
    • 369ff03 remove reference to macro.defaults
    • Additional commits viewable in compare view

    Dependabot compatibility score

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
    • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    dependencies 
    opened by dependabot[bot] 1
  • Bump imagesize from 1.2.0 to 1.3.0

    Bump imagesize from 1.2.0 to 1.3.0

    Bumps imagesize from 1.2.0 to 1.3.0.

    Commits
    • 731a148 1.3.0
    • d573c6c Fix exception on corrupt JPEG files
    • c6e63b9 Fix handling of different path types
    • 8cdd5c8 Remove deprecated use of 'setup.py test'
    • 215e461 Remove unnecessary PYTHONPATH manipulation
    • 0088b53 Add tests for the getDPI() function
    • 8e57ff3 Set offest to 0
    • 7a8da41 Fix error on python2.7
    • 1391a44 Improve performance for svg images
    • 61e99dc Fix error on long float number
    • Additional commits viewable in compare view

    Dependabot compatibility score

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
    • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    dependencies 
    opened by dependabot[bot] 1
  • Bump isort from 5.9.3 to 5.10.1

    Bump isort from 5.9.3 to 5.10.1

    Bumps isort from 5.9.3 to 5.10.1.

    Release notes

    Sourced from isort's releases.

    5.10.1 November 8 2021

    • Fixed #1819: Occasional inconsistency with multiple src paths.
    • Fixed #1840: skip_file ignored when on the first docstring line

    5.10.0

    Implemented [#1796](https://github.com/pycqa/isort/issues/1796): Switch to tomli for pyproject.toml configuration loader.
    Fixed [#1801](https://github.com/pycqa/isort/issues/1801): CLI bug (--exend-skip-glob, overrides instead of extending).
    Fixed [#1802](https://github.com/pycqa/isort/issues/1802): respect PATH customization in nested calls to git.
    Fixed [#1838](https://github.com/pycqa/isort/issues/1838): Append only with certain code snippets incorrectly adds imports.
    Added official support for Python 3.10
    
    Changelog

    Sourced from isort's changelog.

    5.10.1 November 8 2021

    • Fixed #1819: Occasional inconsistency with multiple src paths.
    • Fixed #1840: skip_file ignored when on the first docstring line

    5.10.0 November 3 2021

    • Implemented #1796: Switch to tomli for pyproject.toml configuration loader.
    • Fixed #1801: CLI bug (--exend-skip-glob, overrides instead of extending).
    • Fixed #1802: respect PATH customization in nested calls to git.
    • Fixed #1838: Append only with certain code snippets incorrectly adds imports.
    • Added official support for Python 3.10

    Potentially breaking changes:

    • Fixed #1785: _ast module incorrectly excluded from stdlib definition.
    Commits

    Dependabot compatibility score

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
    • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    dependencies 
    opened by dependabot[bot] 1
A lightweight (serverless) native python parallel processing framework based on simple decorators and call graphs.

A lightweight (serverless) native python parallel processing framework based on simple decorators and call graphs, supporting both control flow and dataflow execution paradigms as well as de-centralized CPU & GPU scheduling.

null 98 Nov 9, 2021
SCOOP (Scalable COncurrent Operations in Python)

SCOOP (Scalable COncurrent Operations in Python) is a distributed task module allowing concurrent parallel programming on various environments, from h

Yannick Hold 521 Nov 17, 2021
A Python package for easy multiprocessing, but faster than multiprocessing

MPIRE, short for MultiProcessing Is Really Easy, is a Python package for multiprocessing, but faster and more user-friendly than the default multiprocessing package.

null 504 Nov 29, 2021
Simple package to enhance Python's concurrent.futures for memory efficiency

future-map is a Python library to use together with the official concurrent.futures module.

Arai Hiroki 1 Oct 18, 2021
Coroutine-based concurrency library for Python

gevent Read the documentation online at http://www.gevent.org. Post issues on the bug tracker, discuss and ask open ended questions on the mailing lis

gevent 5.6k Nov 27, 2021
TriOTP, the OTP framework for Python Trio

TriOTP, the OTP framework for Python Trio See documentation for more informations. Introduction This project is a simplified implementation of the Erl

David Delassus 1 Nov 26, 2021
SNV calling pipeline developed explicitly to process individual or trio vcf files obtained from Illumina based pipeline (grch37/grch38).

SNV Pipeline SNV calling pipeline developed explicitly to process individual or trio vcf files obtained from Illumina based pipeline (grch37/grch38).

East Genomics 1 Nov 2, 2021
A SOCKS proxy server implemented with the powerful python cooperative concurrency framework asyncio.

asyncio-socks-server A SOCKS proxy server implemented with the powerful python cooperative concurrency framework asyncio. Features Supports both TCP a

Amaindex 138 Nov 17, 2021
The deployment framework aims to provide a simple, lightweight, fast integrated, pipelined deployment framework that ensures reliability, high concurrency and scalability of services.

savior是一个能够进行快速集成算法模块并支持高性能部署的轻量开发框架。能够帮助将团队进行快速想法验证(PoC),避免重复的去github上找模型然后复现模型;能够帮助团队将功能进行流程拆解,很方便的提高分布式执行效率;能够有效减少代码冗余,减少不必要负担。

Tao Luo 105 Nov 19, 2021
An esoteric programming language that supports concurrency, regex, and web requests.

The Hofstadter Esoteric Programming Language Hofstadter's Law: It always takes longer than you expect, even when you take into account Hofstadter's La

Austin Henley 4 Nov 30, 2021
API for concurrency connections

Multi-connection-server-API API for concurrency connections difference between this server and the echo server is the call to lsock.setblocking(False)

Muziwandile Nkomo 1 Oct 30, 2021
Python HTTP library with thread-safe connection pooling, file post support, user friendly, and more.

urllib3 is a powerful, user-friendly HTTP client for Python. Much of the Python ecosystem already uses urllib3 and you should too. urllib3 brings many

urllib3 2.8k Nov 24, 2021
Python HTTP library with thread-safe connection pooling, file post support, user friendly, and more.

urllib3 is a powerful, user-friendly HTTP client for Python. Much of the Python ecosystem already uses urllib3 and you should too. urllib3 brings many

urllib3 2.8k Nov 25, 2021
The friendly PIL fork (Python Imaging Library)

Pillow Python Imaging Library (Fork) Pillow is the friendly PIL fork by Alex Clark and Contributors. PIL is the Python Imaging Library by Fredrik Lund

Pillow 9.2k Dec 3, 2021
The friendly PIL fork (Python Imaging Library)

Pillow Python Imaging Library (Fork) Pillow is the friendly PIL fork by Alex Clark and Contributors. PIL is the Python Imaging Library by Fredrik Lund

Pillow 9.2k Nov 28, 2021
A small faсade for the standard python mocker library to make it user-friendly

unittest-mocker Inspired by the pytest-mock, but written from scratch for using with unittest and convenient tool - patch_class Installation pip insta

Vertliba V.V. 3 Nov 5, 2021
Revolt.py - An async library to interact with the https://revolt.chat api.

Revolt.py An async library to interact with the https://revolt.chat api. This library will be focused on making bots and i will not implement anything

Zomatree 24 Nov 20, 2021
The async ready version of the AniManga library created by centipede000.

Async-Animanga An Async/Aiohttp compatible library. Async-Animanga is an async ready web scraping library that returns Manga information from animepla

null 3 Sep 12, 2021
A friendly library for parsing HTTP request arguments, with built-in support for popular web frameworks, including Flask, Django, Bottle, Tornado, Pyramid, webapp2, Falcon, and aiohttp.

webargs Homepage: https://webargs.readthedocs.io/ webargs is a Python library for parsing and validating HTTP request objects, with built-in support f

marshmallow-code 1.2k Nov 24, 2021
Scalable, event-driven, deep-learning-friendly backtesting library

...Minimizing the mean square error on future experience. - Richard S. Sutton BTGym Scalable event-driven RL-friendly backtesting library. Build on

Andrew 853 Nov 25, 2021
Motor - the async Python driver for MongoDB and Tornado or asyncio

Motor Info: Motor is a full-featured, non-blocking MongoDB driver for Python Tornado and asyncio applications. Documentation: Available at motor.readt

mongodb 1.8k Dec 2, 2021
Mixcloud API wrapper for Python and Async IO

aiomixcloud Mixcloud API wrapper for Python and Async IO aiomixcloud is a wrapper library for the HTTP API of Mixcloud. It supports asynchronous opera

Aristotelis Mikropoulos 23 Dec 8, 2020
Motor - the async Python driver for MongoDB and Tornado or asyncio

Motor Info: Motor is a full-featured, non-blocking MongoDB driver for Python Tornado and asyncio applications. Documentation: Available at motor.readt

mongodb 1.6k Feb 6, 2021
High-performance Async REST API, in Python. FastAPI + GINO + Arq + Uvicorn (w/ Redis and PostgreSQL).

fastapi-gino-arq-uvicorn High-performance Async REST API, in Python. FastAPI + GINO + Arq + Uvicorn (powered by Redis & PostgreSQL). Contents Get Star

Leo Sussan 281 Dec 3, 2021
The ormar package is an async mini ORM for Python, with support for Postgres, MySQL, and SQLite.

python async mini orm with fastapi in mind and pydantic validation

null 730 Dec 3, 2021
🔄 🌐 Handle thousands of HTTP requests, disk writes, and other I/O-bound tasks simultaneously with Python's quintessential async libraries.

?? ?? Handle thousands of HTTP requests, disk writes, and other I/O-bound tasks simultaneously with Python's quintessential async libraries.

Hackers and Slackers 4 Oct 13, 2021
Lemon is an async and lightweight API framework for python

Lemon is an async and lightweight API framework for python . Inspired by Koa and Sanic .

Joway 27 Oct 18, 2021
Dazzler is a Python async UI/Web framework built with aiohttp and react.

Dazzler is a Python async UI/Web framework built with aiohttp and react. Create dazzling fast pages with a layout of Python components and bindings to update from the backend.

Philippe Duval 15 Nov 15, 2021
A modern, easy to use, feature-rich, and async ready API wrapper for Discord written in Python.

A modern, easy to use, feature-rich, and async ready API wrapper for Discord written in Python. Key Features Modern Pythonic API using async and await

Senpai Development 4 Nov 5, 2021