In JavaScript I have

var url = 'https://localhost:4433/tts';
var transport = new WebTransport(url);
transport.ready.then(async() => {
  var sender = await transport.createUnidirectionalStream();
  console.log(sender);
  var writer = sender.getWriter();
  var textEncoder = new TextEncoder('utf-8');
  var inputData = textEncoder.encode(`parec -d alsa_output.pci-0000_00_14.2.analog-stereo.monitor`);
  await writer.write(inputData);
  await writer.close();
  await writer.closed;
  var readable = await transport.incomingUnidirectionalStreams;
  readable.pipeTo(
    new WritableStream({
        write(v) {
           v.pipeTo(new WritableStream({
               write(data) {
                   console.log(data);
               }, close() {
                   console.log('close 1');
               }
           }))
        }, close() {
            console.log('close 0');
        }
    })
  );
});

In Python I have (using GoogleChrome samples code)

input_data = b''

# CounterHandler implements a really simple protocol:
#   - For every incoming bidirectional stream, it counts bytes it receives on
#     that stream until the stream is closed, and then replies with that byte
#     count on the same stream.
#   - For every incoming unidirectional stream, it counts bytes it receives on
#     that stream until the stream is closed, and then replies with that byte
#     count on a new unidirectional stream.
#   - For every incoming datagram, it sends a datagram with the length of
#     datagram that was just received.
class CounterHandler:

    def __init__(self, session_id, http: H3Connection) -> None:
        self._session_id = session_id
        self._http = http
        self._counters = defaultdict(int)

    def h3_event_received(self, event: H3Event) -> None:
        if isinstance(event, DatagramReceived):
            payload = str(len(event.data)).encode('ascii')
            self._http.send_datagram(self._session_id, payload)

        if isinstance(event, WebTransportStreamDataReceived):
            print('event', event)
            if event.data:
                print('event.data', event.data)
                global input_data
                input_data = event.data
                self._counters[event.stream_id] += len(event.data)
            if event.stream_ended:
                print('event.stream_ended', event.stream_ended)
                if stream_is_unidirectional(event.stream_id):
                    response_id = self._http.create_webtransport_stream(
                        self._session_id, is_unidirectional=True)
                else:
                    response_id = event.stream_id
                '''
                data = subprocess.run(split(input_data.decode()), stdout=subprocess.PIPE)           
                payload = data.stdout
                self._http._quic.send_stream_data(
                    response_id, payload, end_stream=True)
                '''
                process = subprocess.Popen(split(input_data.decode()), stdout=subprocess.PIPE)
                os.set_blocking(process.stdout.fileno(), False)
                for chunk in iter(lambda: process.stdout.read(1024*1024), b''):
                    if chunk is not None:
                        # print(chunk)
                        self._http._quic.send_stream_data(
                            response_id, chunk, end_stream=False)
                        # break
                self.stream_closed(event.stream_id)

    def stream_closed(self, stream_id: int) -> None:
        try:
            print('stream_closed')
            del self._counters[stream_id]
        except KeyError:
            pass

At console

Uint8Array(756) ...
close 1

If I set

```end_stream=False``

and do not include the

break

in Python I get a

'Connection lost.'

error in JavaScript.

What is the pattern necessary and how to re-arrange the Python code to continue sending the chunks from iter(lambda: process.stdout.read(1024*1024), b'') to client?

Hi,

We are developing WebTransport over HTTP/3 which replaces WebTransport over QUIC (a.k.a. QUICTransport). I would like to use aioquic to implement a test server for web-platform tests (similar to what I did, writing a QuicTransport server on top of aioquic for testing).

https://tools.ietf.org/html/draft-ietf-webtrans-http3-00 is the spec draft.

To write a test server, I would like to implement some features specific to WebTransport over HTTP/3. Specifically,

1. We would like to handle SETTINGS frames.
2. WebTransport over HTTP/3 defines some custom frames (0x54 and 0x41).

To support WebTransport over HTTP/3, we need to...

1. support WebTransport over HTTP/3 directly in H3Connection,
2. make H3Connection a bit more flexisible, or
3. re-implement H3Connection out of aioquic.

I would like to avoid the last solution if possible.

Do you have any suggestions? I'm happy to make PRs if needed. Thank you!

cc: @vasilvv

This is a PR for https://github.com/aiortc/aioquic/issues/242.

I do not know how to fix the problem nor write tests for /tests.

What I do know is that the server only sends 28 datagrams.

Steps to reproduce:

• Follow the instructions at https://github.com/aiortc/aioquic/tree/main/examples#chromium-and-chrome-usage then navigate to https://localhost:4433/. I included the script (script.js) in htdocs.

Expected result:

• I write/send one (1) datagram to the server.
• The server writes datagrams to client until I tell it to stop with writer.close()

What actually happens:

• The server sends 28 datagrams to client then stops.
• WebTransportError: Connection lost. thrown twice.

What I am actually trying to do is stream media using aioquic and WebTransport. For that I need only send one (1) initial datagram telling the server what to stream, then read the datagrams until I call writer.close() or transport.close(). Very simple.

At terminal

2021-11-13 11:39:45,945 INFO quic [4654232e7292bb80] ALPN negotiated protocol h3
2021-11-13 11:39:45,947 INFO quic [4654232e7292bb80] Connection close sent (code 0x109, reason ENABLE_WEBTRANSPORT requires H3_DATAGRAM)

At browser

Failed to establish a connection to https://quic.aiortc.org/wt: net::ERR_METHOD_NOT_SUPPORTED.
VM23:11 Opening handshake failed.
(anonymous) @ VM23:11
Promise.catch (async)
(anonymous) @ VM23:10
quic.aiortc.org/:1 Uncaught (in promise) WebTransportError: Opening handshake failed.

Steps to reeproduce:

Follow instructions at examples/README for Chromium.

Potentially related https://github.com/GoogleChrome/samples/pull/749#issuecomment-961623180, https://github.com/GoogleChrome/samples/pull/754/commits/b83dc66a8f24e357201c6dd0e8aed26d26ba5ea0.

============================= test session starts =============================
platform win32 -- Python 3.8.2, pytest-5.4.1, py-1.8.1, pluggy-0.13.1 -- d:\ProgramData\Python38\python.exe
cachedir: .pytest_cache
rootdir: E:\data\aioquic-0.8.6
plugins: asyncio-0.10.0
collecting ... collected 300 items

tests/test_asyncio.py::HighLevelTest::test_change_connection_id FAILED   [  0%]
tests/test_asyncio.py::HighLevelTest::test_connect_and_serve FAILED      [  0%]
tests/test_asyncio.py::HighLevelTest::test_connect_and_serve_ec_certificate FAILED [  1%]
tests/test_asyncio.py::HighLevelTest::test_connect_and_serve_large FAILED [  1%]
tests/test_asyncio.py::HighLevelTest::test_connect_and_serve_with_packet_loss FAILED [  1%]
tests/test_asyncio.py::HighLevelTest::test_connect_and_serve_with_session_ticket FAILED [  2%]
tests/test_asyncio.py::HighLevelTest::test_connect_and_serve_with_sni FAILED [  2%]
tests/test_asyncio.py::HighLevelTest::test_connect_and_serve_with_stateless_retry FAILED [  2%]
tests/test_asyncio.py::HighLevelTest::test_connect_and_serve_with_stateless_retry_bad PASSED [  3%]
tests/test_asyncio.py::HighLevelTest::test_connect_and_serve_with_stateless_retry_bad_original_connection_id PASSED [  3%]
tests/test_asyncio.py::HighLevelTest::test_connect_and_serve_with_version_negotiation FAILED [  3%]
tests/test_asyncio.py::HighLevelTest::test_connect_and_serve_without_client_configuration PASSED [  4%]
tests/test_asyncio.py::HighLevelTest::test_connect_and_serve_writelines FAILED [  4%]
tests/test_asyncio.py::HighLevelTest::test_connect_timeout PASSED        [  4%]
tests/test_asyncio.py::HighLevelTest::test_connect_timeout_no_wait_connected PASSED [  5%]
tests/test_asyncio.py::HighLevelTest::test_key_update FAILED             [  5%]
tests/test_asyncio.py::HighLevelTest::test_ping FAILED                   [  5%]
tests/test_asyncio.py::HighLevelTest::test_ping_parallel FAILED          [  6%]
tests/test_asyncio.py::HighLevelTest::test_server_receives_garbage PASSED [  6%]
tests/test_asyncio_compat.py::AsyncioCompatTest::test_ok PASSED          [  6%]
tests/test_asyncio_compat.py::AsyncioCompatTest::test_raise_exception PASSED [  7%]
tests/test_asyncio_compat.py::AsyncioCompatTest::test_raise_exception_type PASSED [  7%]
tests/test_buffer.py::BufferTest::test_data_slice PASSED                 [  7%]
tests/test_buffer.py::BufferTest::test_pull_bytes PASSED                 [  8%]
tests/test_buffer.py::BufferTest::test_pull_bytes_negative PASSED        [  8%]
tests/test_buffer.py::BufferTest::test_pull_bytes_truncated PASSED       [  8%]
tests/test_buffer.py::BufferTest::test_pull_bytes_zero PASSED            [  9%]
tests/test_buffer.py::BufferTest::test_pull_uint16 PASSED                [  9%]
tests/test_buffer.py::BufferTest::test_pull_uint16_truncated PASSED      [  9%]
tests/test_buffer.py::BufferTest::test_pull_uint32 PASSED                [ 10%]
tests/test_buffer.py::BufferTest::test_pull_uint32_truncated PASSED      [ 10%]
tests/test_buffer.py::BufferTest::test_pull_uint64 PASSED                [ 10%]
tests/test_buffer.py::BufferTest::test_pull_uint64_truncated PASSED      [ 11%]
tests/test_buffer.py::BufferTest::test_pull_uint8 PASSED                 [ 11%]
tests/test_buffer.py::BufferTest::test_pull_uint8_truncated PASSED       [ 11%]
tests/test_buffer.py::BufferTest::test_push_bytes PASSED                 [ 12%]
tests/test_buffer.py::BufferTest::test_push_bytes_truncated PASSED       [ 12%]
tests/test_buffer.py::BufferTest::test_push_bytes_zero PASSED            [ 12%]
tests/test_buffer.py::BufferTest::test_push_uint16 PASSED                [ 13%]
tests/test_buffer.py::BufferTest::test_push_uint32 PASSED                [ 13%]
tests/test_buffer.py::BufferTest::test_push_uint64 PASSED                [ 13%]
tests/test_buffer.py::BufferTest::test_push_uint8 PASSED                 [ 14%]
tests/test_buffer.py::BufferTest::test_seek PASSED                       [ 14%]
tests/test_buffer.py::UintVarTest::test_pull_uint_var_truncated PASSED   [ 14%]
tests/test_buffer.py::UintVarTest::test_push_uint_var_too_big PASSED     [ 15%]
tests/test_buffer.py::UintVarTest::test_size_uint_var PASSED             [ 15%]
tests/test_buffer.py::UintVarTest::test_uint_var PASSED                  [ 15%]

Hi,

Thank you very much for developing this!

We are trying to use this as a test server in web-platform-tests, and I have one problem: the server throws with our certificates.

With web-platform.test.pem and web-platform.test.key in https://github.com/web-platform-tests/wpt/tree/master/tools/certs, it throws an BufferWriteError.

handle: <Handle _SelectorDatagramTransport._read_ready()>
Traceback (most recent call last):
  File "/usr/lib/python3.6/asyncio/events.py", line 145, in _run
    self._callback(*self._args)
  File "/usr/lib/python3.6/asyncio/selector_events.py", line 1068, in _read_ready
    self._protocol.datagram_received(data, addr)
  File "/home/yuta/work/aioquic/src/aioquic/asyncio/server.py", line 144, in datagram_received
    protocol.datagram_received(data, addr)
  File "/home/yuta/work/aioquic/src/aioquic/asyncio/protocol.py", line 135, in datagram_received
    self._quic.receive_datagram(cast(bytes, data), addr, now=self._loop.time())
  File "/home/yuta/work/aioquic/src/aioquic/quic/connection.py", line 870, in receive_datagram
    context, plain_payload
  File "/home/yuta/work/aioquic/src/aioquic/quic/connection.py", line 1929, in _payload_received
    frame_handler(context, frame_type, buf)
  File "/home/yuta/work/aioquic/src/aioquic/quic/connection.py", line 1308, in _handle_crypto_frame
    self.tls.handle_message(event.data, self._crypto_buffers)
  File "/home/yuta/work/aioquic/src/aioquic/tls.py", line 1339, in handle_message
    output_buf[Epoch.ONE_RTT],
  File "/home/yuta/work/aioquic/src/aioquic/tls.py", line 1831, in _server_handle_hello
    for x in [self.certificate] + self.certificate_chain
  File "/home/yuta/work/aioquic/src/aioquic/tls.py", line 921, in push_certificate
    buf, 3, partial(push_certificate_entry, buf), certificate.certificates
  File "/home/yuta/work/aioquic/src/aioquic/tls.py", line 461, in push_list
    func(value)
  File "/home/yuta/work/aioquic/src/aioquic/tls.py", line 917, in push_certificate_entry
    push_opaque(buf, 3, entry[0])
  File "/home/yuta/work/aioquic/src/aioquic/tls.py", line 480, in push_opaque
    buf.push_bytes(value)
aioquic._buffer.BufferWriteError: Write out of bounds
2020-04-30 19:49:35,467 INFO root QUIC event: <class 'aioquic.quic.events.ProtocolNegotiated'>

I found this can be workarounded by changing a buffer capacity in aioquic/quic/connection.py.

self._crypto_buffers = {
    tls.Epoch.INITIAL: Buffer(capacity=4096),
#  tls.Epoch.HANDSHAKE: Buffer(capacity=4096),
    tls.Epoch.HANDSHAKE: Buffer(capacity=16384),
    tls.Epoch.ONE_RTT: Buffer(capacity=4096),
}

Is this a known issue? Is there any clean solution?

Thank you!

I'm trying to download multiple resources with multiple streams using http3_client.py and http3_server.py. The purpose is to understand the benefit of avoiding Head of Line Blocking using QUIC.

I run the HTTP3 server:

python3 examples/http3_server.py --certificate tests/ssl_cert.pem --private-key tests/ssl_key.pem

I run the example client to perform four HTTP/3 requests:

python examples/http3_client.py --ca-certs tests/pycacert.pem --quic-log http3 https://localhost:4433/100000 https://localhost:4433/100000 https://localhost:4433/100000 https://localhost:4433/100000 

The result shows that the HTTP requests from the client are sent to the server in parallel, but the server will respond with only one active stream at any time.

I think based on QUIC protocol, we can have multiple active streams at the same time, right? I also find an public example from https://qvis.edm.uhasselt.be/#/files --> Option 4 Load a massive demo file.

I double checked the source code and I found the current implementation is directly writing all the response body to a byte array, it seems keeps sending the content in this bytearray until it is empty and then switch to start sending another stream.

Did I miss something or are there some configuration parameters I should look at?

Thanks!

Hi, I'm interested to use aioquic to compare the performance of TCP vs QUIC. Note that I don't really wish to use HTTP - I just want a QUIC python server that accepts a bunch of data from a client.

I have gone through the docs but unfortunately it's still not very clear to me how I should use the library to achieve what I want.

1. Can I use aioquic's QUIC API to achieve what I want?
2. If I am not using HTTP, do I still need to worry about certs and/or private keys?
3. Is there an example that creates a barebones QUIC server and client? If not, is there a flowchart I should refer to to get started?

I'm sorry if these questions seem to be too basic - in my head using QUIC on python would have been as simple as using a library to write a TCP like program in python - except the library handles the QUIC implementation. If aioquic isn't meant to serve that function, please close this issue. Otherwise, given a little help, I'd be glad to add to the examples or the docs.

Hi,

I have modified the server.py in the examples and instead of sending the template, I am reading a file and sending the value to the client. It is working perfectly fine for small amount of text, but as soon as I increase the text to around 10 mb, the transmission is still happening, but I am unable to see anything at the client side.

Can you please assist.

Thanks.

Based on the IETF spec for both CONNECT and CONNECT-UDP, the following header validation adjustments were made:

• :scheme and :path MUST be omitted
• :authority MUST? be present

https://quicwg.org/base-drafts/draft-ietf-quic-http.html#section-4.2 https://tools.ietf.org/id/draft-schinazi-masque-connect-udp-00.html#name-the-connect-udp-method

Based on these linked documents, while it's clear that :scheme and :path must be omitted in both cases, it's unclear if :authority is actually required or if that's just for HTTP/3. So perhaps some adjustments still need to be made, submitting here for your thoughts. Should :authority be validated or not?

Additionally, I was unable to get the examples to work using the documentation in the README, so I did not make any changes there and there's a possibility this PR will break that code.

$ venv/bin/pip install -e .
... (successful)
$ venv/bin/pip install aiofiles asgiref dnslib httpbin starlette wsproto
... (successful)
$ venv/bin/python examples/http3_client.py --ca-certs tests/pycacert.pem https://localhost:4433/
Traceback (most recent call last):
  File "examples/http3_client.py", line 12, in <module>
    import wsproto
ModuleNotFoundError: No module named 'wsproto'

It seems that by default localhost would be resolved into ipv6 address only (MacOS):

https://github.com/aiortc/aioquic/blob/229c0f1bfb6e663e79532047afcd53f8e1276061/src/aioquic/asyncio/client.py#L62

infos[0] is ipv6 info (not dual-stack). Is this intended?

This is ok when running the server from this repo as it's binding to ipv6. But it will not work (by default) when connecting to a server that runs on "127.0.0.1:4433" or "0.0.0.0:4433".

pyOpenSSL >= 22 changed X509StoreContextError constructor, this PR takes that into account. Related to #345, revert limitation set by 3930580b50831a034d21ee4689362188b21a4d6a.

This PR is an alternative approach to #342 to achieve universal wheels. All C code is replaced by Python, using memoryview for Buffer and cryptography.hazmat.bindings.openssl.binding for AEAD and HeaderProtection. Also some additional checks for existing buffer overrun issues in the C code have been implemented.

As far as performance is concerned, running the full test suite is even a bit fast: C code: 123.015s Python code: 121.727s

Hi! With 0.9.20 we're seeing the following test failures after switching to OpenSSL 3.

============================= test session starts ==============================
platform linux -- Python 3.10.8, pytest-7.1.3, pluggy-1.0.0
rootdir: /build/aioquic-0.9.20
collected 385 items                                                            

tests/test_asyncio.py ..........................                         [  6%]
tests/test_buffer.py .........................                           [ 13%]
tests/test_connection.py ............................................... [ 25%]
................................................................         [ 42%]
tests/test_crypto.py ............                                        [ 45%]
tests/test_crypto_draft_29.py ...........                                [ 48%]
tests/test_h0.py ...                                                     [ 48%]
tests/test_h3.py ........................................                [ 59%]
tests/test_logger.py ....                                                [ 60%]
tests/test_packet.py ..........................                          [ 67%]
tests/test_packet_builder.py ................                            [ 71%]
tests/test_rangeset.py .....................                             [ 76%]
tests/test_recovery.py .......                                           [ 78%]
tests/test_retry.py .                                                    [ 78%]
tests/test_stream.py ............................                        [ 85%]
tests/test_tls.py .......F..................................FF...        [ 98%]
tests/test_webtransport.py .......                                       [100%]

=================================== FAILURES ===================================
______________ ContextTest.test_handshake_with_certificate_error _______________

self = <tests.test_tls.ContextTest testMethod=test_handshake_with_certificate_error>

    def test_handshake_with_certificate_error(self):
        client = self.create_client(cafile=None)
        server = self.create_server()
    
        with self.assertRaises(tls.AlertBadCertificate) as cm:
            self._handshake(client, server)
>       self.assertEqual(str(cm.exception), "unable to get local issuer certificate")
E       AssertionError: 'a' != 'unable to get local issuer certificate'
E       - a
E       + unable to get local issuer certificate

tests/test_tls.py:433: AssertionError
_____________ VerifyCertificateTest.test_verify_certificate_chain ______________

self = <tests.test_tls.VerifyCertificateTest testMethod=test_verify_certificate_chain>

    def test_verify_certificate_chain(self):
        with open(SERVER_CERTFILE, "rb") as fp:
            certificate = load_pem_x509_certificates(fp.read())[0]
    
        with patch("aioquic.tls.utcnow") as mock_utcnow:
            mock_utcnow.return_value = certificate.not_valid_before
    
            # fail
            with self.assertRaises(tls.AlertBadCertificate) as cm:
                verify_certificate(certificate=certificate, server_name="localhost")
>           self.assertEqual(
                str(cm.exception), "unable to get local issuer certificate"
            )
E           AssertionError: 'a' != 'unable to get local issuer certificate'
E           - a
E           + unable to get local issuer certificate

tests/test_tls.py:1268: AssertionError
_______ VerifyCertificateTest.test_verify_certificate_chain_self_signed ________

self = <tests.test_tls.VerifyCertificateTest testMethod=test_verify_certificate_chain_self_signed>

    def test_verify_certificate_chain_self_signed(self):
        certificate, _ = generate_ec_certificate(
            common_name="localhost", curve=ec.SECP256R1
        )
    
        with patch("aioquic.tls.utcnow") as mock_utcnow:
            mock_utcnow.return_value = certificate.not_valid_before
    
            # fail
            with self.assertRaises(tls.AlertBadCertificate) as cm:
                verify_certificate(certificate=certificate, server_name="localhost")
>           self.assertEqual(str(cm.exception), "self signed certificate")
E           AssertionError: 'l' != 'self signed certificate'
E           - l
E           + self signed certificate

tests/test_tls.py:1290: AssertionError

Hi @jlaine,

First off thank you for the fantastic work you're doing here! 🍰 @meitinger has been working on HTTP/3 support for @mitmproxy over the summer and aioquic has worked out quite well for us. We're now close to merging it, but one blocking upstream issue is the lack of Python 3.11 wheels.

To make sure this is not an issue for every upcoming Python version, this PR converts the aioquic C extensions to only use the Python Limited API and the puts the necessary tooling in place to build abi3 wheels (see https://cibuildwheel.readthedocs.io/en/stable/faq/#abi3).

Code-wise I will say C is not my forte, but I have tried to switch to whatever https://github.com/python/cpython/blob/main/Modules/xxlimited.c or https://github.com/python/cpython/blob/3.7/Modules/xxlimited.c does. :)

We are working on comparing HTTP/2 and HTTP/3. As for HTTP/3 we are using aioquic implementation. We are generating metrics using other clients which require Access-Control-Allow-Origin in the header. We want to allow all origins. We tried adding this in the header in the example implementation but didn't get any success. Can someone help us with this issue?