WebRTC and ORTC implementation for Python using asyncio

Last update: Jan 7, 2023

Related tags

Networking python webrtc asyncio ortc webrtc-libraries data-channel

Overview

aiortc

What is `aiortc`?

aiortc is a library for Web Real-Time Communication (WebRTC) and Object Real-Time Communication (ORTC) in Python. It is built on top of asyncio, Python's standard asynchronous I/O framework.

The API closely follows its Javascript counterpart while using pythonic constructs:

promises are replaced by coroutines
events are emitted using pyee.EventEmitter

To learn more about aiortc please read the documentation.

Why should I use `aiortc`?

The main WebRTC and ORTC implementations are either built into web browsers, or come in the form of native code. While they are extensively battle tested, their internals are complex and they do not provide Python bindings. Furthermore they are tightly coupled to a media stack, making it hard to plug in audio or video processing algorithms.

In contrast, the aiortc implementation is fairly simple and readable. As such it is a good starting point for programmers wishing to understand how WebRTC works or tinker with its internals. It is also easy to create innovative products by leveraging the extensive modules available in the Python ecosystem. For instance you can build a full server handling both signaling and data channels or apply computer vision algorithms to video frames using OpenCV.

Furthermore, a lot of effort has gone into writing an extensive test suite for the aiortc code to ensure best-in-class code quality.

Implementation status

aiortc allows you to exchange audio, video and data channels and interoperability is regularly tested against both Chrome and Firefox. Here are some of its features:

SDP generation / parsing
Interactive Connectivity Establishment, with half-trickle and mDNS support
DTLS key and certificate generation
DTLS handshake, encryption / decryption (for SCTP)
SRTP keying, encryption and decryption for RTP and RTCP
Pure Python SCTP implementation
Data Channels
Sending and receiving audio (Opus / PCMU / PCMA)
Sending and receiving video (VP8 / H.264)
Bundling audio / video / data channels
RTCP reports, including NACK / PLI to recover from packet loss

Installing

Since release 0.9.28 binary wheels are available on PyPI for Linux, Mac and Windows. The easiest way to install aiortc is to run:

pip install aiortc

Building from source

If there are no wheels for your system or if you wish to build aiortc from source you will need a couple of libraries installed on your system:

OpenSSL 1.0.2 or greater
FFmpeg 4.0 or greater
LibVPX for video encoding / decoding
Opus for audio encoding / decoding

Linux

On Debian/Ubuntu run:

apt install libavdevice-dev libavfilter-dev libopus-dev libvpx-dev pkg-config

pylibsrtp comes with binary wheels for most platforms, but if it needs to be built from you will also need to run:

apt install libsrtp2-dev

OS X

On OS X run:

brew install ffmpeg opus libvpx pkg-config

License

aiortc is released under the BSD license.

Comments

feat: creating media player that works without transcoding

I've tested janus example. However, I noticed that always transcode process happens, even though input codec is the same of output. I checked cpu processing consumption with transcode is ~60%.

In this PR I add an option in example that transcode does not happens. With this, cpu processing reduce to ~3% and has the same visual result. Also, latency reduce ~1.5s for video in Full HD (1920x1080). In this implementation, native media player uses h264 codec.

opened by rprata 47
Data Channel transfer rates are a bit low

Data Channel transfer rates are somewhat low, even when sending locally on the same machine.

Using the datachannel-filexfer example to send a compressed 64-megabyte file, it takes about 90 to 120-seconds. I suppose that is about 5-mbps. A chrome-to-chrome local implementation with the same file is about 45-mbps. ie) https://webrtc.github.io/samples/src/content/datachannel/filetransfer/

My goal is to offer a hosted quality-of-service webRTC-focused speed test, offering insights into maximum webRTC UDP connection speeds and packet loss information with both upload and download. I do not yet understand how I might be able to access connection analytics, but I suppose perhaps aiortc is not the ideal solution for what I am trying to do anyways? Either way, I am loving what I am seeing so far regardless. This is Fantastic stuff!
enhancement question

opened by steveseguin 45
Video playback delay in server example

Hi, I am experiencing a delay, especially in chrome web browser while running the server example as is. I am running the server and on the same network (wifi) trying to view the video on a web client. It seems that over time there is an "accumulated" delay of the video, although i do not apply any filter on the frames of the video data so the only delay here is the round trip of the frame.

Should I fine-tune the frame rate of the video data that is being sent? Maybe the issue is with a queue of the frames to render on the client side, in which case, can i drop the queue?

Thanks!

opened by mikelibg 26

Created audio tracks never call `recv()`

I'll preface this by saying the default server example included with the repo works fine.

I have a webapp that originally used websockets to transfer audio along with other data. I'm trying to move the audio portion over to WebRTC, with the session signalling being done via the existing websocket connection. I'm essentially just adapting the server example to do bidirectional audio and removing the video portion entirely.

I might be misunderstanding how to do things correctly, but here is the python code on the server that responds to the SDP offer and sets up the audio tracks. It's almost identical to the example server.py:

class MicStreamTrack(MediaStreamTrack):
    """
    An audio stream object for the mic audio from the client
    """
    kind = "audio"

    def __init__(self, track):
        super().__init__()
        self.track = track

    async def recv(self):
        logger.logVerbose("mic recv()")

        # Get a new PyAV frame
        frame = await self.track.recv()

        # Convert to float32 numpy array
        floatArray = frame.to_ndarray(format="float32")

        # Put these samples into the mic queue
        micSampleQueue.put_nowait(floatArray)

        logger.logVerbose("Put {} samples to mic queue".format(len(floatArray)))

class SpkrStreamTrack(MediaStreamTrack):
    """
    An audio stream object for the speaker data from the server
    """
    kind = "audio"

    def __init__(self):
        super().__init__()

    async def recv():
        logger.logVerbose("spkr recv()")

        # Get samples from speaker queue if available
        floatArray = await spkrSampleQueue.get()

        # Convert to audio 
        frame = AudioFrame.from_ndarray(floatArray, format='float', layout='mono')

        logger.logVerbose("Got {} speaker samples".format(frame.samples))

        # Return
        return frame

async def gotRtcOffer(offerObj):
    """
    Called when we receive a WebRTC offer from the client

    Args:
        offerObj (dict): WebRTC SDP offer object
    """

    global rtcPeer

    logger.logInfo("Got WebRTC offer")
    
    # Create SDP offer and peer connection objects
    offer = RTCSessionDescription(sdp=offerObj["sdp"], type=offerObj["type"])
    rtcPeer = RTCPeerConnection()

    # Create UUID for peer
    pcUuid = "PeerConnection({})".format(uuid.uuid4())
    logger.logVerbose("Creating peer connection {}".format(pcUuid))

    # ICE connection state callback
    @rtcPeer.on("iceconnectionstatechange")
    async def onIceConnectionStateChange():
        logger.logVerbose("Ice connection state is now {}".format(rtcPeer.iceConnectionState))
        if rtcPeer.iceConnectionState == "failed":
            await rtcPeer.close()
            logger.logError("WebRTC peer connection {} failed".format(pcUuid))
    
    # Audio track callback
    @rtcPeer.on("track")
    def onTrack(track):

        global micStream
        global rtcPeer

        logger.logVerbose("Got {} track from peer {}".format(track.kind, pcUuid))

        # make sure it's audio
        if track.kind != "audio":
            logger.logError("Got non-audio track from peer {}".format(pcUuid))
            return
        
        # Create the mic stream for this track
        micStream = MicStreamTrack(track)
        logger.logVerbose("Added mic track")

        # Send the speaker stream back
        spkrTrack = SpkrStreamTrack()
        rtcPeer.addTrack(spkrTrack)
        logger.logVerbose("Added speaker track")

        # Track ended handler
        @track.on("ended")
        async def onEnded():
            logger.logVerbose("Audio track from {} ended".format(pcUuid))

    # Handle the received offer
    logger.logVerbose("Creating remote description from offer")
    await rtcPeer.setRemoteDescription(offer)

    # Create answer
    logger.logVerbose("Creating WebRTC answer")
    answer = await rtcPeer.createAnswer()

    # Set local description
    logger.logVerbose("setting local SDP")
    await rtcPeer.setLocalDescription(answer)

    # Send answer
    logger.logVerbose("sending SDP answer")
    message = '{{ "webRtcAnswer": {{ "type": "{}", "sdp": {} }} }}'.format(rtcPeer.localDescription.type, json.dumps(rtcPeer.localDescription.sdp))
    messageQueue.put_nowait(message)

    logger.logVerbose("done")

All of the expected SDP log messages occur, and it appears on both ends (the web browser and the python script) that everything is set up properly. Both tracks are added and it looks like the connection goes fully active:

[09/22 13:09:56] (  gotRtcOffer   ) INFO: Got WebRTC offer
[09/22 13:09:56] (  gotRtcOffer   ) VERB: Creating peer connection PeerConnection(514e2bad-a415-412f-b741-1e16ab82b023)
[09/22 13:09:56] (  gotRtcOffer   ) VERB: Creating remote description from offer
[09/22 13:09:56] (    onTrack     ) VERB: Got audio track from peer PeerConnection(514e2bad-a415-412f-b741-1e16ab82b023)
[09/22 13:09:56] (    onTrack     ) VERB: Added mic track
[09/22 13:09:56] (    onTrack     ) VERB: Added speaker track
[09/22 13:09:56] (  gotRtcOffer   ) VERB: Creating WebRTC answer
[09/22 13:09:56] (  gotRtcOffer   ) VERB: setting local SDP
[09/22 13:10:01] (  gotRtcOffer   ) VERB: sending SDP answer
[09/22 13:10:01] (  gotRtcOffer   ) VERB: done
[09/22 13:10:01] (producer_hander ) INFO: sending WebRTC answer to ::1
[09/22 13:10:01] (onIceConnectionStateChange) VERB: Ice connection state is now checking
[09/22 13:10:02] (onIceConnectionStateChange) VERB: Ice connection state is now completed

I can even see the data being sent from the webpage to the server via the chrome WebRTC internals. However, the recv() callbacks on the python side for either track are never called. I'd expect to see a ton of prints as data is flowing every time the tracks get new data, but nothing ever happens after the RTC session connects. No errors occur on either the webpage or the python script either.

My limited understanding of WebRTC is preventing me from being able to figure out what's going wrong. Help would be greatly appreciated.

opened by W3AXL 24

record with MediaRecorder from janus h264 Stream Invalid data found when processing input; last error log: [h264] no frame!

Hello,

i try to record a h264 stream from janus-gateway (master version)

after i get the video track the encoder shows many

non-existing PPS 17 referenced
decode_slice_header error
non-existing PPS 17 referenced
decode_slice_header error
non-existing PPS 17 referenced
decode_slice_header error
non-existing PPS 17 referenced
decode_slice_header error
non-existing PPS 17 referenced
decode_slice_header error
non-existing PPS 17 referenced
decode_slice_header error
non-existing PPS 17 referenced
decode_slice_header error
non-existing PPS 17 referenced
decode_slice_header error
non-existing PPS 17 referenced
decode_slice_header error
non-existing PPS 17 referenced
decode_slice_header error
non-existing PPS 17 referenced
decode_slice_header error
non-existing PPS 17 referenced
decode_slice_header error
non-existing PPS 17 referenced
decode_slice_header error
non-existing PPS 17 referenced
decode_slice_header error
non-existing PPS 17 referenced
decode_slice_header error
non-existing PPS 17 referenced
decode_slice_header error
non-existing PPS 17 referenced
decode_slice_header error
non-existing PPS 17 referenced
decode_slice_header error
non-existing PPS 17 referenced
decode_slice_header error
non-existing PPS 17 referenced
decode_slice_header error
non-existing PPS 17 referenced
decode_slice_header error
non-existing PPS 17 referenced
decode_slice_header error
non-existing PPS 17 referenced
decode_slice_header error
non-existing PPS 17 referenced
decode_slice_header error
non-existing PPS 17 referenced
decode_slice_header error
non-existing PPS 17 referenced
decode_slice_header error
non-existing PPS 17 referenced
decode_slice_header error
non-existing PPS 17 referenced
decode_slice_header error
non-existing PPS 17 referenced
decode_slice_header error
non-existing PPS 17 referenced
decode_slice_header error
non-existing PPS 17 referenced
decode_slice_header error
non-existing PPS 17 referenced
decode_slice_header error
non-existing PPS 17 referenced
decode_slice_header error
non-existing PPS 17 referenced
decode_slice_header error
non-existing PPS 17 referenced
decode_slice_header error
non-existing PPS 17 referenced
decode_slice_header error
non-existing PPS 17 referenced
decode_slice_header error
non-existing PPS 17 referenced
decode_slice_header error
non-existing PPS 17 referenced
decode_slice_header error
non-existing PPS 17 referenced
decode_slice_header error
non-existing PPS 17 referenced
decode_slice_header error
non-existing PPS 17 referenced
decode_slice_header error
non-existing PPS 17 referenced
decode_slice_header error
non-existing PPS 17 referenced
decode_slice_header error
non-existing PPS 17 referenced
decode_slice_header error
non-existing PPS 17 referenced
decode_slice_header error
non-existing PPS 17 referenced
decode_slice_header error
non-existing PPS 17 referenced
decode_slice_header error
non-existing PPS 17 referenced
decode_slice_header error
non-existing PPS 17 referenced
decode_slice_header error
non-existing PPS 17 referenced
decode_slice_header error
non-existing PPS 17 referenced
decode_slice_header error
non-existing PPS 17 referenced
decode_slice_header error
non-existing PPS 17 referenced
decode_slice_header error
non-existing PPS 17 referenced
decode_slice_header error
non-existing PPS 17 referenced
decode_slice_header error
non-existing PPS 17 referenced
decode_slice_header error
non-existing PPS 17 referenced
decode_slice_header error
non-existing PPS 17 referenced
decode_slice_header error
non-existing PPS 17 referenced
decode_slice_header error
non-existing PPS 17 referenced
decode_slice_header error
non-existing PPS 17 referenced
decode_slice_header error
non-existing PPS 17 referenced
decode_slice_header error
non-existing PPS 17 referenced
decode_slice_header error
non-existing PPS 17 referenced
decode_slice_header error
non-existing PPS 17 referenced
decode_slice_header error
non-existing PPS 17 referenced
decode_slice_header error
non-existing PPS 17 referenced
decode_slice_header error
no frame!
failed to decode, skipping package: [Errno 1094995529] Invalid data found when processing input; last error log: [h264] no frame!
non-existing PPS 17 referenced
decode_slice_header error
no frame!
failed to decode, skipping package: [Errno 1094995529] Invalid data found when processing input; last error log: [h264] no frame!
non-existing PPS 17 referenced
decode_slice_header error
no frame!
failed to decode, skipping package: [Errno 1094995529] Invalid data found when processing input; last error log: [h264] no frame!
non-existing PPS 17 referenced
decode_slice_header error
no frame!
failed to decode, skipping package: [Errno 1094995529] Invalid data found when processing input; last error log: [h264] no frame!
non-existing PPS 17 referenced
decode_slice_header error
no frame!
failed to decode, skipping package: [Errno 1094995529] Invalid data found when processing input; last error log: [h264] no frame!

any hint how i can debug or fix this?

Python 3.7.6 aiortc 0.9.26

stale

opened by arpu 24

Add a Websocket-based signaling method for examples

aiortc datachannel-filexfer example is very very nice example for developers interested in WebRTC and needs reliable p2p direct communication with udp hole punching (data channel is over SCTP and reliable transport) !!!. But it can not run on Windows platform due to asyncio restriction of pipe. So I implemented datachannel-filexfer example using websocket signaling server. (signaling server is copy of code on aioice example )

https://github.com/ryogrid/aiortc_filexfer_sample_for_win32/commit/e69c055abe21c3abf9f5913ab0dca609c9e23a5b

if it has problem for pull request, please comment me :-) (if this kind post to issues is bad action, sorry...)
enhancement

opened by ryogrid 23
Using preferred codecs list to configure local RTP stream. Adding possibility to use an encoded stream track without re-encoding

this commit allows setting up the preferred codecs to be used for the local RTP stream. By default aiortc uses VP8 but on some platform one may enforce the use of H264 to minimize CPU load (on embedded platforms such as the raspberry pi).

opened by jpiat 22
Webcam.py --video-codec video/H264 --play-without-decoding weird behavior
I encountered a strange behavior regarding the standard webcam.py example. Im using the newest feature " --video-codec video/H264 --play-without-decoding" on my raspberry pi. If im connecting from any client over the local network (android/ios/chome/firefox) it works the first time, after that only chrome on my windows machine works but the image quality gets signficantly worse than the first connect. Steps to reproduce:

start python webcam.py --video-codec video/H264 --play-without-decoding

connect from firefox/ios/android browser with start

press stop button

refresh and press start again

I verified ice gathering is working and even the h264_omx encoder is starting to run but no feed is shown on the client side. Only Chrome is working and is displaying a much blurrier image .
stale
opened by unitw 17
Performance optimization for `_split_bitstream`
In profiling performance of #559, I noticed that a ton of time is spent in _split_bitstream. So I set out to speed it up. In my test case (passing through un-transcoded h264 1080p), it's now 280x faster than it was. I think it's also a lot easier to read.

Here's what I'm using to profile

python -m cProfile -o out.profile webcam.py snakeviz -s -H 0.0.0.0 out.profile
opened by johnboiles 17
aiortc failing websocket connection in the closed state 1006

aiortc failed after working 8 minutes when I connect it to chrome in controlling mode using google apprtc server. Howerver aiortc works well for a long time if I use chrome login in the room at the beginning and aiortc login in the same room secondly. aiortc failed because it's websocket heart-beat check failed and exit the connection. It doesn't reconnect the apprtc server. Is there a reconnect setting ? I attached part of the log here.

DEBUG:rtp:receiver(video) > RtcpPsfbPacket(fmt=15, ssrc=2596041375, media_ssrc=0, fci=b'REMB\x01\x00gh\\x90\x0c\x9f') DEBUG:ice:Connection(0) protocol(1) > ('192.168.12.46', 55563) DATA 38 DEBUG:ice:Connection(0) protocol(1) < ('192.168.12.46', 55563) DATA 164 DEBUG:rtp:sender(video) > RtpPacket(seq=18088, ts=2365362046, marker=0, payload=96, 1300 bytes) DEBUG:ice:Connection(0) protocol(1) > ('192.168.12.46', 55563) DATA 1330 DEBUG:rtp:sender(video) > RtpPacket(seq=18089, ts=2365362046, marker=1, payload=96, 406 bytes) DEBUG:ice:Connection(0) protocol(1) > ('192.168.12.46', 55563) DATA 436 DEBUG:rtp:receiver(video) < RtpPacket(seq=46785, ts=4163426291, marker=1, payload=96, 130 bytes) DEBUG:ice:Connection(0) protocol(1) < ('192.168.12.46', 55563) DATA 94 DEBUG:ice:Connection(0) protocol(1) < ('192.168.12.46', 55563) DATA 158 DEBUG:rtp:receiver(video) < RtcpSrPacket(ssrc=1552944287, sender_info=RtcpSenderInfo(ntp_timestamp=16134187896596091724, rtp_timestamp=4163428361, packet_count=31048, octet_count=25439093), reports=[]) DEBUG:rtp:receiver(video) < RtcpSdesPacket(chunks=[RtcpSourceInfo(ssrc=1552944287, items=[(1, b'fu4pTxKHM5aZ9ynp')])]) DEBUG:rtp:receiver(video) < RtcpPsfbPacket(fmt=15, ssrc=1552944287, media_ssrc=0, fci=b'REMB\x01\x06\xb8\xb9\x9a\xbcr\x9f') DEBUG:rtp:sender(video) - receiver estimated maximum bitrate 356722 bps DEBUG:rtp:receiver(video) < RtpPacket(seq=46786, ts=4163429171, marker=1, payload=96, 124 bytes) DEBUG:ice:Connection(0) protocol(1) < ('192.168.12.46', 55563) DATA 230 DEBUG:rtp:receiver(video) < RtpPacket(seq=46787, ts=4163432051, marker=1, payload=96, 196 bytes) DEBUG:rtp:receiver(video) > RtcpPsfbPacket(fmt=15, ssrc=2596041375, media_ssrc=0, fci=b'REMB\x01\x00i\xb1\\x90\x0c\x9f') DEBUG:ice:Connection(0) protocol(1) > ('192.168.12.46', 55563) DATA 38 DEBUG:rtp:sender(video) > RtpPacket(seq=18090, ts=2365365046, marker=0, payload=96, 1300 bytes) DEBUG:ice:Connection(0) protocol(1) > ('192.168.12.46', 55563) DATA 1330 DEBUG:rtp:sender(video) > RtpPacket(seq=18091, ts=2365365046, marker=1, payload=96, 48 bytes) DEBUG:ice:Connection(0) protocol(1) > ('192.168.12.46', 55563) DATA 78 DEBUG:rtp:sender(video) > RtpPacket(seq=18092, ts=2365368046, marker=0, payload=96, 1300 bytes) DEBUG:ice:Connection(0) protocol(1) > ('192.168.12.46', 55563) DATA 1330 DEBUG:rtp:sender(video) > RtpPacket(seq=18093, ts=2365368046, marker=1, payload=96, 300 bytes) DEBUG:ice:Connection(0) protocol(1) > ('192.168.12.46', 55563) DATA 330 DEBUG:ice:Connection(0) protocol(1) < ('192.168.12.46', 55563) DATA 132 DEBUG:rtp:receiver(video) < RtpPacket(seq=46788, ts=4163435291, marker=1, payload=96, 98 bytes) DEBUG:ice:Connection(0) protocol(1) < ('192.168.12.46', 55563) DATA 166 DEBUG:rtp:receiver(video) < RtpPacket(seq=46789, ts=4163438171, marker=1, payload=96, 132 bytes) DEBUG:rtp:sender(video) > RtcpSrPacket(ssrc=2596041375, sender_info=RtcpSenderInfo(ntp_timestamp=16134188147075088323, rtp_timestamp=2365368046, packet_count=13842, octet_count=14083911), reports=[]) DEBUG:rtp:sender(video) > RtcpSdesPacket(chunks=[RtcpSourceInfo(ssrc=2596041375, items=[(1, b'{0e1b3b93-8830-437d-b8a0-367f2c1f4d8f}')])]) DEBUG:ice:Connection(0) protocol(1) > ('192.168.12.46', 55563) DATA 94 DEBUG:ice:Connection(0) protocol(1) < ('192.168.12.46', 55563) DATA 201 DEBUG:rtp:receiver(video) < RtpPacket(seq=46790, ts=4163441051, marker=1, payload=96, 167 bytes) DEBUG:rtp:sender(video) > RtpPacket(seq=18094, ts=2365371046, marker=1, payload=96, 1177 bytes) DEBUG:ice:Connection(0) protocol(1) > ('192.168.12.46', 55563) DATA 1207 DEBUG:rtp:sender(video) > RtpPacket(seq=18095, ts=2365374046, marker=0, payload=96, 1300 bytes) DEBUG:ice:Connection(0) protocol(1) > ('192.168.12.46', 55563) DATA 1330 DEBUG:rtp:sender(video) > RtpPacket(seq=18096, ts=2365374046, marker=1, payload=96, 346 bytes) DEBUG:ice:Connection(0) protocol(1) > ('192.168.12.46', 55563) DATA 376 DEBUG:ice:Connection(0) protocol(1) < ('192.168.12.46', 55563) DATA 160 DEBUG:rtp:receiver(video) < RtpPacket(seq=46791, ts=4163444291, marker=1, payload=96, 126 bytes) DEBUG:ice:Connection(0) protocol(1) < ('192.168.12.46', 55563) DATA 211 DEBUG:rtp:receiver(video) < RtpPacket(seq=46792, ts=4163447171, marker=1, payload=96, 177 bytes) DEBUG:rtp:sender(video) > RtpPacket(seq=18097, ts=2365377046, marker=1, payload=96, 1156 bytes) DEBUG:ice:Connection(0) protocol(1) > ('192.168.12.46', 55563) DATA 1186 DEBUG:ice:Connection(0) protocol(1) < ('192.168.12.46', 55563) DATA 94 DEBUG:ice:Connection(0) protocol(1) < ('192.168.12.46', 55563) DATA 154 DEBUG:rtp:receiver(video) < RtcpSrPacket(ssrc=1552944287, sender_info=RtcpSenderInfo(ntp_timestamp=16134187897622417109, rtp_timestamp=4163449871, packet_count=31055, octet_count=25440113), reports=[]) DEBUG:rtp:receiver(video) < RtcpSdesPacket(chunks=[RtcpSourceInfo(ssrc=1552944287, items=[(1, b'fu4pTxKHM5aZ9ynp')])]) DEBUG:rtp:receiver(video) < RtcpPsfbPacket(fmt=15, ssrc=1552944287, media_ssrc=0, fci=b'REMB\x01\x06\xb8\xb9\x9a\xbcr\x9f') DEBUG:rtp:sender(video) - receiver estimated maximum bitrate 356722 bps DEBUG:rtp:receiver(video) < RtpPacket(seq=46793, ts=4163450051, marker=1, payload=96, 120 bytes) DEBUG:ice:Connection(0) protocol(1) < ('192.168.12.46', 55563) DATA 123 DEBUG:rtp:receiver(video) < RtpPacket(seq=46794, ts=4163453291, marker=1, payload=96, 89 bytes) DEBUG:rtp:sender(video) > RtpPacket(seq=18098, ts=2365380046, marker=0, payload=96, 1300 bytes) DEBUG:ice:Connection(0) protocol(1) > ('192.168.12.46', 55563) DATA 1330 DEBUG:rtp:sender(video) > RtpPacket(seq=18099, ts=2365380046, marker=1, payload=96, 460 bytes) DEBUG:ice:Connection(0) protocol(1) > ('192.168.12.46', 55563) DATA 490 DEBUG:ice:Connection(0) protocol(1) < ('192.168.12.46', 55563) DATA 168 DEBUG:rtp:receiver(video) < RtpPacket(seq=46795, ts=4163456171, marker=1, payload=96, 134 bytes) DEBUG:rtp:receiver(video) > RtcpPsfbPacket(fmt=15, ssrc=2596041375, media_ssrc=0, fci=b'REMB\x01\x00k<\\x90\x0c\x9f') DEBUG:ice:Connection(0) protocol(1) > ('192.168.12.46', 55563) DATA 38 DEBUG:rtp:sender(video) > RtpPacket(seq=18100, ts=2365383046, marker=1, payload=96, 1227 bytes) DEBUG:ice:Connection(0) protocol(1) > ('192.168.12.46', 55563) DATA 1257 DEBUG:rtp:receiver(video) > RtcpRrPacket(ssrc=2596041375, reports=[RtcpReceiverInfo(ssrc=1552944287, fraction_lost=0, packets_lost=0, highest_sequence=46795, jitter=1783, lsr=644417840, dlsr=5232)]) DEBUG:ice:Connection(0) protocol(1) > ('192.168.12.46', 55563) DATA 46 DEBUG:ice:Connection(0) protocol(1) < ('192.168.12.46', 55563) DATA 172 DEBUG:dtls:server x DTLS handling timeout DEBUG:rtp:receiver(video) < RtpPacket(seq=46796, ts=4163459051, marker=1, payload=96, 138 bytes) DEBUG:ice:Connection(0) protocol(1) < ('192.168.12.46', 55563) DATA 233 DEBUG:rtp:receiver(video) < RtpPacket(seq=46797, ts=4163462291, marker=1, payload=96, 199 bytes) DEBUG:rtp:sender(video) > RtpPacket(seq=18101, ts=2365386046, marker=0, payload=96, 1300 bytes) DEBUG:ice:Connection(0) protocol(1) > ('192.168.12.46', 55563) DATA 1330 DEBUG:rtp:sender(video) > RtpPacket(seq=18102, ts=2365386046, marker=1, payload=96, 469 bytes) DEBUG:ice:Connection(0) protocol(1) > ('192.168.12.46', 55563) DATA 499 DEBUG:ice:Connection(0) protocol(1) < ('192.168.12.46', 55563) DATA 46 DEBUG:ice:Connection(0) protocol(1) < ('192.168.12.46', 55563) DATA 141 DEBUG:rtp:receiver(video) < RtcpRrPacket(ssrc=1552944287, reports=[RtcpReceiverInfo(ssrc=2596041375, fraction_lost=0, packets_lost=0, highest_sequence=18102, jitter=2118, lsr=648224186, dlsr=16801)]) DEBUG:rtp:receiver(video) < RtpPacket(seq=46798, ts=4163465171, marker=1, payload=96, 107 bytes) DEBUG:ice:Connection(0) protocol(1) < ('192.168.12.46', 55563) DATA 108 DEBUG:rtp:receiver(video) < RtpPacket(seq=46799, ts=4163468051, marker=1, payload=96, 74 bytes) DEBUG:rtp:sender(video) > RtpPacket(seq=18103, ts=2365389046, marker=0, payload=96, 1300 bytes) DEBUG:ice:Connection(0) protocol(1) > ('192.168.12.46', 55563) DATA 1330 DEBUG:rtp:sender(video) > RtpPacket(seq=18104, ts=2365389046, marker=1, payload=96, 47 bytes) DEBUG:ice:Connection(0) protocol(1) > ('192.168.12.46', 55563) DATA 77 DEBUG:ice:Connection(0) protocol(1) < ('192.168.12.46', 55563) DATA 153 DEBUG:ice:Connection(0) protocol(1) < ('192.168.12.46', 55563) DATA 94 DEBUG:ice:Connection(0) protocol(1) < ('192.168.12.46', 55563) DATA 142 DEBUG:rtp:receiver(video) < RtpPacket(seq=46800, ts=4163471291, marker=1, payload=96, 119 bytes) DEBUG:rtp:receiver(video) > RtcpPsfbPacket(fmt=15, ssrc=2596041375, media_ssrc=0, fci=b'REMB\x01\x00e\xb5\\x90\x0c\x9f') DEBUG:ice:Connection(0) protocol(1) > ('192.168.12.46', 55563) DATA 38 DEBUG:rtp:receiver(video) < RtcpSrPacket(ssrc=1552944287, sender_info=RtcpSenderInfo(ntp_timestamp=16134187898695072306, rtp_timestamp=4163472371, packet_count=31063, octet_count=25441093), reports=[]) DEBUG:rtp:receiver(video) < RtcpSdesPacket(chunks=[RtcpSourceInfo(ssrc=1552944287, items=[(1, b'fu4pTxKHM5aZ9ynp')])]) DEBUG:rtp:receiver(video) < RtcpPsfbPacket(fmt=15, ssrc=1552944287, media_ssrc=0, fci=b'REMB\x01\x06\xb8\xb9\x9a\xbcr\x9f') DEBUG:rtp:sender(video) - receiver estimated maximum bitrate 356722 bps DEBUG:rtp:receiver(video) < RtpPacket(seq=46801, ts=4163474171, marker=1, payload=96, 108 bytes) DEBUG:rtp:receiver(video) > RtcpPsfbPacket(fmt=15, ssrc=2596041375, media_ssrc=0, fci=b'REMB\x01\x00d\xa5\\x90\x0c\x9f') DEBUG:ice:Connection(0) protocol(1) > ('192.168.12.46', 55563) DATA 38 DEBUG:ice:Connection(0) protocol(1) < ('192.168.12.46', 55563) DATA 154 DEBUG:ice:Connection(0) protocol(1) > ('192.168.12.46', 55563) Message(message_method=Method.BINDING, message_class=Class.REQUEST, transaction_id=b'\xdb?B6\xb6QzE)\x0b~{') DEBUG:rtp:receiver(video) < RtpPacket(seq=46802, ts=4163477051, marker=1, payload=96, 120 bytes) DEBUG:ice:Connection(0) protocol(1) < ('192.168.12.46', 55563) Message(message_method=Method.BINDING, message_class=Class.RESPONSE, transaction_id=b'\xdb?B6\xb6QzE)\x0b~{') DEBUG:rtp:sender(video) > RtpPacket(seq=18105, ts=2365392046, marker=0, payload=96, 1300 bytes) DEBUG:ice:Connection(0) protocol(1) > ('192.168.12.46', 55563) DATA 1330 DEBUG:rtp:sender(video) > RtpPacket(seq=18106, ts=2365392046, marker=1, payload=96, 288 bytes) DEBUG:ice:Connection(0) protocol(1) > ('192.168.12.46', 55563) DATA 318 DEBUG:ice:Connection(0) protocol(1) < ('192.168.12.46', 55563) DATA 148 DEBUG:rtp:receiver(video) < RtpPacket(seq=46803, ts=4163480291, marker=1, payload=96, 114 bytes) DEBUG:ice:Connection(0) protocol(1) < ('192.168.12.46', 55563) DATA 174 DEBUG:rtp:receiver(video) < RtpPacket(seq=46804, ts=4163483171, marker=1, payload=96, 140 bytes) DEBUG:rtp:sender(video) > RtpPacket(seq=18107, ts=2365395046, marker=1, payload=96, 1045 bytes) DEBUG:ice:Connection(0) protocol(1) > ('192.168.12.46', 55563) DATA 1075 DEBUG:ice:Connection(0) protocol(1) < ('192.168.12.46', 55563) DATA 139 DEBUG:rtp:receiver(video) < RtpPacket(seq=46805, ts=4163486051, marker=1, payload=96, 105 bytes) DEBUG:rtp:sender(video) > RtpPacket(seq=18108, ts=2365398046, marker=0, payload=96, 1300 bytes) DEBUG:ice:Connection(0) protocol(1) > ('192.168.12.46', 55563) DATA 1330 DEBUG:rtp:sender(video) > RtpPacket(seq=18109, ts=2365398046, marker=1, payload=96, 649 bytes) DEBUG:ice:Connection(0) protocol(1) > ('192.168.12.46', 55563) DATA 679 DEBUG:ice:Connection(0) protocol(1) < ('192.168.12.46', 55563) DATA 190 DEBUG:ice:Connection(0) protocol(1) < ('192.168.12.46', 55563) Message(message_method=Method.BINDING, message_class=Class.REQUEST, transaction_id=b'k8ZKJu/T8jxQ') DEBUG:ice:Connection(0) protocol(1) > ('192.168.12.46', 55563) Message(message_method=Method.BINDING, message_class=Class.RESPONSE, transaction_id=b'k8ZKJu/T8jxQ') DEBUG:rtp:receiver(video) < RtpPacket(seq=46806, ts=4163489291, marker=1, payload=96, 156 bytes) DEBUG:ice:Connection(0) protocol(1) < ('192.168.12.46', 55563) DATA 174 DEBUG:rtp:sender(video) > RtpPacket(seq=18110, ts=2365401046, marker=1, payload=96, 1295 bytes) DEBUG:ice:Connection(0) protocol(1) > ('192.168.12.46', 55563) DATA 1325 DEBUG:rtp:receiver(video) < RtpPacket(seq=46807, ts=4163492171, marker=1, payload=96, 140 bytes) DEBUG:ice:Connection(0) protocol(1) < ('192.168.12.46', 55563) DATA 94 DEBUG:rtp:receiver(video) < RtcpSrPacket(ssrc=1552944287, sender_info=RtcpSenderInfo(ntp_timestamp=16134187899775097667, rtp_timestamp=4163495051, packet_count=31070, octet_count=25441976), reports=[]) DEBUG:rtp:receiver(video) < RtcpSdesPacket(chunks=[RtcpSourceInfo(ssrc=1552944287, items=[(1, b'fu4pTxKHM5aZ9ynp')])]) DEBUG:rtp:receiver(video) < RtcpPsfbPacket(fmt=15, ssrc=1552944287, media_ssrc=0, fci=b'REMB\x01\x06\xb8\xb9\x9a\xbcr\x9f') DEBUG:rtp:sender(video) - receiver estimated maximum bitrate 356722 bps DEBUG:ice:Connection(0) protocol(1) < ('192.168.12.46', 55563) DATA 175 DEBUG:rtp:receiver(video) < RtpPacket(seq=46808, ts=4163495051, marker=1, payload=96, 141 bytes) DEBUG:ice:Connection(0) protocol(1) < ('192.168.12.46', 55563) DATA 143 DEBUG:rtp:receiver(video) < RtpPacket(seq=46809, ts=4163498291, marker=1, payload=96, 109 bytes) DEBUG:rtp:sender(video) > RtpPacket(seq=18111, ts=2365404046, marker=0, payload=96, 1300 bytes) DEBUG:ice:Connection(0) protocol(1) > ('192.168.12.46', 55563) DATA 1330 DEBUG:rtp:sender(video) > RtpPacket(seq=18112, ts=2365404046, marker=1, payload=96, 282 bytes) DEBUG:ice:Connection(0) protocol(1) > ('192.168.12.46', 55563) DATA 312 DEBUG:ice:Connection(0) protocol(1) < ('192.168.12.46', 55563) DATA 189 DEBUG:rtp:receiver(video) < RtpPacket(seq=46810, ts=4163501171, marker=1, payload=96, 155 bytes) DEBUG:ice:Connection(0) protocol(1) < ('192.168.12.46', 55563) DATA 184 DEBUG:rtp:sender(video) > RtpPacket(seq=18113, ts=2365407046, marker=1, payload=96, 1110 bytes) DEBUG:ice:Connection(0) protocol(1) > ('192.168.12.46', 55563) DATA 1140 DEBUG:rtp:receiver(video) < RtpPacket(seq=46811, ts=4163504051, marker=1, payload=96, 150 bytes) DEBUG:ice:Connection(0) protocol(1) < ('192.168.12.46', 55563) DATA 227 DEBUG:rtp:sender(video) > RtpPacket(seq=18114, ts=2365410046, marker=0, payload=96, 1300 bytes) DEBUG:ice:Connection(0) protocol(1) > ('192.168.12.46', 55563) DATA 1330 DEBUG:rtp:sender(video) > RtpPacket(seq=18115, ts=2365410046, marker=1, payload=96, 390 bytes) DEBUG:ice:Connection(0) protocol(1) > ('192.168.12.46', 55563) DATA 420 DEBUG:rtp:receiver(video) < RtpPacket(seq=46812, ts=4163507291, marker=1, payload=96, 193 bytes) DEBUG:rtp:receiver(video) > RtcpPsfbPacket(fmt=15, ssrc=2596041375, media_ssrc=0, fci=b'REMB\x01\x00j[\\x90\x0c\x9f') DEBUG:ice:Connection(0) protocol(1) > ('192.168.12.46', 55563) DATA 38 DEBUG:ice:Connection(0) protocol(1) < ('192.168.12.46', 55563) DATA 220 DEBUG:rtp:receiver(video) < RtpPacket(seq=46813, ts=4163510171, marker=1, payload=96, 186 bytes) DEBUG:rtp:receiver(video) > RtcpPsfbPacket(fmt=15, ssrc=2596041375, media_ssrc=0, fci=b'REMB\x01\x00oL\\x90\x0c\x9f') DEBUG:ice:Connection(0) protocol(1) > ('192.168.12.46', 55563) DATA 38 DEBUG:websockets.protocol:client > Frame(fin=True, opcode=9, data=b'\x08\xbd\xe6\x05', rsv1=False, rsv2=False, rsv3=False) DEBUG:ice:Connection(0) protocol(1) < ('192.168.12.46', 55563) DATA 150 DEBUG:websockets.protocol:client - event = connection_lost(None) DEBUG:websockets.protocol:client - state = CLOSED DEBUG:websockets.protocol:client x code = 1006, reason = [no reason] DEBUG:websockets.protocol:client - aborted pending ping: 08bde605 DEBUG:websockets.protocol:client ! failing WebSocket connection in the CLOSED state: 1006 [no reason] DEBUG:rtp:receiver(video) < RtpPacket(seq=46814, ts=4163513051, marker=1, payload=96, 116 bytes) DEBUG:websockets.protocol:client x closing TCP connection Traceback (most recent call last): File "/home/test/anaconda3/lib/python3.6/site-packages/websockets/protocol.py", line 674, in transfer_data message = yield from self.read_message() File "/home/test/anaconda3/lib/python3.6/site-packages/websockets/protocol.py", line 742, in read_message frame = yield from self.read_data_frame(max_size=self.max_size) File "/home/test/anaconda3/lib/python3.6/site-packages/websockets/protocol.py", line 815, in read_data_frame frame = yield from self.read_frame(max_size) File "/home/test/anaconda3/lib/python3.6/site-packages/websockets/protocol.py", line 884, in read_frame extensions=self.extensions, File "/home/test/anaconda3/lib/python3.6/site-packages/websockets/framing.py", line 99, in read data = yield from reader(2) File "/home/test/anaconda3/lib/python3.6/asyncio/streams.py", line 672, in readexactly raise IncompleteReadError(incomplete, n) asyncio.streams.IncompleteReadError: 0 bytes read on a total of 2 expected bytes

The above exception was the direct cause of the following exception:

Traceback (most recent call last): File "apprtc.py", line 186, in loop.run_until_complete(signaling.close()) File "/home/test/anaconda3/lib/python3.6/asyncio/base_events.py", line 473, in run_until_complete return future.result() File "apprtc.py", line 47, in close await self.send(None) File "apprtc.py", line 64, in send 'msg': message, File "/home/test/anaconda3/lib/python3.6/site-packages/websockets/protocol.py", line 462, in send yield from self.ensure_open() File "/home/test/anaconda3/lib/python3.6/site-packages/websockets/protocol.py", line 646, in ensure_open ) from self.transfer_data_exc websockets.exceptions.ConnectionClosed: WebSocket connection is closed: code = 1006 (connection closed abnormally [internal]), no reason
invalid

opened by zhiweny1122 17
Firefox fails in server example if remote

Hi, I'm trying to run the server example from a remote browser. It works with Chrome once the server is secured, but not with Firefox. It works with Firefox across the local network but not outside. Ice gathering state complete / Signaling state stable; but connection state failed.

On debug, firefox is not responding to:

INFO:ice:Connection(2) Check CandidatePair(('192.168.1.100', 44767) -> ('82.240.164.123', 40654)) State.FROZEN -> State.IN_PROGRESS DEBUG:ice:Connection(2) protocol(2) > ('82.240.164.123', 40654) Message(message_method=Method.BINDING, message_class=Class.REQUEST, transaction_id=b'~\xd6\xfb\xd9~@\xd9\x98\x8a\x11\x92\xa0')

There are answers with chrome and the treatment continues without difficulties.

A timeout is triggered on server when it is firefox: ERROR: asyncio: Task exception was never retrieved future: <Task finished coro = <RTCPeerConnection .__ connect () done, defined at /home/middleware/.local/lib/python3.5/site-packages/aiortc/rtcpeerconnection.py:543> exception = ConnectionError ('ICE negotiation failed ')> Traceback (most recent call last): File "/usr/lib/python3.5/asyncio/tasks.py", line 239, in _step result = coro.send (None) File "/home/middleware/.local/lib/python3.5/site-packages/aiortc/rtcpeerconnection.py", line 548, in connect await iceTransport.start (self . remoteIce [transceiver]) File "/home/middleware/.local/lib/python3.5/site-packages/aiortc/rtcicetransport.py", line 270, in start await self._connection.connect () File "/home/middleware/.local/lib/python3.5/site-packages/aioice/ice.py", line 385, in connect raise ConnectionError ('ICE negotiation failed')

I do not understand what blocks firefox and what prevents it from working well.

opened by nfeybesse 17
AttributeError: 'builtins.Certificate' object has no attribute '_x509'

I am working on the WebRTC application. It is working fine on other Linux versions. But when I installed it in OpenSuse, it give me this error related to certificates.

Error

INFO:aiohttp.access:185.134.139.187 [03/Jan/2023:10:14:09 +0000] "GET / HTTP/1.1" 200 2304 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/108.0.0.0 Safari/537.36" INFO:aiohttp.access:185.134.139.187 [03/Jan/2023:10:14:09 +0000] "GET /client.js HTTP/1.1" 200 6577 "http://193.25.X.X/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/108.0.0.0 Safari/537.36" ERROR:aiohttp.server:Error handling request Traceback (most recent call last): File "/home/sysadmin/.local/lib/python3.10/site-packages/aiohttp/web_protocol.py", line 433, in _handle_request resp = await request_handler(request) File "/home/sysadmin/.local/lib/python3.10/site-packages/aiohttp/web_app.py", line 504, in _handle resp = await handler(request) File "/home/sysadmin/abdullah/webrtc-server/src/server.py", line 105, in return_offer pc.addTransceiver(trackOrKind="video") File "/home/sysadmin/.local/lib/python3.10/site-packages/aiortc/rtcpeerconnection.py", line 471, in addTransceiver return self.__createTransceiver( File "/home/sysadmin/.local/lib/python3.10/site-packages/aiortc/rtcpeerconnection.py", line 1067, in __createTransceiver dtlsTransport = self.__createDtlsTransport() File "/home/sysadmin/.local/lib/python3.10/site-packages/aiortc/rtcpeerconnection.py", line 1044, in __createDtlsTransport dtlsTransport = RTCDtlsTransport(iceTransport, self.__certificates) File "/home/sysadmin/.local/lib/python3.10/site-packages/aiortc/rtcdtlstransport.py", line 375, in init self.__ctx = certificate._create_ssl_context() File "/home/sysadmin/.local/lib/python3.10/site-packages/aiortc/rtcdtlstransport.py", line 211, in _create_ssl_context _openssl_assert(lib.SSL_CTX_use_certificate(ctx, self._cert._x509) == 1) # type: ignore AttributeError: 'builtins.Certificate' object has no attribute '_x509' INFO:aiohttp.access:185.134.139.187 [03/Jan/2023:10:14:14 +0000] "GET /data/CLIENT HTTP/1.1" 500 245 "http://193.25.X.X/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/108.0.0.0 Safari/537.36"

Versions

openssl = OpenSSL 1.1.1s 1 Nov 2022 OS = OpenSuse-Linux. python3 -c 'import OpenSSL; print(OpenSSL.version)' = 23.0.0

opened by abmuslim 1
Package loss huge increase when adding multiple video streams

Hello, first of all, thank you for the library. I am having an issue when sending multiple streams from the same client. In my use case I will have one client sending no media (just receiving) and another client sending up to 15 video streams at the same time. So far we have tested with 6 video streams at maximum. When sending only 3 video streams (with low resolution) or less it works great, low latency and great quality, however when increasing resolution or when adding more streams we start getting a huge packet loss in all video streams. We have tested it locally and over the internet and the problem remains the same even when running locally, so we either think it's a performace issue (we are testing in a quite powerful computer and all it's cores don't peak consumption over 50% at any time) or some wrong configuration we are lacking regarding the aiortc library. Has someone tested with this amount of video streams? Is there any limit in the aiortc library which could be limiting the bandwith?

Also, when sending 6 video streams, we have checked that the bandwith doesn't go up the 3 Mb/s which I would say is pretty low.

Any help is appreciated, thanks!

opened by andres-solares 0
Certificate error when creating RTCDtlsTransport with Cryptography 39.0.0
With Cryptography 39.0.0, RTCPeerConnection fails to create RTCDtlsTransport

RTCCertificate can't find self._cert._x509 while running _create_ssl_context()

This seems to due to one of the backwards incompatible changes from Crytography 39.0.0 updates (https://cryptography.io/en/latest/changelog/).

I don't have much background in crypto, so I'm not sure if this is just some change of the procedure acquiring a certificate or Cryptography's removing our necessary function.

Maybe related to #766, but Cryptography is not deprecated yet.

Please use Cryptography < 39 for now.
opened by khlukekim 1
Remote/Local transceiver order can cause unexpected full crashes due to createOffer
Odd behavior occurs when you try to use aiortc as a standalone RTC video receiver which acts as the offerer. When the order of the media information in the offer that is sent by the video receiver to the video sender does not match the order that the video receiver stores locally, when each iceTransport object is started, the ICE connection fails to complete on all video channels. This is because when an ICE connection needs to be established, STUN messages are sent from the "client" to the "server" (I use these terms loosely because the rest of RTC is peer to peer), which include self identification. This self identification consists of:

Username: consists of a remote username and a local username, where the remote username is determined by the offer/answer that is received by either end

Password: randomly generated and should have no issues if the packets are not tampered with

ICE must ensure that the usernames match before proceeding in the connection (as a safety measure against tampered packets). Note that in aiortc, the usernames are sent/received with simple for iteration:

On the "client" (rtcpeerconnection.py in createOffer):

for i in range(max(len(local_media), len(remote_media))): ... description.media.append( create_media_description_for_transceiver( transceiver, cname=self.__cname, direction=transceiver.direction, mid=mid, ) ) ...

On the "server" (rtcpeerconnection.py in __connect):

for transceiver in self.__transceivers: ... await iceTransport.start(self.__remoteIce[transceiver]) ...

which eventually calls:

rx_username = "%s:%s" % (self.local_username, self.remote_username) if message.attributes.get("USERNAME") != rx_username: raise ValueError("Wrong username")

For some reason, the order in which the offer is parsed when sent to the "server" is the reverse of the order in which the transceivers are added to the peer connection, which means if there is more than 1 transceiver, the username expected by the "server" during the first iteration no longer matches the username sent by the "client" during the first iteration. As a result, the connection is abandoned by the "server", despite another transceiver's data may match what is being sent by the "client".

Here is an example of the different usernames that may be compared Though both usernames are valid, they are not being matched to the correct transceiver and the ICE gathering never completes.

So far, the behavior I have observed shows that the order of the transceivers in the offer is always the reverse of the order of transceivers in the RTCPeerConnection.__transceivers property, which is why I have implemented a temporary fix using the following code:

peer_connection._RTCPeerConnection__transceivers = peer_connection._RTCPeerConnection__transceivers[::-1]

that simply reverses the order that the transceivers are stored. This is not a robust fix however, as I am uncertain of the consistency with which the order is mixed, so my suggestion for a fix would be to sort the transceivers on both ends by their mids, which is information constant on both sides.

Note that Chromium's implementation seems to have no trouble with this. This leads me to believe that their implementation checks all of its transceivers' usernames when forming connections, so the state where the "server" is checking for a transceiver's username that has yet to be sent is avoided. Perhaps this would be a more robust fix for this issue.
opened by kennytheeggman 0
Weird behavior while trying to connect to livekit
Hello,

I'm developing a python client SDK for livekit, and I ran into a weird problem.

In rctpeerconnection.py, in the __connect() method, I have to patch line 1018 to disable SCTP to make it work.

As far as I know, SCTP is used for datachannels. I don't need to use the datachannel, but livekit sends it in its initial offer.

The problem I'm seeing, is that the sctp transport is being connecting, somehow preventing the DTLS/RTP transport to connect.

I'm not sure my explanation is clear, since I'm very new to webrtc, but I've put an example here: https://github.com/dguerizec/livekit-client-sdk-python

In this repo, the subscriber example only works when I disable SCTP in RTCPeerConnection.__connect().

If I don't disable SCTP, the dtlsTransport.state is always "connecting", thus never receiving from the transceiver.

Is there something I'm doing wrong ?

Initial offer from livekit-server:

v=0 o=- 4261650069995738642 1672227076 IN IP4 0.0.0.0 s=- t=0 0 a=fingerprint:sha-256 5F:CF:73:AD:E0:DE:D0:8B:7A:B4:FF:38:57:7A:3F:BD:99:1B:82:CB:04:DB:A4:50:C3:04:1C:0B:17:BC:47:9A a=ice-lite a=extmap-allow-mixed a=group:BUNDLE 0 m=application 9 UDP/DTLS/SCTP webrtc-datachannel c=IN IP4 0.0.0.0 a=setup:actpass a=mid:0 a=sendrecv a=sctp-port:5000 a=ice-ufrag:qJHytwgzRTlkmPiM a=ice-pwd:yXckeNWryDvwkHPMQXcOaZrPsASfqaGo

I can provide other debug information if needed, along with network traces.
opened by dguerizec 0