BBB streaming without Xorg and Pulseaudio and Chromium and other nonsense (heavily WIP)

Overview

BBB Streamer NG?

Makes a conference like this...

...streamable like this!

I also recorded a small video showing the basic features: https://www.youtube.com/watch?v=u9pTmzowIPc

Big Blue Button streaming without pressing a virtual camera against a remote controlled webbrowser..

Heavily work-in-progress, but kinda functional.

The whiteboard is definitively the hardest part to get right. Everything else just kinda depends on gstreamer not breaking every 5 minutes :D

Example usage: python test.py your.bbb.server room-id-from-greenlight rtmp://server/app/path?auth=foobar

Or alternatively docker run -t -i --rm=true lukas2511/bbb-streaming your.bbb.server room-id-from-greenlight rtmp://server/app/path?auth=foobar

Working:

  • Capturing audio
  • Capturing all cameras
  • Capturing screen captures
  • Generating presentation canvas (including annotations) and converting it into an internal video stream
  • Automatic switching between presentation and screenshare
  • Simple side-by-side scene with exactly 1 active webcam and the presentation/screenshare
  • Tracking camera of active speaker
  • Background image for streams
  • Selection of multiple scenes (side-by-side, fullscreen cam/presentation) using chat commands (!view <sbs|pip|cam|pres>)

Output is streamed using rtmp for now. How this is implemented will probably change.

Todo:

  • Fixing the gstreamer webrtc video glitches (recovery on packetloss is b0rked, currently enabling slight fec and requesting a keyframe every second)
  • Support different camera selections (follow-speaking/follow-presenter/manual selection)
  • Lots of error handling + recovery foo
  • Finishing the todo list
Comments
  • youtube complaining keyframe <4 needed,

    youtube complaining keyframe <4 needed,

    Great work Lukas..., fired up no problem, wanted to be the first issue on you new project,

    image

    will look at code to see where this is set.

    regards, Stephen

    opened by hostbbb 2
  • Allow to Sponsor

    Allow to Sponsor

    I would like to sponsor this project because I really like the idea.

    I see that @lukas2511 already has the profile enabled. Can you enable the Sponsor button also in this specific repository?

    Thank you so much for your work!

    opened by valerio-bozzolan 1
  • remove /b/ between server and room-id-from-greenlight

    remove /b/ between server and room-id-from-greenlight

    Thank you for your work

    to access my server I don't have /b/between my bbb server address and the room id from greenlight. bbb-streaming can't connect because it rebuilds the url with this /b/ (it's glin my case, see example below)

    The error message:

    During handling of the above exception, another exception occurred:
    
    Traceback (most recent call last):
      File "/opt/stream/test.py", line 15, in <module>
        join_url = session.greenlight_join(sys.argv[1], sys.argv[2])
      File "/opt/stream/lib/session.py", line 17, in greenlight_join
        authenticity_token = session.get("https://" + server + "/b/" + room).text.split('name="authenticity_token"')[1].split('"')[1]
      File "/usr/lib/python3.9/site-packages/requests/sessions.py", line 555, in get
        return self.request('GET', url, **kwargs)
      File "/usr/lib/python3.9/site-packages/requests/sessions.py", line 542, in request
        resp = self.send(prep, **send_kwargs)
      File "/usr/lib/python3.9/site-packages/requests/sessions.py", line 655, in send
        r = adapter.send(request, **kwargs)
      File "/usr/lib/python3.9/site-packages/requests/adapters.py", line 516, in send
        raise ConnectionError(e, request=request)
    requests.exceptions.ConnectionError: HTTPSConnectionPool(host='https', port=443): Max retries exceeded with url: //demo.bigbluebutton.org/gl//b/thi-mqt-cqa (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x7f2fcb4e59a0>: Failed to establish a new connection: [Errno -2] Name or service not known'))
    

    Jeau

    opened by jeau 1
  • wiki

    wiki

    Hi lukas, A really good project.

    Did you plan to do a wiki to try your hard work ?

    I would to try, but I don't how to begin :)

    Any help will be good :)

    Regards, Aurélien.

    opened by aguerson 0
  • Feature request: multiple guests

    Feature request: multiple guests

    Maybe this is more of a question: is it possible to stream multiple guests? Podium discussions or Barcamp session planning? I guess then it is not possible to move positions of the guests. Great tool and happy it has been developed in Bonn/Rhein-Sieg. 🙂

    opened by saschafoerster 0
  • Stream only with audio and background image

    Stream only with audio and background image

    Hi, tried to stream a BBB Session with the following command:

    docker run -t -i --rm=true lukas2511/bbb-streaming --greenlight-url https://meet.piratensommer.de/b/xxx-xxx-xxx-xxx --greenlight-name stream --rtmp-url rtmp://example.de:1935/live/xxxxxxxxx --debug

    The RTMP Stream is reaching my plattform with audio and a (standard?) backround image but without the video. Am I doing something wrong?

    INFO:bbb-streamer:Joining using Greenlight frontend DEBUG:bbb-streamer:Trying to acquire join url from greenlight: https://meet.piratensommer.de/b/xxx-xxx-xxx-xxx (without access code) DEBUG:bbb-streamer:Got join url: https://meet.piratensommer.de/bigbluebutton/api/join?... INFO:bbb-streamer:Found STUN server: stun.freeswitch.org DEBUG:bbb-streamer:Connecting to websocket DEBUG:bbb-streamer:Sending initial control messages DEBUG:bbb-streamer:Subscribing to annotations messages DEBUG:bbb-streamer:Subscribing to current-user messages DEBUG:bbb-streamer:Subscribing to group-chat messages DEBUG:bbb-streamer:Subscribing to group-chat-msg messages DEBUG:bbb-streamer:Subscribing to guestUser messages DEBUG:bbb-streamer:Subscribing to local-settings messages DEBUG:bbb-streamer:Subscribing to meetings messages DEBUG:bbb-streamer:Subscribing to network-information messages DEBUG:bbb-streamer:Subscribing to note messages DEBUG:bbb-streamer:Subscribing to ping-pong messages DEBUG:bbb-streamer:Subscribing to presentations messages DEBUG:bbb-streamer:Subscribing to screenshare messages DEBUG:bbb-streamer:Subscribing to slide-positions messages DEBUG:bbb-streamer:Subscribing to slides messages DEBUG:bbb-streamer:Subscribing to users messages DEBUG:bbb-streamer:Subscribing to users-infos messages DEBUG:bbb-streamer:Subscribing to video-streams messages DEBUG:bbb-streamer:Subscribing to voice-call-states messages DEBUG:bbb-streamer:Subscribing to voiceUsers messages DEBUG:bbb-streamer:Session manager is ready DEBUG:bbb-streamer:Initializing audio connection DEBUG:bbb-streamer:Attaching listener to session manager: <bound method CameraManager.listener of <lib.camera.CameraManager object at 0x7f2314269520>> DEBUG:bbb-streamer:Connecting to bbb-webrtc-sfu DEBUG:bbb-streamer:Attaching listener to session manager: <bound method Presentation.listener of <lib.presentation.Presentation object at 0x7f2314269730>> DEBUG:bbb-streamer:Attaching listener to session manager: <bound method ScreenshareManager.listener of <lib.screenshare.ScreenshareManager object at 0x7f23141b5940>> DEBUG:bbb-streamer:Attaching listener to session manager: <function start..chatmsg at 0x7f23141b45e0> (bbb) DEBUG:bbb-streamer:Starting audio webrtc pipeline INFO:bbb-streamer:Sending offer for audio DEBUG:bbb-streamer:Fixing sdp offer, broken opus codec line: a=rtpmap:111 opus/48000 DEBUG:bbb-streamer:Sending audio SDP offer: 'v=0\r\no=- 1236737925857283059 0 IN IP4 0.0.0.0\r\ns=-\r\nt=0 0\r\na=ice-options:trickle\r\na=group:BUNDLE audio0\r\nm=audio 9 UDP/TLS/RTP/SAVPF 111\r\nc=IN IP4 0.0.0.0\r\na=setup:actpass\r\na=ice-ufrag:HVioQASYnwRZotieCvmN7EkQQlKO9i/W\r\na=ice-pwd:BaFMNwibUYtZ3bDlDviFwXIETqhFE895\r\na=rtcp-mux\r\na=rtcp-rsize\r\na=recvonly\r\na=rtpmap:111 opus/48000/2\r\na=rtcp-fb:111 nack pli\r\na=fmtp:111 fec-type=ulp-red\r\na=mid:audio0\r\na=fingerprint:sha-256 06:D5:23:2A:33:5E:7A:CD:85:8E:AF:3A:06:A4:44:90:FE:7E:2F:BB:94:DF:82:FA:86:62:6D:74:09:BB:6A:E0' Exception in thread Thread-1: Traceback (most recent call last): File "/usr/lib/python3.9/threading.py", line 954, in _bootstrap_inner self.run() File "/opt/stream/lib/session.py", line 94, in run listener(msg) File "/opt/stream/lib/camera.py", line 68, in listener self.voice_users[msg['id']].update(msg['fields']) KeyError: 'GKGGh84eh9gfcT4Yj' INFO:bbb-streamer:WebRTC <Audio(Thread-6, started daemon 139787084228160)> ready

    opened by roere 1
Owner
Lukas Schauer
Student, Hacker, Cyber
Lukas Schauer
Streaming Anomaly Detection Framework in Python (Outlier Detection for Streaming Data)

Python Streaming Anomaly Detection (PySAD) PySAD is an open-source python framework for anomaly detection on streaming multivariate data. Documentatio

Selim Firat Yilmaz 181 Dec 18, 2022
A machine learning package for streaming data in Python. The other ancestor of River.

scikit-multiflow is a machine learning package for streaming data in Python. creme and scikit-multiflow are merging into a new project called River. W

null 670 Dec 30, 2022
Source code and data from the RecSys 2020 article "Carousel Personalization in Music Streaming Apps with Contextual Bandits" by W. Bendada, G. Salha and T. Bontempelli

Carousel Personalization in Music Streaming Apps with Contextual Bandits - RecSys 2020 This repository provides Python code and data to reproduce expe

Deezer 48 Jan 2, 2023
Securetar - A streaming wrapper around python tarfile and allow secure handling files and support encryption

Secure Tar Secure Tarfile library It's a streaming wrapper around python tarfile

Pascal Vizeli 2 Dec 9, 2022
Scikit-event-correlation - Event Correlation and Forecasting over High Dimensional Streaming Sensor Data algorithms

scikit-event-correlation Event Correlation and Changing Detection Algorithm Theo

Intellia ICT 5 Oct 30, 2022
An implementation of the 1. Parallel, 2. Streaming, 3. Randomized SVD using MPI4Py

PYPARSVD This implementation allows for a singular value decomposition which is: Distributed using MPI4Py Streaming - data can be shown in batches to

Romit Maulik 44 Dec 31, 2022
PyTorch implementation of the Deep SLDA method from our CVPRW-2020 paper "Lifelong Machine Learning with Deep Streaming Linear Discriminant Analysis"

Lifelong Machine Learning with Deep Streaming Linear Discriminant Analysis This is a PyTorch implementation of the Deep Streaming Linear Discriminant

Tyler Hayes 41 Dec 25, 2022
Code for Towards Streaming Perception (ECCV 2020) :car:

sAP — Code for Towards Streaming Perception ECCV Best Paper Honorable Mention Award Feb 2021: Announcing the Streaming Perception Challenge (CVPR 2021

Martin Li 85 Dec 22, 2022
Streaming over lightweight data transformations

Description Data augmentation libarary for Deep Learning, which supports images, segmentation masks, labels and keypoints. Furthermore, SOLT is fast a

Research Unit of Medical Imaging, Physics and Technology 256 Jan 8, 2023
Music library streaming app written in Flask & VueJS

djtaytay This is a little toy app made to explore Vue, brush up on my Python, and make a remote music collection accessable through a web interface. I

Ryan Tasson 6 May 27, 2022
Real-time Object Detection for Streaming Perception, CVPR 2022

StreamYOLO Real-time Object Detection for Streaming Perception Jinrong Yang, Songtao Liu, Zeming Li, Xiaoping Li, Sun Jian Real-time Object Detection

Jinrong Yang 237 Dec 27, 2022
A fast, scalable, high performance Gradient Boosting on Decision Trees library, used for ranking, classification, regression and other machine learning tasks for Python, R, Java, C++. Supports computation on CPU and GPU.

Website | Documentation | Tutorials | Installation | Release Notes CatBoost is a machine learning method based on gradient boosting over decision tree

CatBoost 6.9k Jan 4, 2023
A fast, scalable, high performance Gradient Boosting on Decision Trees library, used for ranking, classification, regression and other machine learning tasks for Python, R, Java, C++. Supports computation on CPU and GPU.

Website | Documentation | Tutorials | Installation | Release Notes CatBoost is a machine learning method based on gradient boosting over decision tree

CatBoost 5.7k Feb 12, 2021
Use deep learning, genetic programming and other methods to predict stock and market movements

StockPredictions Use classic tricks, neural networks, deep learning, genetic programming and other methods to predict stock and market movements. Both

Linda MacPhee-Cobb 386 Jan 3, 2023
Tool for installing and updating MiSTer cores and other files

MiSTer Downloader This tool installs and updates all the cores and other extra files for your MiSTer. It also updates the menu core, the MiSTer firmwa

null 72 Dec 24, 2022
Projects for AI/ML and IoT integration for games and other presented at re:Invent 2021.

Playground4AWS Projects for AI/ML and IoT integration for games and other presented at re:Invent 2021. Architecture Minecraft and Lamps This project i

Vinicius Senger 5 Nov 30, 2022
PyMove is a Python library to simplify queries and visualization of trajectories and other spatial-temporal data

Use PyMove and go much further Information Package Status License Python Version Platforms Build Status PyPi version PyPi Downloads Conda version Cond

Insight Data Science Lab 64 Nov 15, 2022
Face and other object detection using OpenCV and ML Yolo

Object-and-Face-Detection-Using-Yolo- Opencv and YOLO object and face detection is implemented. You only look once (YOLO) is a state-of-the-art, real-

Happy  N. Monday 3 Feb 15, 2022