A Linux webcam plugin for BGMv2 as used in our demos.

Overview

The goal of this repository is to supplement the main Real-Time High Resolution Background Matting repo with a working demo of a videoconferencing plugin (e.g. the code used in our promotional demos).

Prerequisites

This plugin requires Linux, because it relies on the v4l2loopback kernel module to create and stream to virtual video devices. We welcome and encourage community adaptations to other platforms.

  1. Install v4l2loopback. On Debian/Ubuntu, the command is likely sudo apt-get install v4l2loopback-utils.
  2. Install required packages in requirements.txt
    • If starting from the main repository, just install pyfakewebcam (Python interface for v4l2loopback, pip install pyfakewebcam). You can move the script to the root level in that repository.

Directions

Before running the plugin, the virtual web camera device needs to be created.

sudo modprobe v4l2loopback devices=1

The above command should create a single virtual webcam at /dev/video1 (the number may change), which is the default stream output for the plugin script. This webcam can now be selected by software such as Zoom, browsers, etc.

After downloading the TorchScript weights of your choice here, launch the pluging with e.g.: python demo_webcam.py --model-checkpoint torchscript_resnet50_fp32.pth

Once the plugin is launched, a simple OpenCV-based UI will show two buttons:

  • Toggle between background selection view and (after snapping a background) actual matting
  • Cycle through target background options:
    1. Plain white
    2. Gaussian blur of the background frame
    3. Supplied target video background (samples included in this repo)
    4. Supplied target image background At any point, press Q to exit.

Guidelines

For best results:

  1. Avoid blocking bright light, especially during background capture.
  2. Avoid changing light conditions. For prolonged use, it may be helpful to re-take the background as light conditions drift.

Thanks to CK FreeVideoTemplates on YouTube for the seasonal video target background.

Comments
  • Some Question

    Some Question

    I try to run demo_webcam.py on cpu, linux. But I get some question, eg: OSError:[Error 22] invalid arguments FATAL: exception not rethrown Abolished (core saved) I'm not sure what went wrong. Support only NVIDIA? Looking forward to your reply!!! Thank you!

    opened by Godlikemandyy 2
  • Added ability to tweak the model params in real time.

    Added ability to tweak the model params in real time.

    This project is really great. Great job all-around.

    Problem:
    Understanding how the different model checkpoints and parameters impact the quality of the model is difficult to discern. These must be specified ahead of time, making it difficult to test various combinations of parameters and checkpoints in an easy fashion. One would have to start and stop the video for each change, which can make it difficult to compare quality levels.

    Solution: Enable the model parameters and checkpoints to be modified in real time so one can see the immediate impact of the changes.

    To accomplish the above, I expanded the control panel window to enable the appropriate buttons for tweaking the model. This required a bit of refactoring, but I tried to preserve the original simplicity of the code. I also did some light reformatting, fixed an issue where playing videos would crash, and hid all buttons until a user selects a background. Example screenshot of the updated control panel below. model_params

    The UI changes depending on the refine mode selected. The "Full" refine mode, for example, hides the row of buttons beneath it since those wouldn't apply. To enable switching of the model checkpoint I added a new optional --model-checkpoint-dir parameter where a user can specify the directory which contains the model checkpoint(s) that have been downloaded. The code will look for known models based on name in that directory and enable the user to cycle through them. If the --model-checkpoint-dir is not specified, then the bottom button is hidden.

    opened by kevinlester 1
  • Add Windows support

    Add Windows support

    This request contains some minor changes to add Windows support for BGMv2 plugin using pyvirtualcam module.

    demo_webcam_win.py and installation description added.

    opened by KostiaChorny 1
  • Error: AttributeError: 'NoneType' object has no attribute 'copy'

    Error: AttributeError: 'NoneType' object has no attribute 'copy'

    I have installed newer version of pytorch, than asked, as it was no longer available. I have 2 fake devices (video0-1) and will be sharing logs of attempts on both of them. Sudo or not, output is the same.

    Command 1: sudo python demo_webcam.py --camera-device /dev/video0 --model-checkpoint torchscript_resnet50_fp16.pth

    Output: [ WARN:0] global /tmp/pip-req-build-ddpkm6fn/opencv/modules/videoio/src/cap_v4l.cpp (1004) tryIoctl VIDEOIO(V4L2:/dev/video0): select() timeout. Traceback (most recent call last): File "/home/albert/git/BGMv2-webcam-plugin-linux/demo_webcam.py", line 586, in fake_camera = pyfakewebcam.FakeWebcam(args.camera_device, cam.width, cam.height) File "/usr/lib/python3.9/site-packages/pyfakewebcam/pyfakewebcam.py", line 54, in init fcntl.ioctl(self._video_device, _v4l2.VIDIOC_S_FMT, self._settings) OSError: [Errno 22] Invalid argument

    Command 2: sudo python demo_webcam.py --camera-device /dev/video1 --model-checkpoint torchscript_resnet50_fp16.pth

    Output: [ WARN:0] global /tmp/pip-req-build-ddpkm6fn/opencv/modules/videoio/src/cap_v4l.cpp (1004) tryIoctl VIDEOIO(V4L2:/dev/video0): select() timeout. Traceback (most recent call last): File "/home/albert/git/BGMv2-webcam-plugin-linux/demo_webcam.py", line 594, in if app_step(): File "/home/albert/git/BGMv2-webcam-plugin-linux/demo_webcam.py", line 524, in app_step frame = cam.read() File "/home/albert/git/BGMv2-webcam-plugin-linux/demo_webcam.py", line 73, in read frame = self.frame.copy() AttributeError: 'NoneType' object has no attribute 'copy'

    opened by Redo11 0
  • Ampere(RTX 30-series, A100) GPU Support.

    Ampere(RTX 30-series, A100) GPU Support.

    All I have done is update torch to 1.8.1 and set cudatoolkit to 11. I could not install torch using pip from the instructions page on pytorch.org for CUDA 11, which is required for Ampere, so I used Conda.

    opened by charitarthchugh 0
  • Invalid argument (pyfakewebcam.py, line 54)

    Invalid argument (pyfakewebcam.py, line 54)

    My Garuda Arch couldn't install the thing (some Nvidia issues), and Debian does this - and this is the issue -

    python3 demo_webcam.py --model-checkpoint torchscript_mobilenetv2_fp16.pth
    Traceback (most recent call last):
      File "/root/demo_webcam.py", line 586, in <module>
        fake_camera = pyfakewebcam.FakeWebcam(args.camera_device, cam.width, cam.height)
      File "/usr/local/lib/python3.9/dist-packages/pyfakewebcam/pyfakewebcam.py", line 54, in __init__
        fcntl.ioctl(self._video_device, _v4l2.VIDIOC_S_FMT, self._settings)
    OSError: [Errno 22] Invalid argument
    FATAL: exception not rethrown
    Aborted
    

    I tried pyfakewebcam from both apt & git

    opened by thatkotov 5
Owner
Andrey Ryabtsev
Computer Science MS 2021
Andrey Ryabtsev
The repository for my video "Playing MINECRAFT with a WEBCAM"

This is the official repo for my video "Playing MINECRAFT with a WEBCAM" on YouTube Original video can be found here: https://youtu.be/701TPxL0Skg Red

Rishabh 27 Jun 7, 2022
LSO, also known as Linux Swap Operator, is a software with both GUI and terminal versions that you can manage the Swap area for Linux operating systems.

LSO - Linux Swap Operator Türkçe - LSO Nedir? LSO, diğer adıyla Linux Swap Operator Linux işletim sistemleri için Swap alanını yönetebileceğiniz hem G

Eren İnce 4 Feb 9, 2022
List of Linux Tools I put on almost every linux / Debian host

Linux-Tools List of Linux Tools I put on almost every Linux / Debian host Installed: geany --> GUI editor/ notepad++ like chkservice --> TUI Linux ser

Stew Alexander 20 Jan 2, 2023
Organize seu linux - organize your linux

OrganizeLinux Organize seu linux - organize your linux Organize seu linux Uma forma rápida de separar arquivos dispersos em pastas. formatos a serem c

Marcus Vinícius Ribeiro Andrade 1 Nov 30, 2021
HPomb Is Socail Engineering Tool , Used For Bombing , Spoofing and Anonymity Available For Linux And Android(Termux)

HPomb v2020.02 Coming Soon Created By Secanonm HPomb Is Socail Engineering Tool , Used For Bombing , Spoofing and Anonymity Available For Linux And An

Secanonm 10 Jul 25, 2022
You can easily send campaigns, e-marketing have actually account using cash will thank you for using our tools, and you can support our Vodafone Cash +201090788026

*** Welcome User Sorry I Mean Hello Brother ✓ Devolper and Design : Mokhtar Abdelkreem ========================================== You Can Follow Us O

Mo Code 1 Nov 3, 2021
This is a multi-app executor that it used when we have some different task in a our applications and want to run them at the same time

This is a multi-app executor that it used when we have some different task in a our applications and want to run them at the same time. It uses SQLAlchemy for ORM and Alembic for database migrations.

Majid Iranpour 5 Apr 16, 2022
Originally used during Marketplace.tf's open period, this program was used to get the profit of items bought with keys and sold for dollars.

Originally used during Marketplace.tf's open period, this program was used to get the profit of items bought with keys and sold for dollars. Practically useless for me now, but can be used as an example of tkinter.

BoggoTV 1 Dec 11, 2021
Python based scripts for obtaining system information from Linux.

sysinfo Python based scripts for obtaining system information from Linux. Python2 and Python3 compatible Output in JSON format Simple scripts and exte

Petr Vavrin 70 Dec 20, 2022
Tool to generate wrappers for Linux libraries allowing for dlopen()ing them without writing any boilerplate

Dynload wrapper This program will generate a wrapper to make it easy to dlopen() shared objects on Linux without writing a ton of boilerplate code. Th

Hein-Pieter van Braam 25 Oct 24, 2022
Hook and simulate global keyboard events on Windows and Linux.

keyboard Take full control of your keyboard with this small Python library. Hook global events, register hotkeys, simulate key presses and much more.

BoppreH 3.2k Jan 1, 2023
Python screenshot library, replacement for the Pillow ImageGrab module on Linux.

tldr: Use Pillow The pyscreenshot module is obsolete in most cases. It was created because PIL ImageGrab module worked on Windows only, but now Linux

null 455 Dec 24, 2022
The earliest beta version of pytgcalls on Linux x86_64 and ARM64! Use in production at your own risk!

Public beta test. Use in production at your own risk! tgcalls - a python binding for tgcalls (c++ lib by Telegram); pytgcalls - library connecting pyt

Il'ya 21 Jan 13, 2022
pspsps(1) is a compyuter software to call an online catgirl to the Linux terminyal.

pspsps(1): call a catgirl from the Internyet to the Linux terminyal show processes: ps show catgirls: pspsps —@[email protected] pspsps(1) is a compyute

Melissa Boiko 32 Dec 19, 2022
Python3 Interface to numa Linux library

py-libnuma is python3 interface to numa Linux library so that you can set task affinity and memory affinity in python level for your process which can help you to improve your code's performence.

Dalong 13 Nov 10, 2022
GNU/Linux'u yeni kurulumu bitirmiş olarak açtığınızda sizi karşılayacak bir uygulama.

Hoş Geldiniz GNU/Linux'u yeni kurulumu bitirmiş olarak açtığınızda sizi karşılayacak bir uygulama.

Alperen İsa 96 Oct 30, 2022
Um Script De Mensagem anonimas Para linux e Termux Feito em python

Um Script De Mensagem anonimas Para linux e Termux Feito em python feito em um celular

null 6 Sep 9, 2021
Runs macOS on linux with qemu.

mac-on-linux-with-qemu Runs macOS on linux with qemu. Pre-requisites qemu-system-x86_64 dmg2img pulseaudio python[click] Usage After cloning the repos

Arindam Das 177 Dec 26, 2022
Tomador de ramos UC automatico para Windows, Linux y macOS

auto-ramos v2.0 Tomador de ramos UC automatico para Windows, Linux y macOS Funcion Este script de Python tiene como principal objetivo hacer que la to

Open Source eUC 13 Jun 29, 2022