Pyeventbus: a publish/subscribe event bus

Related tags

Deep Learning python benchmark decorators concurrency parallel eventbus parallelism concurrent-processes event-driven publish-subscribe threads python27 eventbus-queue eventbus-library
Overview

pyeventbus

https://travis-ci.org/n89nanda/pyeventbus.svg?branch=master

pyeventbus is a publish/subscribe event bus for Python 2.7.

  • simplifies the communication between python classes
  • decouples event senders and receivers
  • performs well threads, greenlets, queues and concurrent processes
  • avoids complex and error-prone dependencies and life cycle issues
  • makes code simpler
  • has advanced features like delivery threads, workers and spawning different processes, etc.
  • is tiny (3KB archive)

pyeventbus in 3 steps:

  1. Define events:

    class MessageEvent:
    # Additional fields and methods if needed
    def __init__(self):
        pass

  2. Prepare subscribers: Declare and annotate your subscribing method, optionally specify a thread mode:

    from pyeventbus import *

@subscribe(onEvent=MessageEvent)
def func(self, event):
    # Do something
    pass

    Register your subscriber. For example, if you want to register a class in Python:

    from pyeventbus import *

class MyClass:
    def __init__(self):
        pass

    def register(self, myclass):
        PyBus.Instance().register(myclass, self.__class__.__name__)

# then during initilization

myclass = MyClass()
myclass.register(myclass)

  3. Post events:

    from pyeventbus import *

class MyClass:
    def __init__(self):
        pass

    def register(self, myclass):
        PyBus.Instance().register(myclass, self.__class__.__name__)

    def postingAnEvent(self):
        PyBus.Instance().post(MessageEvent())

 myclass = MyClass()
 myclass.register(myclass)
 myclass.postingAnEvent()

Modes: pyeventbus can run the subscribing methods in 5 different modes

  1. POSTING:

    Runs the method in the same thread as posted. For example, if an event is posted from main thread, the subscribing method also runs in the main thread. If an event is posted in a seperate thread, the subscribing method runs in the same seperate method

This is the default mode, if no mode has been provided::

@subscribe(threadMode = Mode.POSTING, onEvent=MessageEvent)
def func(self, event):
    # Do something
    pass

  2. PARALLEL:

    Runs the method in a seperate python thread::

@subscribe(threadMode = Mode.PARALLEL, onEvent=MessageEvent)
def func(self, event):
    # Do something
    pass

  3. GREENLET:

    Runs the method in a greenlet using gevent library::

@subscribe(threadMode = Mode.GREENLET, onEvent=MessageEvent)
def func(self, event):
    # Do something
    pass

  4. BACKGROUND:

    Adds the subscribing methods to a queue which is executed by workers::

@subscribe(threadMode = Mode.BACKGROUND, onEvent=MessageEvent)
def func(self, event):
    # Do something
    pass

  1. CONCURRENT:

    Runs the method in a seperate python process::

@subscribe(threadMode = Mode.CONCURRENT, onEvent=MessageEvent)
def func(self, event):
    # Do something
    pass

Adding pyeventbus to your project:

pip install pyeventbus

Example:

git clone https://github.com/n89nanda/pyeventbus.git

cd pyeventbus

virtualenv venv

source venv/bin/activate

pip install pyeventbus

python example.py

Benchmarks and Performance:

Refer /pyeventbus/tests/benchmarks.txt for performance benchmarks on CPU, I/O and networks heavy tasks.

Run /pyeventbus/tests/test.sh to generate the same benchmarks.

Performance comparison between all the modes with Python and Cython

alternate text

Inspiration

Inspired by Eventbus from greenrobot: https://github.com/greenrobot/EventBus
You might also like...
Code for the paper
Code for the paper "Unsupervised Contrastive Learning of Sound Event Representations", ICASSP 2021.

Unsupervised Contrastive Learning of Sound Event Representations This repository contains the code for the following paper. If you use this code or pa

Eduardo Fonseca 81 Dec 22, 2022
Repo for "Event-Stream Representation for Human Gaits Identification Using Deep Neural Networks"

Summary This is the code for the paper Event-Stream Representation for Human Gaits Identification Using Deep Neural Networks by Yanxiang Wang, Xian Zh

zhangxian 54 Jan 3, 2023
Cross-media Structured Common Space for Multimedia Event Extraction (ACL2020)
Cross-media Structured Common Space for Multimedia Event Extraction (ACL2020)

Cross-media Structured Common Space for Multimedia Event Extraction Table of Contents Overview Requirements Data Quickstart Citation Overview The code

Manling Li 49 Nov 21, 2022
CVPRW 2021: How to calibrate your event camera
CVPRW 2021: How to calibrate your event camera

E2Calib: How to Calibrate Your Event Camera This repository contains code that implements video reconstruction from event data for calibration as desc

Robotics and Perception Group 104 Nov 16, 2022
Repository relating to the CVPR21 paper TimeLens: Event-based Video Frame Interpolation
Repository relating to the CVPR21 paper TimeLens: Event-based Video Frame Interpolation

TimeLens: Event-based Video Frame Interpolation This repository is about the High Speed Event and RGB (HS-ERGB) dataset, used in the 2021 CVPR paper T

Robotics and Perception Group 544 Dec 19, 2022
An implementation for `Text2Event: Controllable Sequence-to-Structure Generation for End-to-end Event Extraction`

Text2Event An implementation for Text2Event: Controllable Sequence-to-Structure Generation for End-to-end Event Extraction Please contact Yaojie Lu (@

Roger 153 Jan 7, 2023
Official PyTorch implementation of N-ImageNet: Towards Robust, Fine-Grained Object Recognition with Event Cameras (ICCV 2021)
Official PyTorch implementation of N-ImageNet: Towards Robust, Fine-Grained Object Recognition with Event Cameras (ICCV 2021)

N-ImageNet: Towards Robust, Fine-Grained Object Recognition with Event Cameras Official PyTorch implementation of N-ImageNet: Towards Robust, Fine-Gra

null 32 Dec 26, 2022
SurvITE: Learning Heterogeneous Treatment Effects from Time-to-Event Data

SurvITE: Learning Heterogeneous Treatment Effects from Time-to-Event Data SurvITE: Learning Heterogeneous Treatment Effects from Time-to-Event Data Au

null 14 Nov 28, 2022
Weakly Supervised Dense Event Captioning in Videos, i.e. generating multiple sentence descriptions for a video in a weakly-supervised manner.
Weakly Supervised Dense Event Captioning in Videos, i.e. generating multiple sentence descriptions for a video in a weakly-supervised manner.

WSDEC This is the official repo for our NeurIPS paper Weakly Supervised Dense Event Captioning in Videos. Description Repo directories ./: global conf

Melon(Xuguang Duan) 96 Nov 1, 2022
Comments
  • Same method name for multiple subscribers bug

    Same method name for multiple subscribers bug

    Please see the code below. To summarize:

    • Define one event
    • Define two subscriber listening for the event above. Each subscriber has a listener method with the name on_event
    • Each of the subscriber classes above defines an instance field, but with unique name (self.something in the first class, self.something2 in the second class)
    • Define another class that posts an event

    Run this scenario and get the error below:

    Exception in thread thread-on_event:
Traceback (most recent call last):
  File "C:\Anaconda2\envs\python\lib\threading.py", line 801, in __bootstrap_inner
    self.run()
  File "C:\Anaconda2\envs\python\lib\site-packages\pyeventbus\pyeventbus.py", line 112, in run
    self.method(self.subscriber, self.event)
  File "C:/FractureID/projects/python/ui/spectraqc/PyEventBusBug.py", line 16, in on_event
    print (self.something)
AttributeError: Subscriber2 instance has no attribute 'something'

Exception in thread thread-on_event:
Traceback (most recent call last):
  File "C:\Anaconda2\envs\python\lib\threading.py", line 801, in __bootstrap_inner
    self.run()
  File "C:\Anaconda2\envs\python\lib\site-packages\pyeventbus\pyeventbus.py", line 112, in run
    self.method(self.subscriber, self.event)
  File "C:/FractureID/projects/python/ui/spectraqc/PyEventBusBug.py", line 26, in on_event
    print (self.something_else)
AttributeError: Subscriber1 instance has no attribute 'something_else'

    It complains about the variable in class two not having the attribute in the first class and the other way around.

    If I change on of the on_event to something else like on_event2 then the issue is gone.

    from pyeventbus import *


class SomeEvent:
    def __init__(self):
        pass


class Subscriber1:
    def __init__(self):
        self.something = 'First subscriber'
        PyBus.Instance().register(self, self.__class__.__name__)

    @subscribe(threadMode=Mode.PARALLEL, onEvent=SomeEvent)
    def on_event(self, event):
        print (self.something)


class Subscriber2:
    def __init__(self):
        self.something_else = 'Second subscriber'
        PyBus.Instance().register(self, self.__class__.__name__)

    @subscribe(threadMode=Mode.PARALLEL, onEvent=SomeEvent)
    def on_event(self, event):
        print (self.something_else)


class PyEventBusBug:

    def __init__(self):
        Subscriber1()
        Subscriber2()
        PyBus.Instance().post(SomeEvent())


if __name__ == "__main__":
    PyEventBusBug()
    bug 
    opened by ddanny 0
  • Doesn't even start on Windows because 2000 threads is apparently too much

    Doesn't even start on Windows because 2000 threads is apparently too much 

      File "C:\Python27\lib\site-packages\pyeventbus\pyeventbus.py", line 116, in subscribe
    bus = PyBus.Instance()
  File "C:\Python27\lib\site-packages\pyeventbus\Singleton.py", line 30, in Instance
    self._instance = self._decorated()
  File "C:\Python27\lib\site-packages\pyeventbus\pyeventbus.py", line 24, in __init__
    for worker in [lambda: self.startWorkers() for i in range(self.num_threads)]: worker()
  File "C:\Python27\lib\site-packages\pyeventbus\pyeventbus.py", line 24, in <lambda>
    for worker in [lambda: self.startWorkers() for i in range(self.num_threads)]: worker()
  File "C:\Python27\lib\site-packages\pyeventbus\pyeventbus.py", line 30, in startWorkers
    worker.start()
  File "C:\Python27\lib\threading.py", line 736, in start
    _start_new_thread(self.__bootstrap, ())
thread.error: can't start new thread

    See also: https://stackoverflow.com/a/1835043/2583080

    bug 
    opened by PawelTroka 4
Releases(0.2)
Owner
null
GitHub
Event-forecasting - Event Forecasting Algorithms With Python

event-forecasting Event Forecasting Algorithms Theory Correlating events in comp

Intellia ICT 4 Feb 15, 2022
Event sourced bank - A wide-and-shallow example using the Python event sourcing library

Event Sourced Bank A "wide but shallow" example of using the Python event sourci

null 3 Mar 9, 2022
Lane follower: Lane-detector (OpenCV) + Object-detector (YOLO5) + CAN-bus

Lane Follower This code is for the lane follower, including perception and control, as shown below. Environment Hardware Industrial Camera Intel-NUC(1

Siqi Fan 3 Jul 7, 2022
Predict bus arrival time using VertexAI and Nvidia's Jetson Nano

bus_prediction predict bus arrival time using VertexAI and Nvidia's Jetson Nano imagenet the command for imagenet.py look like this python3 /path/to/i

null 10 Dec 22, 2022
This repository allows you to anonymize sensitive information in images/videos. The solution is fully compatible with the DL-based training/inference solutions that we already published/will publish for Object Detection and Semantic Segmentation.

BMW-Anonymization-Api Data privacy and individuals’ anonymity are and always have been a major concern for data-driven companies. Therefore, we design

BMW TechOffice MUNICH 148 Dec 21, 2022
Repo for FUZE project. I will also publish some Linux kernel LPE exploits for various real world kernel vulnerabilities here. the samples are uploaded for education purposes for red and blue teams.

Linux_kernel_exploits Some Linux kernel exploits for various real world kernel vulnerabilities here. More exploits are yet to come. This repo contains

Wei Wu 472 Dec 21, 2022
Code and model benchmarks for "SEVIR : A Storm Event Imagery Dataset for Deep Learning Applications in Radar and Satellite Meteorology"

NeurIPS 2020 SEVIR Code for paper: SEVIR : A Storm Event Imagery Dataset for Deep Learning Applications in Radar and Satellite Meteorology Requirement

USAF - MIT Artificial Intelligence Accelerator 46 Dec 15, 2022
Code for the ECCV2020 paper "A Differentiable Recurrent Surface for Asynchronous Event-Based Data"

A Differentiable Recurrent Surface for Asynchronous Event-Based Data Code for the ECCV2020 paper "A Differentiable Recurrent Surface for Asynchronous

Marco Cannici 21 Oct 5, 2022
Scalable, event-driven, deep-learning-friendly backtesting library

...Minimizing the mean square error on future experience. - Richard S. Sutton BTGym Scalable event-driven RL-friendly backtesting library. Build on

Andrew 922 Dec 27, 2022
Python Implementation of the CoronaWarnApp (CWA) Event Registration

Python implementation of the Corona-Warn-App (CWA) Event Registration This is an implementation of the Protocol used to generate event and location QR

MaZderMind 17 Oct 5, 2022