rosny is a lightweight library for building concurrent systems.

Overview

ROSNY

PyPI version Test CodeFactor codecov Downloads

rosny is a lightweight library for building concurrent systems.

Installation

Tested on:

  • Linux
  • Python >= 3.6

From pip:

pip install rosny

From source:

pip install git+https://github.com/lRomul/rosny.git@master

Example

from multiprocessing import Queue
from rosny import ThreadStream, ProcessStream, ComposeStream


class SenderStream(ThreadStream):  # using threading.Thread
    def __init__(self, queue: Queue):
        super().__init__(loop_rate=30)
        self.queue = queue
        self.count = 0

    # run the method in a loop in a separate thread
    def work(self):
        self.queue.put(self.count)
        self.logger.info(f'put {self.count}')
        self.count += 1


class ReceiverStream(ProcessStream):  # using multiprocessing.Process
    def __init__(self, queue: Queue):
        super().__init__()
        self.queue = queue

    # run the method in a loop in a separate process
    def work(self):
        value = self.queue.get(timeout=1)
        self.logger.info(f'get {value}')


class MainStream(ComposeStream):  # merging several streams
    def __init__(self):
        super().__init__()
        queue = Queue()
        self.sender = SenderStream(queue)
        self.receiver = ReceiverStream(queue)


if __name__ == "__main__":
    stream = MainStream()
    stream.start()
    stream.wait(5)
    stream.stop()
    stream.join()
You might also like...
Functional interface for concurrent futures, including asynchronous I/O.

Futured provides a consistent interface for concurrent functional programming in Python. It wraps any callable to return a concurrent.futures.Future,

A Python script that exports users from one Telegram group to another using one or more concurrent user bots.

ExportTelegramUsers A Python script that exports users from one Telegram group to another using one or more concurrent user bots. Make sure to set all

A wrapper around ffmpeg to make it work in a concurrent and memory-buffered fashion.

Media Fixer Have you ever had a film or TV show that your TV wasn't able to play its audio? Well this program is for you. Media Fixer is a program whi

A concurrent sync tool which works with multiple sources and targets.

Concurrent Sync A concurrent sync tool which works similar to rsync. It supports syncing given sources with multiple targets concurrently. Requirement

API for RL algorithm design & testing of BCA (Building Control Agent) HVAC on EnergyPlus building energy simulator by wrapping their EMS Python API
API for RL algorithm design & testing of BCA (Building Control Agent) HVAC on EnergyPlus building energy simulator by wrapping their EMS Python API

RL - EmsPy (work In Progress...) The EmsPy Python package was made to facilitate Reinforcement Learning (RL) algorithm research for developing and tes

A Python scikit for building and analyzing recommender systems

Overview Surprise is a Python scikit for building and analyzing recommender systems that deal with explicit rating data. Surprise was designed with th

Official repository for
Official repository for "Action-Based Conversations Dataset: A Corpus for Building More In-Depth Task-Oriented Dialogue Systems"

Action-Based Conversations Dataset (ABCD) This respository contains the code and data for ABCD (Chen et al., 2021) Introduction Whereas existing goal-

Repository for a project of the course EP2520 Building Networked Systems Security

EP2520_ACME_Project Repository for a project of the course EP2520 Building Networked Systems Security in Royal Institute of Technology (KTH), Stockhol

Example Python code for building RPi-controlled robotic systems

RPi Example Code Example Python code for building RPi-controlled robotic systems These python files have been compiled / developed by the Neurobionics

Sierra is a lightweight Python framework for building and integrating web applications
Sierra is a lightweight Python framework for building and integrating web applications

A lightweight Python framework for building and Integrating Web Applications. Sierra is a Python3 library for building and integrating web applications with HTML and CSS using simple enough syntax. You can develop your web applications with Python, taking advantage of its functionalities and integrating them to the fullest.

An advanced multi-threaded, multi-client python reverse shell for hacking linux systems. There's still more work to do so feel free to help out with the development. Disclaimer: This reverse shell should only be used in the lawful, remote administration of authorized systems. Accessing a computer network without authorization or permission is illegal. Python toolkit for defining+simulating+visualizing+analyzing attractors, dynamical systems, iterated function systems, roulette curves, and more
Python toolkit for defining+simulating+visualizing+analyzing attractors, dynamical systems, iterated function systems, roulette curves, and more

Attractors A small module that provides functions and classes for very efficient simulation and rendering of iterated function systems; dynamical syst

generate HPC scheduler systems jobs input scripts and submit these scripts to HPC systems and poke until they finish

DPDispatcher DPDispatcher is a python package used to generate HPC(High Performance Computing) scheduler systems (Slurm/PBS/LSF/dpcloudserver) jobs in

Distributed-systems-algos - Distributed Systems Algorithms For Python

Distributed Systems Algorithms ISIS algorithm In an asynchronous system that kee

Code for Private Recommender Systems: How Can Users Build Their Own Fair Recommender Systems without Log Data? (SDM 2022)

Private Recommender Systems: How Can Users Build Their Own Fair Recommender Systems without Log Data? (SDM 2022) We consider how a user of a web servi

solsim is the Solana complex systems simulator. It simulates behavior of dynamical systems—DeFi protocols, DAO governance, cryptocurrencies, and more—built on the Solana blockchain
solsim is the Solana complex systems simulator. It simulates behavior of dynamical systems—DeFi protocols, DAO governance, cryptocurrencies, and more—built on the Solana blockchain

solsim is the Solana complex systems simulator. It simulates behavior of dynamical systems—DeFi protocols, DAO governance, cryptocurrencies, and more—built on the Solana blockchain

Control System Packer is a lightweight, low-level program to transform energy equations into the compact libraries for control systems.
Control System Packer is a lightweight, low-level program to transform energy equations into the compact libraries for control systems.

Control System Packer is a lightweight, low-level program to transform energy equations into the compact libraries for control systems. Packer supports Python 🐍 , C 💻 and C++ 💻 libraries.

Endpoints is a lightweight REST api framework written in python and used in multiple production systems that handle millions of requests daily.

Endpoints Quickest API builder in the West! Endpoints is a lightweight REST api framework written in python and used in multiple production systems th

Releases(v0.0.6)
  • v0.0.6(Oct 17, 2022)

    Docs

    • Setup material MkDocs.
    • Add logo.

    Examples

    • Multiprocessing pose estimation with mediapipe example (link).
    • Threads inside process example (link).

    Breaking Changes

    • Rename work_loop method to loop.
    • Rename on_work_loop_begin callback method to on_loop_begin.
    • Rename on_work_loop_end callback method to on_loop_end.
    Source code(tar.gz)
    Source code(zip)
  • v0.0.5(Jun 28, 2021)

    New Features

    • Loop rate profiling via parameter profile_interval of ThreadStream and ProcessStream.
    • ProcessStream now works with any methods to start a process: spawn, fork, and forkserver. User should use multiprocessing.set_start_method() function before starting streams.
    • New callback methods: on_compile_begin, on_compile_end, on_work_loop_begin, on_work_loop_end.
    • Internal state is now a common_state and can be set by user via compile method.
    • Add handle_signals parameter of compile method. A boolean value indicating whether handling systems signals.

    Examples

    • Video pipeline with mediapipe (link).

    Breaking Changes

    • Remove callback methods: on_wait_begin, on_wait_end.
    • Remove SIGQUIT handling.
    Source code(tar.gz)
    Source code(zip)
  • v0.0.4(Jun 22, 2021)

    New Features

    • rosny.process.ProcessStream like ThreadStream runs a task in a loop with the ability to set the loop frequency but in a separate process.
    • rosny.signal - set of functions for handling system signals (SIGINT, SIGTERM, SIGQUIT).
    • rosny.timing.LoopRateManager limits rate of a loop.
    • daemon parameter for ThreadStream/ProcessStream indicating whether thread/process will a daemon or not.

    Examples

    • Threads vs processes on CPU-bound task (link).
    Source code(tar.gz)
    Source code(zip)
  • v0.0.3(Jun 19, 2021)

    New Features

    • compiled, stopped, joined predicate methods of streams.
    • Edit logging messages.

    Docs

    • Add badges to README.md. PyPI versionTestCodeFactorcodecovDownloads

    CI/CD

    • Tests.
    • Upload code coverage to codecov (codecov).
    • Type checking with mypy.

    Breaking Changes

    • rosny.thread.ThreadStream: make rate_manager as a public attribute.
    Source code(tar.gz)
    Source code(zip)
  • v0.0.2(Jun 17, 2021)

    rosny is a simple library for building concurrency systems.

    Features

    • rosny.thread.ThreadStream runs a task in a loop with the ability to set the loop frequency.
    • rosny.compose.ComposeStream orchestrates several streams.
    • rosny.timing.LoopRateManager limits rate of a loop.

    Examples

    • Common counter (link).
    • Sender and receiver (link).

    CI/CD

    • Publish а package to PyPI.
    • Check code style with flake8.
    • Run tests with pytest.
    Source code(tar.gz)
    Source code(zip)
Owner
Ruslan Baikulov
Tech Lead at osai.ai
Ruslan Baikulov
Simple package to enhance Python's concurrent.futures for memory efficiency

future-map is a Python library to use together with the official concurrent.futures module.

Arai Hiroki 2 Nov 15, 2022
A concurrent sync tool which works with multiple sources and targets.

Concurrent Sync A concurrent sync tool which works similar to rsync. It supports syncing given sources with multiple targets concurrently. Requirement

Halit Şimşek 2 Jan 11, 2022
A lightweight (serverless) native python parallel processing framework based on simple decorators and call graphs.

A lightweight (serverless) native python parallel processing framework based on simple decorators and call graphs, supporting both control flow and dataflow execution paradigms as well as de-centralized CPU & GPU scheduling.

null 102 Jan 6, 2023
Trio – a friendly Python library for async concurrency and I/O

Trio – a friendly Python library for async concurrency and I/O The Trio project aims to produce a production-quality, permissively licensed, async/awa

null 5k Jan 7, 2023
A "gym" style toolkit for building lightweight Neural Architecture Search systems

A "gym" style toolkit for building lightweight Neural Architecture Search systems

Jack Turner 12 Nov 5, 2022
A lightweight python module for building event driven distributed systems

Eventify A lightweight python module for building event driven distributed systems. Installation pip install eventify Problem Developers need a easy a

Eventify 16 Aug 18, 2022
A library to make concurrent selenium tests that automatically download and setup webdrivers

AutoParaSelenium A library to make parallel selenium tests that automatically download and setup webdrivers Usage Installation pip install autoparasel

Ronak Badhe 8 Mar 13, 2022
SCOOP (Scalable COncurrent Operations in Python)

SCOOP (Scalable COncurrent Operations in Python) is a distributed task module allowing concurrent parallel programming on various environments, from h

Yannick Hold 573 Dec 27, 2022
Fast as FUCK nvim completion. SQLite, concurrent scheduler, hundreds of hours of optimization.

Fast as FUCK nvim completion. SQLite, concurrent scheduler, hundreds of hours of optimization.

i love my dog 2.8k Jan 5, 2023
Simple package to enhance Python's concurrent.futures for memory efficiency

future-map is a Python library to use together with the official concurrent.futures module.

Arai Hiroki 2 Nov 15, 2022