A Python 3 client for the beanstalkd work queue

Overview

Greenstalk

Greenstalk is a small and unopinionated Python client library for communicating with the beanstalkd work queue. The API provided mostly maps one-to-one with commands in the protocol.

https://github.com/justinmayhew/greenstalk/workflows/CI/badge.svg?branch=main

Quickstart

>>> import greenstalk
>>> client = greenstalk.Client(('127.0.0.1', 11300))
>>> client.put(b'hello')
1
>>> job = client.reserve()
>>> job.id
1
>>> job.body
b'hello'
>>> client.delete(job)
>>> client.close()

Documentation is available on Read the Docs.

Comments
  • Fix tests.

    Fix tests.

    Tests are failing (reproduced on 1.0.0 and master) with :

    ========================================== FAILURES ===========================================
    ______________________________ test_delete_job_reserved_by_other ______________________________
    
        def wrapper() -> None:
            cmd = ('beanstalkd', '-l', '127.0.0.1', '-p', str(PORT))
            with subprocess.Popen(cmd) as beanstalkd:
                time.sleep(0.1)
                try:
                    with Client(port=PORT, **kwargs) as c:
    >                   test(c)
    
    tests.py:27: 
    _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
    tests.py:360: in test_delete_job_reserved_by_other
        c.delete(job)
    greenstalk.py:253: in delete
        self._send_cmd(b'delete %d' % _to_id(job), b'DELETED')
    greenstalk.py:172: in _send_cmd
        return _parse_response(line, expected)
    _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
    
    line = b'NOT_FOUND', expected = b'DELETED'
    
        def _parse_response(line: bytes, expected: bytes) -> List[bytes]:
            if not line:
                raise ConnectionError("Unexpected EOF")
        
            assert line[-2:] == b'\r\n'
            line = line[:-2]
        
            status, *values = line.split()
        
            if status == expected:
                return values
        
            if status in ERROR_RESPONSES:
    >           raise ERROR_RESPONSES[status](values)
    E           greenstalk.NotFoundError: []
    
    greenstalk.py:396: NotFoundError
    ============================ 1 failed, 35 passed in 11.09 seconds =============================
    

    I guess the other job must be deleted by its owner, so here is a patch for it.

    opened by dargor 4
  • BadFormatError for reserve_job

    BadFormatError for reserve_job

    I can't help but think I'm missing something obvious, but I can't for the life of me get reserve_job working. Using the basic code below to reserve a job ID I know exists and is available in the watched tube:

    import greenstalk
    client = greenstalk.Client(('192.168.1.2', 11300), watch='testtube')
    client.reserve_job(1)
    

    I get a BadFormatError which the documentation suggests shouldn't be hit unless something is amiss in the library.

    Traceback (most recent call last):
      File "/Users/ropetin/greenstalk/./testgreen.py", line 3, in <module>
        client.reserve_job(1)
      File "/Users/ropetin/.local/share/virtualenvs/greenstalk-evfV_dNi/lib/python3.9/site-packages/greenstalk.py", line 261, in reserve_job
        return self._job_cmd(b'reserve-job %d' % id, b'RESERVED')
      File "/Users/ropetin/.local/share/virtualenvs/greenstalk-evfV_dNi/lib/python3.9/site-packages/greenstalk.py", line 187, in _job_cmd
        id, size = (int(n) for n in self._send_cmd(cmd, expected))
      File "/Users/ropetin/.local/share/virtualenvs/greenstalk-evfV_dNi/lib/python3.9/site-packages/greenstalk.py", line 176, in _send_cmd
        return _parse_response(line, expected)
      File "/Users/ropetin/.local/share/virtualenvs/greenstalk-evfV_dNi/lib/python3.9/site-packages/greenstalk.py", line 411, in _parse_response
        raise ERROR_RESPONSES[status](values)
    greenstalk.BadFormatError: []
    
    opened by ropetin 2
  • Add destructor to Client.

    Add destructor to Client.

    This adds the destructor to the client to clean up connections.

    Without it, unittest may return errors:

    /usr/lib/python3.6/socket.py:657: ResourceWarning: unclosed <socket.socket fd=15, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=6, laddr=('127.0.0.1', 42328), raddr=('127.0.0.1', 11300)>
      self._sock = None
    
    opened by jordanmack 2
  • Add typing to dependencies

    Add typing to dependencies

    After installing this module (version 0.5.0) from PyPI on Debian Stretch, I got the following error:

    >>> import greenstalk
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
      File "/home/user/programming/queue-test/venv/lib/python3.4/site-packages/greenstalk.py", line 2, in <module>
        from typing import Any, BinaryIO, Dict, Iterable, List, Optional, Tuple, Union
    ImportError: No module named 'typing'
    

    After installing the typing module (at version 3.6.1) it worked fine.

    opened by natruiz3555 2
  • Document for binary data

    Document for binary data

    Most of the usage could be inferred via beanstalkd. But the encoding control needs more documents.

    sample

    It is controled by the encoding of Client.__init__().

    When encoding is not None, the input of Client.put() should be str, and the output of Client.reserve() would be str too. Consistent encoding for both producer and consumer is neccessary.

    When encoding is None, the input of Client.put() should be bytes, and the output of Client.reserve() would be bytes too. Consistent encoding (None) for both producer and consumer is neccessary.

    Behaviours under other settings are undefined.

    opened by zhengwx11 2
  • Documentation?

    Documentation?

    I've been looking for a beanstalkd python engine, and yours seems to be the only one with Python 3.5+ support. However I don't really know how to use it due to the lack of documentation. If you could spin up some documentation on how to set up a basic implementation of it I'd greatly appreciate it. Especially because then I could use it, and possibly, contribute

    Edit: I've been reading through the source code, and I understand how it works now. However having some documentation would still be very useful

    opened by devinmatte 1
  • Tests failing on beanstalkd master branch

    Tests failing on beanstalkd master branch

    Hey, I've been working on a PR to run test suites of beanstalkd clients on pushes to the beanstalkd source code. (The PR is not yet merged: https://github.com/beanstalkd/beanstalkd/pull/614)

    I'm posting this here since greenstalk tests will fail as soon as a new beanstalkd version is released:

    https://github.com/SamMousa/beanstalkd/runs/4712188954?check_suite_focus=true

    opened by SamMousa 2
Cup Noodle Vending Maching Ordering Queue

Noodle-API Cup Noodle Vending Machine Ordering Queue Install dependencies in virtual environment python3 -m venv

Jonas Kazlauskas 1 Dec 9, 2021
An After Effects render queue for ShotGrid Toolkit.

AEQueue An After Effects render queue for ShotGrid Toolkit. Features Render multiple comps to locations defined by templates in your Toolkit config. C

Brand New School 5 Nov 20, 2022
Iris-client - Python client for DFIR-IRIS

Python client dfir_iris_client offers a Python interface to communicate with IRI

DFIR-IRIS 11 Dec 22, 2022
Block fingerprinting for the beacon chain, for client identification & client diversity metrics

blockprint This is a repository for discussion and development of tools for Ethereum block fingerprinting. The primary aim is to measure beacon chain

Sigma Prime 49 Dec 8, 2022
Datamol is a python library to work with molecules.

Datamol is a python library to work with molecules. It's a layer built on top of RDKit and aims to be as light as possible.

datamol 276 Dec 19, 2022
Python module to work with Magneto Database directly without using broken Magento 2 core

Python module to work with Magneto Database directly without using broken Magento 2 core

Egor Shitikov 13 Nov 10, 2022
Make after-work Mending More flexible In Python

Mending Make after-work Mending More flexible In Python A Lite Package focuses on making project's after-post mending pythonic and flexible. Certainly

null 2 Jun 15, 2022
A Python Perforce package that doesn't bring in any other packages to work.

P4CMD ?? A Python Perforce package that doesn't bring in any other packages to work. Relies on p4cli installed on the system. p4cmd The p4cmd module h

Niels Vaes 13 Dec 19, 2022
A project to work with databases in 4 worksheets, insert, update, select, delete using Python and MySqI

A project to work with databases in 4 worksheets, insert, update, select, delete using Python and MySqI As a small project for school or college hope it is useful

Sina Org 1 Jan 11, 2022
This is an implementation of NeuronJ work with python.

NeuronJ This is an implementation of NeuronJ work with python. NeuronJ is a plug-in for ImageJ that allows you to create and edit neurons masks. Image

Mohammad Mahdi Samei 3 Aug 28, 2022
Your copilot to studies and work (Pomodoro-timer, Translate and Notes app)

Copylot Your copilot to studies and work (Pomodoro-timer, Translate and Notes app) Copylot are three applications in one: Pomodoro Translate Notes Cop

Eduardo Mendes 20 Dec 16, 2022
A step-by-step tutorial for how to work with some of the most basic features of Nav2 using a Jupyter Notebook in a warehouse environment to create a basic application.

This project has a step-by-step tutorial for how to work with some of the most basic features of Nav2 using a Jupyter Notebook in a warehouse environment to create a basic application.

Steve Macenski 49 Dec 22, 2022
A Puzzle A Day Keep the Work Away

A Puzzle A Day Keep the Work Away No moyu again!

P4SSER8Y 5 Feb 12, 2022
✔️ Create to-do lists to easily manage your ideas and work.

Todo List + Add task + Remove task + List completed task + List not completed task + Set clock task time + View task statistics by date Changelog v 1.

Abbas Ataei 30 Nov 28, 2022
Comics/doujinshi reader application. Web-based, will work on desktop and tablet devices with swipe interface.

Yomiko Comics/doujinshi reader application. Web-based, will work on desktop and tablet devices with swipe interface. Scans one or more directories of

Kyubi Systems 26 Aug 10, 2022
Research on how Gboard Stickers work.

Google-Sticker-Mashup-Research Research on how Gboard Stickers work. Contribute Contributing is nice, and you will be listed below for contributing. C

Jeremiah 45 Oct 28, 2022
Blender Light Manipulation - A script that makes it easier to work with light

Blender Light Manipulation A script that makes it easier to work with light 1. Wstęp W poniższej dokumentacji przedstawiony zostanie skrypt, który swo

Tomasz 1 Oct 19, 2021
Script to work around some quirks of the blender obj importer

ObjFix 1.0 (WIP) Script to work around some quirks of the blender obj importer Installation Download this repo In Blender, press "Edit" on the top-bar

Red_3D 4 Nov 20, 2021
This is a small compiler to demonstrate how compilers work.

This is a small compiler to demonstrate how compilers work. It compiles our own dialect to C, while being written in Python.

Md. Tonoy Akando 2 Jul 19, 2022