Python Client for the Etsy NodeJS Statsd Server

Overview

Introduction

Test Status Coverage Status

statsd is a client for Etsy's statsd server, a front end/proxy for the Graphite stats collection and graphing server.

Links

Install

To install simply execute python setup.py install. If you want to run the tests first, run python setup.py nosetests

Usage

To get started real quick, just try something like this:

Basic Usage

Timers

>>> import statsd
>>>
>>> timer = statsd.Timer('MyApplication')
>>>
>>> timer.start()
>>> # do something here
>>> timer.stop('SomeTimer')

Counters

>>> import statsd
>>>
>>> counter = statsd.Counter('MyApplication')
>>> # do something here
>>> counter += 1

Gauge

>>> import statsd
>>>
>>> gauge = statsd.Gauge('MyApplication')
>>> # do something here
>>> gauge.send('SomeName', value)

Raw

Raw strings should be e.g. pre-summarized data or other data that will get passed directly to carbon. This can be used as a time and bandwidth-saving mechanism sending a lot of samples could use a lot of bandwidth (more b/w is used in udp headers than data for a gauge, for instance).

>>> import statsd
>>>
>>> raw = statsd.Raw('MyApplication', connection)
>>> # do something here
>>> raw.send('SomeName', value, timestamp)

The raw type wants to have a timestamp in seconds since the epoch (the standard unix timestamp, e.g. the output of "date +%s"), but if you leave it out or provide None it will provide the current time as part of the message

Average

>>> import statsd
>>>
>>> average = statsd.Average('MyApplication', connection)
>>> # do something here
>>> average.send('SomeName', 'somekey:%d'.format(value))

Connection settings

If you need some settings other than the defaults for your Connection, you can use Connection.set_defaults().

>>> import statsd
>>> statsd.Connection.set_defaults(host='localhost', port=8125, sample_rate=1, disabled=False)

Every interaction with statsd after these are set will use whatever you specify, unless you explicitly create a different Connection to use (described below).

Defaults:

  • host = 'localhost'
  • port = 8125
  • sample_rate = 1
  • disabled = False

Advanced Usage

>>> import statsd
>>>
>>> # Open a connection to `server` on port `1234` with a `50%` sample rate
>>> statsd_connection = statsd.Connection(
...     host='server',
...     port=1234,
...     sample_rate=0.5,
... )
>>>
>>> # Create a client for this application
>>> statsd_client = statsd.Client(__name__, statsd_connection)
>>>
>>> class SomeClass(object):
...     def __init__(self):
...         # Create a client specific for this class
...         self.statsd_client = statsd_client.get_client(
...             self.__class__.__name__)
...
...     def do_something(self):
...         # Create a `timer` client
...         timer = self.statsd_client.get_client(class_=statsd.Timer)
...
...         # start the measurement
...         timer.start()
...
...         # do something
...         timer.intermediate('intermediate_value')
...
...         # do something else
...         timer.stop('total')

If there is a need to turn OFF the service and avoid sending UDP messages, the Connection class can be disabled by enabling the disabled argument:

>>> statsd_connection = statsd.Connection(
...     host='server',
...     port=1234,
...     sample_rate=0.5,
...     disabled=True
... )

If logging's level is set to debug the Connection object will inform it is not sending UDP messages anymore.

Comments
  • Change raw metric message to match format expected by StatsD

    Change raw metric message to match format expected by StatsD

    Send raw messages as "name:value|r|timestamp". This is compatible with https://github.com/jeffminard-ck/statsd/tree/raw_and_averages, currently pending a pull request into https://github.com/etsy/statsd. This format is also more consistent with other Etsy statsd message formats.

    opened by jeremiahshirk 10
  • Allow sending multiple stats + minimize DNS lookups

    Allow sending multiple stats + minimize DNS lookups

    In wrapping the library for use in ChromiumOS's testing infrastructure, there were a couple small changes that are exceedingly hacky to maintain locally. Thus, here they are, I hope you find them agreeable. If you do, I'd much appreciate a pypi package bump afterwards.

    opened by thisismiller 6
  • Minor changes for convenience

    Minor changes for convenience

    This pull request cover two minor changes to make the library slightly easier to use:

    • I have added methods to statsd.client.Client which give back an instance of whichever object type you have requested. I think this gives a cleaner interface than using the existing get_client() way.
    • Made statsd.timer.Timer.start() return the timer instance so you can chain the instantiation and start of the timer, while still assigning the timer to a variable like this:
      timer = Timer('application_name').start()
    opened by Tenzer 5
  • Why is setup_requires=['nose', 'mock', 'coverage'] needed?

    Why is setup_requires=['nose', 'mock', 'coverage'] needed?

    I ask because I ran into issues while trying to install python-statsd through a proxy. e.g., ('pip install --proxy myproxy.net:80 python-statsd'). The install fails because its unable to download the setup_requires packages. If I remove the 'setup_requires', install succeeds.

    Still trying to figure out the issue. Seems to be related with setuptools/ssl_support.py(184)https_open() though.

    So the root cause of my install problem is not the setup_requires=['nose', 'mock', 'coverage'] line, but it made me wonder why this is even needed since these are test dependency modules. Are they needed for the package to function? If it's just for tests, should they be in tests_require?

    opened by victoradan 5
  • Small time deltas are swallowed up by formatting

    Small time deltas are swallowed up by formatting

    I'm trying to measure the time of the invocations of small functions. The format for timers has my actual times as <1ms, so it's sending 0|ms. For my own use I'm sending %0.08f as the format for timers in that particular virtualenv so my problem isn't urgent, but I'd like to know if you've thought about this, e.g. passing in a format string for the timer so that the scale can be set by the caller?

    opened by pcn 5
  • Fixed a bug where ``self.logger`` wouldn't be available in manual use.

    Fixed a bug where ``self.logger`` wouldn't be available in manual use.

    In trying to manually use the following code:

    timer = statsd.Timer(slug, connection=conn)
    timer.send('total', seconds_taken)
    

    ...it would blow up with the exception AttributeError: 'Timer' object has no attribute 'logger'. This commit fixes that.

    opened by toastdriven 5
  • Fix for sending 0ms entries.

    Fix for sending 0ms entries.

    DEBUG: Bad line: 0,ms in msg "production.timer:0|ms" https://github.com/etsy/statsd/blob/5b900711191a9c5cfd536c877a3c119d1df5679a/lib/helpers.js Line 35: return isNumber(fields[0]) && Number(fields[0]) > 0;

    statsD expects a > 0 number for ms counters.

    opened by anthroprose 3
  • please add License / Copying information

    please add License / Copying information

    Hello,

    According to the pypi page at https://pypi.python.org/pypi/python-statsd this software is released under a BSD license. Would you be kind enough to add a copy of the license in the git source code as well as the copyright information? Thank you!

    opened by hashar 3
  • Multi-Metric packets

    Multi-Metric packets

    Hello,

    Are you interested in supporting the multi-metric-packets functionality as described here: https://github.com/etsy/statsd/#multi-metric-packets

    We are concerned with sending too many packets from our python applications.

    I'd be happy to implement it and send a pull, just figured I'd ask if you don't support it for a reason.

    opened by pgroudas 3
  • Naming on metrics is not consistent

    Naming on metrics is not consistent

    I am not sure if this is a problem with how python-statsd is sending its data or how StatsD is processing and relaying to graphite, but there is some inconsistency that makes it hard to maintain a naming convention when sending metrics over.

    For example, for counters, if we use a "my_counter" name, it displays like this on Graphite:

    Graphite
    `-- stats
        `-- my_counter
    

    But for gauges or timers, they would add a prepending name depending on the name of the metric. So the above example will end up looking like this:

    Graphite
    `-- stats
        |-- timers
        |   |-- my_timer
        |-- gauges
        |   |-- my_gauge
        `-- my_counter
    

    So timers and gauges prepend their name, but counters do not. If we try to use a naming convention (e.g. server name) it would be difficult to try and keep an order because of this.

    I wasn't able to effectively find out what is missing here, but as far as I can tell python-statsd is sending the information correctly, without mangling the names we use.

    Any ideas?

    opened by alfredodeza 3
  • Gauges take arbitrary values, not just integers.

    Gauges take arbitrary values, not just integers.

    From https://github.com/etsy/statsd/blob/bf62f646d195f08fb17a37f7352be47366b2eaa9/stats.js#L120, any decimal/floating point number can be stored in a gauge by statsd. However, https://github.com/WoLpH/python-statsd/blob/master/statsd/gauge.py#L18 sends only an integer.

    Example code:

    from decimal import Decimal
    import mock
    import statsd
    
    with mock.patch('statsd.Client') as mock_client:
        instance = mock_client.return_value
        instance._send.return_value = 1
    
        gauge = statsd.Gauge('testing')
        gauge.send('', 10.5)
        mock_client._send.assert_called_with(mock.ANY, {'testing': '10.5|g'}) # Actually sends '10|g'
    
        gauge.send('', Decimal('6.576'))
        mock_client._send.assert_called_with(mock.ANY, {'testing': '6.576|g'}) # Actually sends '6|g'
    
        gauge('', 1)
        mock_client._send.assert_called_with(mock.ANY, {'testing': '1|g'}) # Correctly sends '1|g'
    
    opened by toastdriven 3
  • Conflict with 'statsd' install dir

    Conflict with 'statsd' install dir

    pip install statsd python-statsd results in a mess in site-packages. Unlikely to be a real issue, only noticed because i was trying to figure out the differences between these and pystatsd, statsd-client

    opened by kaidokert 4
Owner
Rick van Hattem
Author of @mastering-python and entrepreneur interested in scaling large and complicated systems.
Rick van Hattem
A next generation HTTP client for Python. 🦋

HTTPX - A next-generation HTTP client for Python. HTTPX is a fully featured HTTP client for Python 3, which provides sync and async APIs, and support

Encode 9.8k Jan 5, 2023
Python requests like API built on top of Twisted's HTTP client.

treq: High-level Twisted HTTP Client API treq is an HTTP library inspired by requests but written on top of Twisted's Agents. It provides a simple, hi

Twisted Matrix Labs 553 Dec 18, 2022
A modern/fast python SOAP client based on lxml / requests

Zeep: Python SOAP client A fast and modern Python SOAP client Highlights: Compatible with Python 3.6, 3.7, 3.8 and PyPy Build on top of lxml and reque

Michael van Tellingen 1.7k Jan 1, 2023
Aiosonic - lightweight Python asyncio http client

aiosonic - lightweight Python asyncio http client Very fast, lightweight Python asyncio http client Here is some documentation. There is a performance

Johanderson Mogollon 93 Jan 6, 2023
As easy as /aitch-tee-tee-pie/ 🥧 Modern, user-friendly command-line HTTP client for the API era. JSON support, colors, sessions, downloads, plugins & more. https://twitter.com/httpie

HTTPie: human-friendly CLI HTTP client for the API era HTTPie (pronounced aitch-tee-tee-pie) is a command-line HTTP client. Its goal is to make CLI in

HTTPie 25.4k Jan 1, 2023
A minimal HTTP client. ⚙️

HTTP Core Do one thing, and do it well. The HTTP Core package provides a minimal low-level HTTP client, which does one thing only. Sending HTTP reques

Encode 306 Dec 27, 2022
An interactive command-line HTTP and API testing client built on top of HTTPie featuring autocomplete, syntax highlighting, and more. https://twitter.com/httpie

HTTP Prompt HTTP Prompt is an interactive command-line HTTP client featuring autocomplete and syntax highlighting, built on HTTPie and prompt_toolkit.

HTTPie 8.6k Dec 31, 2022
Screaming-fast Python 3.5+ HTTP toolkit integrated with pipelining HTTP server based on uvloop and picohttpparser.

Screaming-fast Python 3.5+ HTTP toolkit integrated with pipelining HTTP server based on uvloop and picohttpparser.

Paweł Piotr Przeradowski 8.6k Jan 4, 2023
Python HTTP library with thread-safe connection pooling, file post support, user friendly, and more.

urllib3 is a powerful, user-friendly HTTP client for Python. Much of the Python ecosystem already uses urllib3 and you should too. urllib3 brings many

urllib3 3.2k Dec 29, 2022
Python HTTP library with thread-safe connection pooling, file post support, user friendly, and more.

urllib3 is a powerful, user-friendly HTTP client for Python. Much of the Python ecosystem already uses urllib3 and you should too. urllib3 brings many

urllib3 3.2k Jan 2, 2023
PycURL - Python interface to libcurl

PycURL -- A Python Interface To The cURL library PycURL is a Python interface to libcurl, the multiprotocol file transfer library. Similarly to the ur

PycURL 933 Jan 9, 2023
A toolbelt of useful classes and functions to be used with python-requests

The Requests Toolbelt This is just a collection of utilities for python-requests, but don't really belong in requests proper. The minimum tested reque

null 892 Jan 6, 2023
Asynchronous Python HTTP Requests for Humans using Futures

Asynchronous Python HTTP Requests for Humans Small add-on for the python requests http library. Makes use of python 3.2's concurrent.futures or the ba

Ross McFarland 2k Dec 30, 2022
HTTP/2 for Python.

Hyper: HTTP/2 Client for Python This project is no longer maintained! Please use an alternative, such as HTTPX or others. We will not publish further

Hyper 1k Dec 23, 2022
HTTP request/response parser for python in C

http-parser HTTP request/response parser for Python compatible with Python 2.x (>=2.7), Python 3 and Pypy. If possible a C parser based on http-parser

Benoit Chesneau 334 Dec 24, 2022
Python Simple SOAP Library

PySimpleSOAP / soap2py Python simple and lightweight SOAP library for client and server webservices interfaces, aimed to be as small and easy as possi

PySimpleSOAP 369 Jan 2, 2023
🔄 🌐 Handle thousands of HTTP requests, disk writes, and other I/O-bound tasks simultaneously with Python's quintessential async libraries.

?? ?? Handle thousands of HTTP requests, disk writes, and other I/O-bound tasks simultaneously with Python's quintessential async libraries.

Hackers and Slackers 15 Dec 12, 2022
A Python obfuscator using HTTP Requests and Hastebin.

?? Jawbreaker ?? Jawbreaker is a Python obfuscator written in Python3, using double encoding in base16, base32, base64, HTTP requests and a Hastebin-l

Billy 50 Sep 28, 2022
Single-file replacement for python-requests

mureq mureq is a single-file, zero-dependency replacement for python-requests, intended to be vendored in-tree by Linux systems software and other lig

Shivaram Lingamneni 267 Dec 28, 2022