Bcc2telegraf: An integration that sends ebpf-based bcc histogram metrics to telegraf daemon

Overview

bcc2telegraf

bcc2telegraf is an integration that sends ebpf-based bcc histogram metrics (https://github.com/iovisor/bcc) to telegraf daemon (https://github.com/influxdata/telegraf)

Original bcc/table.py v.0.10 just printed histograms to stdout. This extension instead creates a memory buffer full of text lines formatted to the requirements of telegraf line protocol and sends it by http to telegraf url .

Important difference is bcc helpers create log2 histogram with low-high ends of bucket like 0-1,1-2,3-4,5-8,9-16,17-32 etc. Tools like prometheus or grafana require those histograms be cumulative, with low always starting with 0, like 0-1,0-2,0-4,0-8,0-16,0-32 etc. So bcc2telegraf transforms log2 histograms into cumulative ones.

Sample telegraf lines generated by this module:

ebpf,metric=biolatency,host=abc.example.com,dc=east-1,tag1=dm-14,cloud=Nine,hi=512 value=100 1645026564
ebpf,metric=biolatency,host=abc.example.com,dc=east-1,tag1=dm-14,cloud=Nine,hi=1024 value=777 1645026564
ebpf,metric=biolatency,host=abc.example.com,dc=east-1,tag1=dm-14,cloud=Nine,hi=2048 value=888 1645026564

You may create some function which extends tag1 with some other tags, based on the histogram bucket struct key described in ebpf C program e.g. your_function returns 8.8.8.8,cloud=GCP then pass this function to send_log2_hist (... bucket_fn=your_function...) and telegraf line will contain ... tag1=8.8.8.8,cloud=GCP...

Usage:

  • set TELEGRAF_URL environment variable to your telegraf service
  • copy a python-based bcc program, e.g. biolatency or tcprtt
  • change it by adding an import bcc2telegraf and replace instances of dist.print_log2_hist(label...) with dist.send_log2_hist(label...) or with dist.send_log2_hist(label,bucket_fn=your_function...)
  • think about creating a systemctl-managed service out of that program
You might also like...
cqMore is a CadQuery plugin based on CadQuery 2.1.
cqMore is a CadQuery plugin based on CadQuery 2.1.

cqMore (under construction) cqMore is a CadQuery plugin based on CadQuery 2.1. Installation Please use conda to install CadQuery and its dependencies

The implementation of the paper
The implementation of the paper "HIST: A Graph-based Framework for Stock Trend Forecasting via Mining Concept-Oriented Shared Information".

The HIST framework for stock trend forecasting The implementation of the paper "HIST: A Graph-based Framework for Stock Trend Forecasting via Mining C

A customized interface for single cell track visualisation based on pcnaDeep and napari.
A customized interface for single cell track visualisation based on pcnaDeep and napari.

pcnaDeep-napari A customized interface for single cell track visualisation based on pcnaDeep and napari. 👀 Under construction You can get test image

The Metabolomics Integrator (MINT) is a post-processing tool for liquid chromatography-mass spectrometry (LCMS) based metabolomics.
The Metabolomics Integrator (MINT) is a post-processing tool for liquid chromatography-mass spectrometry (LCMS) based metabolomics.

MINT (Metabolomics Integrator) The Metabolomics Integrator (MINT) is a post-processing tool for liquid chromatography-mass spectrometry (LCMS) based m

MPL Plotter is a Matplotlib based Python plotting library built with the goal of delivering publication-quality plots concisely.
MPL Plotter is a Matplotlib based Python plotting library built with the goal of delivering publication-quality plots concisely.

MPL Plotter is a Matplotlib based Python plotting library built with the goal of delivering publication-quality plots concisely.

A python script editor for napari based on PyQode.
A python script editor for napari based on PyQode.

napari-script-editor A python script editor for napari based on PyQode. This napari plugin was generated with Cookiecutter using with @napari's cookie

Custom Plotly Dash components based on Mantine React Components library

Dash Mantine Components Dash Mantine Components is a Dash component library based on Mantine React Components Library. It makes it easier to create go

A Python-based non-fungible token (NFT) generator built using Samilla and Matplotlib
A Python-based non-fungible token (NFT) generator built using Samilla and Matplotlib

PyNFT A Pythonic NF (non-fungible token) generator built using Samilla and Matplotlib Use python pynft.py [amount] The intention behind this generato

Fast visualization of radar_scenes based on oleschum/radar_scenes
Fast visualization of radar_scenes based on oleschum/radar_scenes

RadarScenes Tools About This python package provides fast visualization for the RadarScenes dataset. The Open GL based visualizer is smoother than ole

Owner
Peter Bobrov
44 y.o. дядя Петя from молодежный коллектив
Peter Bobrov
This plugin plots the time you spent on a tag as a histogram.

This plugin plots the time you spent on a tag as a histogram.

Tom Dörr 7 Sep 9, 2022
Histogramming for analysis powered by boost-histogram

Hist Hist is an analyst-friendly front-end for boost-histogram, designed for Python 3.7+ (3.6 users get version 2.4). See what's new. Installation You

Scikit-HEP Project 97 Dec 25, 2022
Dipto Chakrabarty 7 Sep 6, 2022
An interactive dashboard for visualisation, integration and classification of data using Active Learning.

AstronomicAL An interactive dashboard for visualisation, integration and classification of data using Active Learning. AstronomicAL is a human-in-the-

null 45 Nov 28, 2022
ipyvizzu - Jupyter notebook integration of Vizzu

ipyvizzu - Jupyter notebook integration of Vizzu. Tutorial · Examples · Repository About The Project ipyvizzu is the Jupyter Notebook integration of V

Vizzu 729 Jan 8, 2023
a plottling library for python, based on D3

Hello August 2013 Hello! Maybe you're looking for a nice Python interface to build interactive, javascript based plots that look as nice as all those

Mike Dewar 1.4k Dec 28, 2022
An interactive GUI for WhiteboxTools in a Jupyter-based environment

whiteboxgui An interactive GUI for WhiteboxTools in a Jupyter-based environment GitHub repo: https://github.com/giswqs/whiteboxgui Documentation: http

Qiusheng Wu 105 Dec 15, 2022
Turn a STAC catalog into a dask-based xarray

StackSTAC Turn a list of STAC items into a 4D xarray DataArray (dims: time, band, y, x), including reprojection to a common grid. The array is a lazy

Gabe Joseph 148 Dec 19, 2022
Visual Python is a GUI-based Python code generator, developed on the Jupyter Notebook environment as an extension.

Visual Python is a GUI-based Python code generator, developed on the Jupyter Notebook environment as an extension.

Visual Python 564 Jan 3, 2023
A simple script that displays pixel-based animation on GitHub Activity

GitHub Activity Animator This project contains a simple Javascript snippet that produces an animation on your GitHub activity tracker. The project als

null 16 Nov 15, 2021