"zpool iostats" for humans; find the slow parts of your ZFS pool

Overview

Getting the gist of zfs statistics

vpool-demo.mp4

The ZFS command "zpool iostat" provides a histogram listing of how often it takes to do things in parts of your pool. It's useful to find bottlenecks and problem devices, but it's also kind of hard to read, with the nearly 4000 mixed-unit numbers.

What's a histogram?

A histogram is an arrangement of data to show how measurements can be different and how often. For instance, last year, 60 times it took Joe 3-to-4 hours to drive to Albequerque, and 20 times it took 4-to-5 hours, 80 times it took 5-to-6 hours, and 2 times it took 6-to-7 hours. That 60-times & 20-times & 80-times & 2-times is a histogram showing population of the buckets of driving-times. It can help you understand the pattern of likelihoods. It seems when Joe has a traffic problem, it's usually pretty big; and problems happen more often than not; but sometimes it's a breeze.

count 0 60 20 80 2 0 times
bucket 0 to 3 3 to 4 4 to 5 5 to 6 6 to 7 7 to 8 hours

Mentally sizing the huge absolute numbers from ZFS

The exact number of items in a histogram bucket does not matter at all. 36778213 ZFS operations vs 39214291 isn't a good comparison for humans to try to make. For understanding what's happening in your ZFS pool, the gist and relative sizes is what's important, so the numerical output of the "zpool iostat" tool is harder to understand than it needs to be. That's the problem this tool aims to fix.

The tools here will summarize and display relative sizes of those numbers, side-by-side, to help you see what's important.

Simplified output compares all devices for a given statistic

Simplified output shows all stats of a device at once

You can help!

There are some opinionated coloring of time herein, and those opinions are not well-informed. If you have a suggestion on how to better slice up the six white-to-red colors of time, please file a ticket. Mention the your (at max) five thresholds to the next color plateau, in nanoseconds.

Also, if the tool crashes, please file a ticket with the output. Include the stack trace.

Meanings of output

The histogram uses both colors to show relative population, but also glyphs. The glyphs are "." meaning the smallest but nonzero, "a" up through "z", and "^" represents the highest number among the buckets. The glyphs represent linear-growing, equidistant-apart number ranges. "f" is 4 times farther away from "b" than "c" is from "b".

Usage and prerequisites

To run, have Python 3.4 or greater, and run it with the pool name(s) as parameter.

$ zpool-iostat-viz

$ zpool-iostat-viz tank1 tank2

$ zpool-iostat-viz tank vdev42 vdev90 vdev163

$ zpool-iostat-viz -d

It displays data points in the histogram as letters of the alphabet, scaled and colored to show hot-ness of that bucket. It's scaled so that each column has a most-filled "^", and the rest of the letters show relative population vs the most populous bucket.

Also useful is the differential updates mode, which you can select with "-d" parameter. It's like stats are reset every three seconds, so you can see what's happening right now with your pool, not since boot.

Press arrows to move stats, and press q to quit. "--help" argument to see what other options you have.

You might also like...
A python script fetches all your starred repositories from your GitHub account and clones them to your server so you will never lose important resources
A python script fetches all your starred repositories from your GitHub account and clones them to your server so you will never lose important resources

A python script fetches all your starred repositories from your GitHub account and clones them to your server so you will never lose important resources

One version package to rule them all, One version package to find them, One version package to bring them all, and in the darkness bind them.

AwesomeVersion One version package to rule them all, One version package to find them, One version package to bring them all, and in the darkness bind

Find people to play tennis with.
Find people to play tennis with.

40Love 40Love is a full-stack web application that helps tennis players find hits at public tennis courts. Players can select public courts on the map

OSINT tool to get information from a Github and Gitlab profile and find user's email addresses leaked on commits.

gitrecon OSINT tool to get information from a Github or Gitlab profile and find user's email addresses leaked on commits. 📚 How does this work? GitHu

A pre-attack hacker tool which aims to find out sensitives comments in HTML comment tag and to help on reconnaissance process
A pre-attack hacker tool which aims to find out sensitives comments in HTML comment tag and to help on reconnaissance process

Find Out in Comment Find sensetive comment out in HTML ⚈ About This is a pre-attack hacker tool that searches for sensitives words in HTML comments ta

A listener for RF = 4.0 that prints a Stack Trace to console to faster find the code section where the failure appears.

robotframework-stacktrace A listener for RF = 4.0 that prints a Stack Trace to console to faster find the code section where the failure appears. Ins

domhttpx is a google search engine dorker with HTTP toolkit built with python, can make it easier for you to find many URLs/IPs at once with fast time.
domhttpx is a google search engine dorker with HTTP toolkit built with python, can make it easier for you to find many URLs/IPs at once with fast time.

domhttpx is a google search engine dorker with HTTP toolkit built with python, can make it easier for you to find many URLs/IPs at once with fast time

A Bot To Find Telegram User ID Easily

Telegram ID Bot 🤖 A Bot To Find Telegram User ID Easily Made with Python3 (C) @BXBotz Copyright permission under MIT License License - https://githu

Find the best repos to contribute to, right from Discord!
Find the best repos to contribute to, right from Discord!

repo-finder-bot Find the best repos to contribute to, right from Discord! Add to your server FAQs Hmm. What's this? This is the Repo Finder Bot, a bot

Comments
  • Crash w/ Python 3.6.9,

    Crash w/ Python 3.6.9,

    user@myhost:~# python3 --version
    Python 3.6.9
    user@myhost:~# python3 bin/zpool-iostat-viz
    CRASH! Sorry!
    Please report this error at 
    https://github.com/chadmiller/zpool-iostat-viz/issues/new
    
    Paste the following:
    Traceback (most recent call last):
      File "bin/zpool-iostat-viz", line 303, in <module>
        raise exc
      File "bin/zpool-iostat-viz", line 292, in <module>
        curses.wrapper(lambda window: main(window, parsed_args.pools, parsed_args.file))
      File "/usr/lib/python3.6/curses/__init__.py", line 94, in wrapper
        return func(stdscr, *args, **kwds)
      File "bin/zpool-iostat-viz", line 292, in <lambda>
        curses.wrapper(lambda window: main(window, parsed_args.pools, parsed_args.file))
      File "bin/zpool-iostat-viz", line 280, in main
        display_measurements_for_device(pool, filename, window)
      File "bin/zpool-iostat-viz", line 152, in display_measurements_for_device
        stats = get_stats(pool, filename)
      File "bin/zpool-iostat-viz", line 63, in get_stats
        zpool_cmd = subprocess.run(["zpool", "iostat", "-wvHp"] + (pool_names if pool_names else []), check=True, capture_output=True, encoding="UTF-8")
      File "/usr/lib/python3.6/subprocess.py", line 423, in run
        with Popen(*popenargs, **kwargs) as process:
    TypeError: __init__() got an unexpected keyword argument 'capture_output'
    
    opened by cmtonkinson 6
  • Add new functionality and a man-page in mdoc format

    Add new functionality and a man-page in mdoc format

    I have created a "port" of zpool-iostat-viz in FreeBSD that contains a few changes relative to your version:

    The following functionality has been added:

    --digits option: while letters give high precision output, it is not always easy to remember their order, especially with quickly changing displays. Digits give less resolution, but are easier to interpret.

    --symbols option: even less steps, but even more intuitive. Symbols have been chosen to cover more of the character cell for higher values. This mode could be combined with a blank white color palette, since colors tend to reduce the effect of the non-black fraction of the character cell.

    Two interactive commands have been added:

    'd': switch --diff mode on or off 'm': toggle between per measurement or per device mode

    I have once got a run-time error when toggling modes, but could not reproduce it in later attempts. AFAIU the program, the state between updates should be stored in a way that allows to toggle modes at any time (between updates).

    The wording of the man-page would probably benefit from a review by a native English speaker (which I'm obviously not).

    opened by stesser 0
  • unexpected keyword argument 'encoding'

    unexpected keyword argument 'encoding'

    probably you need some more info, besides this? (system is debian 9 (stretch) updated to latest available)

    Traceback (most recent call last): File "./zpool-iostat-viz", line 290, in raise exc File "./zpool-iostat-viz", line 279, in curses.wrapper(lambda window: main(window, parsed_args["diff"], parsed_args["parts"], parsed_args["file"], parsed_args["by"] or "m")) File "/usr/lib/python3.5/curses/init.py", line 94, in wrapper return func(stdscr, *args, **kwds) File "./zpool-iostat-viz", line 279, in curses.wrapper(lambda window: main(window, parsed_args["diff"], parsed_args["parts"], parsed_args["file"], parsed_args["by"] or "m")) File "./zpool-iostat-viz", line 241, in main render_stats(window, transformation_function, should_show_differential, pool, filename) File "./zpool-iostat-viz", line 132, in render_stats stats = transform(get_stats(pool, filename)) File "./zpool-iostat-viz", line 65, in get_stats zpool_cmd = subprocess.run(["zpool", "iostat", "-wvHp", "--"] + (pool_names if pool_names else []), check=True, stdout=subprocess.PIPE, encoding="UTF-8") File "/usr/lib/python3.5/subprocess.py", line 383, in run with Popen(*popenargs, **kwargs) as process: TypeError: init() got an unexpected keyword argument 'encoding'

    opened by corower 0
  • Fix program crash on

    Fix program crash on "m" with large value of "current"

    Fix program crash if the "m" command is used and "current" is higher than supported by the current stats array length.

    The "m" command has been part of a previous pull request that has been merged. I forgot that the stats array length can change and that the stats_history contents depends on the transform function selected.

    This patch fixes this issue. The value of "current" is reset to one that is valid for all possible stats array lengths, and stats_history is reset to an empty array to prevent incompatible data to be used in a difference calculation.

    opened by stesser 3
Owner
Chad
Probably going out like Aeschylus.
Chad
A Telegram bot to download youtube playlists and upload them to telegram. (may be slow becoz youtube limitations)

YTPlaylistDL ?? A Telegram bot to download youtube playlists and upload them to telegram. (may be slow becoz youtube limitations) ?? Follow me and sta

Anjana Madu 43 Dec 28, 2022
Faster Twitch Alerts is a highly customizable, lightning-fast alternative to Twitch's slow mobile notification system

Faster Twitch Alerts What is "Faster Twitch Alerts"? Faster Twitch Alerts is a highly customizable, lightning-fast alternative to Twitch's slow mobile

null 6 Dec 22, 2022
TON Miner from TON-Pool.com

TON-Pool Miner Miner from TON-Pool.com

null 21 Nov 18, 2022
“ HOLA HUMANS 👋 I'M DAISYX 2.0 „ LATEST VERSION OF DAISYX.. Source Code of @Daisyxbot

DaisyX 2.0 A Powerful, Smart And Simple Group Manager ... Written with AioGram , Pyrogram and Telethon... The first AioGram based modified groupmanage

TeamDaisyX 153 Dec 6, 2022
Wetterdienst - Open weather data for humans

We are a group of like-minded people trying to make access to weather data in Python feel like a warm summer breeze, similar to other projects like rdwd for the R language, which originally drew our interest in this project.

null 226 Jan 4, 2023
“ HOLA HUMANS 👋 I'M DAISYX 2.0 ❤️ „ LATEST VERSION OF DAISYX.. Source Code of @Daisyxbot

❤️ DaisyX 2.0 ❤️ A Powerful, Smart And Simple Group Manager ... Written with AioGram , Pyrogram and Telethon... ⭐️ Thanks to everyone who starred Dais

TeamDaisyX 153 Dec 6, 2022
A liblary whre you can find helpful functions for your discord bot

DBotUtils A liblary whre you can find helpful functions for your discord bot Easy setup Setup is easily and flexible. Change anytime. After setup just

Kondek286 1 Nov 2, 2021
Discord Token Finder - Find half of your target's token with just their ID.

Discord Token Finder - Find half of your target's token with just their ID.

Ttawi 2 Apr 7, 2022
Instagram Brute force attack helps you to find password of an instagram account from your list of provided password.

Instagram Brute force attack Instagram Brute force attack helps you to find password of an instagram account from your list of provided password. Inst

Naman Raj Singh 1 Dec 27, 2021
You can share your Chegg account for answers using this bot with your friends without getting your account blocked/flagged

Chegg-Answer-Bot You can share your Chegg account for answers using this bot with your friends without getting your account blocked/flagged Reuirement

Ammey Saini 27 Dec 24, 2022