A simple cli utility for importing or exporting dashboard json definitions using the Grafana HTTP API.

Overview

grafana-dashboard-manager

CodeQL

A simple cli utility for importing or exporting dashboard json definitions using the Grafana HTTP API.

This may be useful for:

  • Backing up your dashboards that already exist within your Grafana instance, e.g. if you are migrating from the internal sqlite database to MySQL.
  • Updating dashboard files for your Infrastructure-as-Code, for use with Grafana dashboard provisioning.
  • Making tweaks to dashboard JSON files directly and updating Grafana with one command.

Notable features:

  • Mirrors the folder structure between a local set of dashboards and Grafana, creating folders where necessary.
  • Ensures links to dashboards folders in a dashlist Panel are consistent with the Folder IDs - useful for deploying one set of dashboards across mulitple Grafana instances, e.g. for dev, test, prod environments.

Workflow

The intended development process is:

  1. Develop existing dashboard, or create a new one and save it in the web UI.
  2. Ensure the dashboard is in the desired folder.
  3. Use grafana-dashboard-manager to extract the new dashboards and save them to a local directory.
  4. Dashboards can be created/updated from the local directory back into Grafana.

Usage

Installation

Dependencies are managed with poetry.

Install from pypi:

$ poetry run pip install grafana-dashboard-manager

Install from source

$ cd /path/to/grafana-dashboard-manager
$ poetry install

Note that the admin login user and password are required, and its selected organization is correct.

See the full help text with poetry run grafana-dashboard-manager --help

Download dashboards from web to solution-data

poetry run grafana-dashboard-manager \
    --host https://my.grafana.com \
    --username admin --password mypassword \
    download all \
    --destination-dir /path/to/dashboards/

Upload dashboards from solution-data to web

poetry run grafana-dashboard-manager \
    --host https://my.grafana.com \
    --username admin --password mypassword \
    upload all \
    --source-dir /path/to/dashboards/

N.B. if your Grafana is not at port 80/443 as indicated by the protocol prefix, the port needs to be specified as part of the --host argument, e.g. for a locally hosted instance on port 3000: --host http://localhost:3000

Limitations

  • The home dashboard new deployment needs the default home dashboard to be manually set in the web UI, as the API to set the organisation default dashboard seems to be broken, at least on v8.2.3.

  • Currently expects a hardcoded 'home.json' dashboard to set as the home.

  • Does not handle upload of dashboards more deeply nested than Grafana supports.

  • Does not support multi-organization deployments

Comments
  • Add setting home dashboard with API token auth

    Add setting home dashboard with API token auth

    This solves #10

    There was a bug in grafana.api.put method: instead of JSON it was sending the request as application/x-www-form-urlencoded form data, so that's why it didn't work at all, basically resetting the home dashboard to 0.

    Additionally I added Docker and Makefile and while at it I discovered that it doesn't build under Python 3.11, but didn't dig deeper into it.

    opened by meetmatt 5
  • Unable to install

    Unable to install

    Hi, This package seems great but I was unable to install it succesfully.

    With poetry I got this error :

    .poetry/bin/poetry run pip install grafana-dashboard-manager
    
      RuntimeError
    
      Poetry could not find a pyproject.toml file in /home/dtrckd or its parents
    
      at .poetry/lib/poetry/_vendor/py3.9/poetry/core/factory.py:369 in locate
          365│             if poetry_file.exists():
          366│                 return poetry_file
          367│ 
          368│         else:
        → 369│             raise RuntimeError(
          370│                 "Poetry could not find a pyproject.toml file in {} or its parents".format(
          371│                     cwd
          372│                 )
          373│             )
    

    And with pip, it got installed, but can't execute with the following error

    grafana-dashboard-manager     
    Traceback (most recent call last):
      File "/home/dtrckd/.local/bin/grafana-dashboard-manager", line 5, in <module>
        from grafana_dashboard_manager.__main__ import app
      File "/home/dtrckd/.local/lib/python3.9/site-packages/grafana_dashboard_manager/__main__.py", line 21, in <module>
        import grafana_dashboard_manager.dashboard_upload
      File "/home/dtrckd/.local/lib/python3.9/site-packages/grafana_dashboard_manager/dashboard_upload.py", line 29, in <module>
        REPO = git.Repo(search_parent_directories=True)
      File "/home/dtrckd/.local/lib/python3.9/site-packages/git/repo/base.py", line 224, in __init__
        self.working_dir: Optional[PathLike] = self._working_tree_dir or self.common_dir
      File "/home/dtrckd/.local/lib/python3.9/site-packages/git/repo/base.py", line 307, in common_dir
        raise InvalidGitRepositoryError()
    git.exc.InvalidGitRepositoryError
    

    Any hints how to solve this ?

    opened by dtrckd 4
  • Error:

    Error: "Failed to star dashboard" when using API key

    During the upload grafana-dashboard-manager tries to star the home dashboard in order to set it as default for the user.

    It works well with --username/--password, but fails when using API key.

    It's related to this: https://community.grafana.com/t/impossible-to-make-the-http-call-to-star-a-dashboard/300

    opened by meetmatt 2
  • Allow token-based authentication

    Allow token-based authentication

    This change should provide users an option to use a Grafana API token. I also make use of requests.Session to make sure Grafana session cookie persists across requests and for connection pooling; should be helpful when you got a lot of dashboards to download.

    opened by mstyushin 2
  • Update documentation and HTTP error reporting

    Update documentation and HTTP error reporting

    Motivation

    • Following on from the merging of #7, I thought it was worthwhile to update the README.md content to mention this feature.
    • I also found from my usage that the dump of a stack-trace on hitting a HTTP error was not useful. I have tidied this error up to be more legible.

    Changes

    • Update documentation to mention --token flag.
    • Tidy up README.md content generally.
    • Stop HTTP errors from printing a stack-trace to terminal.
    • Print a clear HTTP error code & message.
    opened by LawrenceWarren 1
  • Remove dependecy on `git`

    Remove dependecy on `git`

    Retrieving the version of grafana-dashboard-manager for the purpose of logging should not require that you are running inside of the git repository, or require you to run via poetry to succeed.

    I have switched to using importlib.metadata from the stdlib to retrieve the package metadata which contains the version number.

    Signed-off-by: Christian Witts [email protected]

    opened by ChristianWitts 1
  • Fix error

    Fix error

    Why

    Dashboards can organise panels into rows, which help to group and structure panels. The output JSON would therefore be structured as follows:

    {
      "rows": [
        {
          "panels": [{}]
        },
        {
          "panels": [{}]
        }
      ]
    }
    

    This was resulting in #3, as the program did not account for this edge case. This change resolves this.

    What

    • The change extracts the inner loop logic of update_dashlist_folder_ids out into a function.
    • It introduces a conditional which checks if the dashboard actually has the panels attribute, if it has the rows attribute, or something else:
      • For the panels case, it uses a loop to call the function on each panel.
      • For the rows case, it uses a loop to cycle through each row, and then a nested loop to call the function on each panel.
      • For the something else, error.
    opened by LawrenceWarren 0
  • Add 'selfSignedCert' flag

    Add 'selfSignedCert' flag

    Possible fix for #4 (Self Signed Certs?).

    I also did some cleanup, added Path() to some places (failed otherwise, don't know the reason but I am using Python 3.10.4 and .venv instead of poetry).

    opened by bzgec 0
  • Self Signed Certs?

    Self Signed Certs?

    Any support for self-signed certs?

    (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self signed certificate (_ssl.c:1129)')))

    opened by vitosans 2
  • error downloading dashboard

    error downloading dashboard

    I get this error when i try to download dashboards :

    Found Zeus - Kubernetes App Metrics dashboard in folder API Management dashboard_download.py:80 Traceback (most recent call last): File "", line 1, in File "/home/clement/.cache/pypoetry/virtualenvs/grafana-dashboard-manager-GvvO0-hQ-py3.8/lib/python3.8/site-packages/typer/main.py", line 214, in call return get_command(self)(*args, **kwargs) File "/home/clement/.cache/pypoetry/virtualenvs/grafana-dashboard-manager-GvvO0-hQ-py3.8/lib/python3.8/site-packages/click/core.py", line 1128, in call return self.main(*args, **kwargs) File "/home/clement/.cache/pypoetry/virtualenvs/grafana-dashboard-manager-GvvO0-hQ-py3.8/lib/python3.8/site-packages/click/core.py", line 1053, in main rv = self.invoke(ctx) File "/home/clement/.cache/pypoetry/virtualenvs/grafana-dashboard-manager-GvvO0-hQ-py3.8/lib/python3.8/site-packages/click/core.py", line 1659, in invoke return _process_result(sub_ctx.command.invoke(sub_ctx)) File "/home/clement/.cache/pypoetry/virtualenvs/grafana-dashboard-manager-GvvO0-hQ-py3.8/lib/python3.8/site-packages/click/core.py", line 1659, in invoke return _process_result(sub_ctx.command.invoke(sub_ctx)) File "/home/clement/.cache/pypoetry/virtualenvs/grafana-dashboard-manager-GvvO0-hQ-py3.8/lib/python3.8/site-packages/click/core.py", line 1395, in invoke return ctx.invoke(self.callback, **ctx.params) File "/home/clement/.cache/pypoetry/virtualenvs/grafana-dashboard-manager-GvvO0-hQ-py3.8/lib/python3.8/site-packages/click/core.py", line 754, in invoke return __callback(*args, **kwargs) File "/home/clement/.cache/pypoetry/virtualenvs/grafana-dashboard-manager-GvvO0-hQ-py3.8/lib/python3.8/site-packages/typer/main.py", line 500, in wrapper return callback(**use_params) # type: ignore File "/home/clement/grafana-dashboard-manager/grafana_dashboard_manager/dashboard_download.py", line 52, in all _write_dashboards_to_local_folder_from_grafana_folder(folder, destination_dir) File "/home/clement/grafana-dashboard-manager/grafana_dashboard_manager/dashboard_download.py", line 86, in _write_dashboards_to_local_folder_from_grafana_folder dashboard_definition = update_dashlist_folder_ids(dashboard_definition) File "/home/clement/grafana-dashboard-manager/grafana_dashboard_manager/dashboard.py", line 58, in update_dashlist_folder_ids for panel in dashboard_definition["dashboard"]["panels"]: KeyError: 'panels'

    opened by clement94310 4
Owner
Beam Connectivity
Beam Connectivity
A Python command-line utility for validating that the outputs of a given Declarative Form Azure Portal UI JSON template map to the input parameters of a given ARM Deployment Template JSON template

A Python command-line utility for validating that the outputs of a given Declarative Form Azure Portal UI JSON template map to the input parameters of a given ARM Deployment Template JSON template

Glenn Musa 1 Feb 3, 2022
As easy as /aitch-tee-tee-pie/ 🥧 Modern, user-friendly command-line HTTP client for the API era. JSON support, colors, sessions, downloads, plugins & more. https://twitter.com/httpie

HTTPie: human-friendly CLI HTTP client for the API era HTTPie (pronounced aitch-tee-tee-pie) is a command-line HTTP client. Its goal is to make CLI in

HTTPie 25.4k Dec 30, 2022
This CLI give the possibility to do a queries in Star Wars API and returns a JSON in a terminal.

Star Wars CLI (swcli) This CLI give the possibility to do a queries in Star Wars API and returns a JSON in a terminal. Install $ pip install swcli Qu

Pery Lemke 5 Nov 5, 2021
Yts-cli-streamer - A CLI movie streaming client which works on yts.mx API written in python

YTSP It is a CLI movie streaming client which works on yts.mx API written in pyt

null 1 Feb 5, 2022
Python wrapper and CLI utility to render LaTeX markup and equations as SVG using dvisvgm and svgo.

latex2svg Python wrapper and CLI utility to render LaTeX markup and equations as SVG using dvisvgm and svgo. Based on the original work by Tino Wagner

Matthias C. Hormann 4 Feb 18, 2022
A simple CLI based any Download Tool, that find files and let you stream or download thorugh WebTorrent CLI or Aria or any command tool

Privateer A simple CLI based any Download Tool, that find files and let you stream or download thorugh WebTorrent CLI or Aria or any command tool How

Shreyash Chavan 2 Apr 4, 2022
CLI utility to search and download torrents from major torrent sites

CLI Torrent Downloader About CLI Torrent Downloader provides convenient and quick way to search torrent magnet links (and to run associated torrent cl

x0r0x 86 Dec 19, 2022
CLI Utility to encode and recursively recreate directories with ffmpeg.

FFenmass CLI Utility to encode and recursively recreate directories with ffmpeg. Report Bug · Request Feature Table of Contents Getting Started Prereq

George Av. 8 May 6, 2022
CLI utility for updating the EVE Online static data export in a postgres database

EVE SDE Postgres updater CLI utility for updating the EVE Online static data export postgres database. This has been tested with the Fuzzwork postgres

Markus Juopperi 1 Oct 29, 2021
This is a CLI utility that allows you to view RedFlagDeals.com on the command line.

RFD Description Motivation Installation Usage View Hot Deals View and Sort Hot Deals Search Advanced View Posts Shell Completion bash zsh Description

Dave G 8 Nov 29, 2022
Python CLI utility and library for manipulating SQLite databases

sqlite-utils Python CLI utility and library for manipulating SQLite databases. Some feature highlights Pipe JSON (or CSV or TSV) directly into a new S

Simon Willison 1.1k Jan 4, 2023
AlienFX is a CLI and GUI utility to control the lighting effects of your Alienware computer.

AlienFX is a Linux utility to control the lighting effects of your Alienware computer. At present there is a CLI version (alienfx) and a gtk GUI versi

Stephen Harris 218 Dec 26, 2022
Sink is a CLI tool that allows users to synchronize their local folders to their Google Drives. It is similar to the Git CLI and allows fast and reliable syncs with the drive.

Sink is a CLI synchronisation tool that enables a user to synchronise local system files and folders with their Google Drives. It follows a git C

Yash Thakre 16 May 29, 2022
flora-dev-cli (fd-cli) is command line interface software to interact with flora blockchain.

Install git clone https://github.com/Flora-Network/fd-cli.git cd fd-cli python3 -m venv venv source venv/bin/activate pip install -e . --extra-index-u

null 14 Sep 11, 2022
AWS Interactive CLI - Allows you to execute a complex AWS commands by chaining one or more other AWS CLI dependency

AWS Interactive CLI - Allows you to execute a complex AWS commands by chaining one or more other AWS CLI dependency

Rafael Torres 2 Dec 10, 2021
Python-Stock-Info-CLI: Get stock info through CLI by passing stock ticker.

Python-Stock-Info-CLI Get stock info through CLI by passing stock ticker. Installation Use the following command to install the required modules at on

Ayush Soni 1 Nov 5, 2021
[WIP]An ani-cli like cli tool for movies and webseries

mov-cli A cli to browse and watch movies. Installation This project is a work in progress. However, you can try it out python git clone https://github

null 166 Dec 30, 2022