ClusterMonitor - a very simple python script which monitors and records the CPU and RAM consumption of submitted cluster jobs

Overview

ClusterMonitor

A very simple python script which monitors and records the CPU and RAM consumption of submitted cluster jobs.

Usage

To start recording use the cpu_ram_log.py script. This script requires 3 arguments:

  • -u which corresponds to your username.
  • -o which specifies the output file (in tsv format)
  • --interval which specifies the time (in seconds) over which the CPU and RAM usage gets averaged

Example:

python cpu_ram_log.py -u nickhir -o cpu_ram.log --interval 5

The script can simply be included in your cluster job submittion like this:

#!/bin/bash
#SBATCH --job-name=Example_run

sambamba sort -t 20 some_bam.bam &
MainScript=$!
python cpu_ram_log.py -u nickhir -o cpu_ram.log --interval 5 &
wait $MainScript
pkill -P $$

Afterwards the resulting log file can be plotted (for that matplotlib is needed). For that use the visualize_log.py script which needs 2 arguments:

  • -i which specifies the path to the log file which was created with cpu_ram_log.py
  • -o which specifies the path to the resulting PDF which contains a plot for the CPU and RAM usage.

Example:

python visualize_log.py -i cpu_ram.log -o cpu_ram_visualization.pdf

The resulting plots will look somewhat like this: exampleImage

You might also like...
Simple and versatile logging library for python 3.6 above

Simple and versatile logging library for python 3.6 above

Python logging made (stupidly) simple
Python logging made (stupidly) simple

Loguru is a library which aims to bring enjoyable logging in Python. Did you ever feel lazy about configuring a logger and used print() instead?... I

A simple, transparent, open-source key logger, written in Python, for tracking your own key-usage statistics.
A simple, transparent, open-source key logger, written in Python, for tracking your own key-usage statistics.

A simple, transparent, open-source key logger, written in Python, for tracking your own key-usage statistics, originally intended for keyboard layout optimization.

This is a wonderful simple python tool used to store the keyboard log.

Keylogger This is a wonderful simple python tool used to store the keyboard log. Record your keys. It will capture passwords and credentials in a comp

pyEventLogger - a simple Python Library for making customized Logs of certain events that occur in a program

pyEventLogger is a simple Python Library for making customized Logs of certain events that occur in a program. The logs can be fully customized and can be printed in colored format or can be stored in a file.

Fuzzy-logger - Fuzzy project is here Log all your pc's actions Simple and free to use Security of datas !
Fuzzy-logger - Fuzzy project is here Log all your pc's actions Simple and free to use Security of datas !

Fuzzy-logger - ➡️⭐ Fuzzy ⭐ project is here ! ➡️ Log all your pc's actions ! ➡️ Simple and free to use ➡️ Security of datas !

A simple package that allows you to save inputs & outputs as .log files

wolf_dot_log A simple package that allows you to save inputs & outputs as .log files pip install wolf_dot_log pip3 install wolf_dot_log |Instructions|

👻 - Simple Keylloger with Socket
👻 - Simple Keylloger with Socket

Keyllogs 👻 - Simple Keylloger with Socket Keyllogs 🎲 - Run Keyllogs

Outlog it's a library to make logging a simple task

outlog Outlog it's a library to make logging a simple task!. I'm a lazy python user, the times that i do logging on my apps it's hard to do, a lot of

Owner
null
This is a key logger based in python which when executed records all the keystrokes of the system it has been executed on .

This is a key logger based in python which when executed records all the keystrokes of the system it has been executed on

Purbayan Majumder 0 Mar 28, 2022
This open-source python3 script is a builder to the very popular token logger that is on my github that many people use.

Discord-Logger-Builder This open-source python3 script is a builder to the very popular token logger that is on my github that many people use. This i

Local 4 Nov 17, 2021
metovlogs is a very simple logging library

metovlogs is a very simple logging library. Setup is one line, then you can use it as a drop-in print replacement. Sane and useful log format out of the box. Best for small or early projects.

Azat Akhmetov 1 Mar 1, 2022
A new kind of Progress Bar, with real time throughput, eta and very cool animations!

alive-progress :) A new kind of Progress Bar, with real-time throughput, eta and very cool animations! Ever found yourself in a remote ssh session, do

Rogério Sampaio de Almeida 4k Dec 30, 2022
A very basic esp32-based logic analyzer capable of sampling digital signals at up to ~3.2MHz.

A very basic esp32-based logic analyzer capable of sampling digital signals at up to ~3.2MHz.

Davide Della Giustina 43 Dec 27, 2022
Vibrating-perimeter - Simple helper mod that logs how fast you are mining together with a simple buttplug.io script to control a vibrator

Vibrating Perimeter This project consists of a small minecraft helper mod that writes too a log file and a script that reads said log. Currently it on

Heart[BOT] 0 Nov 20, 2022
Keylogger with Python which logs words into server terminal.

word_logger Experimental keylogger with Python which logs words into server terminal.

Selçuk 1 Nov 15, 2021
A Python package which supports global logfmt formatted logging.

Python Logfmter A Python package which supports global logfmt formatted logging. Install $ pip install logfmter Usage Before integrating this library,

Joshua Taylor Eppinette 15 Dec 29, 2022
Python script to scan log files/system for unauthorized access around system

checkLogs Python script to scan log files/system for unauthorized access around Linux systems Table of contents General info Getting started Usage Gen

James Kelly 1 Feb 25, 2022
Beautifully colored, quick and simple Python logging

Python Quick Logging | QLogging Beautifully colored, quick and simple Python logging. This logger is based on Python logging package Screenshots: Term

null 45 Sep 25, 2022