Yaml - Loggers are like print() statements

Overview

Upgrade your print statements

print statement meme

Loggers are like print() statements

except they also include loads of other metadata:

  • timestamp
  • msg (same as print!)
  • args (values or variables put into functions)
  • function name
  • level (e.g. DEBUG)
  • line number (e.g line 42)
  • module (which python script it came from)
  • processes (getting fancy)
  • threads (getting very fancy)

I want it all WITHOUT CONFIGURATION!

loguru logo

No problems. Check out the Loguru repo!
You can pretty much run with minimal config and loads of features. Loguru has all the features of this repo and more, along with a badass logo.

Where do I start?

This repo comes bundled with tidy logging configuration files saved in src/logconfig/.
There is also a timing decorator in src/utils/ so you can optimize your code by simply decorating your function with @timing above it. There are many online articles but very few clearly explain how to configure complex loggers with filters and yaml files.
To see loggers in action, setup a virtual environment and then run src/mainmodule.py

Requirements

  • Python 3.6+

(Feel free to submit earlier versions that work)

Quickstart - Git clone and virtual env setup

Quickstart Instructions

Windows using powershell or CMD

cd to clone directory. Create virtual env with pip + venv:

git clone https://github.com/izzley/loggerexamples
cd loggerexamples\
py -0p # Optional: check your version and python path
py -m venv .venv
.venv\Scripts\activate
pip install --upgrade pip
pip install -r requirements.txt

# run main script
.\src\main.py

Linux/Mac

cd /to/clone/location
git clone https://github.com/izzley/loggerexamples
cd loggerexamples/
python3 -m venv .venv
source .venv/bin/activate
pip install --upgrade pip
pip install -r requirements.txt

If your relative imports aren't working, create `.pth' and add the parent folder/s to the file:

$ echo $(pwd) >> .venv/lib/python3.8/site-packages/my_p_ext.pth

Play with the logger

Walkthroughs

@TODO add list of suggestions to inspect

Try changing the main logger level from main_module.py

The main logger is first setup from main_module.py. Notice how the level is set to DEBUG?

logger level change


  • Run main_module.py with this DEBUG setting and take a look at src/logconfig/root.log.
  • Now change the level to WARNING and spot the difference in src/logconfig/root.log. Remember this is the __main__ logger. Notice any changes?

logger level change


  • Notice everything from __main__ is greater or equal to WARNING? The logs running from other_module.py are still showing because we haven't changed its log level. Go ahead and change the loglevel in other_module.py to CRITICAL and spot the difference.

Whats in the YAML file??

YAML parts

In short, the conf.YAML file contains all of the instructions for how the logger should behave. Below breaks down the conf yaml file into its parts:

formatters

formatters:
    standard:
        format: "%(asctime)s %(levelname)s - [%(filename)s: line %(lineno)s] - %(funcName)s - %(message)s"

Take this logger for example:

def funccalc(n):
    logger.debug("something executed")
    for _ in range(n):
        i = 0
    return

The output reflects the yaml file format settings:

2021-11-21 15:43:47,689 DEBUG - [module01.py: line 17] - funccalc - something executed

loggers

@TODO describe root loggers and their inheritance

root:
  level: DEBUG
  handlers: [console, debug_file_handler, info_file_handler, warn_file_handler, error_file_handler, critical_file_handler, root_file_handler]

root logger yaml

handlers

@TODO describe handlers and how they redirect bytes

filters

@TODO describe how filters only allow bytes to handlers if a condition is true. reference filter classes in logconfig.py

References

You might also like...
A pyparsing-based library for parsing SOQL statements

CONTRIBUTORS WANTED!! Installation pip install python-soql-parser or, with poetry poetry add python-soql-parser Usage from python_soql_parser import p

An easy python calculator for those who want's to know how if statements, loops, and imports works give it a try!

A usefull calculator for any student or anyone who want's to know how to build a simple 2 mode python based calculator.

fixup: Automatically add and remove python import statements

fixup: Automatically add and remove python import statements The goal is that running fixup my_file.py will automatically add or remove import stateme

Python Module for Tabular Datasets in XLS, CSV, JSON, YAML, &c.

Tablib: format-agnostic tabular dataset library _____ ______ ___________ ______ __ /_______ ____ /_ ___ /___(_)___ /_ _ __/_ __ `/__ _

A command-line tool and Python library and Pytest plugin for automated testing of RESTful APIs, with a simple, concise and flexible YAML-based syntax

1.0 Release See here for details about breaking changes with the upcoming 1.0 release: https://github.com/taverntesting/tavern/issues/495 Easier API t

Type-safe YAML parser and validator.

StrictYAML StrictYAML is a type-safe YAML parser that parses and validates a restricted subset of the YAML specification. Priorities: Beautiful API Re

pytest plugin that let you automate actions and assertions with test metrics reporting executing plain YAML files
pytest plugin that let you automate actions and assertions with test metrics reporting executing plain YAML files

pytest-play pytest-play is a codeless, generic, pluggable and extensible automation tool, not necessarily test automation only, based on the fantastic

Allows including an action inside another action (by preprocessing the Yaml file). This is how composite actions should have worked.

actions-includes Allows including an action inside another action (by preprocessing the Yaml file). Instead of using uses or run in your action step,

Transparently load variables from environment or JSON/YAML file.

A thin wrapper over Pydantic's settings management. Allows you to define configuration variables and load them from environment or JSON/YAML file. Also generates initial configuration files and documentation for your defined configuration.

Lumen provides a framework for visual analytics, which allows users to build data-driven dashboards from a simple yaml specification
Lumen provides a framework for visual analytics, which allows users to build data-driven dashboards from a simple yaml specification

Lumen project provides a framework for visual analytics, which allows users to build data-driven dashboards from a simple yaml specification

Pydantic-ish YAML configuration management.
Pydantic-ish YAML configuration management.

Pydantic-ish YAML configuration management.

A simple tool that updates your pubspec.yaml file, of a Flutter project, without altering the structure of your file.
A simple tool that updates your pubspec.yaml file, of a Flutter project, without altering the structure of your file.

A simple tool that updates your pubspec.yaml file, of a Flutter project, without altering the structure of your file.

Automatizando a criação de DAGs usando Jinja e YAML

Automatizando a criação de DAGs no Airflow usando Jinja e YAML Arquitetura do Repo: Pastas por contexto de negócio (ex: Marketing, Analytics, HR, etc)

ETL flow framework based on Yaml configs in Python
ETL flow framework based on Yaml configs in Python

ETL framework based on Yaml configs in Python A light framework for creating data streams. Setting up streams through configuration in the Yaml file.

A YAML validator for Programming Historian lessons.

phyaml A simple YAML validator for Programming Historian lessons. USAGE: python3 ph-lesson-yaml-validator.py lesson.md The script automatically detect

Simple dataclasses configuration management for Python with hocon/json/yaml/properties/env-vars/dict support.

Simple dataclasses configuration management for Python with hocon/json/yaml/properties/env-vars/dict support, based on awesome and lightweight pyhocon parsing library.

What if home automation was homoiconic? Just transformations of data? No more YAML!

radiale what if home-automation was also homoiconic? The upper or proximal row contains three bones, to which Gegenbaur has applied the terms radiale,

Apt2sbom python package generates SPDX or YAML files

Welcome to apt2sbom This package contains a library and a CLI tool to convert a Ubuntu software package inventory to a software bill of materials. You

Dag-bakery - Dag Bakery enables the capability to define Airflow DAGs via YAML.

DAG Bakery - WIP 🔧 dag-bakery aims to simplify our DAG development by removing all the boilerplate and duplicated code when defining multiple DAG cro

Owner
isaac peterson
Open source || Awesome sauce
isaac peterson
A small utility to pretty-print Python tracebacks. ⛺

TBVaccine TBVaccine is a utility that pretty-prints Python tracebacks. It automatically highlights lines you care about and deemphasizes lines you don

Stavros Korokithakis 365 Nov 11, 2022
ScreenshotLogger works just like a keylogger but instead of capturing keystroke,it captures the screen, stores it or sends via email

ScreenshotLogger works just like a keylogger but instead of capturing keystroke,it captures the screen, stores it or sends via email. Scrapeasy is super easy to use and handles everything for you. Just specify what website you want to scrape and what media you want to receive back, and the rest is done for you.

Ifechukwudeni Oweh 17 Jul 17, 2022
This is discord nitro code generator and checker made with python. This will generate nitro codes and checks if the code is valid or not. If code is valid then it will print the code leaving 2 lines and if not then it will print '*'.

Discord Nitro Generator And Checker ⚙️ Rᴜɴ Oɴ Rᴇᴘʟɪᴛ ??️ Lᴀɴɢᴜᴀɢᴇs Aɴᴅ Tᴏᴏʟs If you are taking code from this repository without a fork, then atleast

Vɪɴᴀʏᴀᴋ Pᴀɴᴅᴇʏ 37 Jan 7, 2023
Leap is an experimental package written to enable the utilization of C-like goto statements in Python functions

Leap is an experimental package written to enable the utilization of C-like goto statements in Python functions

null 6 Dec 26, 2022
Python Fstab Generator is a small Python script to write and generate /etc/fstab files based on yaml file on Unix-like systems.

PyFstab Generator PyFstab Generator is a small Python script to write and generate /etc/fstab files based on yaml file on Unix-like systems. NOTE : Th

Mahdi 2 Nov 9, 2021
Multi-processing capable print-like logger for Python

MPLogger Multi-processing capable print-like logger for Python Requirements and Installation Python 3.8+ is required Pip pip install mplogger Manual P

Eötvös Loránd University Department of Digital Humanities 1 Jan 28, 2022
flake8 plugin which forbids match statements (PEP 634)

flake8-match flake8 plugin which forbids match statements (PEP 634)

Anthony Sottile 25 Nov 1, 2022
A hack for writing switch statements with type annotations in Python.

py_annotation_switch A hack for writing switch statements in type annotations for Python. Why should I use this? You most definitely should not use th

null 6 Oct 17, 2021
flake8 plugin to catch useless `assert` statements

flake8-useless-assert flake8 plugin to catch useless assert statements Download or install on the PyPI page Violations Code Description Example ULA001

null 1 Feb 12, 2022
💰 Import your ING Germany bank statements via FinTS into YNAB.

Import your ING Germany bank statements via FinTS into YNAB. Setup Before setting this up, please register your FinTS product – it's free and takes on

Arne Bahlo 23 Jan 21, 2022