Upgrade your print statements
Loggers are like print() statements
except they also include loads of other metadata:
- timestamp
- msg (same as print!)
- args (values or variables put into functions)
- function name
- level (e.g. DEBUG)
- line number (e.g line 42)
- module (which python script it came from)
- processes (getting fancy)
- threads (getting very fancy)
I want it all WITHOUT CONFIGURATION!
No problems. Check out the Loguru repo!
You can pretty much run with minimal config and loads of features. Loguru has all the features of this repo and more, along with a badass logo.
Where do I start?
This repo comes bundled with tidy logging configuration files saved in src/logconfig/
.
There is also a timing decorator in src/utils/
so you can optimize your code by simply decorating your function with @timing
above it. There are many online articles but very few clearly explain how to configure complex loggers with filters and yaml files.
To see loggers in action, setup a virtual environment and then run src/mainmodule.py
Requirements
- Python 3.6+
(Feel free to submit earlier versions that work)
Quickstart - Git clone and virtual env setup
Quickstart Instructions
Windows using powershell or CMD
cd to clone directory. Create virtual env with pip + venv:
git clone https://github.com/izzley/loggerexamples
cd loggerexamples\
py -0p # Optional: check your version and python path
py -m venv .venv
.venv\Scripts\activate
pip install --upgrade pip
pip install -r requirements.txt
# run main script
.\src\main.py
Linux/Mac
cd /to/clone/location
git clone https://github.com/izzley/loggerexamples
cd loggerexamples/
python3 -m venv .venv
source .venv/bin/activate
pip install --upgrade pip
pip install -r requirements.txt
If your relative imports aren't working, create `.pth' and add the parent folder/s to the file:
$ echo $(pwd) >> .venv/lib/python3.8/site-packages/my_p_ext.pth
Play with the logger
Walkthroughs
@TODO add list of suggestions to inspect
main_module.py
Try changing the main logger level from The main logger is first setup from main_module.py
. Notice how the level is set to DEBUG
?
- Run
main_module.py
with this DEBUG setting and take a look atsrc/logconfig/root.log
. - Now change the level to
WARNING
and spot the difference insrc/logconfig/root.log
. Remember this is the__main__
logger. Notice any changes?
- Notice everything from
__main__
is greater or equal toWARNING
? The logs running fromother_module.py
are still showing because we haven't changed its log level. Go ahead and change the loglevel inother_module.py
toCRITICAL
and spot the difference.
Whats in the YAML file??
YAML parts
In short, the conf.YAML
file contains all of the instructions for how the logger should behave. Below breaks down the conf yaml file into its parts:
formatters
formatters:
standard:
format: "%(asctime)s %(levelname)s - [%(filename)s: line %(lineno)s] - %(funcName)s - %(message)s"
Take this logger for example:
def funccalc(n):
logger.debug("something executed")
for _ in range(n):
i = 0
return
The output reflects the yaml file format settings:
2021-11-21 15:43:47,689 DEBUG - [module01.py: line 17] - funccalc - something executed
loggers
@TODO describe root loggers and their inheritance
root:
level: DEBUG
handlers: [console, debug_file_handler, info_file_handler, warn_file_handler, error_file_handler, critical_file_handler, root_file_handler]
handlers
@TODO describe handlers and how they redirect bytes
filters
@TODO describe how filters only allow bytes to handlers if a condition is true. reference filter classes in logconfig.py
References
- Docs: https://docs.python.org/3/library/logging.html#module-logging
- lots of logging examples: https://zetcode.com/python/logging/
- loguru. Everyone should know about it: https://github.com/Delgan/loguru#readme
- rich logging handler makes it pretty: https://rich.readthedocs.io/en/stable/logging.html
- filters explained: https://gist.github.com/kingspp/9451566a5555fb022215ca2b7b802f19
- timeit vs decorator: https://stackoverflow.com/questions/1622943/timeit-versus-timing-decorator
- real python uses perf_counter: https://realpython.com/lessons/timing-functions-decorators/
- What are decorators: https://gist.github.com/Zearin/2f40b7b9cfc51132851a
- Decorators can be reinforced to accept args: https://stackoverflow.com/questions/653368/how-to-create-a-python-decorator-that-can-be-used-either-with-or-without-paramet