Structured Logging for Python

Overview

structlog

Documentation Status CI Status Test Coverage Code style: black

structlog makes logging in Python faster, less painful, and more powerful by adding structure to your log entries.

It's up to you whether you want structlog to take care about the output of your log entries or whether you prefer to forward them to an existing logging system like the standard library's logging module.

Once you feel inspired to try it out, check out our friendly Getting Started tutorial that also contains detailed installation instructions!

If you prefer videos over reading, check out this DjangoCon Europe 2019 talk by Markus Holtermann: "Logging Rethought 2: The Actions of Frank Taylor Jr.".

Easier Logging

You can stop writing prose and start thinking in terms of an event that happens in the context of key/value pairs:

>>> from structlog import get_logger
>>> log = get_logger()
>>> log.info("key_value_logging", out_of_the_box=True, effort=0)
2020-11-18 09:17.09 [info     ] key_value_logging              effort=0 out_of_the_box=True

Each log entry is a meaningful dictionary instead of an opaque string now!

Data Binding

Since log entries are dictionaries, you can start binding and re-binding key/value pairs to your loggers to ensure they are present in every following logging call:

>>> log = log.bind(user="anonymous", some_key=23)
>>> log = log.bind(user="hynek", another_key=42)
>>> log.info("user.logged_in", happy=True)
2020-11-18 09:18.28 [info     ] user.logged_in                 another_key=42 happy=True some_key=23 user=hynek

Powerful Pipelines

Each log entry goes through a processor pipeline that is just a chain of functions that receive a dictionary and return a new dictionary that gets fed into the next function. That allows for simple but powerful data manipulation:

def timestamper(logger, log_method, event_dict):
    """Add a timestamp to each log entry."""
    event_dict["timestamp"] = time.time()
    return event_dict

There are plenty of processors for most common tasks coming with structlog:

Formatting

structlog is completely flexible about how the resulting log entry is emitted. Since each log entry is a dictionary, it can be formatted to any format:

  • A colorful key/value format for local development,
  • JSON for easy parsing,
  • or some standard format you have parsers for like nginx or Apache httpd.

Internally, formatters are processors whose return value (usually a string) is passed into loggers that are responsible for the output of your message. structlog comes with multiple useful formatters out-of-the-box.

Output

structlog is also very flexible with the final output of your log entries:

  • A built-in lightweight printer like in the examples above. Easy to use and fast.
  • Use the standard library's or Twisted's logging modules for compatibility. In this case structlog works like a wrapper that formats a string and passes them off into existing systems that won't ever know that structlog even exists. Or the other way round: structlog comes with a logging formatter that allows for processing third party log records.
  • Don't format it to a string at all! structlog passes you a dictionary and you can do with it whatever you want. Reported uses cases are sending them out via network or saving them in a database.

Getting Help

Please use the structlog tag on StackOverflow to get help.

Answering questions of your fellow developers is also a great way to help the project!

Project Information

structlog is dual-licensed under Apache License, version 2 and MIT, available from PyPI, the source code can be found on GitHub, the documentation at https://www.structlog.org/.

We collect useful third party extension in our wiki.

structlog targets Python 3.6 and newer, and PyPy3.

If you need support for older Python versions, the last release with support for Python 2.7 and 3.5 was 20.1.0. The package meta data should ensure that you get the correct version.

structlog for Enterprise

Available as part of the Tidelift Subscription.

The maintainers of structlog and thousands of other packages are working with Tidelift to deliver commercial support and maintenance for the open source packages you use to build your applications. Save time, reduce risk, and improve code health, while paying the maintainers of the exact packages you use. Learn more.

Comments
  • Add a JSONifier for stdlib logging

    Add a JSONifier for stdlib logging

    Especially within frameworks, one hasn’t always the power to control every part of logging.

    For Twisted, there is already a solution in place, there should be also one for stdlib.

    enhancement stdlib 
    opened by hynek 23
  • [Feature-Request] Allow copying attrs from stdlib LogRecord into structlog's msg

    [Feature-Request] Allow copying attrs from stdlib LogRecord into structlog's msg

    When ._proxy_to_logger is called, Python's built-in logging calls makeRecord that computes attributes that might be useful for some use cases (the most common example being filename/module and line number)

    All of that information is available in ProcessorFormatter's .format at which structlog just copies the msg attribute (event_dict) and drops the rest.

    My proposal is to allow copying attributes from LogRecord into the event_dict


    EDIT: See https://github.com/hynek/structlog/issues/253#issuecomment-605638088 for a better idea


    Roughly, that would look like

    class ProcessorFormatter(logging.Formatter):
    
        def __init__(self, ..., copy_logrecord_attrs: Tuple[str] = ()):
            ...
            self.copy_logrecord_attrs = copy_logrecord_attrs
    
        def format(self, record):
            record = logging.makeLogRecord(record.__dict__)
            try:
                ...
                ed = {attr: getattr(record, attr, '?') for attr in self.copy_logrecord_attrs}
                ed.update(**record.msg.copy())
            except AttributeError:
                ...
    

    Here, I am assuming

    1. It is better to allow values from attached processors to take precedence over copying from LogRecord
    2. The user of the library understands what attributes when copied are serializable

    Would love to hear your thoughts on this and if it's acceptable I can send a PR too.

    EDIT: After studying the code a bit more, I realized that I can get those LogRecord attributes rendered simply by passing a custom format using fmt keyword arg, but that would mean losing the niceties of using JSONRenderer

    opened by chiragjn 17
  • Traceback missing

    Traceback missing

    My Django 1.8, Python 2 application uses third party middleware:

    import struct
    logger = structlog.get_logger(__name__)
    
    class MyMiddleware(AuthMiddleware):
        def process_view(self, *args, **kwargs):
            print('FOO')
            1/0
            return super(MyMiddleware, self).process_view(*args, **kwargs)
    

    My output misses tracebacks:

    FOO
    2018-07-06 11:06.36 {'events': []}
    

    I can't debug, even see errors!

    I did try zero-config options, other configuration options, structlog.processors.ExceptionPrettyPrinter(), but couldn't find who swallows and exception. I looked for try/excepts with pass, but there's not one for 1/0.

    Please suggest some idea, I'm running out of debug options.

    opened by md1023 17
  • Add ProcessorFormatter

    Add ProcessorFormatter

    First off, I’m terribly sorry I didn’t get around it any sooner. However given how long it took me to dive back into logging, I guess my procrastination was warranted. :(

    Anyhow, @insolite, and @if-fi I’d like your feedback on what I’ve molded based on your PR and comments. Does this solve your problem? I’ve tried to make it more useful by adding the possibility to run stdlib entries through a separate chain, does that make sense to you?

    I hope we’ll be able to close these PRs/issues soon.

    Again: sorry.

    opened by hynek 17
  • Positional args for stdlib logging

    Positional args for stdlib logging

    This PR addresses #19. It's a rebased version of PR#29 by @lakshmi-kannan on top of current master, with additional commits by me that hopefully address all the points raised by @hynek in the review of #29.

    opened by wbolster 17
  • Logging levels for an application

    Logging levels for an application

    Hi.

    I couldn't find a simple example of setting a logging level properly for the whole application. This is something which turned out to be non-trivial.

    If I used wrap at the top level, the loggers for modules would still use PrintLogger, and the logger which has level defined wouldn't be picked up by submodules. The only way I could make it work is to pass a factory to a structlog.config.

    Is there a canonical/easy way out ?

    opened by ayoshi 17
  • AttributeError: 'PrintLogger' object has no attribute 'isEnabledFor'

    AttributeError: 'PrintLogger' object has no attribute 'isEnabledFor'

    Trying to setup structlog after it has already logged a message can put it into a strange state where logging messages causes exceptions.

    import structlog
    log = structlog.get_logger()
    log.msg('greeted', whom='world', more_than_a_string=[1, 2, 3])
    
    from structlog import get_logger, configure
    configure(
        logger_factory=structlog.stdlib.LoggerFactory(),
        wrapper_class=structlog.stdlib.BoundLogger,
        # Setup logger output format.
        processors=[
            structlog.stdlib.filter_by_level,
            structlog.processors.format_exc_info,
        ])
    
    log.warn("Hey, this isn't cool")
    

    Output:

    whom='world' more_than_a_string=[1, 2, 3] event='greeted'
    Traceback (most recent call last):
      File "main.py", line 15, in <module>
        log.warn("Hey, this isn't cool")
      File "/home/ethan/Jobs/Mozilla/tmp/structlog-bug/.venv/lib/python2.7/site-packages/structlog/stdlib.py", line 73, in warning
        return self._proxy_to_logger('warning', event, *args, **kw)
      File "/home/ethan/Jobs/Mozilla/tmp/structlog-bug/.venv/lib/python2.7/site-packages/structlog/stdlib.py", line 119, in _proxy_to_logger
        **event_kw)
      File "/home/ethan/Jobs/Mozilla/tmp/structlog-bug/.venv/lib/python2.7/site-packages/structlog/_base.py", line 176, in _proxy_to_logger
        args, kw = self._process_event(method_name, event, event_kw)
      File "/home/ethan/Jobs/Mozilla/tmp/structlog-bug/.venv/lib/python2.7/site-packages/structlog/_base.py", line 136, in _process_event
        event_dict = proc(self._logger, method_name, event_dict)
      File "/home/ethan/Jobs/Mozilla/tmp/structlog-bug/.venv/lib/python2.7/site-packages/structlog/stdlib.py", line 327, in filter_by_level
        if logger.isEnabledFor(_NAME_TO_LEVEL[name]):
    AttributeError: 'PrintLogger' object has no attribute 'isEnabledFor'
    

    This makes it difficult to log messages during startup, because logging isn't configured yet, and if we log messages before we configure, later log messages will just crash and burn with this exception.

    opened by glasserc 16
  • Issue 19 fix: Pass #2: Pass positional arguments down to stdlib

    Issue 19 fix: Pass #2: Pass positional arguments down to stdlib

    Ref: https://github.com/hynek/structlog/pull/23

    Apologies for opening another PR for same issue. I didn't want to pollute the original author's PR.

    I addressed all of your comments that I agreed with and added a commit. I retained the original author's commit as well. Tox tests pass on my box.

    I disagreed with one comment about adding pass through methods in stdlib BoundLogger (https://github.com/hynek/structlog/pull/23/files#diff-519ce9a9803f8404219d01de0afb0c1fR65). There are more than 5 such methods that are in stdlib logger. I was reading the code for the first time and instantly got what getattr was trying to do. I don't feel like it's magic. If you still insist on explicit pass through methods, I can add a commit. Please let me know.

    opened by lakshmi-kannan 16
  • Add traceback parser for structured tracebacks

    Add traceback parser for structured tracebacks

    Summary

    Parse tracebacks into an object structure than can easily be converted, e.g., to JSON. The code is based on rich (I talked to @willmcgugan and he is okay with that) but simplified and stripped of every "pretty printing" functionality. It’s main goal is to produce structured tracebacks that can be parsed by log aggregators.

    This is still work-in-progress. TODO:

    • Documentation
    • How to integrate it with structlog? I currently use an orjson processor that also calls the traceback parser (similar to what the ConsoleRenderer does with rich):
    import sys
    
    import orjson
    import structlog
    from structlog.processors import _json_fallback_handler as orjson_fallback_handler
    from structlog.types import EventDict, WrappedLogger
    
    
    def render_orjson(_logger: WrappedLogger, _name: str, event_dict: EventDict) -> str:
        """
        Modified version of :class:`structlog.processors.JSONRenderer` that works around
        the issues in https://github.com/hynek/structlog/issues/360
        """
        exc_info = event_dict.pop("exc_info", None)
        if exc_info:
            event_dict["exception"] = get_traceback_dicts(exc_info)
    
        return orjson.dumps(event_dict, default=orjson_fallback_handler).decode()
    
    
    def fail():
        1 / 0
    
    
    def main():
        processors = [
            structlog.processors.add_log_level,
            structlog.processors.TimeStamper(),
        ]
        if sys.stdout.isatty():
            processors.append(structlog.dev.ConsoleRenderer())
        else:
            processors.append(render_orjson)
    
        structlog.configure(
            processors=processors,
        )
        log = structlog.get_logger()
        log.info("ohai")
    
    
        try:
            fail()
        except Exception as e:
            log.exception("onoes", exc_info=e)
    
    
    if __name__ == "__main__":
        main()
    

    Output:

    # python tb.py 2>&1 | jq
    {
      "event": "ohai",
      "level": "info",
      "timestamp": 1648975014.737222
    }
    {
      "event": "onoes",
      "level": "error",
      "timestamp": 1648975014.7372942,
      "exception": [
        {
          "exc_type": "ZeroDivisionError",
          "exc_value": "division by zero",
          "syntax_error": null,
          "is_cause": false,
          "frames": [
            {
              "filename": "/Users/stefan/Projects/typed-settings/tb.py",
              "lineno": 280,
              "name": "main",
              "line": "",
              "locals": {
                "processors": "'[<function add_log_level at 0x109a1dfc0>, <structlog.processors.TimeStamper obje'+60",
                "log": "'<BoundLoggerLazyProxy(logger=None, wrapper_class=None, processors=None, context_'+55",
                "e": "ZeroDivisionError('division by zero')"
              }
            },
            {
              "filename": "/Users/stefan/Projects/typed-settings/tb.py",
              "lineno": 259,
              "name": "fail",
              "line": "",
              "locals": {}
            }
          ]
        }
      ]
    }
    

    Pull Request Check List

    • [x] Added tests for changed code.
      • The CI fails with less than 100% coverage.
    • [x] New APIs are added to typing_examples.py.
    • [x] Updated documentation for changed code.
      • [x] New functions/classes have to be added to docs/api.rst by hand.
      • [x] Changed/added classes/methods/functions have appropriate versionadded, versionchanged, or deprecated directives. Find the appropriate next version in our __init__.py file.
    • [x] Documentation in .rst and .md files is written using semantic newlines.
    • [x] Changes (and possible deprecations) are documented in the changelog.
    • [x] Consider granting push permissions to the PR branch, so maintainers can fix minor issues themselves without pestering you.
    opened by sscherfke 14
  • Public access to threadlocal._get_context()

    Public access to threadlocal._get_context()

    Similarly to #266, I find myself needing to access the thread-local context (I need to re-estabish some bound variables after a clean_threadlocal()).

    I tried to use structlog.get_context() but it's not returning the threadlocal context. If this would be accepted, I might even come up with a PR.

    opened by lucatpa 14
  • Permission to generate stubs for structlog in typeshed

    Permission to generate stubs for structlog in typeshed

    Hi Hynek,

    I would like to add stubs for structlog to https://github.com/python/typeshed, a repository used to store PEP 484 type signatures for the Python stdlib and popular third party libraries. As per PEP 484, permission is required from the library owner before merging type signatures into typeshed - https://www.python.org/dev/peps/pep-0484/#the-typeshed-repo

    These stubs will be used by mypy and Pycharm. I'll start working on creating stubs after I get permission from you and do let me know if you have any suggestions.

    Thanks

    opened by waseem18 14
  • Add String Interpolation to FilteringBoundLogger for exceptions.

    Add String Interpolation to FilteringBoundLogger for exceptions.

    Summary

    👋🏾 I'm extremely new to this project (as in just trying it out today 😅 ).

    I came across this issue as I was playing around with it. Perhaps, I'm doing something wrong? If not, this is my attempt to fix it.

    Before my changes:

    >>> from structlog import get_logger
    >>> logger = get_logger()
    >>> logger.exception("boom! code: %i", 42)
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
    TypeError: exception() takes 2 positional arguments but 3 were given
    

    After:

    >>> from structlog import get_logger
    >>> logger = get_logger()
    >>> logger.exception("boom! code: %i", 42)
    2023-01-07 01:59:28 [error    ] boom! code: 42 
    NoneType: None
    

    If this looks like I'm on the right track I will complete the doc changes as mentioned in the checklist, but I wanted to get some feedback first if that is alright. Thanks!

    Pull Request Check List

    • [x] Added tests for changed code.
      • The CI fails with less than 100% coverage.
    • [ ] New APIs are added to typing_examples.py.
    • [ ] Updated documentation for changed code.
      • [ ] New functions/classes have to be added to docs/api.rst by hand.
      • [ ] Changed/added classes/methods/functions have appropriate versionadded, versionchanged, or deprecated directives. Find the appropriate next version in our __init__.py file.
    • [ ] Documentation in .rst and .md files is written using semantic newlines.
    • [ ] Changes (and possible deprecations) are documented in the changelog.
    • [x] Consider granting push permissions to the PR branch, so maintainers can fix minor issues themselves without pestering you.
    opened by jriggins 0
  • Correctly figure out exc_info in ConsoleRenderer.

    Correctly figure out exc_info in ConsoleRenderer.

    Summary

    If the exc_info key was filled with a raw exception, and not a tuple, the ConsoleRenderer code would call sys.exc_info and overwrite it. This is different to how the other processors handle the exc_info key.

    Closes #482

    Pull Request Check List

    • [ ] Added tests for changed code.
      • The CI fails with less than 100% coverage.
    • [ ] New APIs are added to typing_examples.py.
    • [ ] Updated documentation for changed code.
      • [ ] New functions/classes have to be added to docs/api.rst by hand.
      • [ ] Changed/added classes/methods/functions have appropriate versionadded, versionchanged, or deprecated directives. Find the appropriate next version in our __init__.py file.
    • [ ] Documentation in .rst and .md files is written using semantic newlines.
    • [ ] Changes (and possible deprecations) are documented in the changelog.
    • [ ] Consider granting push permissions to the PR branch, so maintainers can fix minor issues themselves without pestering you.
    opened by aclemons 1
  • ConsoleRenderer does not handle non-tuple exc_info

    ConsoleRenderer does not handle non-tuple exc_info

    First, thank you for this great package.

    We are using structlog for creating canonical logs in a lambda function. The entrypoint of the lambda function calls another method which handles the event. This method never raises, but code within the other method might trap an exception and place it in the contextvar key exc_info .

    In production, we log output as json and everything is fine. For local development, we are using the console renderer, but the exceptions are not handled properly unless the exc_key contains a tuple.

    Here is a minimal reproduction:

    import structlog
    from structlog.contextvars import bind_contextvars
    
    logger = structlog.stdlib.get_logger()
    
    def process():
        bind_contextvars(event_id="123")
        # do some things, but don't propagate the exception
        try:
            print("Running...")
    
            raise ValueError("Boom")
        except Exception as exc:
            bind_contextvars(exc_info=exc)
    
        print("Finished...")
    
    def run():
        try:
            process()
        finally:
            logger.info("canonical-log")
    
    if __name__ == "__main__":
        run()
    

    When running, this explodes while handling the log:

    $ poetry run python3 main.py
    Running...
    Finished...
    Traceback (most recent call last):
      File "/Users/aclemons/worksp/structlog-test/main.py", line 25, in <module>
        run()
      File "/Users/aclemons/worksp/structlog-test/main.py", line 22, in run
        logger.info("canonical-log")
      File "/Users/aclemons/Library/Caches/pypoetry/virtualenvs/structlog-test-wfc9biWa-py3.9/lib/python3.9/site-packages/structlog/_log_levels.py", line 157, in meth
        return self._proxy_to_logger(name, event, **kw)
      File "/Users/aclemons/Library/Caches/pypoetry/virtualenvs/structlog-test-wfc9biWa-py3.9/lib/python3.9/site-packages/structlog/_base.py", line 205, in _proxy_to_logger
        args, kw = self._process_event(method_name, event, event_kw)
      File "/Users/aclemons/Library/Caches/pypoetry/virtualenvs/structlog-test-wfc9biWa-py3.9/lib/python3.9/site-packages/structlog/_base.py", line 162, in _process_event
        event_dict = proc(self._logger, method_name, event_dict)
      File "/Users/aclemons/Library/Caches/pypoetry/virtualenvs/structlog-test-wfc9biWa-py3.9/lib/python3.9/site-packages/structlog/dev.py", line 424, in __call__
        self._exception_formatter(sio, exc_info)
      File "/Users/aclemons/Library/Caches/pypoetry/virtualenvs/structlog-test-wfc9biWa-py3.9/lib/python3.9/site-packages/structlog/dev.py", line 187, in rich_traceback
        Traceback.from_exception(*exc_info, show_locals=True)
      File "/Users/aclemons/Library/Caches/pypoetry/virtualenvs/structlog-test-wfc9biWa-py3.9/lib/python3.9/site-packages/rich/traceback.py", line 292, in from_exception
        rich_traceback = cls.extract(
      File "/Users/aclemons/Library/Caches/pypoetry/virtualenvs/structlog-test-wfc9biWa-py3.9/lib/python3.9/site-packages/rich/traceback.py", line 348, in extract
        exc_type=safe_str(exc_type.__name__),
    AttributeError: 'NoneType' object has no attribute '__name__'
    

    If the exception is propagated it works without problem.

    It looks like the problem is here:

    https://github.com/hynek/structlog/blob/22.3.0/src/structlog/dev.py#L420-L424

    Since exc_info is not a tuple, it calls sys.exc_info, but we are not currently handling an exception and this returns None values.

    Would it be appropriate to replace that with the following code?

            if exc_info:
                self._exception_formatter(sio, _figure_out_exc_info(exc_info))
    

    That is what ExceptionRenderer is using here. This then works for me.

    $ poetry run python3 other.py
    Running...
    Finished...
    2022-12-06 10:40:27 [info     ] canonical-log                  event_id=123
    ╭─────────────────────────────── Traceback (most recent call last) ────────────────────────────────╮
    │ /Users/aclemons/worksp/structlog-test/other.py:25 in process                                     │
    │                                                                                                  │
    │   22 │   try:                                                                                    │
    │   23 │   │   print("Running...")                                                                 │
    │   24 │   │                                                                                       │
    │ ❱ 25 │   │   raise ValueError("Boom")                                                            │
    │   26 │   except Exception as exc:                                                                │
    │   27 │   │   bind_contextvars(exc_info=exc)                                                      │
    ╰──────────────────────────────────────────────────────────────────────────────────────────────────╯
    ValueError: Boom
    
    opened by aclemons 0
  • Rework stdlib compat

    Rework stdlib compat

    Summary

    Reworking a few things to allow make_filtering_bound_logger to provide FilteringBoundLogger's that inherit from BoundLogger instead of BoundLoggerBase which results in stdlib logger compatibility. Additionally adding a wrapper property for filters which was missing from BoundLogger.

    This should close #469 and result in the docs matching what's being provided by make_filtering_bound_logger, so I believe no documentation will need to be updated.

    Pull Request Check List

    • [x] Added tests for changed code.
      • The CI fails with less than 100% coverage.
    • [x] New APIs are added to typing_examples.py.
    • [x] Updated documentation for changed code.
      • [x] New functions/classes have to be added to docs/api.rst by hand.
      • [x] Changed/added classes/methods/functions have appropriate versionadded, versionchanged, or deprecated directives. Find the appropriate next version in our __init__.py file.
    • [x] Documentation in .rst and .md files is written using semantic newlines.
    • [x] Changes (and possible deprecations) are documented in the changelog.
    • [x] Consider granting push permissions to the PR branch, so maintainers can fix minor issues themselves without pestering you.
    opened by segfault 0
  • Improving make_filtering_bound_logger's compatibility with the stdlib logger

    Improving make_filtering_bound_logger's compatibility with the stdlib logger

    While trying to use structlog as a drop in for the stdlib logger usage in slack_bolt, I encountered a variety of errors stemming from missing wrapper implementations.

    Error example:

    AttributeError: 'BoundLoggerFilteringAtDebug' object has no attribute 'disabled'
    

    The root cause appears to be make_filtering_bound_logger's usage of BoundLoggerBase instead of the fully implemented BoundLogger. BoundLogger itself is missing the filters wrapper, but that is trivial to fix. BoundLoggerBase is missing all of the wrapper logic and that appears intentional.

    With an analog to BoundLogger implemented I am able to use structlog as a drop-in replacement for the standard logger.

    I'll be tying in a pull request with a proposed solution.

    opened by segfault 3
Releases(22.3.0)
  • 22.3.0(Nov 24, 2022)

    Highlights

    This is bug-fix release due to overly-zealous string interpolation in the native bound logger. You can now pass anything as event again (but it really should be a string) and log % signs as long as you don't pass positional arguments.

    Special Thanks

    This release would not be possible without my generous sponsors! Thank you to all of you making sustainable maintenance possible! If you would like to join them, go to https://github.com/sponsors/hynek and check out the sweet perks!

    Above and Beyond

    Variomedia AG (@variomedia), Tidelift (@tidelift), Sentry (@getsentry), HiredScore (@HiredScore), FilePreviews (@filepreviews), and Daniel Fortunov (@asqui).

    Maintenance Sustainers

    @rzijp, Adam Hill (@adamghill), Dan Groshev (@si14), Tamir Bahar (@tmr232), Adi Roiban (@adiroiban), Magnus Watn (@magnuswatn), David Cramer (@dcramer), Moving Content AG (@moving-content), Stein Magnus Jodal (@jodal), Iwan Aucamp (@aucampia), ProteinQure (@ProteinQure), Jesse Snyder (@jessesnyder), Rivo Laks (@rivol), Thomas Ballinger (@thomasballinger), @medecau, Ionel Cristian Mărieș (@ionelmc), The Westervelt Company (@westerveltco), Philippe Galvan (@PhilippeGalvan), Birk Jernström (@birkjernstrom), Jannis Leidel (@jezdez), Tim Schilling (@tim-schilling), Chris Withers (@cjw296), and Christopher Dignam (@chdsbd).

    Not to forget 2 more amazing humans who chose to be generous but anonymous!

    Full Changlog

    Changed

    • String interpolation in FilteringBoundLogger (used by default) is now only attempted if positional arguments are passed. This prevents crashes if something different than a string is passed for the event argument. #475

    Fixed

    • String interpolation doesn't cause crashes in filtered log call anymore. #478
    Source code(tar.gz)
    Source code(zip)
  • 22.2.0(Nov 19, 2022)

    Highlights

    This is another (too) big release, but before I go into new features, allow me to beg you to check out structlog's documentation. I've spent easily half of the time on bringing is up to date, restructuring, and adding usage recipes. Not new in this release, but did you know that the standard library chapter has flowcharts that give you as visual explanations of how the various methods work? This is usually the biggest sticking point when starting to use structlog.

    Feature-wise the big thing is that structlog's internal (and extremely fast) loggers (the one created using structlog.make_filtering_bound_logger() got two new features that people have asked for forever:

    1. String interpolation: log.info("hello %s!", "world") works now!
    2. Async! Each logging method has an async version: await log.ainfo("hello %s!", "world") is the same thing as above, but async.

    Special Thanks

    This release would not be possible without my generous sponsors! Thank you to all of you making sustainable maintenance possible! If you would like to join them, go to https://github.com/sponsors/hynek and check out the sweet perks!

    Above and Beyond

    Variomedia AG (@variomedia), Tidelift (@tidelift), Sentry (@getsentry), HiredScore (@HiredScore), FilePreviews (@filepreviews), and Daniel Fortunov (@asqui).

    Maintenance Sustainers

    @rzijp, Adam Hill (@adamghill), Dan Groshev (@si14), Tamir Bahar (@tmr232), Adi Roiban (@adiroiban), Magnus Watn (@magnuswatn), David Cramer (@dcramer), Moving Content AG (@moving-content), Stein Magnus Jodal (@jodal), Iwan Aucamp (@aucampia), ProteinQure (@ProteinQure), Jesse Snyder (@jessesnyder), Rivo Laks (@rivol), Thomas Ballinger (@thomasballinger), @medecau, Ionel Cristian Mărieș (@ionelmc), The Westervelt Company (@westerveltco), Philippe Galvan (@PhilippeGalvan), Birk Jernström (@birkjernstrom), Jannis Leidel (@jezdez), Tim Schilling (@tim-schilling), Chris Withers (@cjw296), and Christopher Dignam (@chdsbd).

    Not to forget 2 more amazing humans who chose to be generous but anonymous!

    Full Changelog

    Deprecated

    • Accessing package metadata as attributes on the structlog module is deprecated (e.g. structlog.__version__). Please use importlib.metadata instead (for Python 3.7: the importlib-metadata PyPI package).
    • The structlog.types module is now deprecated in favor of the structlog.typing module. It seems like the Python typing community is settling on this name.

    Added

    • FilteringBoundLogger (used by default) now allows for string interpolation using positional arguments:

      >>> log.info("Hello %s! The answer is %d.", "World", 42, x=1)
      2022-10-07 10:04.31 [info     ] Hello World! The answer is 42. x=1
      

      #454

    • FilteringBoundLogger now also has support for asyncio-based logging. Instead of a wrapper class like structlog.stdlib.AsyncBoundLogger, async equivalents have been added for all logging methods. So instead of log.info("hello") you can also write await log.ainfo("hello") in async functions and methods.

      This seems like the better approach and if it's liked by the community, structlog.stdlib.BoundLogger will get those methods too. #457

    Changed

    • The documentation has been heavily overhauled. Have a look if you haven't lately! Especially the graphs in the standard library chapter have proven valuable to many.
    • The build backend has been switched to Hatch.

    Fixed

    • The timestamps in the default configuration now use the correct separator (:) for seconds.
    Source code(tar.gz)
    Source code(zip)
  • 22.1.0(Jul 20, 2022)

    Highlights

    This is a (too) big release, so it has many highlights!

    Firstly, rendering exceptions in machine-readable logs (usually JSON) got a big upgrade: thanks to structlog.processors.dict_tracebacks you can now have fully structured exceptions in your logs!

    To ease getting started with structlog, we're now shipping structlog.stdlib.recreate_defaults() that recreates structlog's default behavior, but on top of standard library's logging. The output looks the same, but it runs through logging's machinery and integrates itself easier. The default configuration now also merges your contextvars-based context, so enjoy structlog.contextvars.bind_contextvars() without configuring anything!

    Another request wish that kept coming up is naming the message key differently than event. We're aware that nowadays keys like msg are more common, but structlog pre-dates the software that introduced and popularized it. To allow for more consistency across your platforms, structlog now ships structlog.processors.EventRenamer that allows you to rename the default event key to something else and additionally also allows you to rename another key to event.

    ❤️ Huge thanks to my GitHub sponsors, Tidelift subscribers, and Ko-fi buyers! ❤️

    None of my projects would exist in their current form without you!

    Full Changelog

    Removed

    • Python 3.6 is not supported anymore.
    • Pickling is now only possible with protocol version 3 and newer.

    Deprecated

    • The entire structlog.threadlocal module is deprecated. Please use the primitives from structlog.contextvars instead.

      If you're using the modern APIs (bind_threadlocal() / merge_threadlocal()) it's enough to replace them 1:1 with their contextvars counterparts. The old approach around wrap_dict() has been discouraged for a while.

      Currently there are no concrete plans to remove the module, but no patches against it will be accepted from now on. #409

    Added

    • structlog.processors.StackInfoRenderer now has an additional_ignores parameter that allows you to filter out your own logging layer. #396
    • Added structlog.WriteLogger, a faster – but more low-level – alternative to structlog.PrintLogger. It works the way PrintLogger used to work in previous versions. #403 #404
    • structlog.make_filtering_bound_logger()-returned loggers now also have a log() method to match the structlog.stdlib.BoundLogger signature closer. #413
    • Added structured logging of tracebacks via the structlog.tracebacks module, and most notably the structlog.tracebacks.ExceptionDictTransformer which can be used with the new structlog.processors.ExceptionRenderer to render JSON tracebacks. #407
    • structlog.stdlib.recreate_defaults(log_level=logging.NOTSET) that recreates structlog's defaults on top of standard library's logging. It optionally also configures logging to log to standard out at the passed log level. #428
    • structlog.processors.EventRenamer allows you to rename the hitherto hard-coded event dict key event to something else. Optionally, you can rename another key to event at the same time, too. So adding EventRenamer(to="msg", replace_by="_event") to your processor pipeline will rename the standard event key to msg and then rename the _event key to event. This allows you to use the event key in your own log files and to have consistent log message keys across languages.
    • structlog.dev.ConsoleRenderer(event_key="event") now allows to customize the name of the key that is used for the log message.

    Changed

    • structlog.make_filtering_bound_logger() now returns a method with the same signature for all log levels, whether they are active or not. This ensures that invalid calls to inactive log levels are caught immediately and don't explode once the log level changes. #401
    • structlog.PrintLogger – that is used by default – now uses print() for printing, making it a better citizen for interactive terminal applications. #399
    • structlog.testing.capture_logs now works for already initialized bound loggers. #408
    • structlog.processors.format_exc_info() is no longer a function, but an instance of structlog.processors.ExceptionRenderer. Its behavior has not changed. #407
    • The default configuration now includes the structlog.contextvars.merge_contextvars processor. That means you can use structlog.contextvars features without configuring structlog.

    Fixed

    • Overloaded the bind, unbind, try_unbind and new methods in the FilteringBoundLogger Protocol. This makes it easier to use objects of type FilteringBoundLogger in a typed context. #392
    • Monkeypatched sys.stdouts are now handled more gracefully by ConsoleRenderer (that's used by default). #404
    • structlog.stdlib.render_to_log_kwargs() now correctly handles the presence of exc_info, stack_info, and stackLevel in the event dictionary. They are transformed into proper keyword arguments instead of putting them into the extra dictionary. #424, #427
    Source code(tar.gz)
    Source code(zip)
  • 21.5.0(Dec 16, 2021)

    I didn't expect to make this release but @aucampia and @airwoodix contributed features that I'm sure will excite many users, so here's Santa Hynek 🎅 with a surprise release.

    Changes:

    • Added the structlog.processors.LogfmtRenderer processor to render log lines using the logfmt format. #376
    • Added the structlog.stdlib.ExtraAdder processor that adds extra attributes of logging.LogRecord objects to the event dictionary. This processor can be used for adding data passed in the extra parameter of the logging module's log methods to the event dictionary. #209 #377
    • Added the structlog.processor.CallsiteParameterAdder processor that adds parameters of the callsite that an event dictionary orginated from to the event dictionary. This processor can be used to enrich events dictionaries with information such as the function name, line number and filename that an event dictionary orignated from. #380
    Source code(tar.gz)
    Source code(zip)
  • 21.4.0(Nov 25, 2021)

    This release is mostly about a regression when importing using a Python interpreter running with the PYTHONOPTIMIZE=2 environment variable set, or as python -OO. The one new feature is kinda neat too, though!

    Changes:

    • Fixed import when running in optimized mode (PYTHONOPTIMIZE=2 or python -OO). #373
    • Added the structlog.threadlocal.bound_threadlocal and structlog.contextvars.bound_contextvars decorator/context managers to temporarily bind key/value pairs to a thread-local and context-local context. #371
    Source code(tar.gz)
    Source code(zip)
  • 21.3.0(Nov 20, 2021)

    The main reason for this comparatively timely release is that aiohttp 3.8's new behavior of starting new loops within aiohttp.web.run_app() led to breakage in apps that use structlog.stdlib.AsyncBoundLogger.

    The one big new feature though is the support for much more powerful processor chains within structlog.stdlib.ProcessorFormatter. This took me way too long to get right, but I'm excited to share it with you.

    This is also the first release without a setup.py. Invoking it was never tested and never supported, so now it's gone. Please use standard packaging tools like PyPA's build or flit directly if you want to package structlog yourself.

    Backward-incompatible changes:

    • structlog switched its packaging to flit. Users shouldn't notice a difference, but (re-)packagers might.

    Deprecations:

    none

    Changes:

    • structlog.dev.ConsoleRenderer now has sort_keys boolean parameter that allows to disable the sorting of keys on output. #358

    • structlog.processors.TimeStamper now works well with FreezeGun even when it gets applied before the loggers are configured. #364

    • structlog.stdlib.AsyncBoundLogger now determines the running loop when logging, not on instantiation. That has a minor performance impact, but makes it more robust when loops change (e.g. aiohttp.web.run_app()), or you want to use sync_bl before a loop has started.

    • structlog.stdlib.ProcessorFormatter now has a processors argument that allows to define a processor chain to run over all log entries.

      Before running the chain, two additional keys are added to the event dictionary: _record and _from_structlog. With them it's possible to extract information from logging.LogRecords and differentiate between structlog and logging log entries while processing them.

      The old processor (singular) parameter is now deprecated, but no plans exist to remove it. #365

    Source code(tar.gz)
    Source code(zip)
  • 21.2.0(Oct 12, 2021)

    Highlights

    • Support for for beautiful (and helpful!) exceptions by integrating ConsoleRenderer with rich or better-exceptions.
    • Helpers to access thread-local and context-local context.
    • Deeper contextvars support.

    Backward-incompatible changes:

    • To implement pretty exceptions (see Changes below), structlog.dev.ConsoleRenderer now formats exceptions itself.

      Make sure to remove format_exc_info from your processor chain if you configure structlog manually. This change is not really breaking, because the old use-case will keep working as before. However if you pass pretty_exceptions=True (which is the default if either rich or better-exceptions is installed), a warning will be raised and the exception will be renderered without prettyfication.

    Deprecations:

    none

    Changes:

    • structlog is now importable if sys.stdout is None (e.g. when running using pythonw). #313

    • structlog.threadlocal.get_threadlocal() and structlog.contextvars.get_contextvars() can now be used to get a copy of the current thread-local/context-local context that has been bound using structlog.threadlocal.bind_threadlocal() and structlog.contextvars.bind_contextvars(). #331 #337

    • structlog.threadlocal.get_merged_threadlocal(bl) and structlog.contextvars.get_merged_contextvars(bl) do the same, but also merge the context from a bound logger bl. Same pull requests as previous change.

    • structlog.contextvars.bind_contextvars() now returns a mapping of keys to contextvars.Tokens, allowing you to reset values using the new structlog.contextvars.reset_contextvars(). #339

    • Exception rendering in structlog.dev.ConsoleLogger is now configurable using the exception_formatter setting. If either the rich or the better-exceptions package is present, structlog will use them for pretty-printing tracebacks. rich takes precedence over better-exceptions if both are present.

      This only works if format_exc_info is absent in the processor chain. #330 #349

    • All use of colorama on non-Windows systems has been excised. Thus, colors are now enabled by default in structlog.dev.ConsoleRenderer on non-Windows systems. You can keep using colorama to customize colors, of course. #345

    • The final processor can now return a bytearray (additionally to str and bytes). #344

    Source code(tar.gz)
    Source code(zip)
  • 21.1.0(Feb 18, 2021)

    Backward-incompatible changes:

    none

    Deprecations:

    none

    Changes:

    • structlog.threadlocal.wrap_dict() now has a correct type annotation. #290
    • Fix isolation in structlog.contextvars. #302
    • The default configuration and loggers are pickleable again. #301
    • structlog.dev.ConsoleRenderer will now look for a logger_name key if no logger key is set. #295
    Source code(tar.gz)
    Source code(zip)
  • 20.2.0(Dec 31, 2020)

    Backward-incompatible changes:

    • Python 2.7 and 3.5 aren't supported anymore. The package meta data should ensure that you keep getting 20.1.0 on those versions. #244

    • structlog is now fully type-annotated. This won't break your applications, but if you use Mypy, it will most likely break your CI.

      Check out the new chapter on typing for details.

    Deprecations:

    • Accessing the _context attribute of a bound logger is now deprecated. Please use the new structlog.get_context().

    Changes:

    • structlog has now type hints for all of its APIs! Since structlog is highly dynamic and configurable, this led to a few concessions like a specialized structlog.stdlib.get_logger() whose only difference to structlog.get_logger() is that it has the correct type hints.

      We consider them provisional for the time being – i.e. the backward compatibility does not apply to them in its full strength until we feel we got it right. Please feel free to provide feedback! #223, #282

    • Added structlog.make_filtering_logger that can be used like configure(wrapper_class=make_filtering_bound_logger(logging.INFO)). It creates a highly optimized bound logger whose inactive methods only consist of a return None. This is now also the default logger.

    • As a complement, structlog.stdlib.add_log_level() can now additionally be imported as structlog.processors.add_log_level since it just adds the method name to the event dict.

    • structlog.processors.add_log_level() is now part of the default configuration.

    • structlog.stdlib.ProcessorFormatter no longer uses exceptions for control flow, allowing foreign_pre_chain processors to use sys.exc_info() to access the real exception.

    • Added structlog.BytesLogger to avoid unnecessary encoding round trips. Concretely this is useful with orjson which returns bytes. #271

    • The final processor now also may return bytes that are passed untouched to the wrapped logger.

    • structlog.get_context() allows you to retrieve the original context of a bound logger. #266,

    • structlog.PrintLogger now supports copy.deepcopy(). #268

    • Added structlog.testing.CapturingLogger for more unit testing goodness.

    • Added structlog.stdlib.AsyncBoundLogger that executes logging calls in a thread executor and therefore doesn't block. #245

    Source code(tar.gz)
    Source code(zip)
A colored formatter for the python logging module

Log formatting with colors! colorlog.ColoredFormatter is a formatter for use with Python's logging module that outputs records using terminal colors.

Sam Clements 778 Dec 26, 2022
Colored terminal output for Python's logging module

coloredlogs: Colored terminal output for Python's logging module The coloredlogs package enables colored terminal output for Python's logging module.

Peter Odding 496 Dec 30, 2022
A cool logging replacement for Python.

Welcome to Logbook Travis AppVeyor Supported Versions Latest Version Test Coverage Logbook is a nice logging replacement. It should be easy to setup,

null 1.4k Nov 11, 2022
Robust and effective logging for Python 2 and 3.

Robust and effective logging for Python 2 and 3.

Chris Hager 1k Jan 4, 2023
Python logging package for easy reproducible experimenting in research

smilelogging Python logging package for easy reproducible experimenting in research. Why you may need this package This project is meant to provide an

Huan Wang 20 Dec 23, 2022
A basic logging library for Python.

log.py ?? About: A basic logging library for Python with the capability to: save to files. have custom formats. have custom levels. be used instantiat

Sebastiaan Bij 1 Jan 19, 2022
Small toolkit for python multiprocessing logging to file

Small Toolkit for Python Multiprocessing Logging This is a small toolkit for solving unsafe python mutliprocess logging (file logging and rotation) In

Qishuai 1 Nov 10, 2021
Beautifully colored, quick and simple Python logging

Python Quick Logging | QLogging Beautifully colored, quick and simple Python logging. This logger is based on Python logging package Screenshots: Term

null 45 Sep 25, 2022
A lightweight logging library for python applications

cakelog a lightweight logging library for python applications This is a very small logging library to make logging in python easy and simple. config o

null 2 Jan 5, 2022
Simple and versatile logging library for python 3.6 above

Simple and versatile logging library for python 3.6 above

Miguel 1 Nov 23, 2022
A Python package which supports global logfmt formatted logging.

Python Logfmter A Python package which supports global logfmt formatted logging. Install $ pip install logfmter Usage Before integrating this library,

Joshua Taylor Eppinette 15 Dec 29, 2022
Stand-alone parser for User Access Logging from Server 2012 and newer systems

KStrike Stand-alone parser for User Access Logging from Server 2012 and newer systems BriMor Labs KStrike This script will parse data from the User Ac

BriMor Labs 69 Nov 1, 2022
Logging system for the TPC software.

tpc_logger Logging system for the TPC software. The TPC Logger class provides a singleton for logging information within C++ code or in the python API

UC Davis Machine Learning 1 Jan 10, 2022
Outlog it's a library to make logging a simple task

outlog Outlog it's a library to make logging a simple task!. I'm a lazy python user, the times that i do logging on my apps it's hard to do, a lot of

ZSendokame 2 Mar 5, 2022
metovlogs is a very simple logging library

metovlogs is a very simple logging library. Setup is one line, then you can use it as a drop-in print replacement. Sane and useful log format out of the box. Best for small or early projects.

Azat Akhmetov 1 Mar 1, 2022
Pretty-print tabular data in Python, a library and a command-line utility. Repository migrated from bitbucket.org/astanin/python-tabulate.

python-tabulate Pretty-print tabular data in Python, a library and a command-line utility. The main use cases of the library are: printing small table

Sergey Astanin 1.5k Jan 6, 2023
Progressbar 2 - A progress bar for Python 2 and Python 3 - "pip install progressbar2"

Text progress bar library for Python. Travis status: Coverage: Install The package can be installed through pip (this is the recommended method): pip

Rick van Hattem 795 Dec 18, 2022
The new Python SDK for Sentry.io

sentry-python - Sentry SDK for Python This is the next line of the Python SDK for Sentry, intended to replace the raven package on PyPI. from sentry_s

Sentry 1.4k Dec 31, 2022
A Fast, Extensible Progress Bar for Python and CLI

tqdm tqdm derives from the Arabic word taqaddum (تقدّم) which can mean "progress," and is an abbreviation for "I love you so much" in Spanish (te quie

tqdm developers 23.7k Jan 1, 2023