Display, filter and search log messages in your terminal

Rik Huygen

Last update: Dec 10, 2022

Related tags

Deep Learning textualog

Overview

Textualog

Display, filter and search logging messages in the terminal.

This project is powered by rich and textual.

Some of the ideas and code in this project are based on:

kaskade
textual example code, e.g. code_viewer
cutelog

Installation

The easiest way to install the package is by running the pip command in the Python virtual environment of your project:

$ python -m pip install [--upgrade] textualog

Usage

The textualog app should have been installed in your environment, then run the following command:

$ textualog --log <path to the log file>

In the examples directory of this project, you can find an example log file to inspect and play with.

The main view is divided in three panels, (1) a Records panel that displays all the logging records in a colored view, (2) a Record Info panel that displays more details about the selected logging message (a message can be selected by a mouse click), and (3) a Levels panel that displays the standard logging levels. Logging levels can be switched on or off with a key press, d=debug, i=info, w=warning, e=error, c=critical. When you click inside the Record Info panel, the main view will change in a Record Details view that displays all information associated with the selected logging message. This view is mainly used when the logging message has extra multi-line information attached, and depending on the amount of information, this view is scrollable. When the selected logging message contains extra information, the Record Info panel will have an asterisk in the title. Use the Escape key to return to the main view.

The app can be terminated with the 'q' key or by pressing CTRL-C. If you need a little help on the keyboard shortcuts, press the '?' key to present the Info Help panel on the right side of the terminal. Also here use the Escape key to hide the help panel again.

Pressing the 'n' key will slide in a Namespaces panel on the left side of the Terminal. This panel is currently not functional. The idea is to allow the user to filter the logging messages by selecting one or more namespaces.

Log file formats

The current support is for a key-value type of log file. The log line shall have a fixed format, which is what I currently use in my main other projects. The following key=value pairs shall be there in the given order:

level=<logging level>
ts=<'%Y-%m-%dT%H:%M:%S,%f'>
process=<process name>
process_id=<PID>
caller=<calling function:lineno>
msg=<logging message>

In the future other formats can be supported by implementing a plugin class. Planned formats are the JSON format, ...

Roadmap

Display message details including extra lines that contain further information like e.g. traceback info.
Implement search functionality to search for strings or regular expressions and position the screen at the first match
Start work on filtering log messages based on their namespace

You might also like...

Model search is a framework that implements AutoML algorithms for model architecture search at scale

Model search (MS) is a framework that implements AutoML algorithms for model architecture search at scale. It aims to help researchers speed up their exploration process for finding the right model architecture for their classification problems (i.e., DNNs with different types of layers).

3.2k Dec 31, 2022

Densely Connected Search Space for More Flexible Neural Architecture Search (CVPR2020)

DenseNAS The code of the CVPR2020 paper Densely Connected Search Space for More Flexible Neural Architecture Search. Neural architecture search (NAS)

291 Nov 18, 2022

Simple Python project using Opencv and datetime package to recognise faces and log attendance data in a csv file.

Attendance-System-based-on-Facial-recognition-Attendance-data-stored-in-csv-file- Simple Python project using Opencv and datetime package to recognise

3 Aug 9, 2022

Milano is a tool for automating hyper-parameters search for your models on a backend of your choice.

Milano (This is a research project, not an official NVIDIA product.) Documentation https://nvidia.github.io/Milano Milano (Machine learning autotuner

147 Dec 17, 2022

A simple log parser and summariser for IIS web server logs

IISLogFileParser A basic parser tool for IIS Logs which summarises findings from the log file. Inspired by the Gist https://gist.github.com/wh13371/e7

2 Mar 26, 2022

People log into different sites every day to get information and browse through these sites one by one

HyperLink People log into different sites every day to get information and browse through these sites one by one. And they are exposed to advertisemen

0 Feb 17, 2022

Incorporating Transformer and LSTM to Kalman Filter with EM algorithm

Deep learning based state estimation: incorporating Transformer and LSTM to Kalman Filter with EM algorithm Overview Kalman Filter requires the true p

57 Dec 27, 2022

Simple torch.nn.module implementation of Alias-Free-GAN style filter and resample

Alias-Free-Torch Simple torch module implementation of Alias-Free GAN. This repository including Alias-Free GAN style lowpass sinc filter @filter.py A

64 Dec 22, 2022

Snapchat-filters-app-opencv-python - Here we used opencv and other inbuilt python modules to create filter application like snapchat

Snapchat like filter App using opencv python Backend : opencv and python Library

2 Jul 19, 2022

Comments

ValueError: Indices for islice() must be None or an integer: 0 <= x <= sys.maxsize.

$ textualog --log $PLATO_LOG_FILE_LOCATION/general.log
╭───────────────────────────────────────────────────────────── Traceback (most recent call last) ──────────────────────────────────────────────────────────────╮
│ /home/plato-data/.local/lib/python3.8/site-packages/textualog/__main__.py:148 in on_key                                                                      │
│                                                                                                                                                              │
│   145 │   │   │   self.records.replace(self.loader.get_records(self.cursor, height, self.leve ╭──────────────────── locals ─────────────────────╮            │
│   146 │   │   elif event.key == Keys.End:                                                     │  event = Key(key='end')                         │            │
│   147 │   │   │   self.cursor = size - height                                                 │ height = 22                                     │            │
│ ❱ 148 │   │   │   self.records.replace(self.loader.get_records(self.cursor, height, self.leve │   self = TextualLog(title='Textual Log Viewer') │            │
│   149 │   │   elif event.key == Keys.Home:                                                    │   size = 14                                     │            │
│   150 │   │   │   self.cursor = 0                                                             ╰─────────────────────────────────────────────────╯            │
│   151 │   │   │   self.records.replace(self.loader.get_records(self.cursor, height, self.leve                                                                │
│                                                                                                                                                              │
│ /home/plato-data/.local/lib/python3.8/site-packages/textualog/loader.py:118 in get_records                                                                   │
│                                                                                                                                                              │
│   115 │   │   │   │   │   num_lines: int = DEFAULT_NUM_LINES,                                                                                                │
│   116 │   │   │   │   │   levels: Levels = None) -> List[LogRecord]:                                                                                         │
│   117 │   │                                                                                                                                                  │
│ ❱ 118 │   │   self.process(start, num_lines, levels)                                                                                                         │
│   119 │   │                                                                                                                                                  │
│   120 │   │   return self._records                                                                                                                           │
│   121                                                                                                                                                        │
│                                                                                                                                                              │
│ ╭──────────────────────────────── locals ────────────────────────────────╮                                                                                   │
│ │    levels = Levels(name='Levels#1')                                    │                                                                                   │
│ │ num_lines = 22                                                         │                                                                                   │
│ │      self = <textualog.loader.KeyValueLoader object at 0x7f6c9b4b1400> │                                                                                   │
│ │     start = -8                                                         │                                                                                   │
│ ╰────────────────────────────────────────────────────────────────────────╯                                                                                   │
│                                                                                                                                                              │
│ /home/plato-data/.local/lib/python3.8/site-packages/textualog/loader.py:59 in process                                                                        │
│                                                                                                                                                              │
│    56 │   │   in_sub_message = False                                                                                                                         │
│    57 │   │   records = []                                                                                                                                   │
│    58 │   │   match_count = 0                                                                                                                                │
│ ❱  59 │   │   for count, line in enumerate(islice(self._lines, start, None)):                                                                                │
│    60 │   │   │   if count > MAX_NUM_LINES:                                                                                                                  │
│    61 │   │   │   │   break                                                                                                                                  │
│    62 │   │   │   if not line.startswith("level="):                                                                                                          │
│                                                                                                                                                              │
│ ╭────────────────────────────────── locals ───────────────────────────────────╮                                                                              │
│ │ in_sub_message = False                                                      │                                                                              │
│ │         levels = Levels(name='Levels#1')                                    │                                                                              │
│ │    match_count = 0                                                          │                                                                              │
│ │      num_lines = 22                                                         │                                                                              │
│ │        records = []                                                         │                                                                              │
│ │           self = <textualog.loader.KeyValueLoader object at 0x7f6c9b4b1400> │                                                                              │
│ │          start = -8                                                         │                                                                              │
│ │    sub_message = []                                                         │                                                                              │
│ ╰─────────────────────────────────────────────────────────────────────────────╯                                                                              │
╰──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
ValueError: Indices for islice() must be None or an integer: 0 <= x <= sys.maxsize.

bug

opened by rhuygen 1

textualog v0.1.2 - ImportError: Package 'textual.widgets' has no class 'ScrollView'

error presented trying to run textualog v0.1.2 from a fresh install. There were no errors shown during installation.

(textualog) C:\Users\trevor.hook\Scripts>textualog --version
Traceback (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "C:\Users\trevor.hook\Scripts\textualog\Scripts\textualog.exe\__main__.py", line 4, in <module>
  File "C:\Users\trevor.hook\Scripts\textualog\Lib\site-packages\textualog\__main__.py", line 12, in <module>
    from textual.widgets import ScrollView
  File "C:\Users\trevor.hook\Scripts\textualog\Lib\site-packages\textual\widgets\__init__.py", line 62, in __getattr__
    raise ImportError(f"Package 'textual.widgets' has no class '{widget_class}'")
ImportError: Package 'textual.widgets' has no class 'ScrollView'

opened by trevorhook 1

Owner

Rik Huygen

Self-educated Pythonista. Seriously trying to write clean and Pythonic code.

GitHub

A rule-based log analyzer & filter

Flog 一个根据规则集来处理文本日志的工具。前言在日常开发过程中，由于缺乏必要的日志规范，导致很多人乱打一通，一个日志文件夹解压缩后往往有几十万行。日志泛滥会导致信息密度骤减，给排查问题带来了不小的麻烦。以前都是用grep之类的工具先挑选出有用的，再逐条进行排查，费时费力。在忍无可忍之后决

9 Jun 23, 2022

Deep Image Search is an AI-based image search engine that includes deep transfor learning features Extraction and tree-based vectorized search.

Deep Image Search - AI-Based Image Search Engine Deep Image Search is an AI-based image search engine that includes deep transfer learning features Ex

139 Jan 1, 2023

Search and filter videos based on objects that appear in them using convolutional neural networks

Thingscoop: Utility for searching and filtering videos based on their content Description Thingscoop is a command-line utility for analyzing videos se

354 Dec 4, 2022

Ensemble Knowledge Guided Sub-network Search and Fine-tuning for Filter Pruning

Ensemble Knowledge Guided Sub-network Search and Fine-tuning for Filter Pruning This repository is official Tensorflow implementation of paper: Ensemb

12 Oct 18, 2022

UpChecker is a simple opensource project to host it fast on your server and check is server up, view statistic, get messages if it is down. UpChecker - just run file and use project easy

UpChecker UpChecker is a simple opensource project to host it fast on your server and check is server up, view statistic, get messages if it is down.

4 Apr 7, 2022

Make Watson Assistant send messages to your Discord Server

Make Watson Assistant send messages to your Discord Server Prerequisites Sign up for an IBM Cloud account. Fill in the required information and press

1 Jan 10, 2022

Stock-history-display - something like a easy yearly review for your stock performance

Stock History Display Available on Heroku: https://stock-history-display.herokua

1 Jan 7, 2022

TorchPQ is a python library for Approximate Nearest Neighbor Search (ANNS) and Maximum Inner Product Search (MIPS) on GPU using Product Quantization (PQ) algorithm.

Efficient implementations of Product Quantization and its variants using Pytorch and CUDA

146 Dec 28, 2022

This repo contains the code and data used in the paper "Wizard of Search Engine: Access to Information Through Conversations with Search Engines"

Wizard of Search Engine: Access to Information Through Conversations with Search Engines by Pengjie Ren, Zhongkun Liu, Xiaomeng Song, Hongtao Tian, Zh

19 Oct 27, 2022

Deep Text Search is an AI-powered multilingual text search and recommendation engine with state-of-the-art transformer-based multilingual text embedding (50+ languages).

Deep Text Search - AI Based Text Search & Recommendation System Deep Text Search is an AI-powered multilingual text search and recommendation engine w

19 Sep 29, 2022