🦉
A Modern Observability System
HiQ is a declarative, non-intrusive, dynamic and transparent tracking system for both monolithic application and distributed system. It brings the runtime information tracking and optimization to a new level without compromising with speed and system performance, or hiding any tracking overhead information. HiQ applies for both I/O bound and CPU bound applications.
To explain the four features, declarative means you can declare the things you want to track in a text file, which could be a json, yaml or even csv,and no need to change program code. Non-intrusive means HiQ doesn't requires to modify original python code. Dynamic means HiQ supports tracing metrics featuring at run time, which can be used for adaptive tracing. Transparent means HiQ provides the tracing overhead and doesn't hide it no matter it is huge or tiny.
In addition to latency tracking, HiQ provides memory, disk I/O and Network I/O tracking out of the box. The output can be saved in form of normal line by line log file, or HiQ tree, or span graph.
HiQ's philosophy is to decouple observability logic from business logic. We don't have to enter the black hole to observe it. Do you like the idea? Leave a
Installation
pip install hiq-python
Get Started
Let start with a simplest example by running HiQ against a simple monolithic python code
main.py:
# this is the main.py python source code
import time
def func1():
time.sleep(1.5)
print("func1")
func2()
def func2():
time.sleep(2.5)
print("func2")
def main():
func1()
if __name__ == "__main__":
main()
In this code, there is a simple chain of function calls: main() -> func1 -> func2.
Now we want to trace the functions without modifying its code. Let's run the following:
git clone https://github.com/oracle-samples/hiq.git
cd hiq/examples/quick_start
python main_driver.py
If everything is fine, you should be able to see the output like this:
From the screenshot we can see the timestamp and the latency of each function:
| main | func1 | func2 | tracing overhead | |
|---|---|---|---|---|
| latency(second) | 4.0045 | 4.0044 | 2.5026 | 0.0000163 |
HiQ just traced the main.py file running without touching one line of its code.
Documentation
HTML:
Logging: https://hiq.readthedocs.io/en/latest/4_o_advanced.html#log-monkey-king
Tracing: https://hiq.readthedocs.io/en/latest/5_distributed.html
- Zipkin: https://hiq.readthedocs.io/en/latest/5_distributed.html#zipkin
- Jaeger: https://hiq.readthedocs.io/en/latest/5_distributed.html#jaeger
Metrics:
Streaming:
Jupyter NoteBook
Add Observability to PaddlePaddle (PaddleOCR)
Add Observability to Onnxruntime (AlexNet)
Examples
Please check
Contributing
HiQ welcomes contributions from the community. Before submitting a pull request, please review our
Security
Please consult the
Change Log
dev
- add non-intrusive auto instrumentation for flask: HiQFlaskLatencyOtel (
🔗 example)
v1.0.2
- add non-intrusive auto instrumentation for Onnxruntime, Paddlepaddle, PaddleOCR
License
Copyright (c) 2022 Oracle and/or its affiliates. Released under the Universal Permissive License v1.0 as shown at https://oss.oracle.com/licenses/upl/.
4.7k Jan 8, 2023
1.1k Jan 9, 2023
1.4k Dec 31, 2022
1k Dec 16, 2022
70 Dec 20, 2022
60 Apr 1, 2022
876 Jan 5, 2023
27.5k Jan 4, 2023
8 Nov 17, 2022
827 Jan 4, 2023
39 Nov 8, 2022
4.7k Jan 2, 2023
7 Dec 12, 2021
4 Nov 21, 2021
1 Dec 17, 2021
106 Dec 19, 2022
6 May 26, 2022
332 Dec 25, 2022