Profiling
The profiling package is an interactive continuous Python profiler. It is inspired from Unity 3D profiler. This package provides these features:
- Profiling statistics keep the frame stack.
- An interactive TUI profiling statistics viewer.
- Provides both of statistical and deterministic profiling.
- Utilities for remote profiling.
- Thread or greenlet aware CPU timer.
- Supports Python 2.7, 3.3, 3.4 and 3.5.
- Currently supports only Linux.
Installation
Install the latest release via PyPI:
$ pip install profiling
Profiling
To profile a single program, simply run the profiling command:
$ profiling your-program.py
Then an interactive viewer will be executed:
If your program uses greenlets, choose greenlet timer:
$ profiling --timer=greenlet your-program.py
With --dump option, it saves the profiling result to a file. You can browse the saved result by using the view subcommand:
$ profiling --dump=your-program.prf your-program.py
$ profiling view your-program.prf
If your script reads sys.argv, append your arguments after --. It isolates your arguments from the profiling command:
$ profiling your-program.py -- --your-flag --your-param=42
Live Profiling
If your program has a long life time like a web server, a profiling result at the end of program is not helpful enough. Probably you need a continuous profiler. It can be achived by the live-profile subcommand:
$ profiling live-profile webserver.py
See a demo:
There's a live-profiling server also. The server doesn't profile the program at ordinary times. But when a client connects to the server, it starts to profile and reports the results to the all connected clients.
Start a profling server by the remote-profile subcommand:
$ profiling remote-profile webserver.py --bind 127.0.0.1:8912
And also run a client for the server by the view subcommand:
$ profiling view 127.0.0.1:8912
Statistical Profiling
TracingProfiler, the default profiler, implements a deterministic profiler for deep call graph. Of course, it has heavy overhead. The overhead can pollute your profiling result or can make your application to be slow.
In contrast, SamplingProfiler implements a statistical profiler. Like other statistical profilers, it also has only very cheap overhead. When you profile you can choose it by just --sampling (shortly -S) option:
$ profiling live-profile -S webserver.py
^^
Timeit then Profiling
Do you use timeit to check the performance of your code?
$ python -m timeit -s 'from trueskill import *' 'rate_1vs1(Rating(), Rating())'
1000 loops, best of 3: 722 usec per loop
If you want to profile the checked code, simply use the timeit subcommand:
$ profiling timeit -s 'from trueskill import *' 'rate_1vs1(Rating(), Rating())'
^^^^^^^^^
Profiling from Code
You can also profile your program by profiling.tracing.TracingProfiler or profiling.sampling.SamplingProfiler directly:
from profiling.tracing import TracingProfiler
# profile your program.
profiler = TracingProfiler()
profiler.start()
... # run your program.
profiler.stop()
# or using context manager.
with profiler:
... # run your program.
# view and interact with the result.
profiler.run_viewer()
# or save profile data to file
profiler.dump('path/to/file')
Viewer Key Bindings
- q - Quit.
- space - Pause/Resume.
- \ - Toggle layout between NESTED and FLAT.
- ↑ and ↓ - Navigate frames.
- → - Expand the frame.
- ← - Fold the frame.
- > - Go to the hotspot.
- esc - Defocus.
- [ and ] - Change sorting column.
Columns
Common
FUNCTION- The function name with the code location. (e.g.
my_func (my_code.py:42),my_func (my_module:42)) - Only the location without line number. (e.g.
my_code.py,my_module)
- The function name with the code location. (e.g.
Tracing Profiler
CALLS- Total call count of the function.OWN(Exclusive Time) - Total spent time in the function excluding sub calls./CALLafterOWN- Exclusive time per call.%afterOWN- Exclusive time per total spent time.DEEP(Inclusive Time) - Total spent time in the function./CALLafterDEEP- Inclusive time per call.%afterDEEP- Inclusive time per total spent time.
Sampling Profiler
OWN(Exclusive Samples) - Number of samples which are collected during the direct execution of the function.%afterOWN- Exclusive samples per number of the total samples.DEEP(Inclusive Samples) - Number of samples which are collected during the excution of the function.%afterDEEP- Inclusive samples per number of the total samples.
Testing
There are some additional requirements to run the test code, which can be installed by running the following command.
$ pip install $(python test/fit_requirements.py test/requirements.txt)
Then you should be able to run pytest.
$ pytest -v
Thanks to
- Seungmyeong Yang who suggested this project.
- Pavel who inspired to implement
-moption.
Licensing
Written by Heungsub Lee at What! Studio in Nexon, and distributed under the BSD 3-Clause license.
1k Jan 1, 2023
5k Jan 1, 2023
3.9k Dec 19, 2022
138 Dec 30, 2022
17 Aug 23, 2022
15.6k Jan 2, 2023
996 Jan 1, 2023
3.2k Jan 7, 2023
663 Dec 29, 2022
9k Jan 2, 2023
3.7k Dec 30, 2022
1.6k Dec 31, 2022
1.7k Jan 5, 2023
80 Nov 18, 2022
735 Dec 27, 2022
4 Feb 11, 2022
116 Dec 15, 2022
1 Nov 1, 2021
9.5k Jan 8, 2023