(cudf) user@host:/pysamprof/pyext$ PYTHONPATH=../../pysamprof/3rd_party/protobuf/python/ python ../../pysamprof/trace_writer/python/raw2pprof.py --raw ./result/trace/30920.0.raw --symbols ./result/trace/30920.0.symraw --out ./result/pprof_out/trace.pprof --resolve Traceback (most recent call last): File "../../pysamprof/trace_writer/python/raw2pprof.py", line 16, in TYPE_UNDERFINEDF, TYPE_SAMPLE, TYPE_FUNCTION_INFO, TYPE_MAPPING = [struct.unpack('>i', tag)[0] for tag in 'UNDF SMPL FNCI MAPP'.split()] File "../../pysamprof/trace_writer/python/raw2pprof.py", line 16, in TYPE_UNDERFINEDF, TYPE_SAMPLE, TYPE_FUNCTION_INFO, TYPE_MAPPING = [struct.unpack('>i', tag)[0] for tag in 'UNDF SMPL FNCI MAPP'.split()] TypeError: a bytes-like object is required, not 'str'

Applying patch in Windows: aregm@Barmaley MINGW64 /c/Projects/pysamprof (master) $ cd 3rd_party/protobuf-c && git apply ../protobuf-c-vs2008-support.patch ../protobuf-c-vs2008-support.patch:36: trailing whitespace.

../protobuf-c-vs2008-support.patch:39: trailing whitespace.

warning: protobuf-c/protobuf-c.h has type 100644, expected 100755 warning: protoc-c/c_message.cc has type 100644, expected 100755 warning: 2 lines add whitespace errors.

While executing cmake -DCMAKE_BUILD_TYPE=Release .. -DPYTHON_EXECUTABLE=/usr/bin/python3 on Ubuntu 20.04 I get the following errors:

-- The C compiler identification is GNU 9.3.0
-- The CXX compiler identification is GNU 9.3.0
-- Check for working C compiler: /usr/bin/cc
-- Check for working C compiler: /usr/bin/cc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Check for working CXX compiler: /usr/bin/c++
-- Check for working CXX compiler: /usr/bin/c++ -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Found PythonInterp: /usr/bin/python3 (found version "3.8.2")
-- detected CFLAGS for Python extensions: -fPIC
-- detected LDFLAGS for Python extensions: -Wl,-Bsymbolic-functions  -Wl,-z,relro -g -fwrapv -O2
-- libunwind needs lzma on this platform
-- Found PythonLibs: /usr/lib/x86_64-linux-gnu/libpython3.8.so (found version "3.8.2")
-- Configuring done
CMake Error at /nfs/site/home/gashiman/work/pysamprof/common/CMakeLists.txt:33 (add_library):
  Target "pysamprof-common" links to item "-Wl,-Bsymbolic-functions
  -Wl,-z,relro -g -fwrapv -O2 " which has leading or trailing whitespace.
  This is now an error according to policy CMP0004.


CMake Error at /nfs/site/home/gashiman/work/pysamprof/common/CMakeLists.txt:33 (add_library):
  Target "pysamprof-common" links to item "-Wl,-Bsymbolic-functions
  -Wl,-z,relro -g -fwrapv -O2 " which has leading or trailing whitespace.
  This is now an error according to policy CMP0004.


CMake Error at /nfs/site/home/gashiman/work/pysamprof/common/CMakeLists.txt:33 (add_library):
  Target "pysamprof-common" links to item "-Wl,-Bsymbolic-functions
  -Wl,-z,relro -g -fwrapv -O2 " which has leading or trailing whitespace.
  This is now an error according to policy CMP0004.


CMake Error at /nfs/site/home/gashiman/work/pysamprof/common/CMakeLists.txt:33 (add_library):
  Target "pysamprof-common" links to item "-Wl,-Bsymbolic-functions
  -Wl,-z,relro -g -fwrapv -O2 " which has leading or trailing whitespace.
  This is now an error according to policy CMP0004.


CMake Error at /nfs/site/home/gashiman/work/pysamprof/common/CMakeLists.txt:33 (add_library):
  Target "pysamprof-common" links to item "-Wl,-Bsymbolic-functions
  -Wl,-z,relro -g -fwrapv -O2 " which has leading or trailing whitespace.
  This is now an error according to policy CMP0004.


CMake Error at /nfs/site/home/gashiman/work/pysamprof/common/CMakeLists.txt:33 (add_library):
  Target "pysamprof-common" links to item "-Wl,-Bsymbolic-functions
  -Wl,-z,relro -g -fwrapv -O2 " which has leading or trailing whitespace.
  This is now an error according to policy CMP0004.


-- Generating done
CMake Generate step failed.  Build files cannot be regenerated correctly.

The problem is in Python configuration files so I ended up editing /usr/lib/python3.8/_sysconfigdata__x86_64-linux-gnu.py and removing all whitespace after -O2 where I found this flag (I am not sure exactly which lines goes into cmake config). But I don't think this is a correct way of handling this problem. Here is an answer to a similar problem https://stackoverflow.com/a/51367125/4584975 explaining how to strip whitespace from cmake config variables. I think it should be added to CMakeLists.txt.

Fix Python 3 support. Tested with Python 3.5 on Ubuntu 16.04 and Python 3.6 on Windows 10.

Fix mixed callstacks in release mode - this is done by stopping compiler from doing tail call optimization in PyEval_EvalFrameEx probe so that probe is always present in callstack.

Also drop legacy Makefiles - all is built with CMake now. And fixed a few warnings.

Currently to make profiling a child process possible that child has to manually do import pysamprof. By using proper pth file in site-packages it should be possible to automatically import pysamprof upon Python interpreter startup (see https://docs.python.org/3.7/library/site.html for reference).

Currently _pysamprof.c is both the interface to Python and the implementation of some internals. It should be only implementing the functions exposed to Python interpreter, and other things should be done in separate modules.

pysamprof now supports both Python versions. Most likely adding support for newer versions or other platforms (like Mac) won't break Python 2 support, but, as Python 2 is being deprecated in Jan 2020, it makes sense to deprecate its support (and maybe even remove said support to reduce the amount of ifdef-s around the code).

pysamprof uses signals on POSIX to collect stacks at regular intervals.

According to the spec, incoming signal interrupts a blocking wait of certain system calls (like recv). To make pysamprof less intrusive it needs to probe these system functions and automatically re-launch them if they were interrupted by profiling signal. Note that those functions shouldn't be restarted if they were interrupted by other signals!

Also note that this most likely applies to both Mac and Linux.

Prerequisite: #21.

This most likely requires both posting a source package and Python wheels for most popular cases (32- and 64-bit versions; Windows, Linux, Mac; Python 3.6 and 3.7).