In Pyston 2.2:

$ python
Python 3.8.8 (heads/rel2.2:6287d61, Apr 29 2021, 15:46:12)
[Pyston 2.2.0, GCC 9.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
rocky's .pythonrc loaded
>>> import sys, platform
>>> platform.python_implementation()
'CPython'
>>> "Pyston" in sys.version
True

The code from v1 could be used though: https://github.com/pyston/pyston_v1/blob/6d392ba7a5882139e8a6ad8bf5165abd5f32f2d3/from_cpython/Lib/platform.py#L1465

• Reduce image size by cleaning the package list after running apt-get (341 -> 309 MB)
• Use variables and symlink existing symlinks to reduce diff size of future updates
• Add python symlink

Hi, Thanks for pyston - I was able to run a test suite about 20% faster. It would be great if there was a release for 3.9. PEP-585, made a lot of improvements to pythons type annotations and is part of python 3.9 I had to backport code to work with 3.8 to work with Pyston - it was worth it but it would be nice to not need to.

Best Regards Stuart

I get

23:41:16 θ60° grizzlysmit@pern:~/Projects/python3/pyston $ git submodule update --init pyston/llvm pyston/bolt/bolt pyston/bolt/llvm pyston/LuaJIT pyston/macrobenchmarks
fatal: not a git repository (or any of the parent directories): .git

I guess I need to clone the pyston git hub first but I cannot get the link what gives

I was following https://github.com/pyston/pyston#readme

this allows us to cache stuff in a repl session and at module scope e.g.

def f():
    return 42
for x in range(100000):
    f() # we now cache the "f" lookup here

We forgot to decref a value in the error path of this function. It's easily reproducible by:

for i in range(1000000):
    try:
        int("ca4b9344", 10)
    except:
        pass

This fixes #149

Hello,

I’m using Pyston for the AI scripts, but I've got some errors during the install Pytorch 1.8.0 package.

The error is “wheel is not supported on this platform”. I upgraded the wheel and pip but I failed.

I am using Debian 10 amd64 architecture. Pyston is 2.3.1.

I'm building a Web API using FastAPI framework. It used uvicorn as a server to run the application. Can Pyston run uvicorn server? I'm not sure because I'm getting this error.

uvicorn main:app --reload                                                                                        
INFO:     Uvicorn running on http://127.0.0.1:8000 (Press CTRL+C to quit)
INFO:     Started reloader process [31274] using watchgod
Traceback (most recent call last):
  File "/home/emadmokhtar/.pyenv/versions/fastapi-pyston/bin/uvicorn", line 8, in <module>
    sys.exit(main())
  File "/home/emadmokhtar/.pyenv/versions/pyston-2.2/envs/fastapi-pyston/lib/pyston3.8/site-packages/click/core.py", line 1137, in __call__
    return self.main(*args, **kwargs)
  File "/home/emadmokhtar/.pyenv/versions/pyston-2.2/envs/fastapi-pyston/lib/pyston3.8/site-packages/click/core.py", line 1062, in main
    rv = self.invoke(ctx)
  File "/home/emadmokhtar/.pyenv/versions/pyston-2.2/envs/fastapi-pyston/lib/pyston3.8/site-packages/click/core.py", line 1404, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/home/emadmokhtar/.pyenv/versions/pyston-2.2/envs/fastapi-pyston/lib/pyston3.8/site-packages/click/core.py", line 763, in invoke
    return __callback(*args, **kwargs)
  File "/home/emadmokhtar/.pyenv/versions/pyston-2.2/envs/fastapi-pyston/lib/pyston3.8/site-packages/uvicorn/main.py", line 371, in main
    run(app, **kwargs)
  File "/home/emadmokhtar/.pyenv/versions/pyston-2.2/envs/fastapi-pyston/lib/pyston3.8/site-packages/uvicorn/main.py", line 388, in run
    ChangeReload(config, target=server.run, sockets=[sock]).run()
  File "/home/emadmokhtar/.pyenv/versions/pyston-2.2/envs/fastapi-pyston/lib/pyston3.8/site-packages/uvicorn/supervisors/basereload.py", line 43, in run
    self.startup()
  File "/home/emadmokhtar/.pyenv/versions/pyston-2.2/envs/fastapi-pyston/lib/pyston3.8/site-packages/uvicorn/supervisors/basereload.py", line 64, in startup
    self.process.start()
  File "/home/emadmokhtar/.pyenv/versions/pyston-2.2/lib/pyston3.8/multiprocessing/process.py", line 121, in start
    self._popen = self._Popen(self)
  File "/home/emadmokhtar/.pyenv/versions/pyston-2.2/lib/pyston3.8/multiprocessing/context.py", line 284, in _Popen
    return Popen(process_obj)
  File "/home/emadmokhtar/.pyenv/versions/pyston-2.2/lib/pyston3.8/multiprocessing/popen_spawn_posix.py", line 32, in __init__
    super().__init__(process_obj)
  File "/home/emadmokhtar/.pyenv/versions/pyston-2.2/lib/pyston3.8/multiprocessing/popen_fork.py", line 19, in __init__
    self._launch(process_obj)
  File "/home/emadmokhtar/.pyenv/versions/pyston-2.2/lib/pyston3.8/multiprocessing/popen_spawn_posix.py", line 39, in _launch
    from . import resource_tracker
  File "/home/emadmokhtar/.pyenv/versions/pyston-2.2/lib/pyston3.8/multiprocessing/resource_tracker.py", line 38, in <module>
    import _posixshmem
ImportError: /home/emadmokhtar/.pyenv/versions/pyston-2.2/lib/pyston3.8/lib-dynload/../../../lib/librt.so.1: undefined symbol: __clock_nanosleep, version GLIBC_PRIVATE

The current command to execute pyston is currently pyston3.

I understand that this is done to mirror python itself, but creating a second symlink named pyston or replacing the pyston3 symlink may be a better option, as there is no version of pyston that is using that symlink, unlike the python symlink which is used by py 2.

This significantly enhances our CI tests. And should make it much less likely for regressions to sneak in.

• add new non conda build job which uses regular unopt non-conda build but with ubuntus llvm libs
  • needs to run inside own docker image so we can control exactly what dependencies get installed
  • could not use the conda job because of dependency issues while installing the dependencies for the external tests.
  • we are building own ubuntu 20.04 based image because some tests fail when run as root and I found this the easiest and cleanest solution
• moved test_numpy to the EXTERNAL_TESTSUITES
• only run the external testsuites with default JIT flags to reduce build time
• test_numpy failed for me. It seemed the usercustomize.py script did not get run so I switched to putting in a .pth file

Runtime seems to be about 2h15min - will update when all CI jobs have finished

Hi,

thanks for the great new way of leveraging Pyston optimization via the new packages pyston-lite and pyston-lite-autoload 🚀

When I tried to install the autoload package in a fresh venv it will result in following error

pip install pyston-lite-autoload==2.3.4.1
Collecting pyston-lite-autoload==2.3.4.1
  Downloading pyston_lite_autoload-2.3.4.1.tar.gz (1.3 kB)
  Preparing metadata (setup.py) ... done
Collecting pyston_lite==2.3.4.1
  Downloading pyston_lite-2.3.4.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (466 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 466.1/466.1 KB 7.0 MB/s eta 0:00:00
Using legacy 'setup.py install' for pyston-lite-autoload, since package 'wheel' is not installed.
Installing collected packages: pyston_lite, pyston-lite-autoload
  Running setup.py install for pyston-lite-autoload ... error
  error: subprocess-exited-with-error
  
  × Running setup.py install for pyston-lite-autoload did not run successfully.
  │ exit code: 1
  ╰─> [6 lines of output]
      usage: setup.py [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
         or: setup.py --help [cmd1 cmd2 ...]
         or: setup.py --help-commands
         or: setup.py cmd --help
      
      error: option --single-version-externally-managed not recognized
      [end of output]
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
error: legacy-install-failure

× Encountered error while trying to install package.
╰─> pyston-lite-autoload

it looks like this package has no wheel uploaded to pypi. I tried to figure out, how you create and upload the package, but couldn't find the script for it.

The follow program describe a case of a recursive call. This recursive call occurs after building a weakref, then it crashes pyston and causes a segmentation fault.

test.py

import weakref

class Object:

    def __init__(self, arg):
        self.arg = arg

def test_set_callback_attribute():
    x = Object(1)
    callback = lambda ref: None
    callback = weakref.ref(callback, x)
    with test_set_callback_attribute():
        pass
test_set_callback_attribute()

Behavior on CPython 3.10.8:

>> python3.10 test.py
........
    test_set_callback_attribute()
  File "/home/xxm/Desktop/test.py", line 12, in test_set_callback_attribute
    test_set_callback_attribute()
  File "/home/xxm/Desktop/test.py", line 12, in test_set_callback_attribute
    test_set_callback_attribute()
  File "/home/xxm/Desktop/test.py", line 12, in test_set_callback_attribute
    test_set_callback_attribute()
  [Previous line repeated 994 more times]
  File "/home/xxm/Desktop/test.py", line 9, in test_set_callback_attribute
    x = Object(1)
RecursionError: maximum recursion depth exceeded

Behavior on pyston:

>> pyston test.py
......
TypeError: 'Object' object is not callable
Exception ignored in: <__main__.Object object at 0x41fa7970>
Traceback (most recent call last):
  File "/home/xxm/Desktop/test.py", line 11, in test_set_callback_attribute
    callback = weakref.ref(callback, x)
TypeError: 'Object' object is not callable
Exception ignored in: <__main__.Object object at 0x41fa79d0>
Traceback (most recent call last):
  File "/home/xxm/Desktop/test.py", line 11, in test_set_callback_attribute
    callback = weakref.ref(callback, x)
TypeError: 'Object' object is not callable
Exception ignored in: <__main__.Object object at 0x41fa7a30>
Traceback (most recent call last):
  File "/home/xxm/Desktop/test.py", line 11, in test_set_callback_attribute
Segmentation fault (core dumped)

Test Environment: Pyston: Python 3.8.12 (remotes/origin/release_2.3.5:4b858b5062) [Pyston 2.3.5, GCC 9.4.0] on linux Operating System: Ubuntu 18.04

The following code recursively call a function foo() in try branch. Then it causes the core dumped on pyston.

test.py

def foo():
         try:
            foo()
         except Exception as e:
            pass
foo()

Behavior on CPython 3.10.8: work well

Behavior on pyston:

>> pyston test.py
...
  File "/home/xxm/Desktop/test.py", line 4 in foo
  File "/home/xxm/Desktop/test..py", line 4 in foo
  File "/home/xxm/Desktop/test..py", line 4 in foo
  File "/home/xxm/Desktop/test.", line 4 in foo
  File "/home/xxm/Desktop/test.", line 4 in foo
  File "/home/xxm/Desktop/test.", line 4 in foo
  ...
Aborted (core dumped)

Test Environment: Pyston: Python 3.8.12 (remotes/origin/release_2.3.5:4b858b5062) [Pyston 2.3.5, GCC 9.4.0] on linux Operating System: Ubuntu 18.04

The following 'test.py' shows an example that referencing a cleared list index crashes pyston. This example does not crashes CPython, just reports an IndexError.

test.py

import pickle
import io

class P(pickle.Pickler):
    def persistent_id(self, obj):
        if obj is a[0]:
            a.clear()
        return None

a = [[[[]]]]
P(io.BytesIO()).dump(a)

Behavior on CPython3.10.8

>> python3.10 test.py
Traceback (most recent call last):
  File "/home/xxm/Desktop/test.py", line 11, in <module>
    P(io.BytesIO()).dump(a)
  File"/home/xxm/Desktop/test.py", line 6, in persistent_id
    if obj is a[0]:
IndexError: list index out of range

Behavior on pyston:

>> pyston test.py
Traceback (most recent call last):
  File "/home/xxm/Desktop/test.py", line 11, in <module>
    P(io.BytesIO()).dump(a)
  File"/home/xxm/Desktop/test.py", line 6, in persistent_id
    if obj is a[0]:
IndexError: list index out of range
Segmentation fault (core dumped）

Test Environment: Pyston: Python 3.8.12 (remotes/origin/release_2.3.5:4b858b5062) [Pyston 2.3.5, GCC 9.4.0] on linux Operating System: Ubuntu 18.04

In the follow, we define a function 'a()' then call a function 'throw()' to throw an exception class. Pyston crashes with segmentation fault.

test.py

class E(Exception):
    def __new__( *args):
        pass

def a():
    yield
a().throw(E)

Behavior on Pyston Segmentation fault (core dumped)

Behaviors on CPython 3.10.8:

Traceback (most recent call last):
  File "/home/xxm/Desktop/test.py", line 8, in <module>
    a().throw(E)
  File "/home/xxm/Desktop/test.py", line 6, in a
    def a():
TypeError: calling <class '__main__.E'> should have returned an instance of BaseException, not type

Test Environment: Pyston: Python 3.8.12 (remotes/origin/release_2.3.5:4b858b5062) [Pyston 2.3.5, GCC 9.4.0] on linux Operating System: Ubuntu 18.04

See the following example 'test.py'. We create an object Test() then store it into a shelve object 'store'. The storing process fails and it crashes pyston

test.py

import shelve

store = shelve.open("/tmp/a.db", writeback=True)

class Test(object):
    pass

def main():
    store["a"] = Test()

main()

Behavior on Pyston:

Segmentation fault (core dumped)

Behavior on CPython3.9.2

Exception ignored in: <function Shelf.__del__ at 0x7f40e1442480>
Traceback (most recent call last):
  File "/usr/local/lib/python3.11/shelve.py", line 162, in __del__
  File "/usr/local/lib/python3.11/shelve.py", line 144, in close
  File "/usr/local/lib/python3.11/shelve.py", line 168, in sync
  File "/usr/local/lib/python3.11/shelve.py", line 124, in __setitem__
ImportError: sys.meta_path is None, Python is likely shutting down

Test Environment: Pyston: Python 3.8.12 (remotes/origin/release_2.3.5:4b858b5062) [Pyston 2.3.5, GCC 9.4.0] on linux Operating System: Ubuntu 18.04