Python compiler that massively increases Python's code performance without code changes.

Overview

alt text

Flyable - A python compiler for highly performant code

Flyable is a Python compiler that generates efficient native code. It uses different techniques and aggressive optimizations to make sure your code runs the way you want: the fastest way possible. It also relies on CPython to make sure that all your favorite libraries are still available and fully functional.

Flyable is licensed under the GNU Affero General Public License v3.0 .

Build

Make sure that you have downloaded the GCC compiler and you have an active CPython installation setup on your machine and follow this tutorial to learn how to compile, link and run a Python script with Flyable. Alternatively, you can read the code in the main.py file to see how we do this.

How does it get faster ?

Flyable does multiple things to generate efficient code but most of the performance gains come from the following:

  • Native execution of the code
  • Static function dispatch (enabling direct call and efficient inlining)
  • Type tracking (When possible)
  • Function specialization depending on the signature but also on the usage

Supported platforms

Flyable only generates code for Windows 64 bits x86 for now. It will quickly be extended to Linux and Mac.

How to use it ?

For now, Flyable is only available as a module that reads a Python file and outputs an object file. To run the output, the object file must be linked with any python39.lib file available in your Python installation. To run a Flyable made executable, it is required to have an active CPython installation setup on the machine.

Flyable is still in an early stage. We do not recommend full-scale usage at the moment.

Plan

Our current efforts are primarily focused on stabilizing the compiler to support most Python syntax and optimize the most common Python use cases.

Contributions

If you wish to, you can participate in this project in many ways:

Feel free to contact us at [email protected] if you have any other concerns.

Comments
  • [BUG] infinite recursion in load_lib_and_dependecies

    [BUG] infinite recursion in load_lib_and_dependecies

    library_loader.load_lib_and_dependecies will cause to go into a infinite recursion when ran on linux. Might want to limit how many nested calls can be made.

    bug solved 
    opened by WhoAteDaCake 17
  • Error with the modulo operator

    Error with the modulo operator

    If you take the modulo of a variable while in a for loop (the variable must be the one used by the for loop (i in the example)), it yields a tuple where the first element is the floor division and the second element is the actual modulo Ex:

    for i in range(100)
        print(i % 2) # yields (i // 2, i % 2)
    
    bug solved 
    opened by Ecoral360 7
  • [BUG] load_lib_and_dependecies should check the error before appending to the path

    [BUG] load_lib_and_dependecies should check the error before appending to the path

    My machine does not have GLIBC_2.29, so it causes a crash during the library loading, however, that exception is not handled properly and another attempt is made to load the library, with a messed up path (since the function just appends first piece of the path from error message), which causes a recursive loop, which will eventually crash the program.

    A naive solution to the problem I've came up with is:

    def load_lib_and_dependecies(path, lib):
        try:
            return ctypes.CDLL(path + lib)
        except OSError as excp:
            # Get the name of the library not found
            error_msg = excp.args[0]
            # Should crash for any errors that are not missing object file 
            # errors, since we can't handle them
            if not "cannot open shared" in error_msg:
                # https://stackoverflow.com/questions/24752395/python-raise-from-usage
                # Helps to avoid a massive error message due to recursive calls
                # within an try/except
                raise excp from None
            lib_load = error_msg.split(" ")[0]
            lib_load = lib_load[0:-1]
            # Now load it
            load_lib_and_dependecies(path,lib_load)
            return load_lib_and_dependecies(path,lib)
    
    bug solved 
    opened by WhoAteDaCake 6
  • Debug tools

    Debug tools

    This branch aim was to implement an easy and expandable debug set of tools. All the code related to those tools is found in the flyable/debug directory. There is essentially 2 parts to this system

    1. The debug flag system

    • In main.py, there is now a new constant variable called ENABLED_DEBUG_FLAG where you put all the flags you want to enable during the compilation of the program.
    • Inside the compiler, the programmer can add a call to the function value_if_debug that takes a normal value, a debug value and a debug flag. At runtime, the function will return the debug value if the specified flag is enabled, else the normal value.

    example

    ENABLED_DEBUG_FLAGS: list[DebugFlags] = [
        DebugFlags.SHOW_VISIT_AST,
        DebugFlags.SHOW_OUTPUT_BUILDER,
    ]
    ...
    DebugFlags.enable_debug_flags(*ENABLED_DEBUG_FLAGS)
    

    2. The analysers

    • An analyser extends a particular class and wraps some of its methods to allow a non-intrusive, non-modifying way of debugging the class.
    • It is recommended to create a debug flag for each analyser and instentiate it inside a value_if_debug function along side its non-debug counterpart

    example

    self.__builder = value_if_debug(CodeBuilder(self), CodeBuilderAnalyser(self), DebugFlags.SHOW_OUTPUT_BUILDER)
    
    feature 
    opened by Ecoral360 4
  • Variable scope issue

    Variable scope issue

    When a local variable has the same name as a global variable, flyable thinks we are talking about the global variable.

    example 1:

    x = "global "
    
    def foo():
        x = x * 2
        print(x)
    
    foo()
    print(x)
    

    cpython: error flyable: prints global global

    example 2:

    x = "global "
    
    def foo():
        x = "hello"
        print(x)
    
    foo()
    print(x)
    

    cpython: prints hello global flyable: prints hello hello

    bug solved 
    opened by YohanFinet 4
  • Inheritance & virtual call

    Inheritance & virtual call

    Implement the ability to check / dispatch call for inheritance.

    For now, only single inheritance needs to be supported but some groundwork can be put in place for multiple inheritance.

    feature solved critical 
    opened by ALavallee 4
  • Support Importing External Libraries

    Support Importing External Libraries

    It is possible that external libraries interpret the compiled python program as the location of the python executable instead of the actual path of the system's installed Python executable.

    <module 'sympy' from '/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/sympy/__init__.py'>
    Traceback (most recent call last):
      File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/numpy/core/__init__.py", line 23, in <module>
        from . import multiarray
      File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/numpy/core/multiarray.py", line 10, in <module>
        from . import overrides
      File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/numpy/core/overrides.py", line 6, in <module>
        from numpy.core._multiarray_umath import (
    ImportError: PyCapsule_Import could not import module "datetime"
    
    During handling of the above exception, another exception occurred:
    
    Traceback (most recent call last):
      File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/numpy/__init__.py", line 144, in <module>
        from . import core
      File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/numpy/core/__init__.py", line 49, in <module>
        raise ImportError(msg)
    ImportError:
    
    IMPORTANT: PLEASE READ THIS FOR ADVICE ON HOW TO SOLVE THIS ISSUE!
    
    Importing the numpy C-extensions failed. This error can happen for
    many reasons, often due to issues with your setup or how NumPy was
    installed.
    
    We have compiled some common reasons and troubleshooting tips at:
    
        https://numpy.org/devdocs/user/troubleshooting-importerror.html
    
    Please note and check the following:
    
      * The Python version is: Python3.10 from ".../Flyable/build/macos-arm64/a.out"
      * The NumPy version is: "1.22.1"
    
    and make sure that they are the versions you expect.
    Please carefully study the documentation linked above for further help.
    
    Original error was: PyCapsule_Import could not import module "datetime"
    
    Application ended with code 0
    
    bug invalid 
    opened by daytonjallen 3
  • Centralise build files

    Centralise build files

    Right now runtime object files, python static and dynamic libraries are all prsent in the multiples build folders. Ideally they would all be at the same place to avoid repetition.

    test 
    opened by ALavallee 3
  • Binary operations are not working

    Binary operations are not working

    When you compile your code, the compiler simply takes the argument on the left to resolve the binary operation, whatever it is. Ex:

    3 + 4 # yields 3 
    
    8 % 2 # yields 8
    
    bug solved 
    opened by Ecoral360 3
  • Comprehension with unpack

    Comprehension with unpack

    The compiler raise an exception when trying to handle comprehension with multiple for items. To fix this we need a unpack routine in the code.

    `old_price = {'milk': 1.02, 'coffee': 2.5, 'bread': 2.5}

    dollar_to_pound = 0.76 new_price = {item: value * dollar_to_pound for (item, value) in old_price.items()} print(new_price)`

    bug 
    opened by ALavallee 3
  • Fix Binary_Add for Python Strings

    Fix Binary_Add for Python Strings

    We're currently including the addition of strings in our Number Protocol call PyNumber_Add, however Python has a separate way of adding strings.

    From their handling of the Binary_Add bytecode:

    // code ...
    if (PyUnicode_CheckExact(left) &&
             PyUnicode_CheckExact(right)) {
        sum = unicode_concatenate(tstate, left, right, f, next_instr);
        /* unicode_concatenate consumed the ref to left */
    }
    else {
        sum = PyNumber_Add(left, right);
        Py_DECREF(left);
    }
    // ... code
    
    bug invalid 
    opened by daytonjallen 2
Owner
Flyable
Redefining the development of high-performance programs for Python developers !
Flyable
PerfSpect is a system performance characterization tool based on linux perf targeting Intel microarchitectures

PerfSpect PerfSpect is a system performance characterization tool based on linux perf targeting Intel microarchitectures. The tool has two parts perf

Intel Corporation 139 Dec 30, 2022
guapow is an on-demand and auto performance optimizer for Linux applications.

guapow is an on-demand and auto performance optimizer for Linux applications. This project's name is an abbreviation for Guarana powder (Guaraná is a fruit from the Amazon rainforest with a highly caffeinated seed).

Vinícius Moreira 19 Nov 18, 2022
Pearpy - a Python package for writing multithreaded code and parallelizing tasks across CPU threads.

Pearpy The Python package for (pear)allelizing your tasks across multiple CPU threads. Installation The latest version of Pearpy can be installed with

MLH Fellowship 5 Nov 1, 2021
Pyccel stands for Python extension language using accelerators.

Pyccel stands for Python extension language using accelerators.

Pyccel 242 Jan 2, 2023
Sampling profiler for Python programs

py-spy: Sampling profiler for Python programs py-spy is a sampling profiler for Python programs. It lets you visualize what your Python program is spe

Ben Frederickson 9.5k Jan 1, 2023
How to Create a YouTube Bot that Increases Views using Python Programming Language

YouTube-Bot-in-Python-Selenium How to Create a YouTube Bot that Increases Views using Python Programming Language. The app is for educational purpose

Edna 14 Jan 3, 2023
The venturimeter works on the principle of Bernoulli's equation, i.e., the pressure decreases as the velocity increases.

The venturimeter works on the principle of Bernoulli's equation, i.e., the pressure decreases as the velocity increases. The cross-section of the throat is less than the cross-section of the inlet pipe.The animation is done in python manim

Shankar Mahadevan L 1 Dec 3, 2021
Volta: A Virtual Assistant which increases your productivity with time as you use it…

Volta Official Documentation Overview & Purpose Volta: A Virtual Assistant which increases your productivity with time as you use it… Volta, developed

Abeer Joshi 1 Jan 14, 2022
OpenSea Bulk Uploader And Trader 100000 NFTs (MAC WINDOWS ANDROID LINUX) Automatically and massively upload and sell your non-fungible tokens on OpenSea using Python Selenium

OpenSea Bulk Uploader And Trader 100000 NFTs (MAC WINDOWS ANDROID LINUX) Automatically and massively upload and sell your non-fungible tokens on OpenS

ERC-7211 3 Mar 24, 2022
A :baby: buddy to help caregivers track sleep, feedings, diaper changes, and tummy time to learn about and predict baby's needs without (as much) guess work.

Baby Buddy A buddy for babies! Helps caregivers track sleep, feedings, diaper changes, tummy time and more to learn about and predict baby's needs wit

Baby Buddy 1.5k Jan 2, 2023
Massively parallel self-organizing maps: accelerate training on multicore CPUs, GPUs, and clusters

Somoclu Somoclu is a massively parallel implementation of self-organizing maps. It exploits multicore CPUs, it is able to rely on MPI for distributing

Peter Wittek 239 Nov 10, 2022
Massively parallel self-organizing maps: accelerate training on multicore CPUs, GPUs, and clusters

Somoclu Somoclu is a massively parallel implementation of self-organizing maps. It exploits multicore CPUs, it is able to rely on MPI for distributing

Peter Wittek 239 Nov 10, 2022
squid-dl is a massively parallel yt-dlp-based YouTube downloader.

squid-dl squid-dl is a massively parallel yt-dlp-based YouTube downloader. Installation Run the setup.py, which will install squid-dl and its two depe

tuxlovesyou 51 Jan 5, 2023
Use PaddlePaddle to reproduce the paper:mT5: A Massively Multilingual Pre-trained Text-to-Text Transformer

MT5_paddle Use PaddlePaddle to reproduce the paper:mT5: A Massively Multilingual Pre-trained Text-to-Text Transformer English | 简体中文 mT5: A Massively

null 2 Oct 17, 2021
Nuitka Organization 8k Jan 7, 2023
A HTML-code compiler-thing that lets you reuse HTML code.

RHTML RHTML stands for Reusable-Hyper-Text-Markup-Language, and is pronounced "Rech-tee-em-el" despite how its abbreviation is. As the name stands, RH

Duckie 4 Nov 15, 2021
🎉 🎉 PyComp - Java Code compiler written in python.

?? ?? PyComp Java Code compiler written in python. This is yet another compiler meant for babcock students project which was created using pure python

Alumona Benaiah 5 Nov 30, 2022
Docker image with Uvicorn managed by Gunicorn for high-performance FastAPI web applications in Python 3.6 and above with performance auto-tuning. Optionally with Alpine Linux.

Supported tags and respective Dockerfile links python3.8, latest (Dockerfile) python3.7, (Dockerfile) python3.6 (Dockerfile) python3.8-slim (Dockerfil

Sebastián Ramírez 2.1k Dec 31, 2022
peace-performance (Rust) binding for python. To calculate star ratings and performance points for all osu! gamemodes

peace-performance-python Fast, To calculate star ratings and performance points for all osu! gamemodes peace-performance (Rust) binding for python bas

null 9 Sep 19, 2022