Pyccel stands for Python extension language using accelerators.

Overview

Welcome to Pyccel

build status Codacy Badge

Pyccel stands for Python extension language using accelerators.

The aim of Pyccel is to provide a simple way to generate automatically, parallel low level code. The main uses would be:

  1. Convert a Python code (or project) into a Fortran or C code.
  2. Accelerate Python functions by converting them to Fortran or C functions.

Pyccel can be viewed as:

  • Python-to-Fortran/C converter
  • a compiler for a Domain Specific Language with Python syntax

Pyccel comes with a selection of extensions allowing you to convert calls to some specific python packages to Fortran/C. The following packages will be covered (partially):

  • numpy
  • scipy
  • mpi4py
  • h5py (not available yet)

If you are eager to try Pyccel out, we recommend reading our quick-start guide!

Pyccel Installation Methods

Pyccel can be installed on virtually any machine that provides Python 3, the pip package manager, a C/Fortran compiler, and an Internet connection. Some advanced features of Pyccel require additional non-Python libraries to be installed, for which we provide detailed instructions below.

Alternatively, Pyccel can be deployed through a Linux Docker image that contains all dependencies, and which can be setup with any version of Pyccel. For more information, please read the section on Pyccel container images.

Requirements

First of all, Pyccel requires a working Fortran/C compiler

For Fortran it supports

For C it supports

In order to perform fast linear algebra calculations, Pyccel uses the following libraries:

Finally, Pyccel supports distributed-memory parallel programming through the Message Passing Interface (MPI) standard; hence it requires an MPI library like

We recommend using GFortran/GCC and Open-MPI.

Pyccel also depends on several Python3 packages, which are automatically downloaded by pip, the Python Package Installer, during the installation process. In addition to these, unit tests require the scipy, mpi4py, pytest and coverage packages, while building the documentation requires Sphinx <http://www.sphinx-doc.org/>.

Linux Debian/Ubuntu/Mint

To install all requirements on a Linux Ubuntu machine, just use APT, the Advanced Package Tool:

sudo apt update
sudo apt install gcc
sudo apt install gfortran
sudo apt install libblas-dev liblapack-dev
sudo apt install libopenmpi-dev openmpi-bin

Linux Fedora/CentOS/RHEL

Install all requirements using the DNF software package manager:

su
dnf check-update
dnf install gcc
dnf install gfortran
dnf install blas-devel lapack-devel
dnf install openmpi-devel
exit

Similar commands work on Linux openSUSE, just replace dnf with zypper.

Mac OS X

On an Apple Macintosh machine we recommend using Homebrew <https://brew.sh/>:

brew update
brew install gcc
brew install openblas
brew install lapack
brew install open-mpi

This requires that the Command Line Tools (CLT) for Xcode are installed.

Windows

Support for Windows is still experimental, and the installation of all requirements is more cumbersome. We recommend using Chocolatey <https://chocolatey.org/> to speed up the process, and we provide commands that work in a git-bash shell. In an Administrator prompt install git-bash (if needed), a Python3 Anaconda distribution, and a GCC compiler:

choco install git
choco install anaconda3
choco install mingw

Download x64 BLAS and LAPACK DLLs from https://icl.cs.utk.edu/lapack-for-windows/lapack/:

WEB_ADDRESS=https://icl.cs.utk.edu/lapack-for-windows/libraries/VisualStudio/3.7.0/Dynamic-MINGW/Win64
LIBRARY_DIR=/c/ProgramData/chocolatey/lib/mingw/tools/install/mingw64/lib
curl $WEB_ADDRESS/libblas.dll -o $LIBRARY_DIR/libblas.dll
curl $WEB_ADDRESS/liblapack.dll -o $LIBRARY_DIR/liblapack.dll

Generate static MS C runtime library from corresponding dynamic link library:

cd "$LIBRARY_DIR"
cp $SYSTEMROOT/SysWOW64/vcruntime140.dll .
gendef vcruntime140.dll
dlltool -d vcruntime140.def -l libmsvcr140.a -D vcruntime140.dll
cd -

Download MS MPI runtime and SDK, then install MPI:

WEB_ADDRESS=https://github.com/microsoft/Microsoft-MPI/releases/download/v10.1.1
curl -L $WEB_ADDRESS/msmpisetup.exe -o msmpisetup.exe
curl -L $WEB_ADDRESS/msmpisdk.msi -o msmpisdk.msi
./msmpisetup.exe
msiexec //i msmpisdk.msi

At this point, close and reopen your terminal to refresh all environment variables!

In Administrator git-bash, generate mpi.mod for gfortran according to https://abhilashreddy.com/writing/3/mpi_instructions.html:

cd "$MSMPI_INC"
sed -i 's/mpifptr.h/x64\/mpifptr.h/g' mpi.f90
sed -i 's/mpifptr.h/x64\/mpifptr.h/g' mpif.h
gfortran -c -D_WIN64 -D INT_PTR_KIND\(\)=8 -fno-range-check mpi.f90
cd -

Generate static libmsmpi.a from msmpi.dll:

cd "$MSMPI_LIB64"
cp $SYSTEMROOT/SysWOW64/msmpi.dll .
gendef msmpi.dll
dlltool -d msmpi.def -l libmsmpi.a -D msmpi.dll
cd -

Before installing Pyccel and using it, the Anaconda environment should be activated with:

source /c/tools/Anaconda3/etc/profile.d/conda.sh
conda activate

On Windows and/or Anaconda Python, use pip instead of pip3 for the Installation of pyccel below.

Installation

From PyPi

Simply run, for a user-specific installation:

pip3 install --user pyccel

or:

sudo pip3 install pyccel

for a system-wide installation.

From sources

  • Standard mode:

    git clone [email protected]:pyccel/pyccel.git
    cd pyccel
    pip3 install --user .
    
  • Development mode:

    git clone [email protected]:pyccel/pyccel.git
    cd pyccel
    pip3 install --user -e .
    

this will install a python library pyccel and a binary called pyccel. Any required Python packages will be installed automatically from PyPI.

Additional packages

In order to run the unit tests and to get a coverage report, a few additional Python packages should be installed::

pip3 install --user scipy
pip3 install --user mpi4py
pip3 install --user tblib
pip3 install --user pytest
pip3 install --user astunparse
pip3 install --user coverage

Most of the unit tests can also be run in parallel. This can be done by installing one additional package:

pip3 install --user pytest-xdist

Testing

To test your Pyccel installation please run the script tests/run_tests_py3.sh (Unix), or tests/run_tests.bat (Windows).

Continuous testing runs on github actions: <https://github.com/pyccel/pyccel/actions?query=branch%3Amaster>

Pyccel Container Images

Pyccel container images are available through both Docker Hub (docker.io) and the GitHub Container Registry (ghcr.io).

The images:

  • are based on ubuntu:latest
  • use distro packaged python3, gcc, gfortran, blas and openmpi
  • support all pyccel releases except the legacy "0.1"

Image tags match pyccel releases.

In order to implement your pyccel accelerated code, you can use a host based volume during the pyccel container creation.

For example:

docker pull pyccel/pyccel:v1.0.0
docker run -it -v $PWD:/data:rw  pyccel/pyccel:v1.0.0 bash

If you are using SELinux, you will need to set the right context for your host based volume. Alternatively you may have docker or podman set the context using -v $PWD:/data:rwz instead of -v $PWD:/data:rw .

Comments
  • testing new Windows configuration

    testing new Windows configuration

    Dear Pyccel team, I'm working on a Windows pc, and apparently I have all the pyccel-requirements already installed. Everything is easily provided for free by:

    • Visual Studio Community (2017 or 2019), with Python 3.7 64 bit [Python Software Fundation] ... (not Conga, or Anaconda or others...);
    • Intel oneAPI Base Toolkit (compilers, pre-optimized libraries (MKL included), analysis tools,....)
    • Intel oneAPI HPC Toolkit (C++,Fortran,OpenMP,MPI,...)

    I don't know if you provide support for testing Pyccel on new platforms, just let me know and eventually close this issue. In principle, if I had info about the meaning of the error-messages, I could try to fix something...

    Now: I installed Pyccel with pip and I'm running sequentially the pyccel-tests provided in the tests\run_tests.bat. I passed all the test until the following:

    python -m pytest -rx --ignore=symbolic --ignore=ndarrays -v --maxfail=1 -v ../tests/epyccel -m "not parallel"
    

    when I get the following error message (I append only the first and last lines):

    ==================================================================================================================== test session starts ====================================================================================================================
    platform win32 -- Python 3.7.8, pytest-6.2.3, py-1.10.0, pluggy-0.13.1 -- C:\Users\myuser\AppData\Local\Programs\Python\Python37\python.exe
    cachedir: .pytest_cache
    rootdir: C:\Users\myuser\Documents\00_Tools\Pyccel, configfile: pytest.ini
    plugins: forked-1.3.0, xdist-2.2.1
    collected 2659 items / 13 deselected / 2646 selected
    
    epyccel\test_array_as_func_args.py::test_array_int_1d_scalar_add[fortran] FAILED                                                                                                                                                                       [  0%]
    
    ========================================================================================================================= FAILURES ==========================================================================================================================
    ___________________________________________________________________________________________________________ test_array_int_1d_scalar_add[fortran] ___________________________________________________________________________________________________________
    
    language = 'fortran'
    
        def test_array_int_1d_scalar_add(language):
            @types( 'int8[:]' , 'int8' , 'int')
            @types( 'int16[:]', 'int16', 'int')
            @types( 'int32[:]', 'int32', 'int')
            @types( 'int64[:]', 'int64', 'int')
            def array_int_1d_scalar_add( x, a, x_len ):
                for i in range(x_len):
                    x[i] += a
            f1 = array_int_1d_scalar_add
    >       f2 = epyccel(f1, language=language)
    
    epyccel\test_array_as_func_args.py:24:
    _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
    C:\Users\myuser\AppData\Local\Programs\Python\Python37\lib\site-packages\pyccel\epyccel.py:335: in epyccel
        mod, fun = epyccel_seq( python_function_or_module, **kwargs )
    C:\Users\myuser\AppData\Local\Programs\Python\Python37\lib\site-packages\pyccel\epyccel.py:180: in epyccel_seq
        output_name = module_name)
    C:\Users\myuser\AppData\Local\Programs\Python\Python37\lib\site-packages\pyccel\codegen\pipeline.py:458: in execute_pyccel
        verbose)
    _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
    
    codegen = <pyccel.codegen.codegen.Codegen object at 0x00000251FF5E8B48>, language = 'fortran', pyccel_dirpath = 'C:\\Users\\myuser\\Documents\\00_Tools\\Pyccel\\tests\\__epyccel__\\__pyccel__', compiler = 'gfortran', mpi_compiler = None
    accelerator = None
    dep_mods = ('C:\\Users\\myuser\\Documents\\00_Tools\\Pyccel\\tests\\__epyccel__\\__pyccel__\\bind_c_mod_a9gxdwww2u2p_a9gxdwww2u2p', 'C:\\Users\\myuser\\Documents\\00_Tools\\Pyccel\\tests\\__epyccel__\\__pyccel__\\mod_a9gxdwww2u2p_a9gxdwww2u2p')
    libs = [], libdirs = [], includes = ['C:\\Users\\myuser\\Documents\\00_Tools\\Pyccel\\tests\\__epyccel__\\__pyccel__', 'C:\\Users\\myuser\\AppData\\Local\\Programs\\Python\\Python37\\lib\\site-packages\\numpy\\core\\include']
    flags = ' -O3 -fPIC  -I"C:\\Users\\myuser\\Documents\\00_Tools\\Pyccel\\tests\\__epyccel__\\__pyccel__"', sharedlib_modname = 'mod_a9gxdwww2u2p_a9gxdwww2u2p', verbose = False
    
        def create_shared_library(codegen,
                                  language,
                                  pyccel_dirpath,
                                  compiler,
                                  mpi_compiler,
                                  accelerator,
                                  dep_mods,
                                  libs,
                                  libdirs,
                                  includes='',
                                  flags = '',
                                  sharedlib_modname=None,
                                  verbose = False):
    ... code lines
    ... code lines
    ... code lines
    ... code lines
                if verbose:
                    print(out)
                if p.returncode != 0:
                    err_msg = "Failed to build module"
                    if verbose:
                        err_msg += "\n" + err
    >               raise RuntimeError(err_msg)
    E               RuntimeError: Failed to build module
    
    C:\Users\myuser\AppData\Local\Programs\Python\Python37\lib\site-packages\pyccel\codegen\python_wrapper.py:148: RuntimeError
    ------------------------------------------------------------------------------------------------------------------- Captured stdout call --------------------------------------------------------------------------------------------------------------------
    
    ERROR at shared library generation stage
    ----------------------------------------------------------------------------------------------------------------- Captured stdout teardown ------------------------------------------------------------------------------------------------------------------
    Tearing down!
    

    this message does not help me to fix the problem, should I run other basic tests before this?

    many thanks and best regards

    opened by ffambri 26
  • Numpy-pyccel

    Numpy-pyccel

    Describe the bug Hello,

    I create a program in fortran and then I transformed it to python and I use Pyccel to accelerate Python functions, but when I use numpy functions my code pyccel is slow than pure fortran , and when I don't use numpy functions pyccel be faster than my fortran code .

    So is there a possibility to combine the tow together (fortran code + pyccel ) ?

    To Reproduce Provide code to reproduce the behavior:

    from pyccel.decorators import types
    @types('float[:,:]','float[:,:](order=C)','float[:,:](order=C)')
    def qr_mgs(A,Q,R):
        from numpy import matmul,norm,zeros
        n,p=A.shape
        Qt=zeros((A.shape[1],A.shape[0]))
        s=zeros((Qt.shape[0],Q.shape[1]))
    
        for  k in range(p): 
            Q[:,k] = A[:,k]
            Qt[k,:]=A[:,k]
            s=matmul(Qt,Q)
            for  i in range(k):
                R[i,k] =s[k,i]
                Q[:,k] = Q[:,k] - R[i,k]*Q[:,i]
            R[k,k] = norm(Q[:,k])
            Q[:,k] = Q[:,k]/R[k,k]
        return 0
    from pyccel.epyccel import epyccel
    qr_f90 = epyccel(qr_mgs)
    def qr_fp(a):
        n,p=a.shape
        q= np.zeros((n,p),order='C')
        r= np.zeros((p,p),order='C')
       # qt=np.zeros((p,n),order='C')
        qr_f90(a,q,r)
        return q,r
    """"fortran version using f2py """
    %load_ext fortranmagic
    %%fortran --fcompiler=gnu95
    
    subroutine  mgs(A,n,p,Q,R)
        integer(8) ,intent(in) ::n,p   
        real(8) ,intent(in) ::A(n,p)    
        real(8), intent(out) :: R(p,p)
        real(8), intent(out) ::   Q(n,p)
        integer(8):: k,i
        Q = 0.0d+00
        R = 0.0d+00
        do k = 1,p
            Q(:,k) = A(:,k)
            do i = 1,k-1
                R(i,k) = dot_product(Q(:,i),Q(:,k))
                Q(:,k) = Q(:,k) - R(i,k)*Q(:,i)
            end do
            R(k,k) = sqrt(dot_product(Q(:,k),Q(:,k)))
            Q(:,k) = Q(:,k)/R(k,k)
        end do
    end subroutine  mgs
    

    Provide the generated code, or the error message:

    import numpy as np
    
    A=np.random.rand(700,700)
    q,r=qr_fp(A)
    Q,R=mgs(A)
    np.linalg.norm(A-q@r) =2.8754496870175957e-13
    np.linalg.norm(A-Q@R) = 2.870136485142167e-13
    %timeit qr_fp(A)
    22.9 s ± 242 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)
    %timeit mgs(A)
    247 ms ± 2.84 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)
    

    Expected behavior A clear and concise description of what you expected to happen.

    Language Please specify which language the python code is translated to (Fortran by default)

    Additional context Add any other context about the problem here.

    bug 
    opened by mohamedlaminebabou 25
  • Add transpose function

    Add transpose function

    Implement the transpose function. In fortran when used as a temporary variable the transpose function is used. When the variable is saved, the ordering is used to ensure pointers are created. In C the function is handled through unravelling or with a new function transpose_alias_assign.

    The transpose function is also used to implement the ndarray property .T and method .transpose()

    Related to #806 and #458

    Ready_to_merge 
    opened by EmilyBourne 22
  • Segmentation fault when doing a

    Segmentation fault when doing a "for reduction" with openmp

    When pyccelizing a function that does a for reduction of two arrays with openmp, a segmentation fault occurs. The same code with only one array in the for reduction does not encounter this error.

    As an example take a file pyccel_test.py with two functions mult_reduct1 and mult_reduct2:

    def mult_reduct1(arr : 'float[:,:]', arr1 : 'float[:,:]'):
        """
        Parameters:
            2 arrays of the same size
        """
        nx = arr.shape[1]
        ny = arr.shape[0]
    
        #$ omp parallel private (i, j)
        #$ omp for reduction(+: arr1)
        for i in range(nx):
            j = i * i
    
            arr1[i,0] = arr[i,0]**2
            arr1[i,1] = arr[i,1]**2
    
        #$ omp end parallel
    
    def mult_reduct2(arr : 'float[:,:]', arr1 : 'float[:,:]', arr2 : 'float[:,:]'):
        """
        Parameters:
            3 arrays of the same size
        """
        nx = arr.shape[1]
        ny = arr.shape[0]
    
        #$ omp parallel private (i, j)
        #$ omp for reduction(+: arr1, arr2)
        for i in range(nx):
            j = i * i
    
            arr1[i,0] = arr[i,0]**2
            arr1[i,1] = arr[i,1]**2
    
            arr2[i,0] = arr[i,0]**2
            arr2[i,1] = arr[i,1]**2
    
    
        #$ omp end parallel
    

    which is pyccelized with the command pyccel --openmp pyccel_test.py. The pyccelization itself gives no errors.

    When calling these functions in another python file with e.g.

    import pyccel_test as pt
    
    nx = 1000
    ny = 1000
    
    array = np.random.rand(nx,ny)
    array1 = np.zeros(np.shape(array), dtype=float)
    array2 = np.zeros(np.shape(array), dtype=float)
    
    pt.mult_reduct1(array, array1)
    print('\n results \n')
    print(np.sum(array1))
    
    pt.mult_reduct2(array, array1, array2)
    print('\n results \n')
    print(np.sum(array1))
    print(np.sum(array2))
    

    the output reads

    
     results 
    
    674.2675057646197
    Segmentation fault (core dumped)
    

    I am using the newest pyccel version 1.5.2

    bug 
    opened by dominikbell 20
  • Name collisions in C

    Name collisions in C

    Describe the bug C has no way to rename functions, nor does it have namespaces. This makes collisions much more likely when using more than one file. Furthermore there is no way to import part of a file. This makes collisions even more likely.

    To fix this problem I suggest pre-pending the name of the module to each function/variable/class defined within that module in the C code. I.e: f -> mod__f

    This should prevent almost all collisions. Collisions are still possible if the user names his variables using the following pattern:

    NAME1__NAME2.py containing: - function : f() NAME1.py containing: - function NAME2__f()

    But this seems unlikely, especially due to the double _

    Test Code Test files can be found here. When working they can be activated for C in this file

    Language C

    Additional Information I suspect the difficulty here will be at the function call. The namespace must be used to find the Import which created the object in order to find the module it has come from and it's original name

    enhancement Language:C discussion_needed 
    opened by EmilyBourne 20
  • Avoid printing temporary variables for scalar single return in c

    Avoid printing temporary variables for scalar single return in c

    When a function in C returns 1 scalar variable a temporary variable is created to mimic fortran. This is not necessary, to avoid that we should make _print_return in ccode.py prints directly the return of scalar/variable without assigning a temporary variable,

    Language:C 
    opened by MostafaMamouni 20
  • Numpy_version.h not generated

    Numpy_version.h not generated

    Describe the bug Under very specific circumstances the code will not compile as the file numpy_version.h is not created.

    To Reproduce There are several steps required to reproduce this bug:

    1. Compile code requiring a cwrapper using ndarrays (generates numpy_version.h)
    2. Modify pyccel/stdlib/cwrapper/cwrapper_ndarrays.h
    3. Recompile code

    In the last step, the following happens:

    1. Check to copy cwrapper info (this is the stage that generates numpy_version.h). Nothing happens as cwrapper.h and cwrapper.c exist and are up to date
    2. Check to copy cwrapper_ndarray info. The code recognises that the file pyccel/stdlib/cwrapper/cwrapper_ndarrays.h needs updating so it deletes the __pyccel__/cwrapper folder (containing numpy_version.h)
    3. Copy the cwrapper folder to collect the updated pyccel/stdlib/cwrapper/cwrapper_ndarrays.h file. This time the numpy_version.h is not corrected

    To fix the problem we simply need to move the cwrapper_ndarray files to their own folder

    Extra notes This problem could theoretically be triggered by a clock skew

    bug 
    opened by EmilyBourne 19
  • Clean compilation

    Clean compilation

    Overhaul the compilation method. A class Compiler is now used to handle all compilation steps. A class CompileObj is used to store all information required to compile a given file. Default compile configurations for basic compilers (gcc, gfortran, icc, ifort) are saved in json files in pyccel/compilers these are determined thanks to the sysconfig module. Fixes #541.

    After this PR, tests run much faster. The times below show the last 3 tests run on the master and ebourne_compiling branches, followed by the average times Master branch |Linux (python3.6)|MacOs|Windows| |-------------------|--------|----------| |19m37 | 16m54 | 28m34 | |18m46 | 21m13 | 24m28 | |21m13 | 19m53 | 23m54 | | | | | |19m52 | 19m20 | 25m39 |

    ebourne_compiling |Linux (python3.6)|MacOs|Windows| |-------------------|--------|----------| |13m13 | 17m23 | 15m17 | |11m59 | 16m17 | 15m25 | |15m30 | 17m52 | 18m01 | | | | | |13m34 | 17m11 | 16m11 |

    Supported compilers: The following compilers have been tested:

    • GNU : gcc/gfortran
    • intel : icc/ifort (tested by @EmilyBourne )
    • PGI : pgcc/pgfortran (tested by @saidctb )
    • nvidia : nvc/nvfort (tested by @rakati )

    Commit Summary

    • Create CompileObj to hold all information required to compile a given file
    • Create Compiler to hold all information about a given compiler
    • Remove cwrapper.py. The setup_X.py file is no longer required
    • Use Compiler and CompileObj to simplify pipeline.py and python_wrapper.py
    • Allow multiple accelerators
    • Make mpi an accelerator instead of specifying the mpi_compiler
    • Specify the compiler family (e.g. GNU) instead of the compiler name
    • Add a wrapper_flags argument to separate the flags for the file (fortran) from the flags for the wrapper (c)
    • When compiling a program, compile everything as modules then link separately. This procedure is closer to the procedure used for shared libraries
    • Rename pyc_math to pyc_math_c and pyc_math_f90. This allows us to differentiate between the .o files created from the source files which means we don't need to systematically recompile the library
    • Only overwrite stdlib files if they are out of date
    • Only recompile stdlib files if the source files have been updated
    • Add pyccel.compilers.generate_default to generate the default json files
    • Lock file during pickling. Fixes #917
    • Add a file describing how to use different compilers
    Ready_to_merge 
    opened by EmilyBourne 19
  • Drop Sympy's Symbol

    Drop Sympy's Symbol

    • Create class PyccelSymbol
    • Create functions symbols that return a tuple of instances of PyccelSymbol
    • Change imports in all code
    • Remove function allocatable_like not used anymore.
    • Remove unnecessary cast to a string.
    • Add magic methods __eq__ and __hash__ to Variable and DottedVariable
    Ready_to_merge 
    opened by MenoIy 19
  • Use correct format string when printing long integers in C

    Use correct format string when printing long integers in C

    Fix #1124: use correct static cast when printing integers in C.

    List of modifications:

    • Add f suffix to float32 float literals
    • Use macro INT64_C for appending suffix to int64 integer literals
    • Use macros PRId64 , PRId16, PRId8 to get correct conversion specifier (%ld %lld %hd ...) for int64, int16 and int8
    • Introduce new CMacro class representing a C Macro name
    • Introduce new CStringExpression representing a holder for a list of CMacros and LiteralStrings
    Ready_to_merge 
    opened by Hellio404 18
  • Extend OpenMP support by adding more pragmas

    Extend OpenMP support by adding more pragmas

    This PR is about extending the support of OpenMP by adding more Construct and Directives and their clauses:

    • Cover all the constructs that are inside the documentation;
    • Add clauses to the new constructs;
    • Support combined constructs;
    • Add tests for new pragmas;
    • Update the documentation to fit new changes and fixes.
    Ready_to_merge 
    opened by nhamidn 18
  • Developer documentation for GPU integration with Pyccel

    Developer documentation for GPU integration with Pyccel

    This PR serves as a starting point for discussing the developer documentation for GPU integration with Pyccel. The goal of the documentation is to help developers understand and use Pyccel effectively when building applications that run on GPUs.

    We would like to focus on the following topics in the documentation:

    device and kernel decorators CUDA Built-in variables

    We welcome feedback and suggestions on these topics, as well as any other ideas you may have for improving the developer documentation on GPU integration with Pyccel. Please share your thoughts on how we can make the documentation more useful and user-friendly for developers working with GPUs.

    documentation discussion_needed Language:Ccuda 
    opened by bauom 0
  • C functions don't correctly handle arrays with strides

    C functions don't correctly handle arrays with strides

    Describe the bug

    Passing a non-contiguous array to a function which has been translated to C gives unexpected results.

    To Reproduce

    Provide code to reproduce the behaviour:

    import numpy as np
    
    def f(a : 'float[:]'):
        a[:] = 2
    
    if __name__ == '__main__':
        arr = np.ones((2,3,4))
    
        f(arr[0,:,3])
        print(arr)
    

    Error details

    Python output:

    [[[1. 1. 1. 2.]
      [1. 1. 1. 2.]
      [1. 1. 1. 2.]]
    
     [[1. 1. 1. 1.]
      [1. 1. 1. 1.]
      [1. 1. 1. 1.]]]
    

    C output:

    [[[1.000000000000 1.000000000000 1.000000000000 2.000000000000]
    [1.000000000000 1.000000000000 1.000000000000 1.000000000000]
    [1.000000000000 1.000000000000 1.000000000000 1.000000000000]]
    [[1.000000000000 1.000000000000 1.000000000000 1.000000000000]
    [1.000000000000 1.000000000000 1.000000000000 1.000000000000]
    [1.000000000000 1.000000000000 1.000000000000 1.000000000000]]]
    

    The generated code for the function is:

    void f(t_ndarray a)
    {
        int64_t i;
        for (i = INT64_C(0); i < a.shape[INT64_C(0)]; i += INT64_C(1))
        {
            GET_ELEMENT(a, nd_double, (int64_t)i) = INT64_C(2);
        }
    }
    

    so either GET_ELEMENT is incorrectly implemented, or array_slicing is incorrectly implemented.

    Language

    C

    Additional context

    This problem may also exist for other code snippets using different structures

    bug Language:C 
    opened by EmilyBourne 0
  • adding sphinx doc

    adding sphinx doc

    This commit aims to add the autogenerated sphinx documention. a build has already been performed to generate html pages. At this point, each module has its own separate page.

    documentation discussion_needed 
    opened by jalalium 9
  • Fix wrong interpretation of AugAssign to AliasAssign

    Fix wrong interpretation of AugAssign to AliasAssign

    Fixes #1281

    This issue is solving the wrong interpretation of AugAssign into AliasAssign.

    import numpy as np
    
    if __name__ == '__main__':
        a = np.ones(5)
        b = a
        b += 3
    

    The code above, when converted to C, used to generate this:

    #include "test.h"
    #include <stdlib.h>
    #include "ndarrays.h"
    #include <stdint.h>
    int main()
    {
        t_ndarray a = {.shape = NULL};
        t_ndarray b = {.shape = NULL};
        a = array_create(1, (int64_t[]){INT64_C(5)}, nd_double, false);
        array_fill((double)1.0, a);
        alias_assign(&b, a);
        free_pointer(b);
        b = INT64_C(3);
        free_array(a);
        free_pointer(b);
        return 0;
    }
    

    Now it generates this:

    #include "test.h"
    #include <stdlib.h>
    #include "ndarrays.h"
    #include <stdint.h>
    #include <stdio.h>
    #include <inttypes.h>
    int main()
    {
        t_ndarray a = {.shape = NULL};
        t_ndarray b = {.shape = NULL};
        int64_t i;
        int64_t i_0001;
        a = array_create(1, (int64_t[]){INT64_C(5)}, nd_double, false);
        array_fill((double)1.0, a);
        alias_assign(&b, a);
        for (i = INT64_C(0); i < b.shape[INT64_C(0)]; i += INT64_C(1))
        {
            GET_ELEMENT(b, nd_double, (int64_t)i) += INT64_C(3);
        }
        free_array(a);
        free_pointer(b);
        return 0;
    }
    

    which removes an unnecessary de-allocation of b after the call to alias_assign(&b, a) and replaces b = INT64_C(3) with for loop to broadcast += operation on all elements of b, which point on a.

    For Fortran:

    program prog_prog_test
    
      use test_0001
    
      use, intrinsic :: ISO_C_Binding, only : f64 => C_DOUBLE , i64 => &
            C_INT64_T
      implicit none
    
      real(f64), allocatable, target :: a(:)
      real(f64), pointer :: b(:)
    
      allocate(a(0:4_i64))
      a = 1.0_f64
      b(0:) => a
      b(0:) => 3_i64
      if (allocated(a)) then
        deallocate(a)
      end if
    
    end program prog_prog_test
    

    the line b(0:) => 3_i64 is replaced with b = b + 3_i64

    bug Ready_to_merge 
    opened by aihya 3
  • Add developer docs for the syntactic stage

    Add developer docs for the syntactic stage

    Add developer docs for the syntactic stage. As per the docs, fatal errors should be used sparingly to give the user as much information as possible. This was not the case, so this PR also reduces the severity of most errors issued by the syntactic stage

    documentation Ready_for_review good-first-issue 
    opened by EmilyBourne 2
Releases(v1.7.0)
Owner
Pyccel
Pyccel
Sampling profiler for Python programs

py-spy: Sampling profiler for Python programs py-spy is a sampling profiler for Python programs. It lets you visualize what your Python program is spe

Ben Frederickson 9.5k Jan 1, 2023
Pearpy - a Python package for writing multithreaded code and parallelizing tasks across CPU threads.

Pearpy The Python package for (pear)allelizing your tasks across multiple CPU threads. Installation The latest version of Pearpy can be installed with

MLH Fellowship 5 Nov 1, 2021
Pytorch Lightning Distributed Accelerators using Ray

Distributed PyTorch Lightning Training on Ray This library adds new PyTorch Lightning accelerators for distributed training using the Ray distributed

null 166 Dec 27, 2022
Pytorch Lightning Distributed Accelerators using Ray

Distributed PyTorch Lightning Training on Ray This library adds new PyTorch Lightning plugins for distributed training using the Ray distributed compu

null 167 Jan 2, 2023
SCodeScanner stands for Source Code scanner where the user can scans the source code for finding the Critical Vulnerabilities.

The SCodeScanner stands for Source Code Scanner, where you can scan your source code files like PHP and get identify the vulnerabilities inside it. The tool can use by Pentester, Developer to quickly identify the weakness.

null 136 Dec 13, 2022
A Python library for differentiable optimal control on accelerators.

A Python library for differentiable optimal control on accelerators.

Google 80 Dec 21, 2022
CoSA: Scheduling by Constrained Optimization for Spatial Accelerators

CoSA is a scheduler for spatial DNN accelerators that generate high-performance schedules in one shot using mixed integer programming

UC Berkeley Architecture Research 44 Dec 13, 2022
An exploration of log domain "alternative floating point" for hardware ML/AI accelerators.

This repository contains the SystemVerilog RTL, C++, HLS (Intel FPGA OpenCL to wrap RTL code) and Python needed to reproduce the numerical results in

Facebook Research 373 Dec 31, 2022
Azure MLOps (v2) solution accelerators.

Azure MLOps (v2) solution accelerator Welcome to the MLOps (v2) solution accelerator repository! This project is intended to serve as the starting poi

Microsoft Azure 233 Jan 1, 2023
This is a file deletion program that asks you for an extension of a file (.mp3, .pdf, .docx, etc.) to delete all of the files in a dir that have that extension.

FileBulk This is a file deletion program that asks you for an extension of a file (.mp3, .pdf, .docx, etc.) to delete all of the files in a dir that h

Enoc Mena 1 Jun 26, 2022
VSCode extension to sort and refactor python imports using reorder-python-imports.

reorder-python-imports VSCode extension to sort and refactor python imports using reorder-python-imports. Unlike other import organizers, reorder-pyth

Ryan Butler 3 Aug 26, 2022
Nuitka Organization 8k Jan 7, 2023
A flask extension using pyexcel to read, manipulate and write data in different excel formats: csv, ods, xls, xlsx and xlsm.

Flask-Excel - Let you focus on data, instead of file formats Support the project If your company has embedded pyexcel and its components into a revenu

null 247 Dec 27, 2022
You can use the mvc pattern in your flask application using this extension.

You can use the mvc pattern in your flask application using this extension. Installation Run the follow command to install mvc_flask: $ pip install mv

Marcus Pereira 37 Dec 17, 2022
This is just a GUI that detects your file's real extension using the filetype module.

Real-file.extnsn This is just a GUI that detects your file's real extension using the filetype module. Requirements Python 3.4 and above filetype modu

null 1 Aug 8, 2021
Python module (C extension and plain python) implementing Aho-Corasick algorithm

pyahocorasick pyahocorasick is a fast and memory efficient library for exact or approximate multi-pattern string search meaning that you can find mult

Wojciech Muła 763 Dec 27, 2022
Python module (C extension and plain python) implementing Aho-Corasick algorithm

pyahocorasick pyahocorasick is a fast and memory efficient library for exact or approximate multi-pattern string search meaning that you can find mult

Wojciech Muła 579 Feb 17, 2021
Visual Python is a GUI-based Python code generator, developed on the Jupyter Notebook environment as an extension.

Visual Python is a GUI-based Python code generator, developed on the Jupyter Notebook environment as an extension.

Visual Python 564 Jan 3, 2023