There is a bug in the referenced CoreCLR submodule in master.

The NuGet config in CoreCLR/tests/src/runtime (tests) references a feed on MyGet that no longer exists. https://github.com/dotnet/coreclr/blob/release/1.0.0-rc2/tests/src/NuGet.Config

When you run BuildDeps.bat it will try and build CoreCLR which fails at the restore packages phase, since the feed no longer exists.

Im trying to build Pyjion using VS 2015 Update 3, cl Compiler Version 19.00.24224. When I build pyjion.sln, the build fails on _Py_IDENTIFIER macros. (The first one being here) It seems the designated initializer the macro expands to is not valid. This seems strange to me, and Im not sure where to proceed. The full output log is https://gist.github.com/ethanhs/2538016e4b8e9b80b20d8cd56e06a996

Im not sure why this would fail exactly. The same macro is used without issue in CPython sources, which worked fine with the same compiler. I was hoping to get some guidance on what the issue could be. I have tried re-installing Visual Studio to no avail.

This partially fixes #88 by improving how code for ROT_TWO is generated. In particular, optimized locals are used for integers and floats. A similar fix will be made for ROT_THREE in a following patch.

API in CPython becomes:

1. JitInit()
2. EvalFrame()
3. Code objects gain co_extra
4. PyEval_EvalFrameEx_NoJit() (which should be named something else eventually)
5. Interpreter state gains eval_frame

Everything else is contained within Pyjion itself.

This is the beginning of my work on moving Pyjion to CoreCLR 2.0. Currently I am having some issues with the "real jit" not existing, but I just need to read through things a bit more. I just wanted to put this out there and get feedback (if there is any) on my work so far.

WIP - DO NOT MERGE

After this (hopefully) gets merged, I will move on to porting to a cmake build system.

FAIL: test_first_completed (test.test_concurrent_futures.ProcessPoolWaitTests)

Traceback (most recent call last): File "C:\Users\brcan\Documents\Repositories\Pyjion\Python\lib\test\test_concurrent_futures.py", line 204, in test_firs t_completed self.assertEqual(set([future1]), done) AssertionError: Items in the first set but not the second: <Future at 0x1dc452a8468 state=running> Items in the second set but not the first: <Future at 0x1dc45820740 state=finished returned NoneType>

FAIL: test_first_exception (test.test_concurrent_futures.ProcessPoolWaitTests)

Traceback (most recent call last): File "C:\Users\brcan\Documents\Repositories\Pyjion\Python\lib\test\test_concurrent_futures.py", line 228, in test_firs t_exception self.assertEqual(set([future1, future2]), finished) AssertionError: Items in the second set but not the first: <Future at 0x1dc45820740 state=finished returned NoneType>

Since no activity for 2 months , this project is stopped? Shouldn't microsoft just contribute to PyPy development ? PyPy team is already working back on CPyext compatiblity and it is a lot improving.

This patch changes the implementation of ROT_THREE to only escape its operands when the types are not all ints or all floats. Furthermore, the ROT_THREE code generator has been enhanced to use the appropriate type for defining locals.

This is the companion patch to 539866f0ec5e3595bfd5e8c9117084f2517dbc58 and is the last patch to properly resolve issue #88.

I'm a bit of a know-nothing, so pardon if this is a stupid question.

Are there plans to make this buildable on (or at least for) Linux? A cursory glance through some of the source makes this seem non-trivial, but I figure I'd ask.

Thanks much!

Triggers an assertion failure at https://github.com/Microsoft/Pyjion/blob/master/Pyjion/absint.h#L176. If you run with test -n you end up with:

PS C:\Users\brcan\Documents\Repositories\Pyjion\Python> .\python.bat -m test -n
Running Debug|x64 interpreter...
== CPython 3.6.0a0 (default, Oct 26 2015, 11:12:33) [MSC v.1900 64 bit (AMD64)]
c:\users\brcan\documents\repositories\pyjion\pyjion\absint.h(176) : Assertion failed: !(valueInfo.Value == &Undefined && !isUndefined)
==   Windows-10.0.10240 little-endian
==   hash algorithm: siphash24 64bit
==   C:\Users\brcan\Documents\Repositories\Pyjion\Python\build\test_python_6592
Testing with flags: sys.flags(debug=0, inspect=0, interactive=0, optimize=0, dont_write_bytecode=0, no_user_site=0, no_site=0, ignore_environment=0, verbose=0, bytes_warning=0, quiet=0, hash_randomization=1, isolated=0)
[  1/398] test_grammar
Unknown unsupported opcode: YIELD_FROMC:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\include\vector(159) : Assertion failed: vector iterator + offset out of range
C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\include\vector(160) : Assertion failed: "Standard C++ Libraries Out of Range" && 0

g++ (6.3.1) and clang++ (3.9.1), I've got

/home/ryuta/packages/pyjion/src/pyjion-git/Pyjion/taggedptr.h:60:10: error: pasting "*" and "tmpIn" does not give a valid preprocessing token
  PyObject* ##name = init_number(tmp_##name, value);
          ^
/home/ryuta/packages/pyjion/src/pyjion-git/Pyjion/intrins.cpp:2445:9: note: in expansion of macro ‘INIT_TMP_NUMBER’
         INIT_TMP_NUMBER(tmpIn, UNTAG_IT(inI));
         ^~~~~~~~~~~~~~~
        size_t tmp_tmpIn[(((sizeof(PyVarObject) + sizeof(uint32_t) * ((((sizeof(tagged_ptr) * 8) - 1) + 30 - 1) / 30)) / sizeof(size_t)) + sizeof(size_t))]; PyObject*tmpIn = init_number(tmp_tmpIn, ((inI) >> 1));;
        *out = PyLong_AsDouble(tmpIn);
    }
    else {
        *out = PyLong_AsDouble(in);
    }
    return *out == -1.0 && PyErr_Occurred();
}

PyObject* ##name = init_number(tmp_##name, value);

get preprocessed to

PyObject*tmpIn = init_number(tmp_tmpIn, ((inI) >> 1))

## of ##name in PyObject* ##name = init_number(tmp_##name, value); should be removed. Is this change valid? I don't have a Windows machine to check this change won't break the build.

Hi, I was unable to compile the following snippet of code:

    map<std::u16string, int> intSettings;

        int getIntConfigValue(const WCHAR* name, int defaultValue) override
        {
        if (intSettings.find(name) != intSettings.end())
            return intSettings[name];
        return defaultValue;
        }

from Pyjion\cee.h. I'm not quite sure how to fix this easily...

Here's the full log:

[ 33%] Building CXX object CMakeFiles/pyjionlib.dir/Pyjion/pycomp.cpp.obj
pycomp.cpp
C:\msys64\home\ryuta\packages\pyjion\pyjion\Pyjion\cee.h(86): error C2664: 'std::_Tree_const_iterator<std::_Tree_val<std::_Tree_simple_types<std::pair<const std::u16string,int>>>> std::_Tree<std::_Tmap_traits<_Kty,_Ty,_Pr,_Alloc,false>>::find(const std::basic_string<char16_t,std::char_traits<char16_t>,std::allocator<char16_t>> &) const': cannot convert argument 1 from 'const WCHAR *' to 'const std::basic_string<char16_t,std::char_traits<char16_t>,std::allocator<char16_t>> &'
        with
        [
            _Kty=std::u16string,
            _Ty=int,
            _Pr=std::less<std::u16string>,
            _Alloc=std::allocator<std::pair<const std::u16string,int>>
        ]
C:\msys64\home\ryuta\packages\pyjion\pyjion\Pyjion\cee.h(86): note: Reason: cannot convert from 'const WCHAR *' to 'const std::basic_string<char16_t,std::char_traits<char16_t>,std::allocator<char16_t>>'
C:\msys64\home\ryuta\packages\pyjion\pyjion\Pyjion\cee.h(86): note: No constructor could take the source type, or constructor overload resolution was ambiguous
C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.27.29110\include\xtree(1467): note: see declaration of 'std::_Tree<std::_Tmap_traits<_Kty,_Ty,_Pr,_Alloc,false>>::find'
        with
        [
            _Kty=std::u16string,
            _Ty=int,
            _Pr=std::less<std::u16string>,
            _Alloc=std::allocator<std::pair<const std::u16string,int>>
        ]
C:\msys64\home\ryuta\packages\pyjion\pyjion\Pyjion\cee.h(87): error C2679: binary '[': no operator found which takes a right-hand operand of type 'const WCHAR *' (or there is no acceptable conversion)
C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.27.29110\include\map(324): note: could be 'int &std::map<std::u16string,int,std::less<std::u16string>,std::allocator<std::pair<const std::u16string,int>>>::operator [](const std::basic_string<char16_t,std::char_traits<char16_t>,std::allocator<char16_t>> &)'
C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.27.29110\include\map(153): note: or       'int &std::map<std::u16string,int,std::less<std::u16string>,std::allocator<std::pair<const std::u16string,int>>>::operator [](std::basic_string<char16_t,std::char_traits<char16_t>,std::allocator<char16_t>> &&)'
C:\msys64\home\ryuta\packages\pyjion\pyjion\Pyjion\cee.h(87): note: while trying to match the argument list '(std::map<std::u16string,int,std::less<std::u16string>,std::allocator<std::pair<const std::u16string,int>>>, const WCHAR *)'
C:\msys64\home\ryuta\packages\pyjion\pyjion\Pyjion\cee.h(93): error C2664: 'std::_Tree_const_iterator<std::_Tree_val<std::_Tree_simple_types<std::pair<const std::u16string,const char16_t *>>>> std::_Tree<std::_Tmap_traits<_Kty,_Ty,_Pr,_Alloc,false>>::find(const std::basic_string<char16_t,std::char_traits<char16_t>,std::allocator<char16_t>> &) const': cannot convert argument 1 from 'const WCHAR *' to 'const std::basic_string<char16_t,std::char_traits<char16_t>,std::allocator<char16_t>> &'
        with
        [
            _Kty=std::u16string,
            _Ty=const char16_t *,
            _Pr=std::less<std::u16string>,
            _Alloc=std::allocator<std::pair<const std::u16string,const char16_t *>>
        ]
C:\msys64\home\ryuta\packages\pyjion\pyjion\Pyjion\cee.h(93): note: Reason: cannot convert from 'const WCHAR *' to 'const std::basic_string<char16_t,std::char_traits<char16_t>,std::allocator<char16_t>>'
C:\msys64\home\ryuta\packages\pyjion\pyjion\Pyjion\cee.h(93): note: No constructor could take the source type, or constructor overload resolution was ambiguous
C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.27.29110\include\xtree(1467): note: see declaration of 'std::_Tree<std::_Tmap_traits<_Kty,_Ty,_Pr,_Alloc,false>>::find'
        with
        [
            _Kty=std::u16string,
            _Ty=const char16_t *,
            _Pr=std::less<std::u16string>,
            _Alloc=std::allocator<std::pair<const std::u16string,const char16_t *>>
        ]
C:\msys64\home\ryuta\packages\pyjion\pyjion\Pyjion\cee.h(94): error C2679: binary '[': no operator found which takes a right-hand operand of type 'const WCHAR *' (or there is no acceptable conversion)
C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.27.29110\include\map(324): note: could be 'const char16_t *&std::map<std::u16string,const char16_t *,std::less<std::u16string>,std::allocator<std::pair<const std::u16string,const char16_t *>>>::operator [](const std::basic_string<char16_t,std::char_traits<char16_t>,std::allocator<char16_t>> &)'
C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.27.29110\include\map(153): note: or       'const char16_t *&std::map<std::u16string,const char16_t *,std::less<std::u16string>,std::allocator<std::pair<const std::u16string,const char16_t *>>>::operator [](std::basic_string<char16_t,std::char_traits<char16_t>,std::allocator<char16_t>> &&)'
C:\msys64\home\ryuta\packages\pyjion\pyjion\Pyjion\cee.h(94): note: while trying to match the argument list '(std::map<std::u16string,const char16_t *,std::less<std::u16string>,std::allocator<std::pair<const std::u16string,const char16_t *>>>, const WCHAR *)'
C:\msys64\home\ryuta\packages\pyjion\pyjion\Pyjion\pycomp.h(248): warning C4267: 'return': conversion from 'size_t' to 'int', possible loss of data
NMAKE : fatal error U1077: 'C:\PROGRA~2\MICROS~4\2019\COMMUN~1\VC\Tools\MSVC\1427~1.291\bin\Hostx64\x64\cl.exe' : return code '0x2'
Stop.
NMAKE : fatal error U1077: '"C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.27.29110\bin\HostX64\x64\nmake.exe"' : return code '0x2'
Stop.
NMAKE : fatal error U1077: '"C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.27.29110\bin\HostX64\x64\nmake.exe"' : return code '0x2'
Stop.

• [x] Convert build process to CMake
• [x] Link .NET 5 stdlib libraries directly instead of copying dll files
• [x] Update API to implement the merged PEP for FrameEval
• [x] Remove the Python submodule as now uses the Python3 libraries directly
• [x] Validate C++11/Clang support and compilation errors to ensure builds correctly on Linux and MacOS
• [x] Remove MSBuild related files and scripts
• [x] Remove IExecutionEngine/CExecutionEngine shim classes as they're removed from .NET and no longer needed for init
• [x] Remove legacy Opcodes
• [x] Replace Windows TLS API with the "new" PEP539 TLS API
• [x] Add new opcodes
• [x] Test it?

Opcodes to implement

• [x] JUMP_IF_NOT_EXC_MATCH
• [x] DICT_MERGE
• [x] LOAD_ASSERTION_ERROR
• [x] RERAISE
• [x] SETUP_ANNOTATIONS
• [x] IS_OP
• [x] DICT_UPDATE
• [x] CONTAINS_OP
• [x] SET_UPDATE
• [x] LIST_EXTEND
• [x] LIST_TO_TUPLE
• [x] ROT_FOUR

Unsupported

• [ ] WITH_EXCEPT_START
• [ ] END_ASYNC_FOR
• [ ] GET_AITER

Consider this function:

def simple_func(a, b, c):
    if a > b:
        return b if b > c else c
    else:
        return 0

To get good performance, we want to generate code that operates on machine integers, not Python objects. That means that we must:

• Profile and record that (as is likely here) a, b, and c are all integers normally.
• Generate optimized machine code that
  • tests that a, b, and c are machine integers
  • if not, bail out to the interpreter and deoptimize the generated code
  • otherwise, use the optimized machine code.

After the death of pyston, the community, which had great hopes for pyjion, is starting to feel nervous about the future of pyjion. The repository is not very active, and little communication is made of it.

Pyjion is cited as the most promising attempt to speed up Python on HN, /r/python, twitter regularly, so enthusiasm is here.

So how is it going on ? Be honest. We all can understand people at MS have many important things to do and nobody would blame you if you couldn't work on it.

On the other hand, if you need help (money, people) to carry on, we could spread the word.

What is JIT?

... yeah, I can google it ... but why not mention it at least once in README?

JIT --> Just In Time Compilation

Never take anything for granted.