Nuitka is a Python compiler written in Python. It's fully compatible with Python 2.6, 2.7, 3.3, 3.4, 3.5, 3.6, 3.7, 3.8, and 3.9. You feed it your Python app, it does a lot of clever things, and spits out an executable or extension module.

Overview

Nuitka User Manual

doc/images/Nuitka-Logo-Symbol.png

Overview

This document is the recommended first read if you are interested in using Nuitka, understand its use cases, check what you can expect, license, requirements, credits, etc.

Nuitka is the Python compiler. It is written in Python. It is a seamless replacement or extension to the Python interpreter and compiles every construct that CPython 2.6, 2.7, 3.3, 3.4, 3.5, 3.6, 3.7, 3.8, 3.9 have, when itself run with that Python version.

It then executes uncompiled code and compiled code together in an extremely compatible manner.

You can use all Python library modules and all extension modules freely.

Nuitka translates the Python modules into a C level program that then uses libpython and static C files of its own to execute in the same way as CPython does.

All optimization is aimed at avoiding overhead, where it's unnecessary. None is aimed at removing compatibility, although slight improvements will occasionally be done, where not every bug of standard Python is emulated, e.g. more complete error messages are given, but there is a full compatibility mode to disable even that.

Usage

Requirements

  • C Compiler: You need a compiler with support for C11 or alternatively for C++03 [1]

    Currently this means, you need to use one of these compilers:

    • The gcc compiler of at least version 5.1, or the g++ compiler of at least version 4.4 as an alternative.
    • The clang compiler on macOS X or FreeBSD.
    • The MinGW64 C11 compiler on Windows, must be based on gcc 8 or higher. It will be automatically downloaded if not found, which is the recommended way of installing it.
    • Visual Studio 2019 or higher on Windows [2], older versions will work but only supported for commercial users. Configure to use the English language pack for best results (Nuitka filters away garbage outputs, but only for that language).
    • On Windows the clang-cl compiler on Windows can be used if provided by the Visual Studion installer.
  • Python: Version 2.6, 2.7 or 3.3, 3.4, 3.5, 3.6, 3.7, 3.8, 3.9

    For Python 3.3, and 3.4 and only those versions, we need other Python versions as a compile time dependency.

    Nuitka itself is fully compatible with all listed versions, but Scons as an internally used tool is not.

    For these versions, you need a Python2 or Python 3.5 or higher installed as well, but only during the compile time only. That is for use with Scons (which orchestrates the C compilation), which does not support the same Python versions as Nuitka.

    In addition, on Windows, Python2 cannot be used because clcache does not work with it, there a Python 3.5 or higher needs to be installed.

    Nuitka finds these needed Python versions (on Windows via registry) and you shouldn't notice it as long as they are installed.

    Moving binaries to other machines

    The created binaries can be made executable independent of the Python installation, with --standalone and --onefile options.

    Binary filename suffix

    The created binaries have an .exe suffix on Windows. On other platforms they have no suffix for standalone mode, or .bin suffix, that you are free to remove or change, or specify with the -o option.

    The suffix for acceleration mode is added just to be sure that the original script name and the binary name do not ever collide, so we can safely do an overwrite without destroying the original source file.

    It has to be CPython, Anaconda Python.

    You need the standard Python implementation, called "CPython", to execute Nuitka, because it is closely tied to implementation details of it.

    On Windows, for Python not installed system-wide and acceleration mode, you need to copy the PythonXX.DLL alongside it, something Nuitka does automatically.

    It cannot be from Windows app store

    It is known that Windows app store Python definitely does not work, it's checked against. And on macOS "pyenv" likely does not work.

  • Operating System: Linux, FreeBSD, NetBSD, macOS X, and Windows (32/64 bits).

    Others may work as well. The portability is expected to be generally good, but the e.g. Scons usage may have to be adapted. Make sure to match Windows Python and C compiler architecture, or else you will get cryptic error messages.

  • Architectures: x86, x86_64 (amd64), and arm, likely many more

    Other architectures are expected to also work, out of the box, as Nuitka is generally not using any hardware specifics. These are just the ones tested and known to be good. Feedback is welcome. Generally, the architectures that Debian supports can be considered good and tested too.

[1]

Support for this C11 is a given with gcc 5.x or higher or any clang version.

The MSVC compiler doesn't do it yet. But as a workaround, as the C++03 language standard is very overlapping with C11, it is then used instead where the C compiler is too old. Nuitka used to require a C++ compiler in the past, but it changed.

[2]

Download for free from http://www.visualstudio.com/en-us/downloads/download-visual-studio-vs.aspx (the Express editions work just fine).

The latest version is recommended if not required. There is no need to use older versions, they might in fact not work.

Command Line

The recommended way of executing Nuitka is <the_right_python> -m nuitka to be absolutely certain which Python interpreter you are using, so it is easier to match with what Nuitka has.

The next best way of executing Nuitka bare that is from a source checkout or archive, with no environment variable changes, most noteworthy, you do not have to mess with PYTHONPATH at all for Nuitka. You just execute the nuitka and nuitka-run scripts directly without any changes to the environment. You may want to add the bin directory to your PATH for your convenience, but that step is optional.

Moreover, if you want to execute with the right interpreter, in that case, be sure to execute <the_right_python> bin/nuitka and be good.

Pick the right Interpreter

If you encounter a SyntaxError you absolutely most certainly have picked the wrong interpreter for the program you are compiling.

Nuitka has a --help option to output what it can do:

nuitka --help

The nuitka-run command is the same as nuitka, but with a different default. It tries to compile and directly execute a Python script:

nuitka-run --help

This option that is different is --run, and passing on arguments after the first non-option to the created binary, so it is somewhat more similar to what plain python will do.

Installation

For most systems, there will be packages on the download page of Nuitka. But you can also install it from source code as described above, but also like any other Python program it can be installed via the normal python setup.py install routine.

License

Nuitka is licensed under the Apache License, Version 2.0; you may not use it except in compliance with the License.

You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.

Tutorial Setup and build on Windows

This is basic steps if you have nothing installed, of course if you have any of the parts, just skip it.

Setup

Install Python

  • Download and install from https://www.python.org/downloads/windows
  • Select one of Windows x86-64 web-based installer (64 bits Python, recommended) or x86 executable (32 bits Python) installer.
  • Verify using command python --version.

Install Nuitka

  • python -m pip install nuitka
  • Verify using command python -m nuitka --version

Write some code and test

Create a folder for the Python code

  • mkdir HelloWorld
  • make a python file named hello.py
def talk(message):
    return "Talk " + message

def main():
    print( talk("Hello World"))

if __name__ == "__main__":
    main()

Test your program

Do as you normally would. Running Nuitka on code that works incorrectly is not easier to debug.

python hello.py

Build it using

python -m nuitka --mingw64 hello.py

Note

This will prompt you to download a C caching tool (to speed up repeated compilation of generated C code) and a MinGW64 based C compiler. Say yes to those.

If you like to have full output add --show-progress and --show-scons.

Run it

Execute the hello.exe created near hello.py.

Distribute

To distribute, build with --standalone option, which will not output a single executable, but a whole folder. Copy the resulting hello.dist folder to the other machine and run it.

You may also try --onefile which does create a single file, but make sure that the mere standalone is working, before turning to it, as it will make the debugging only harder, e.g. in case of missing data files.

Use Cases

Use Case 1 - Program compilation with all modules embedded

If you want to compile a whole program recursively, and not only the single file that is the main program, do it like this:

python -m nuitka --follow-imports program.py

Note

There are more fine grained controls than --follow-imports available. Consider the output of nuitka --help. Including less modules into the compilation, but instead using normal Python for it will make it faster to compile.

In case you have a source directory with dynamically loaded files, i.e. one which cannot be found by recursing after normal import statements via the PYTHONPATH (which would be the recommended way), you can always require that a given directory shall also be included in the executable:

python -m nuitka --follow-imports --include-plugin-directory=plugin_dir program.py

Note

If you don't do any dynamic imports, simply setting your PYTHONPATH at compilation time is what you should do.

Use --include-plugin-directory only if you make __import__() calls that Nuitka cannot predict, because they e.g. depend on command line parameters. Nuitka also warns about these, and point to the option.

Note

The resulting filename will be program.exe on Windows, program.bin on other platforms.

Note

The resulting binary still depend on CPython and used C extension modules being installed.

If you want to be able to copy it to another machine, use --standalone and copy the created program.dist directory and execute the program.exe (Windows) or program (other platforms) put inside.

Use Case 2 - Extension Module compilation

If you want to compile a single extension module, all you have to do is this:

python -m nuitka --module some_module.py

The resulting file some_module.so can then be used instead of some_module.py.

Note

It's left as an exercise to the reader, to find out what happens if both are present.

Note

The option --follow-imports and other variants work as well, but the included modules will only become importable after you imported the some_module name.

Note

The resulting extension module can only be loaded into a CPython of the same version and doesn't include other extension modules.

Use Case 3 - Package compilation

If you need to compile a whole package and embed all modules, that is also feasible, use Nuitka like this:

python -m nuitka --module some_package --include-package=some_package

Note

The recursion into the package directory needs to be provided manually, otherwise, the package is empty. Data files located inside the package will not be embedded yet.

Use Case 4 - Program Distribution

For distribution to other systems, there is the standalone mode which produces a folder for which you can specify --standalone.

python -m nuitka --standalone program.py

Follow all imports is default in this mode. You can selectively exclude modules by specifically saying --nofollow-import-to, but then an ImportError will be raised when import of it is attempted at program runtime.

For data files to be included, use the option --include-data-file=<source>=<target> where the source is a file system path, but target has to be specified relative. For standalone you can also copy them manually, but this can do extra checks, and for onefile mode, there is no manual copying possible.

For package data, there is a better way, using --include-package-data which detects data files of packages automatically and copies them over. It even accepts patterns in shell style.

With data files, you are largely on your own. Nuitka keeps track of ones that are needed by popular packages, but it might be incomplete. Raise issues if you encounter something in these.

When that is working, you can use the onefile if you so desire.

python -m nuitka --onefile program.py

This will create a single binary, which on Linux will not even unpack itself, but instead loop back mount its contents as a filesystem and use that. On Windows, there are two modes, one which is copying it to the AppData of your company specified, to also use it as a cache, and one which does it in the temporary directory.

Typical Problems

Dynamic sys.path

If your script modifies sys.path to e.g. insert directories with source code relative to it, Nuitka will currently not be able to see those. However, if you set the PYTHONPATH to the resulting value, you will be able to compile it.

Missing data files in standalone

If your program fails to file data, it can cause all kinds of different behaviours, e.g. a package might complain it is not the right version, because a VERSION file check defaulted to unknown. The absence of icon files or help texts, may raise strange errors.

Often the error paths for files not being present are even buggy and will reveal programming errors like unbound local variables. Please look carefully at these exceptions keeping in mind that this can be the cause. If you program works without standalone, chances are data files might be cause.

Missing DLLs in standalone

Nuitka has plugins that deal with copying DLLs. For NumPy, SciPy, Tkinter, etc.

These need special treatment to be able to run on other systems. Manually copying them is not enough and will given strange errors. Sometimes newer version of packages, esp. NumPy can be unsupported. In this case you will have to raise an issue, and use the older one.

Dependency creep in standalone

Some packages are a single import, but to Nuitka mean that more than a thousand packages (literally) are to be included. The prime example of Pandas, which does want to plug and use just about everything you can imagine. Multiple frameworks for syntax highlighting everything imaginable take time.

Nuitka will have to learn effective caching to deal with this in the future. Right now, you will have to deal with huge compilation times for these.

Tips

Python command line flags

For passing things like -O or -S to Python, to your compiled program, there is a command line option name --python-flag= which makes Nuitka emulate these options.

The most important ones are supported, more can certainly be added.

Caching compilation results

The C compiler, when invoked with the same input files, will take a long time and much CPU to compile over and over. Make sure you are having ccache installed and configured when using gcc (even on Windows). It will make repeated compilations much faster, even if things are not yet not perfect, i.e. changes to the program can cause many C files to change, requiring a new compilation instead of using the cached result.

On Windows, with gcc Nuitka supports using ccache.exe which it will offer to download from an official source and it automatically. This is the recommended way of using it on Windows, as other versions can e.g. hang.

Nuitka will pick up ccache if it's in found in system PATH, and it will also be possible to provide if by setting NUITKA_CCACHE_BINARY to the full path of the binary, this is for use in CI systems.

For the Visual Studio compilers, you are just one pip install clcache command away. To make Nuitka use those, set NUITKA_CLCACHE_BINARY to the full path of clcache.exe, which will be in the scripts folder of the Python, you installed it into.

Runners

Avoid running the nuitka binary, doing python -m nuitka will make a 100% sure you are using what you think you are. Using the wrong Python will make it give you SyntaxError for good code or ImportError for installed modules. That is happening, when you run Nuitka with Python2 on Python3 code and vice versa. By explicitly calling the same Python interpreter binary, you avoid that issue entirely.

Fastest C Compilers

The fastest binaries of pystone.exe on Windows with 64 bits Python proved to be significantly faster with MinGW64, roughly 20% better score. So it is recommended for use over MSVC. Using clang-cl.exe of Clang7 was faster than MSVC, but still significantly slower than MinGW64, and it will be harder to use, so it is not recommended.

On Linux for pystone.bin the binary produced by clang6 was faster than gcc-6.3, but not by a significant margin. Since gcc is more often already installed, that is recommended to use for now.

Differences in C compilation times have not yet been examined.

Unexpected Slowdowns

Using the Python DLL, like standard CPython does can lead to unexpected slowdowns, e.g. in uncompiled code that works with Unicode strings. This is because calling to the DLL rather than residing in the DLL causes overhead, and this even happens to the DLL with itself, being slower, than a Python all contained in one binary.

So if feasible, aim at static linking, which is currently only possible with Anaconda Python on non-Windows.

Windows Standalone executables and dependencies

The process of making standalone executables for Windows traditionally involves using an external dependency walker in order to copy necessary libraries along with the compiled executables to the distribution folder.

Using the external dependency walker is quite a time consuming, and may copy some unnecessary libraries along the way (better have too much than missing).

There's also an experimental alternative internal dependency walker that relies on pefile which analyses PE imports of executables and libraries.

This implementation shall create smaller Standalone distributions since it won't include Windows' equivalent of the standard library, and will speed-up first Nuitka compilations by an order of magnitude.

In order to use it, you may enable the internal dependency walker by using the following switch:

python -m nuitka --standalone --windows-dependency-tool=pefile myprogram.py

Note

The pefile dependency walker will test all dependencies of the distribution folder.

Optionally, it is also possible to check all recursive dependencies of included libraries using the following switch along with the above one:

python -m nuitka --standalone --windows-dependency-tool=pefile --experimental=use_pefile_recurse myprogram.py

Note

Some modules may have hidden dependencies outside of their directory. In order for the pefile dependency walker to find them, you may also scan the whole site-packages directory for missing dependencies using the following switch along with the two above:

python -m nuitka --standalone --windows-dependency-tool=pefile --experimental=use_pefile_recurse --experimental=use_pefile_fullrecurse myprogram.py

Note

Be aware that using this switch will increase compilation time a lot.

Windows errors with resources

On Windows, the Windows Defender tool and the Windows Indexing Service both scan the freshly created binaries, while Nuitka wants to work with it, e.g. adding more resources, and then preventing operations randomly due to holding locks. Make sure to exclude your compilation stage from these services.

Windows standalone program redistribuation

Whether compiling with MingW or MSVC, the standalone programs have external dependencies to Visual C Runtime libraries. Nuitka tries to ship those dependent DLLs by copying them from your system.

Beginning with Microsoft Windows 10, Microsoft ships ucrt.dll (Universal C Runtime libraries) which rehook calls to api-ms-crt-*.dll.

With earlier Windows platforms (and wine/ReactOS), you should consider installing Visual C Runtime libraries before executing a Nuitka standalone compiled program.

Depending on the used C compiler, you'll need the following redist versions:

Visual C version Redist Year CPython
14.2 2019 3.5, 3.6, 3.7, 3.8, 3.9
14.1 2017 3.5, 3.6, 3.7, 3.8
14.0 2015 3.5, 3.6, 3.7, 3.8
10.0 2010 3.3, 3.4
9.0 2008 2.6, 2.7, 3.0, 3.1, 3.2

When using MingGW64, you'll need the following redist versions:

MingGW64 version Redist Year CPython
8.1.0 2015 3.5, 3.6, 3.7, 3.8, 3.9

Once the corresponding runtime libraries are installed on the target system, you may remove all api-ms-crt-*.dll files from your Nuitka compiled dist folder.

Detecting Nuitka at run time

It doesn't set sys.frozen unlike other tools. For Nuitka, we have the module attribute __compiled__ to test if a specific module was compiled.

Where to go next

Remember, this project is not completed yet. Although the CPython test suite works near perfect, there is still more work needed, esp. to make it do more optimization. Try it out.

Follow me on Twitter

Nuitka announcements and interesting stuff is pointed to on the Twitter account, but obviously with no details. @KayHayen.

I will not answer Nuitka issues via Twitter though, rather make occasional polls, and give important announcements, as well as low-level posts about development ongoing.

Report issues or bugs

Should you encounter any issues, bugs, or ideas, please visit the Nuitka bug tracker and report them.

Best practices for reporting bugs:

  • Please always include the following information in your report, for the underlying Python version. You can easily copy&paste this into your report.

    python -m nuitka --version
  • Try to make your example minimal. That is, try to remove code that does not contribute to the issue as much as possible. Ideally come up with a small reproducing program that illustrates the issue, using print with different results when that programs runs compiled or native.

  • If the problem occurs spuriously (i.e. not each time), try to set the environment variable PYTHONHASHSEED to 0, disabling hash randomization. If that makes the problem go away, try increasing in steps of 1 to a hash seed value that makes it happen every time, include it in your report.

  • Do not include the created code in your report. Given proper input, it's redundant, and it's not likely that I will look at it without the ability to change the Python or Nuitka source and re-run it.

  • Do not send screenshots of text, that is bad and lazy. Instead, capture text outputs from the console.

Word of Warning

Consider using this software with caution. Even though many tests are applied before releases, things are potentially breaking. Your feedback and patches to Nuitka are very welcome.

Join Nuitka

You are more than welcome to join Nuitka development and help to complete the project in all minor and major ways.

The development of Nuitka occurs in git. We currently have these 3 branches:

  • master

    This branch contains the stable release to which only hotfixes for bugs will be done. It is supposed to work at all times and is supported.

  • develop

    This branch contains the ongoing development. It may at times contain little regressions, but also new features. On this branch, the integration work is done, whereas new features might be developed on feature branches.

  • factory

    This branch contains unfinished and incomplete work. It is very frequently subject to git rebase and the public staging ground, where my work for develop branch lives first. It is intended for testing only and recommended to base any of your own development on. When updating it, you very often will get merge conflicts. Simply resolve those by doing git reset --hard origin/factory and switch to the latest version.

Note

The Developer Manual explains the coding rules, branching model used, with feature branches and hotfix releases, the Nuitka design and much more. Consider reading it to become a contributor. This document is intended for Nuitka users.

Donations

Should you feel that you cannot help Nuitka directly, but still want to support, please consider making a donation and help this way.

Unsupported functionality

The co_code attribute of code objects

The code objects are empty for native compiled functions. There is no bytecode with Nuitka's compiled function objects, so there is no way to provide it.

PDB

There is no tracing of compiled functions to attach a debugger to.

Optimization

Constant Folding

The most important form of optimization is the constant folding. This is when an operation can be fully predicted at compile time. Currently, Nuitka does these for some built-ins (but not all yet, somebody to look at this more closely will be very welcome!), and it does it e.g. for binary/unary operations and comparisons.

Constants currently recognized:

5 + 6     # binary operations
not 7     # unary operations
5 < 6     # comparisons
range(3)  # built-ins

Literals are the one obvious source of constants, but also most likely other optimization steps like constant propagation or function inlining will be. So this one should not be underestimated and a very important step of successful optimizations. Every option to produce a constant may impact the generated code quality a lot.

Status

The folding of constants is considered implemented, but it might be incomplete in that not all possible cases are caught. Please report it as a bug when you find an operation in Nuitka that has only constants as input and is not folded.

Constant Propagation

At the core of optimizations, there is an attempt to determine the values of variables at run time and predictions of assignments. It determines if their inputs are constants or of similar values. An expression, e.g. a module variable access, an expensive operation, may be constant across the module of the function scope and then there needs to be none or no repeated module variable look-up.

Consider e.g. the module attribute __name__ which likely is only ever read, so its value could be predicted to a constant string known at compile time. This can then be used as input to the constant folding.

if __name__ == "__main__":
   # Your test code might be here
   use_something_not_use_by_program()

Status

From modules attributes, only __name__ is currently actually optimized. Also possible would be at least __doc__. In the future, this may improve as SSA is expanded to module variables.

Built-in Name Lookups

Also, built-in exception name references are optimized if they are used as a module level read-only variables:

try:
   something()
except ValueError: # The ValueError is a slow global name lookup normally.
   pass

Status

This works for all built-in names. When an assignment is done to such a name, or it's even local, then, of course, it is not done.

Built-in Call Prediction

For built-in calls like type, len, or range it is often possible to predict the result at compile time, esp. for constant inputs the resulting value often can be precomputed by Nuitka. It can simply determine the result or the raised exception and replace the built-in call with that value, allowing for more constant folding or code path reduction.

type("string") # predictable result, builtin type str.
len([1, 2])    # predictable result
range(3, 9, 2) # predictable result
range(3, 9, 0) # predictable exception, range raises due to 0.

Status

The built-in call prediction is considered implemented. We can simply during compile time emulate the call and use its result or raised exception. But we may not cover all the built-ins there are yet.

Sometimes the result of a built-in should not be predicted when the result is big. A range() call e.g. may give too big values to include the result in the binary. Then it is not done.

range( 100000 ) # We do not want this one to be expanded

Status

This is considered mostly implemented. Please file bugs for built-ins that are pre-computed, but should not be computed by Nuitka at compile time with specific values.

Conditional Statement Prediction

For conditional statements, some branches may not ever be taken, because of the conditions being possible to predict. In these cases, the branch not taken and the condition check is removed.

This can typically predict code like this:

if __name__ == "__main__":
   # Your test code might be here
   use_something_not_use_by_program()

or

if False:
   # Your deactivated code might be here
   use_something_not_use_by_program()

It will also benefit from constant propagations, or enable them because once some branches have been removed, other things may become more predictable, so this can trigger other optimization to become possible.

Every branch removed makes optimization more likely. With some code branches removed, access patterns may be more friendly. Imagine e.g. that a function is only called in a removed branch. It may be possible to remove it entirely, and that may have other consequences too.

Status

This is considered implemented, but for the maximum benefit, more constants need to be determined at compile time.

Exception Propagation

For exceptions that are determined at compile time, there is an expression that will simply do raise the exception. These can be propagated upwards, collecting potentially "side effects", i.e. parts of expressions that were executed before it occurred, and still have to be executed.

Consider the following code:

print(side_effect_having() + (1 / 0))
print(something_else())

The (1 / 0) can be predicted to raise a ZeroDivisionError exception, which will be propagated through the + operation. That part is just Constant Propagation as normal.

The call side_effect_having() will have to be retained though, but the print does not and can be turned into an explicit raise. The statement sequence can then be aborted and as such the something_else call needs no code generation or consideration anymore.

To that end, Nuitka works with a special node that raises an exception and is wrapped with a so-called "side_effects" expression, but yet can be used in the code as an expression having a value.

Status

The propagation of exceptions is mostly implemented but needs handling in every kind of operations, and not all of them might do it already. As work progresses or examples arise, the coverage will be extended. Feel free to generate bug reports with non-working examples.

Exception Scope Reduction

Consider the following code:

try:
    b = 8
    print(range(3, b, 0))
    print("Will not be executed")
except ValueError as e:
    print(e)

The try block is bigger than it needs to be. The statement b = 8 cannot cause a ValueError to be raised. As such it can be moved to outside the try without any risk.

b = 8
try:
    print(range(3, b, 0))
    print("Will not be executed")
except ValueError as e:
    print(e)

Status

This is considered done. For every kind of operation, we trace if it may raise an exception. We do however not track properly yet, what can do a ValueError and what cannot.

Exception Block Inlining

With the exception propagation, it then becomes possible to transform this code:

try:
    b = 8
    print(range(3, b, 0))
    print("Will not be executed!")
except ValueError as e:
    print(e)
try:
    raise ValueError("range() step argument must not be zero")
except ValueError as e:
    print(e)

Which then can be lowered in complexity by avoiding the raise and catch of the exception, making it:

e = ValueError("range() step argument must not be zero")
print(e)

Status

This is not implemented yet.

Empty Branch Removal

For loops and conditional statements that contain only code without effect, it should be possible to remove the whole construct:

for i in range(1000):
    pass

The loop could be removed, at maximum, it should be considered an assignment of variable i to 999 and no more.

Status

This is not implemented yet, as it requires us to track iterators, and their side effects, as well as loop values, and exit conditions. Too much yet, but we will get there.

Another example:

if side_effect_free:
   pass

The condition check should be removed in this case, as its evaluation is not needed. It may be difficult to predict that side_effect_free has no side effects, but many times this might be possible.

Status

This is considered implemented. The conditional statement nature is removed if both branches are empty, only the condition is evaluated and checked for truth (in cases that could raise an exception).

Unpacking Prediction

When the length of the right-hand side of an assignment to a sequence can be predicted, the unpacking can be replaced with multiple assignments.

a, b, c = 1, side_effect_free(), 3
a = 1
b = side_effect_free()
c = 3

This is of course only really safe if the left-hand side cannot raise an exception while building the assignment targets.

We do this now, but only for constants, because we currently have no ability to predict if an expression can raise an exception or not.

Status

Not implemented yet. Will need us to see through the unpacking of what is an iteration over a tuple, we created ourselves. We are not there yet, but we will get there.

Built-in Type Inference

When a construct like in xrange() or in range() is used, it is possible to know what the iteration does and represent that so that iterator users can use that instead.

I consider that:

for i in xrange(1000):
    something(i)

could translate xrange(1000) into an object of a special class that does the integer looping more efficiently. In case i is only assigned from there, this could be a nice case for a dedicated class.

Status

Future work, not even started.

Quicker Function Calls

Functions are structured so that their parameter parsing and tp_call interface is separate from the actual function code. This way the call can be optimized away. One problem is that the evaluation order can differ.

def f(a, b, c):
    return a, b, c

f(c = get1(), b = get2(), a = get3())

This will have to evaluate first get1(), then get2() and only then get3() and then make the function call with these values.

Therefore it will be necessary to have a staging of the parameters before making the actual call, to avoid a re-ordering of the calls to get1(), get2(), and get3().

Status

Not even started. A re-formulation that avoids the dictionary to call the function, and instead uses temporary variables appears to be relatively straight forward once we do that kind of parameter analysis.

Lowering of iterated Container Types

In some cases, accesses to list constants can become tuple constants instead.

Consider that:

for x in [a, b, c]:
    something(x)

Can be optimized into this:

for x in (a, b, c):
     something(x)

This allows for simpler, faster code to be generated, and fewer checks needed, because e.g. the tuple is clearly immutable, whereas the list needs a check to assert that. This is also possible for sets.

Status

Implemented, even works for non-constants. Needs other optimization to become generally useful, and will itself help other optimization to become possible. This allows us to e.g. only treat iteration over tuples, and not care about sets.

In theory, something similar is also possible for dict. For the later, it will be non-trivial though to maintain the order of execution without temporary values introduced. The same thing is done for pure constants of these types, they change to tuple values when iterated.

Credits

Contributors to Nuitka

Thanks go to these individuals for their much-valued contributions to Nuitka. Contributors have the license to use Nuitka for their own code even if Closed Source.

The order is sorted by time.

  • Li Xuan Ji: Contributed patches for general portability issue and enhancements to the environment variable settings.
  • Nicolas Dumazet: Found and fixed reference counting issues, import packages work, improved some of the English and generally made good code contributions all over the place, solved code generation TODOs, did tree building cleanups, core stuff.
  • Khalid Abu Bakr: Submitted patches for his work to support MinGW and Windows, debugged the issues, and helped me to get cross compile with MinGW from Linux to Windows. This was quite difficult stuff.
  • Liu Zhenhai: Submitted patches for Windows support, making the inline Scons copy actually work on Windows as well. Also reported import related bugs, and generally helped me make the Windows port more usable through his testing and information.
  • Christopher Tott: Submitted patches for Windows, and general as well as structural cleanups.
  • Pete Hunt: Submitted patches for macOS X support.
  • "ownssh": Submitted patches for built-ins module guarding, and made massive efforts to make high-quality bug reports. Also the initial "standalone" mode implementation was created by him.
  • Juan Carlos Paco: Submitted cleanup patches, creator of the Nuitka GUI, creator of the Ninja IDE plugin for Nuitka.
  • "Dr. Equivalent": Submitted the Nuitka Logo.
  • Johan Holmberg: Submitted patch for Python3 support on macOS X.
  • Umbra: Submitted patches to make the Windows port more usable, adding user provided application icons, as well as MSVC support for large constants and console applications.
  • David Cortesi: Submitted patches and test cases to make macOS port more usable, specifically for the Python3 standalone support of Qt.
  • Andrew Leech: Submitted github pull request to allow using "-m nuitka" to call the compiler. Also pull request to improve "bist_nuitka" and to do the registration.
  • Paweł K: Submitted github pull request to remove glibc from standalone distribution, saving size and improving robustness considering the various distributions.
  • Orsiris de Jong: Submitted github pull request to implement the dependency walking with pefile under Windows.
  • Jorj X. McKie: Submitted github pull requests with NumPy plugin to retain its accelerating libraries, and Tkinter to include the TCL distribution on Windows.

Projects used by Nuitka

  • The CPython project

    Thanks for giving us CPython, which is the base of Nuitka. We are nothing without it.

  • The GCC project

    Thanks for not only the best compiler suite but also thanks for making it easy supporting to get Nuitka off the ground. Your compiler was the first usable for Nuitka and with very little effort.

  • The Scons project

    Thanks for tackling the difficult points and providing a Python environment to make the build results. This is such a perfect fit to Nuitka and a dependency that will likely remain.

  • The valgrind project

    Luckily we can use Valgrind to determine if something is an actual improvement without the noise. And it's also helpful to determine what's actually happening when comparing.

  • The NeuroDebian project

    Thanks for hosting the build infrastructure that the Debian and sponsor Yaroslav Halchenko uses to provide packages for all Ubuntu versions.

  • The openSUSE Buildservice

    Thanks for hosting this excellent service that allows us to provide RPMs for a large variety of platforms and make them available immediately nearly at release time.

  • The MinGW64 project

    Thanks for porting the gcc to Windows. This allowed portability of Nuitka with relatively little effort.

  • The Buildbot project

    Thanks for creating an easy to deploy and use continuous integration framework that also runs on Windows and is written and configured in Python code. This allows running the Nuitka tests long before release time.

  • The isort project

    Thanks for making nice import ordering so easy. This makes it so easy to let your IDE do it and clean up afterward.

  • The black project

    Thanks for making a fast and reliable way for automatically formatting the Nuitka source code.

Updates for this Manual

This document is written in REST. That is an ASCII format which is readable as ASCII, but used to generate PDF or HTML documents.

You will find the current source under: https://nuitka.net/gitweb/?p=Nuitka.git;a=blob_plain;f=README.rst

And the current PDF under: https://nuitka.net/doc/README.pdf

Comments
  • Nuitka one file standalone option

    Nuitka one file standalone option

    Project description: Nuitka has a mode meant for distribution to another system that puts everything needed in a single folder.

    One complaint often raised is that it's a folder rather than a single file, for alternative packaging methods, e.g. py2exe and pyinstaller, these do actually exist, and this project would be about integrating with that.

    In a first stage, you would identify the code of these tools that is doing it and try to port it to Nuitka for one or more platforms.

    Skills: Python programming, Linux and/or Windows installs of Python, both would be great.

    enhancement 
    opened by kayhayen 79
  • Problems compiling tensorflow --standalone

    Problems compiling tensorflow --standalone

    Hello,

    I'm trying to run a very simple tensorflow script through nuitka with the --standalone option. It builds fine BUT when going to run it complains about several packages being missing. packaging and setuptools are the first two, but using --include-package for those seems to fix those issues. However, it can't seems to get around missing the mock package. This is unfortunate as that's only needed by tests in tensorflow which we won't be running from this binary. Here are the details of my setup.

    A fresh virtualenv with only the following run in it:

    pip install tensorflow-gpu==1.10.0
    pip install nuitka
    
    $ python -m nuitka --version
    0.6.3
    Python: 2.7.12 (default, Dec  4 2017, 14:50:18)
    Executable: /mnt/nfs/zeiler/virtualenv/v1/bin/python
    OS: Linux
    Arch: x86_64
    

    Ubuntu 16.04 OS.

    This is the command I ran to compile:

    python -m nuitka --show-progress --show-scons --show-modules --include-package=setuptools --include-plugin-directory=/mnt/nfs/zeiler/virtualenv/v3/local/lib/python2.7/site-packages/pbr --include-plugin-directory=/mnt/nfs/zeiler/virtualenv/v3/local/lib/python2.7/site-packages/mock --include-package=mock --standalone main.py
    

    And this is the main.py script, tried to make it a minimal possible thing in tensorflow:

    import tensorflow as tf
    
    def main():
      hello = tf.constant('Hello,world!')
      sess = tf.Session()
      result = sess.run(hello)
      sess.close()
      print(result)
    
    if __name__ == '__main__':
      main()
    

    Originally I ran the above nuitka call without --include-package=pbr (which seems to have a packaging module in it) and --include-package=setuptools but that failed to find both of those. Now it's just stuck on mock like:

    (v3) zeiler@host:~/virtualenv/v3$ ./main.dist/main
    /mnt/nfs/zeiler/virtualenv/v3/main.dist/distutils/__init__.py:14: UserWarning: The virtualenv distutils package at %s appears to be in the same location as the system distutils?
    Traceback (most recent call last):
      File "/mnt/nfs/zeiler/virtualenv/v3/main.dist/main.py", line 4, in <module>
      File "/mnt/nfs/zeiler/virtualenv/v3/main.dist/tensorflow/__init__.py", line 22, in <module tensorflow>
      File "/mnt/nfs/zeiler/virtualenv/v3/main.dist/tensorflow/python/__init__.py", line 120, in <module tensorflow.python>
      File "/mnt/nfs/zeiler/virtualenv/v3/main.dist/tensorflow/python/platform/test.py", line 47, in <module tensorflow.python.platform.test>
      File "/mnt/nfs/zeiler/virtualenv/v3/main.dist/mock/__init__.py", line 2, in <module mock>
      File "/mnt/nfs/zeiler/virtualenv/v3/main.dist/mock/mock.py", line 71, in <module mock.mock>
      File "/mnt/nfs/zeiler/virtualenv/v3/main.dist/pbr/version.py", line 462, in semantic_version
      File "/mnt/nfs/zeiler/virtualenv/v3/main.dist/pbr/version.py", line 449, in _get_version_from_pkg_resources
      File "/mnt/nfs/zeiler/virtualenv/v3/main.dist/pbr/packaging.py", line 843, in get_version
    Exception: Versioning for this project requires either an sdist tarball, or access to an upstream git repository. It's also possible that there is a mismatch between the package name in setup.cfg and the argument given to pbr.version.VersionInfo. Project name mock was given, but was not able to be found.
    

    Without the --standalone option and where the virtualenv is still in place, it seems to find mock ok so it just seems like some packaging part isn't quite working for me.

    As a side note, I am pretty confused by the difference between many of the command line flags. The --help is the best resource it seems compared the docs, but it's very limited. For example to get mock work (or other ones) should I use:

    --include-package=mock
    --include-plugin-directory={...virtual env site-packages...}/mock
    --enable-plugin=mock
    

    I don't see much around the definition of a plugin and when do use that, which plugins are available, and how it's different from a package/modules? Answers to some of these might not only solve my problem but provide some better docs we can collectively contribute back.

    As a second side note, the compile time is very long with --standalone. Easily > 30 minutes, sometimes > 1 hour. I have ccache setup but it seems most time is spent in the python recursive importing and the very last gcc call for MetaPathBasedLoader.c. That last one along seems to sit there for about 50% of the time. Is that typical?

    Thanks so much for the help, I really love this tool I came across last week and I think it can be a game changer for us!

    bug 
    opened by zeiler 67
  • Not including tkinter for standalone exe

    Not including tkinter for standalone exe

    Cannot create standalone exe for scripts using tkinter.

    • nuitka pip installation
    • Compiler installed VS 2017
    • output of nuitka generation:
    Microsoft Windows [Version 10.0.16299.665]                                          
                                                                                        
    Jorj@ARRAKIS C:\Users\Jorj\Desktop\test                                             
    $ python -m nuitka --portable test.py                                               
                                                                                        
    Jorj@ARRAKIS C:\Users\Jorj\Desktop\test                                             
    $ python -m nuitka --version                                                        
    0.5.33                                                                              
    Python: 3.7.0 (v3.7.0:1bf9cc5093, Jun 27 2018, 04:59:51) [MSC v.1914 64 bit (AMD64)]
    Executable: C:\Users\Jorj\AppData\Local\Programs\Python\Python37\python.exe         
    OS: Windows                                                                         
    Arch: x86_64                                                                        
    
    • Script:
    import tkinter
    root = tkinter.Tk()
    root.bbox()
    root.bell()
    

    Output when running test.exe:

    Jorj@ARRAKIS C:\Users\Jorj\Desktop\test\test.dist                                                                                                                                                                                              
    $ test.exe                                                                                                                                                                                                                                     
    Traceback (most recent call last):                                                                                                                                                                                                             
     File "C:\Users\Jorj\Desktop\test\test.dist\test.py", line 2, in <module>                                                                                                                                                                     
      File "C:\Users\Jorj\Desktop\test\test.dist\tkinter\__init__.py", line 2020, in __init__                                                                                                                                                      
    _tkinter.TclError: Can't find a usable init.tcl in the following directories:                                                                                                                                                                  
        {C:\Users\Jorj\AppData\Local\Programs\Python\Python37\tcl} C:/Users/Jorj/AppData/Local/Programs/Python/Python37/tcl8.6 C:/Users/Jorj/Desktop/test/lib/tcl8.6 C:/Users/Jorj/Desktop/test/lib/tcl8.6 C:/Users/Jorj/Desktop/lib/tcl8.6 C:/User
    s/Jorj/Desktop/test/library C:/Users/Jorj/Desktop/library C:/Users/Jorj/Desktop/tcl8.6.8/library C:/Users/Jorj/tcl8.6.8/library                                                                                                                
                                                                                                                                                                                                                                                   
                                                                                                                                                                                                                                                   
                                                                                                                                                                                                                                                   
    This probably means that Tcl wasn't installed properly. 
    
    

    Problem goes away when I (e.g.) copy the two directories C:\Users\Jorj\AppData\Local\Programs\Python\Python37\tcl\tcl8.6 and C:\Users\Jorj\AppData\Local\Programs\Python\Python37\tcl\tk8.6 side-by side in a common directory called lib on the same level as test.dist.

    What am I missing / doing wrong?

    bug 
    opened by JorjMcKie 61
  • GSOC 2019: Make sure top 50 PyPI packages are supported

    GSOC 2019: Make sure top 50 PyPI packages are supported

    Nuitka works with most software. The aim of this project is to make sure it's true for the top 50 packages on PyPI, by compiling and using their example codes.

    In a first stage, you would identify and report the issues to the bug tracker, in a second stage develop tools that help to narrow down issues, e.g. what extension module fails to load precisely, even with a segfault happening, and put them to use and try to fix a few of the simpler issues.

    Setting up these as automated tests would be the ultimate goal, so we can follow these top 50 packages with Nuitka over time and make sure they continue to work.

    In the past it has happened e.g. that Jinja2 was breaking for Python3.7, and it would be cool to discover this immediately.

    Skills: Python programming, pip installation, Linux and/or Windows installs of Python, one is good, both would be great.

    enhancement z_historic_gsoc2019 
    opened by kayhayen 55
  • Issue about

    Issue about "Can't perform this operation for unregistered loader type"

    My script runs correctly and output what I wanted. But I packed my python scripts to exe file, it raised "Can't perform this operation for unregistered loader type" when I run the exe file.

    my nuitka command: python -m nuitka --mingw64 --standalone --show-progress --recurse-all --enable-plugin=numpy --include-package=openpyxl --include-package=markupsafe --include-package=jinja2 --follow-imports --output-dir=exeFiles mytest.py

    my envirionmnet: Windows 7 I installed all the packages(nuitka, jinja2,...) by pip

    python -m nuitka --version 0.6.8.4 Python: 3.6.8 (tags/v3.6.8:3c6b436a57, Dec 23 2018, 23:31:17) [MSC v.1916 32 bit (Intel)] Executable: C:\Program Files (x86)\Python36-32\python.exe OS: Windows Arch: x86

    my source code: #===============mytest.py=========================

    import pandas as pd
    import numpy as np
    
    
    
    def highlight(df, color = "yellow"):
    
        attr = 'background-color: {}'.format(color)
        df_bool = pd.DataFrame(df.apply(lambda x: [True if x.iloc[0] < v else False for v in x],axis=1).apply(pd.Series),
                          index=df.index)
        df_bool.columns =df.columns
        return pd.DataFrame(np.where(df_bool, attr, ""),
                           index= df.index, columns=df.columns)
                           
    if __name__ == "__main__":
        print('writing....')
        dict = {'A': [1, 1, 1, 1, 1], 'B':[2,1,2,1,2], 'C':[1,2,1,2,1]}
        df = pd.DataFrame(dict)
        try:
            df.style. \
                apply(highlight, axis=None).\
                to_excel("teststyled.xlsx", engine="openpyxl")
            print('Done!')
        except Exception as e:
            print('write error')
            print(str(e))
    
    bug 
    opened by jfzsl 50
  • [glfw] ImportError: Failed to load GLFW3 shared library

    [glfw] ImportError: Failed to load GLFW3 shared library

    After compiling and starting the excecutable via CMD, we get:

    Traceback (most recent call last):
      File "C:\Users\ADMINI~1\PYCHAR~1\NUITKA~1\APP~1.DIS\app.py", line 6, in <module>
      File "C:\Users\ADMINI~1\PYCHAR~1\NUITKA~1\APP~1.DIS\concur\__init__.py", line 17, in <module concur>
      File "C:\Users\ADMINI~1\PYCHAR~1\NUITKA~1\APP~1.DIS\concur\integrations\__init__.py", line 8, in <module concur.integrations>
      File "C:\Users\ADMINI~1\PYCHAR~1\NUITKA~1\APP~1.DIS\concur\integrations\glfw.py", line 4, in <module concur.integrations.glfw>
      File "C:\Users\ADMINI~1\PYCHAR~1\NUITKA~1\APP~1.DIS\glfw\__init__.py", line 43, in <module glfw>
    ImportError: Failed to load GLFW3 shared library.
    

    (The missing DLL is located in venv\glfw\glfw3.dll) If we manually copy it over and attempt to run again we get:

    Traceback (most recent call last):
      File "C:\Users\chwba\PYCHAR~1\NUITKA~1\APP~1.DIS\OpenGL\latebind.py", line 43, in __call__
    TypeError: 'NoneType' object is not callable
    
    During handling of the above exception, another exception occurred:
    
    Traceback (most recent call last):
      File "C:\Users\chwba\PYCHAR~1\NUITKA~1\APP~1.DIS\app.py", line 23, in <module>
      File "C:\Users\chwba\PYCHAR~1\NUITKA~1\APP~1.DIS\concur\integrations\glfw.py", line 118, in main
      File "C:\Users\chwba\PYCHAR~1\NUITKA~1\APP~1.DIS\imgui\integrations\glfw.py", line 13, in __init__
      File "C:\Users\chwba\PYCHAR~1\NUITKA~1\APP~1.DIS\imgui\integrations\opengl.py", line 91, in __init__
      File "C:\Users\chwba\PYCHAR~1\NUITKA~1\APP~1.DIS\imgui\integrations\opengl.py", line 23, in __init__
      File "C:\Users\chwba\PYCHAR~1\NUITKA~1\APP~1.DIS\imgui\integrations\opengl.py", line 115, in _create_device_objects
      File "C:\Users\chwba\PYCHAR~1\NUITKA~1\APP~1.DIS\OpenGL\latebind.py", line 47, in __call__
      File "C:\Users\chwba\PYCHAR~1\NUITKA~1\APP~1.DIS\OpenGL\wrapper.py", line 668, in wrapperCall
      File "C:\Users\chwba\PYCHAR~1\NUITKA~1\APP~1.DIS\OpenGL\wrapper.py", line 471, in calculate_cArgs
      File "C:\Users\chwba\PYCHAR~1\NUITKA~1\APP~1.DIS\OpenGL\converters.py", line 251, in __call__
      File "C:\Users\chwba\PYCHAR~1\NUITKA~1\APP~1.DIS\OpenGL\converters.py", line 196, in __call__
      File "C:\Users\chwba\PYCHAR~1\NUITKA~1\APP~1.DIS\OpenGL\arrays\arraydatatype.py", line 177, in zeros
      File "C:\Users\chwba\PYCHAR~1\NUITKA~1\APP~1.DIS\OpenGL\arrays\arraydatatype.py", line 84, in get_output_handler
    RuntimeError: ('Unable to find any output handler at all (not even ctypes/numpy ones!)', 'Failure in cConverter <OpenGL.converters.SizedOutputOrInput object at 0x00000000610C61C0>', (GL_TEXTURE_BINDING_2D, <object object at 0x0000000060774750>), 1, <OpenGL.platform.baseplatform.glGetIntegerv object at 0x00000000610C5C80>)
    

    Compiled with:

    python -m nuitka --follow-imports^
                     --show-progress^
                     --show-scons^
                     --assume-yes-for-downloads^
                     --warn-unusual-code^
                     --plugin-enable=pylint-warnings^
                     --plugin-enable=numpy^
                     --windows-dependency-tool=pefile^
                     --experimental=use_pefile_recurse^
                     --experimental=use_pefile_fullrecurse^
                     --standalone^
                     --mingw64^
                     app.py
    

    Log: https://nextcloud.mnwd.in/s/XN434Jf4Sz5yzwd

    Used requirements:

    concur
    concur-imgui >= 1.3.9
    glfw
    PyOpenGL
    imageio
    imageio-ffmpeg
    

    Code snippet to reproduce:

    import time
    from concurrent.futures import ThreadPoolExecutor
    
    import concur as c
    
    executor = ThreadPoolExecutor()
    
    
    def timer():
    	yield from c.orr([c.text(""), c.button("Start timer")])
    	yield
    	future = executor.submit(lambda: time.sleep(3))
    	yield from c.orr([c.text("waiting for 3s..."), c.button("Cancel"), c.Block(future)])
    
    
    def app():
    	return c.orr([c.forever(timer) for _ in range(3)])
    
    
    if __name__ == "__main__":
    	c.main(app(), "Timers")
    

    Nuitka

    0.6.11.3
    Python: 3.9.1 (tags/v3.9.1:1e5d33e, Dec  7 2020, 17:08:21) [MSC v.1927 64 bit (AMD64)]
    Executable: C:\Users\Administrator\.pyenv\pyenv-win\versions\3.9.1\python.exe
    OS: Windows
    Arch: x86_64
    

    Installed via python -m pip install nuitka

    Important note If we try to run this code on a machine which cannot render opengl and the above error would be fixed, we would get GLFWError: (65542) b'WGL: The driver does not appear to support OpenGL' warnings.warn(message, GLFWError) which is then considered 'normal'. To efficiently test it we need to run the compiled code on a machine which can render opengl (any > i5 processor or a proper GPU will suffice).

    bug 
    opened by sla-te 47
  • Cannot get cryptodome module to work

    Cannot get cryptodome module to work

    Hello Kay,

    I am trying to compile a program that uses pycryptodome module as standalone, and of course running into issues. I've tried with latest nuitka stable and latest rc so far, and tried to find some topics before opening the issue.

    • Nuitka version, full Python version and Platform (Windows, OSX, Linux ...) 0.6.2rc1 Python: 3.7.1 (v3.7.1:260ec2c36a, Oct 20 2018, 14:05:16) [MSC v.1915 32 bit (Intel)] Executable: C:\Python37-32\python.exe OS: Windows Arch: x86

    Using MingW64 8.1.0 with posix/dwarf

    • How did you install Nuitka and Python (pip, anaconda, deb, rpm, from source, what is a virtualenv ...), this is very important usually. Installed latest nuitka as site package with http://nuitka.net/releases/Nuitka-6.0.210.win32.py37.msi By the way, there seems to be a naming problem with those 6.0.210 develop packages which naming make them look elder than the 6.1.10 stable packages.

    • If possible please supply a Short, Self Contained, Correct, Example Example test.py file:

    from Crypto.Cipher import AES
    from Crypto.Random import get_random_bytes
    
    print(str(get_random_bytes(1337)))
    

    Compiled with

    python -m nuitka --exe --follow-imports --recurse-to=Crypto --standalone c:\PythonApps\test\test.py
    

    Compilation produces no errors.

    • If this is a regression (used to work in an earlier version of Nuitka), Didn't find anything about this.

    Anything I may have missed ? Best regards.

    bug 
    opened by deajan 46
  • GSOC 2019: Use this issue for other GSoC questions

    GSOC 2019: Use this issue for other GSoC questions

    First of all: Welcome to Nuitka!

    So this issue is intended for all questions of general nature. Introduce yourself here, ask about the process, or about available tools.

    This is absolutely open to all kinds of questions related to the GSoC, esp. if you are interested in creating your own idea too. Do not hesitate to ask, as long as you can live with us not answering immediately around the clock. :)

    z_historic_gsoc2019 
    opened by kayhayen 45
  • MacOS - ld: symbol(s) not found for architecture x86_64

    MacOS - ld: symbol(s) not found for architecture x86_64

    Hello,

    I'm trying to compile a simple Hello World module and I'm getting this error:

    ld: symbol(s) not found for architecture x86_64 clang: error: linker command failed with exit code 1 (use -v to see invocation) scons: *** [prueba.exe] Error 1

    I've tested it with both Python 3.7 and 3.6.6 with the same results. Compilation works, but there is a problem with the linker.

    I'm working with MacOS High Sierra and XCode is installed.

    What am I missing?

    wontfix 
    opened by pposca 42
  • Generating specialize C helper code automatically

    Generating specialize C helper code automatically

    This issue is about generating specialized code helpers automatically. A code helper is considered specialized, if it doesn't work on the most general form, e.g. two PyObject *, but instead e.g. on two PyUnicode_Object instead, where the task can be identified more immediately as PyUnicode_Concat, just concatenating the two strings.

    In Nuitka as a result of compile time analysis during optimization we have increasingly more shape knowledge. Often with relatively concrete types. A shape is a set of characteristics that the uses of a expose, e.g. could mean it's "iterable", "indexable", as example shapes. Many times however, shapes are known to be one or two types specifically, int, bool, unicode, etc. but often also one of the other, e.g. unicode or str.

    Currently in Nuitka there are helper functions used for code that does all sort of operations. There are very general ones, and then the more specialized ones. The general ones use OBJECT as a type indicator in the function names (where there is specialization). As of writing this, this is only the case for '+' and += (_INPLACE) are optimized for a subset of types and shapes. And there are comparison operations, <, <=, ==, !=, >=,>, but they are only specialized forINT`, i.e. Python2 integers.

    These helpers are hand optimized versions of more general code that represents what e.g. PyNumber_Add does. It looks at type slots and implements special behavior like the concatenation that + also does in Python. Many of their forms are not optimized, or not yet fully optimized. The current form is more of a demonstration of feasibility than a practical approach. This will only be sufficient to get few operations optimized, but it will never scale and doing this by hand is error prone and could lead to bugs.

    Instead, what we need is a code generator with sufficient type knowledge about the two arguments, so that it can make decisions automatically. For instance, having a nb_add is a given in for long, float, but not so for a list. So instead of making checks, and executing branched code, the code generator would not generate that check, or do so in a way that the C compiler will know it's not happening.

    Other examples are checks like type1 == type2. Many times, this comes down to branch decisions. And the different shapes allow some of these, some do not.

    enhancement help wanted 
    opened by kayhayen 41
  • Adding support for

    Adding support for "any" built-in

    What does this PR do?

    Enhancements

    • [x] Any builtin Node added
    • [x] Specialized optimizing algorithm
    • [x] Tests for Any built-in

    Fixes

    • [x] #349

    Why was it initiated? Any relevant Issues?

    • optimizes the any built-in from #232
    • fixes #349

    PR Checklist

    • [x] Correct base branch selected? develop for new features and bug fixes too. If it's part of a hotfix, it will be moved to master during it.
    • [x] All tests still pass. Check the developer manual about Running the Tests. There are Travis tests that cover the most important things however, and you are welcome to rely on those, but they might not cover enough.
    • [x] Ideally new features or fixed regressions ought to be covered via new tests.
    • [x] Ideally new or changed features have documentation updates.
    z_historic_gsoc2019 
    opened by bksahu 39
  • Can't compile tls-client

    Can't compile tls-client

    • Nuitka version 0.8.4 Commercial: None Python: 3.9.0rc2 (tags/v3.9.0rc2:2bd31b5, Sep 17 2020, 00:58:12) [MSC v.1927 64 bit (AMD64)] Flavor: Unknown Executable: C:\Users\xCrysti\AppData\Local\Programs\Python\Python39\python.exe OS: Windows Arch: x86_64 WindowsRelease: 10

    • The specific PyPI names and versions tls-client==0.1.8

    main.py: import tls_client session = tls_client.Session( client_identifier="chrome_108" ) res = session.get( "https://httpbin.org/get", headers={ "trixtm": "true", } ) print(res.text) input() nuitka command: nuitka --standalone --follow-imports --include-data-files=C:/Users/xCrysti/AppData/Local/Programs/Python/Python39/Lib/site-packages/tls_client/dependencies/tls-client-64.dll=tls-client-64.dll main.py

    And i get this error: FileNotFoundError: Could not find module 'C:\Users\xCrysti\Desktop\TLSTES~1\MAIN~1.DIS\tls_client\dependencies\tls-client-64.dll' (or one of its dependencies). Try using the full path with constructor syntax. So i include the dll but it still can't find it

    needs_example 
    opened by TrixTM 9
  • ModuleNotFoundError: No module named 'huggingface_hub.hf_api'

    ModuleNotFoundError: No module named 'huggingface_hub.hf_api'

    • Nuitka version, full Python version, flavor, OS, etc. as output by this command.

    $ python -m nuitka --version 1.3.5 Commercial: None Python: 3.10.7 (tags/v3.10.7:6cc6b13, Sep 5 2022, 14:08:36) [MSC v.1933 64 bit (AMD64)] Flavor: Unknown Executable: E:\PythonProjects\GPT3_Assistant\.env\Scripts\python.exe OS: Windows Arch: x86_64 WindowsRelease: 7 Version C compiler: D:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.27.29110\bin\HostX64\x64\cl.exe (cl 14.2).

    • How did you install Nuitka and Python

      Nuitka installed via pip in venv. ordered-sets and dependency walker installed.

    • The specific PyPI names and versions

    $ python -m pip freeze
    altgraph==0.17.3
    certifi==2022.12.7
    charset-normalizer==2.1.1
    colorama==0.4.6
    et-xmlfile==1.1.0
    filelock==3.9.0
    future==0.18.2
    huggingface-hub==0.11.1
    idna==3.4
    Nuitka==1.4rc6
    numpy==1.24.1
    openai==0.25.0
    openpyxl==3.0.10
    ordered-set==4.1.0
    packaging==22.0
    pandas==1.5.2
    pandas-stubs==1.5.2.221213
    pefile==2022.5.30
    pygubu==0.28
    pyinstaller==5.7.0
    pyinstaller-hooks-contrib==2022.14
    python-dateutil==2.8.2
    pytz==2022.7
    pywin32-ctypes==0.2.0
    PyYAML==6.0
    regex==2022.10.31
    requests==2.28.1
    six==1.16.0
    tokenizers==0.13.2
    tqdm==4.64.1
    transformers==4.25.1
    types-pytz==2022.7.0.0
    typing_extensions==4.4.0
    urllib3==1.26.13

    • Many times when you get an error from Nuitka, your setup may be special

      Project compiles and starts successfully without huggingface transformers.

    • Also supply a Short, Self Contained, Correct, Example

    #!/usr/bin/python3
    # nuitka-project: --include-package=tqdm
    # nuitka-project: --include-package=regex
    # nuitka-project: --include-package=requests
    # nuitka-project: --include-package=packaging
    # nuitka-project: --include-package=filelock
    # nuitka-project: --include-package=numpy
    # nuitka-project: --include-package=tokenizers
    
    from transformers import GPT2TokenizerFast
    
    if __name__ == "__main__":
        print('hello')
    
    • Provide in your issue the Nuitka options used
    Project settings:
    # nuitka-project: --include-data-files={MAIN_DIRECTORY}/editor.ui=editor.ui
    # nuitka-project: --enable-plugin=tk-inter
    # nuitka-project: --include-package=pygubu
    # nuitka-project: --include-package=tqdm
    # nuitka-project: --include-package=regex
    # nuitka-project: --include-package=requests
    # nuitka-project: --include-package=packaging
    # nuitka-project: --include-package=filelock
    # nuitka-project: --include-package=numpy
    # nuitka-project: --include-package=tokenizers
    

    In my venv terminal:

    python -m nuitka --follow-imports --show-progress --show-modules --standalone init.py

    At first, before adding all --include-package but pygubu I ended up with an error:

    importlib.metadata.PackageNotFoundError: No package metadata was found for tqdm

    I got the same error in pyinstaller, so ended up adding one more package after each failed compilation (or freezing? whatever the correct name is). pyinstaller works 10x faster so it was more feasible to test it step by step than in Nuitka. At this moment I don't know better way to find hidden imports.

    The point is, pyinstaller works with this configuration but the program made with Nuitka returned: Traceback (most recent call last):

    Traceback (most recent call last):
      File "E:\PYTHON~1\GPT3_A~1\__INIT~1.DIS\__init__.py", line 31, in <module>
      File "E:\PYTHON~1\GPT3_A~1\__INIT~1.DIS\transformers\__init__.py", line 30, in <module transformers>
      File "E:\PYTHON~1\GPT3_A~1\__INIT~1.DIS\transformers\dependency_versions_check.py", line 17, in <module transformers.dependency_versions_check>
      File "E:\PYTHON~1\GPT3_A~1\__INIT~1.DIS\transformers\utils\__init__.py", line 59, in <module transformers.utils>
      File "E:\PYTHON~1\GPT3_A~1\__INIT~1.DIS\transformers\utils\hub.py", line 32, in <module transformers.utils.hub>
      File "E:\PYTHON~1\GPT3_A~1\__INIT~1.DIS\huggingface_hub\__init__.py", line 254, in __getattr__
      File "importlib.py", line 126, in import_module
    ModuleNotFoundError: No module named 'huggingface_hub.hf_api'
    
    opened by StyriamMZ 0
  • something wrong with support for gevent

    something wrong with support for gevent

    error:

    Traceback (most recent call last):
      File "C:\Users\admin\code\out\TEST~1.DIS\test.py", line 3, in <module>
      File "C:\Users\admin\code\out\TEST~1.DIS\gevent\monkey.py", line 1285, in patch_all
      File "C:\Users\admin\code\out\TEST~1.DIS\gevent\monkey.py", line 200, in ignores
      File "C:\Users\admin\code\out\TEST~1.DIS\gevent\monkey.py", line 1165, in patch_signal
      File "C:\Users\admin\code\out\TEST~1.DIS\gevent\monkey.py", line 462, in _patch_module
      File "C:\Users\admin\code\out\TEST~1.DIS\gevent\monkey.py", line 448, in _check_availability
    ModuleNotFoundError: No module named 'gevent.signal'
    

    code: test.py

    import gevent
    from gevent import monkey
    monkey.patch_all(thread=False)
    

    compile

    python -m nuitka --standalone  --output-dir=out test.py
    

    env:

    windows 10/64bit
    python 3.9.15
    nuitka 1.3.5
    gevent 22.10.2
    

    #1933

    opened by arthurchan1906 1
  • ModuleNotFoundError: No module named 'requests.packages.urllib3.contrib.appengine'

    ModuleNotFoundError: No module named 'requests.packages.urllib3.contrib.appengine'

    version --

    1.3.5 Commercial: None Python: 3.9.1 (tags/v3.9.1:1e5d33e, Dec 7 2020, 17:08:21) [MSC v.1927 64 bit (AMD64)] Flavor: CPython Official Executable: C:\Python39\python.exe OS: Windows Arch: x86_64 WindowsRelease: 10 Version C compiler: C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.29.30133\bin\HostX64\x64\cl.exe (cl 14.2).

    pip

    > 2captcha-python==1.1.0
    > absl-py==0.12.0
    > account==0.1.0
    > action==1.4.4
    > adder==1.0
    > aiohttp==3.7.4.post0
    > airtest==1.2.6
    > alembic==1.7.7
    > altgraph==0.17
    > amazoncaptcha==0.5.6
    > amqp==2.6.1
    > androidemu==0.0.2
    > ansicon==1.89.0
    > anticaptchaofficial==1.0.31
    > anyio==3.6.1
    > appdirs==1.4.4
    > Appium-Python-Client==2.6.0
    > apptools==5.2.0
    > APScheduler==3.6.3
    > asgiref==3.5.2
    > astunparse==1.6.3
    > async-generator==1.10
    > async-timeout==4.0.2
    > atomicwrites==1.4.0
    > attmap==0.13.0
    > attrs==21.2.0
    > AuthGG==0.9
    > Babel==2.10.1
    > backcall==0.2.0
    > bcrypt==3.2.2
    > bearer==3.1.0
    > beautifulsoup4==4.9.3
    > billiard==3.6.4.0
    > blackboxprotobuf==1.0.1
    > blessed==1.19.1
    > blessings==1.7
    > blinker==1.4
    > blowfish==0.6.1
    > Brotli==1.0.9
    > browser-cookie3==0.12.1
    > bs4==0.0.1
    > CacheControl==0.12.11
    > cached-property==1.5.2
    > cachetools==4.2.2
    > captchatools==1.2.1
    > celery==4.4.7
    > certifi==2022.9.24
    > cffi==1.14.6
    > chardet==3.0.4
    > charset-normalizer==2.0.4
    > chromedriver==2.24.1
    > cli-helpers==2.2.1
    > click==7.1.2
    > click-help-colors==0.9.1
    > clickable==1.5
    > clicking==0.2.0
    > colorama==0.4.4
    > coloredlogs==15.0.1
    > comtypes==1.1.10
    > configobj==5.0.6
    > crypto==1.4.1
    > cryptography==36.0.2
    > csrf==0.1b1
    > cssselect==1.1.0
    > cssutils==2.4.0
    > cycler==0.11.0
    > Cython==0.29.32
    > cytoolz==0.11.0
    > dateparser==1.1.0
    > DateTime==4.3
    > decompyle3==3.9.0
    > decorator==5.0.9
    > defusedxml==0.7.1
    > delegator.py==0.1.1
    > Deprecated==1.2.12
    > discord-webhook==0.14.0
    > divvy==0.5.0
    > Django==4.1.4
    > dnspython==2.2.1
    > docopt==0.6.2
    > duration==1.1.1
    > easygui==0.98.2
    > ecdsa==0.17.0
    > eido==0.1.5
    > email-validator==1.2.1
    > emails==0.5.15
    > envisage==6.1.0
    > eth-hash==0.3.1
    > eth-typing==2.2.2
    > eth-utils==1.10.0
    > expand==0.2.2
    > facebook-wda==1.4.3
    > fake-useragent==0.1.11
    > Faker==13.7.0
    > fastapi==0.70.1
    > ffmpeg==1.4
    > filelock==3.3.2
    > firebase==3.0.1
    > firebase-admin==6.0.1
    > Flask==2.0.1
    > Flask-Cors==3.0.10
    > flatbuffers==1.12
    > fonttools==4.33.3
    > free-proxy==1.0.3
    > frida==15.1.17
    > frida-tools==10.6.1
    > ftfy==6.1.1
    > future==0.18.2
    > gast==0.4.0
    > gcloud==0.17.0
    > gdown==4.6.0
    > geographiclib==1.52
    > geopy==2.2.0
    > getmac==0.8.3
    > google==3.0.0
    > google-api-core==2.11.0
    > google-api-python-client==2.68.0
    > google-auth==2.15.0
    > google-auth-httplib2==0.1.0
    > google-auth-oauthlib==0.4.4
    > google-cloud-core==2.3.2
    > google-cloud-firestore==2.7.2
    > google-cloud-storage==2.7.0
    > google-crc32c==1.5.0
    > google-pasta==0.2.0
    > google-resumable-media==2.4.0
    > googleapis-common-protos==1.57.0
    > greenlet==1.1.2
    > grpcio==1.51.1
    > grpcio-status==1.51.1
    > gspread==4.0.1
    > gunicorn==20.1.0
    > h11==0.12.0
    > h2==4.0.0
    > h5py==3.1.0
    > hexdump==3.3
    > hpack==4.0.0
    > hrpc==1.0.9
    > httpcore==0.15.0
    > httplib2==0.19.1
    > httpx==0.23.0
    > humanfriendly==10.0
    > hyperframe==6.0.1
    > idna==2.10
    > ifaddr==0.1.7
    > imap-tools==0.45.0
    > importlib-metadata==4.6.4
    > inflect==5.6.0
    > inquirer==2.10.0
    > install==1.3.4
    > ipython==7.29.0
    > iso8601==1.0.2
    > itsdangerous==2.0.1
    > jedi==0.18.0
    > jeepney==0.7.1
    > Jinja2==2.11.3
    > jinxed==1.2.0
    > joblib==1.1.0
    > Js2Py==0.71
    > jsonschema==4.0.1
    > jws==0.1.3
    > kaitaistruct==0.9
    > keras-nightly==2.5.0.dev2021032900
    > Keras-Preprocessing==1.1.2
    > keyring==23.1.0
    > keystone-engine==0.9.2
    > kiwisolver==1.4.2
    > kombu==4.6.11
    > latest-user-agents==0.0.3
    > ldap3==2.9.1
    > libpython==0.2
    > librespot==0.0.7
    > litecli==1.8.0
    > loading==0.0.1
    > logmuse==0.2.7
    > logomaker==0.8
    > looper==1.3.1
    > lxml==4.9.2
    > lz4==3.1.3
    > m3u8==2.0.0
    > Mako==1.2.0
    > Markdown==3.3.4
    > MarkupSafe==2.0.1
    > matplotlib==3.5.2
    > matplotlib-inline==0.1.3
    > maturin==0.13.2
    > mayavi==4.8.0
    > mitmproxy==7.0.2
    > more-itertools==8.13.0
    > MouseInfo==0.1.3
    > msgpack==1.0.2
    > mss==6.1.0
    > multidict==5.1.0
    > mutagen==1.44.0
    > Naked==0.1.31
    > names==0.3.0
    > Nuitka==1.3.5
    > num2words==0.5.10
    > number2words==0.1.3
    > numpy==1.19.3
    > oauth2client==3.0.0
    > oauthlib==3.1.0
    > objection==1.11.0
    > opencv-contrib-python==4.6.0.66
    > opencv-python==4.5.5.64
    > opt-einsum==3.3.0
    > orderedset==2.0.3
    > outcome==1.1.0
    > oyaml==1.0
    > packaging==21.3
    > pandas==1.3.3
    > parso==0.8.2
    > passlib==1.7.4
    > password==0.2
    > pbkdf2==1.3
    > Pebble==4.6.3
    > pefile==2022.5.30
    > peppercorn==0.6
    > peppy==0.31.1
    > perform==0.0.8
    > pexpect==4.8.0
    > pickleshare==0.7.5
    > Pillow==9.0.1
    > pipwin==0.5.2
    > platformdirs==2.5.2
    > pluggy==0.13.1
    > plyer==2.0.0
    > pocoui==1.0.87
    > premailer==3.10.0
    > prices==1.1.0
    > problem==0.2.3
    > prompt-toolkit==3.0.30
    > proto-plus==1.22.1
    > protobuf==3.20.1
    > proxy==0.0.1
    > proxy-extractor==0.1.1
    > Proxy-List-Scrapper==0.2.2
    > psutil==5.8.0
    > psycopg2-binary==2.9.3
    > ptyprocess==0.7.0
    > publicsuffix2==2.20191221
    > pubproxpy==2.0.2
    > py==1.10.0
    > pyaes==1.6.1
    > pyarmor==7.6.1
    > pyarmor-webui==1.3.3
    > pyasn1==0.4.8
    > pyasn1-modules==0.2.8
    > PyAutoGUI==0.9.53
    > pycairo==1.21.0
    > pychrome==0.2.3
    > pycparser==2.20
    > pycryptodome==3.15.0
    > pycryptodomex==3.15.0
    > pycuda @ file:///C:/Users/A.T/pipwin/pycuda-2022.1%2Bcuda116-cp39-cp39-win_amd64.whl
    > pydantic==1.9.1
    > pydivert==2.1.0
    > pyelftools==0.27
    > pyface==7.4.2
    > pyffmpeg==2.3
    > pyfiglet==0.8.post1
    > PyGetWindow==0.0.9
    > Pygments==2.10.0
    > pyheartradio==0.0.1a1
    > pyimport==0.5.6
    > PyInquirer==1.0.3
    > pyinstaller==4.6
    > pyinstaller-hooks-contrib==2022.8
    > pyjsparser==2.7.1
    > PyJWT==2.6.0
    > pymongo==4.2.0
    > PyMsgBox==1.0.9
    > PyOgg==0.6.14a1
    > pyOpenSSL==22.0.0
    > pyparsing==2.4.7
    > pyperclip==1.8.2
    > PyPrind==2.11.3
    > PyQt5==5.15.7
    > PyQt5-Qt5==5.15.2
    > PyQt5-sip==12.11.0
    > PyQt5-stubs==5.15.6.0
    > pyreadline3==3.4.1
    > Pyrebase==3.0.27
    > PyRect==0.2.0
    > pyrsistent==0.18.0
    > PyScreeze==0.1.28
    > PySide6==6.3.1
    > PySide6-Addons==6.3.1
    > PySide6-Essentials==6.3.1
    > pySmartDL==1.3.4
    > PySocks==1.7.1
    > PySoundCloud==2020.6.2
    > pyTelegramBotAPI==4.2.0
    > pytesseract==0.3.9
    > pytest==5.4.3
    > python-anticaptcha==1.0.0
    > python-binance==1.0.15
    > python-dateutil==2.8.2
    > python-editor==1.0.4
    > python-ffmpeg-video-streaming==0.1.15
    > python-generate-mac==1.3.1
    > python-jose==3.3.0
    > python-jwt==2.0.1
    > python-multipart==0.0.5
    > python-telegram-bot==13.8.1
    > python-vlc==2.2.6100
    > pytools==2022.1.12
    > pytweening==1.0.4
    > pytz==2021.1
    > pytz-deprecation-shim==0.1.0.post0
    > pywin32==303
    > pywin32-ctypes==0.2.0
    > pywinauto==0.6.3
    > PyYAML==5.4.1
    > pyzipcode==3.0.1
    > random-password-generator==2.2.0
    > random-user-agent==1.0.1
    > raven==6.10.0
    > readchar==4.0.2
    > regex==2021.11.10
    > report==0.0.1
    > requests==2.28.1
    > requests-oauthlib==1.3.0
    > requests-toolbelt==0.7.0
    > retry==0.9.2
    > retrying==1.3.3
    > rfc3986==1.5.0
    > rlp==1.2.0
    > rsa==4.7.2
    > ruamel.yaml==0.17.10
    > ruamel.yaml.clib==0.2.6
    > scikit-learn==1.0.1
    > scipy==1.7.2
    > SecretStorage==3.3.1
    > selenium==4.5.0
    > selenium-stealth==1.0.6
    > selenium-wire==4.4.1
    > self==2020.12.3
    > semver==2.13.0
    > shellescape==3.8.1
    > shiboken6==6.3.1
    > six==1.16.0
    > sklearn==0.0
    > sniffio==1.2.0
    > SocksipyChain==2.1.2.post1
    > sockslib==1.7.3
    > software==0.0.0
    > Solution==5.4.0
    > sortedcontainers==2.4.0
    > soundground==0.1
    > soupsieve==2.2.1
    > spark-parser==1.8.9
    > spotipy==2.19.0
    > SQLAlchemy==1.4.36
    > sqlparse==0.4.2
    > starlette==0.16.0
    > tabulate==0.8.9
    > tenacity==6.3.1
    > tensorboard==2.4.1
    > tensorboard-plugin-wit==1.8.0
    > tensorflow==2.5.0rc1
    > tensorflow-estimator==2.5.0rc0
    > tensorflow-gpu==2.5.0rc1
    > termcolor==1.1.0
    > text-unidecode==1.3
    > threadpoolctl==3.0.0
    > tidalapi==0.6.10
    > tomli==2.0.1
    > toolz==0.11.1
    > tornado==6.1
    > tqdm==4.62.3
    > traitlets==5.1.1
    > traits==6.4.1
    > traitsui==7.4.0
    > trie==1.4.0
    > trio==0.19.0
    > trio-websocket==0.9.2
    > typing_extensions==4.3.0
    > tzdata==2021.5
    > tzlocal==4.1
    > ubiquerg==0.6.1
    > ujson==4.2.0
    > uncompyle6==3.7.4
    > undetected-chromedriver==3.1.5.post4
    > unicorn==1.0.3
    > unpy2exe==0.3
    > unpyclib==0.8.1
    > until==0.1.2
    > uritemplate==4.1.1
    > urllib3==1.26.13
    > urwid==2.1.2
    > utils==1.0.1
    > uvicorn==0.11.8
    > val==0.6
    > vboxapi==1.0
    > vine==1.3.0
    > vtk==9.1.0
    > wcwidth==0.2.5
    > webapp2==2.5.2
    > websocket-client==1.4.2
    > websockets==8.1
    > Werkzeug==2.0.1
    > windows-curses==2.3.0
    > WMI==1.5.1
    > wordcloud==1.8.2.2
    > wrapt==1.12.1
    > wslink==1.7.0
    > wsproto==1.0.0
    > xdis==6.0.4
    > xmltodict==0.13.0
    > yacman==0.8.3
    > yarl==1.6.3
    > youtube-dl==2021.12.17
    > ytmusicapi==0.21.0
    > zeroconf==0.39.4
    > zipp==3.5.0
    > zope.interface==5.4.0
    > zstandard==0.15.2
    

    after compiling with this command

    python -m nuitka --follow-imports main.py --standalone --include-package=requests --include-data-files=logo2.txt=. --windows-icon-from-ico=C:\aa.ico --enable-console

    and running exe

    PS D:\Newfolder\PROJECT\compile> ./main.exe
    Traceback (most recent call last):
      File "C:\Users\A.T\AppData\Local\Temp\ONEFIL~1\constanttemp.py", line 7, in <module constanttemp>
        import pyrebase
      File "C:\Users\A.T\AppData\Local\Temp\ONEFIL~1\pyrebase\__init__.py", line 1, in <module pyrebase>
      File "C:\Users\A.T\AppData\Local\Temp\ONEFIL~1\pyrebase\pyrebase.py", line 19, in <module pyrebase.pyrebase>
    ModuleNotFoundError: No module named 'requests.packages.urllib3.contrib.appengine'
    
    bug factory 
    opened by Pranavprana 5
  • Plugin warnings causes tests to fail

    Plugin warnings causes tests to fail

    I am packaging Nuitka 1.3.5 (last version packaged was 1.0.8) and finding that plugin warnings during the tests are causing the test runner to fail.

    i.e.

    $ CC=clang $python ./tests/run-tests --skip-basic-tests --skip-syntax-tests --skip-program-tests --skip-package-tests --skip-plugins-tests --skip-reflection-test --no-other-python
    

    causes

    [  216s] Consider output of standalone mode compiled program: GlfwUsing.py
    [  216s] Comparing output of 'GlfwUsing.py' using '/usr/bin/python3.10' with flags silent, expect_success, --standalone, remove_output, cpython_cache, timing, --nowarn-mnemonic=debian-dist-packages, plugin_enable:numpy ...
    [  373s] --- /usr/bin/python3.10 (stderr)
    [  373s] +++ nuitka (stderr)
    [  373s] 
    [  373s] @@ -0,0 +1 @@
    [  373s] 
    [  373s] +Nuitka-Plugins:WARNING: numpy: This plugin has been deprecated, do not enable it anymore.
    [  373s] Updating CPython cache by force due to non-matching comparison results.
    [  374s] --- /usr/bin/python3.10 (stderr)
    [  374s] +++ nuitka (stderr)
    [  374s] 
    [  374s] @@ -0,0 +1 @@
    [  374s] 
    [  374s] +Nuitka-Plugins:WARNING: numpy: This plugin has been deprecated, do not enable it anymore.
    [  374s] Error, results differed (stderr).
    ...
    

    and

    [  621s] Comparing output of 'PyQt5Plugins.py' using '/usr/bin/python3.10' with flags silent, expect_success, --standalone, remove_output, cpython_cache, timing, --nowarn-mnemonic=debian-dist-packages ...
    [  649s] --- /usr/bin/python3.10 (stderr)
    [  649s] +++ nuitka (stderr)
    [  649s] 
    [  649s] @@ -0,0 +1,2 @@
    [  649s] 
    [  649s] +Nuitka-Plugins:WARNING: pyqt5: For the obsolete PyQt5 the Nuitka support is incomplete. Threading, callbacks to compiled functions, etc. may not be working.
    [  649s] +Nuitka-Plugins:WARNING:     Complex topic! More information can be found at https://nuitka.net/info/pyqt5.html
    [  649s] Updating CPython cache by force due to non-matching comparison results.
    [  649s] --- /usr/bin/python3.10 (stderr)
    [  649s] +++ nuitka (stderr)
    [  649s] 
    [  649s] @@ -0,0 +1,2 @@
    [  649s] 
    [  649s] +Nuitka-Plugins:WARNING: pyqt5: For the obsolete PyQt5 the Nuitka support is incomplete. Threading, callbacks to compiled functions, etc. may not be working.
    [  649s] +Nuitka-Plugins:WARNING:     Complex topic! More information can be found at https://nuitka.net/info/pyqt5.html
    [  649s] Error, results differed (stderr).
    ...
    
    opened by jayvdb 2
  • Webpage displayed incorrectly when running pywebview[cef]

    Webpage displayed incorrectly when running pywebview[cef]

    Nuitka version

    1.4rc5
    Commercial: None
    Python: 3.8.8 (tags/v3.8.8:024d805, Feb 19 2021, 13:18:16) [MSC v.1928 64 bit (AMD64)]
    Flavor: CPython Official
    Executable: C:\Soultion-Applications\test\.venv\Scripts\python.exe
    OS: Windows
    Arch: x86_64
    WindowsRelease: 10
    Version C compiler: C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.29.30037\bin\HostX64\x64\cl.exe (cl 14.2).
    

    Packages

    cefpython3==66.1
    cffi==1.15.1
    click==8.1.3
    clr-loader==0.2.4
    colorama==0.4.6
    Flask==2.2.2
    importlib-metadata==5.2.0
    itsdangerous==2.1.2
    Jinja2==3.1.2
    MarkupSafe==2.1.1
    Nuitka==1.4rc5
    orderedset==2.0.3
    proxy-tools==0.1.0
    pycparser==2.21
    pythonnet==3.0.1
    pywebview==3.7.2
    Werkzeug==2.2.2
    zipp==3.11.0
    
    pip install pywebview[cef]
    pip install flask
    

    Webview Code

    import webview
    
    webview.create_window(
        "test webview", "http://127.0.0.1:5000/", text_select=True, min_size=(1200, 800)
    )
    
    webview.start(gui="cef")
    

    Flask App Code

    from flask import Flask
    
    app = Flask(__name__)
    
    
    @app.route('/')
    def hello():
        return 'Hello, World!'*1000
    
    if __name__ == '__main__':
        app.run()
    

    Error

    Works fine when using python webview_test.py. Opens a small empty webview window with the webpage displayed as expected. The following is how it is displayed when using the exe generated.

    image

    Looks like a window inside a window or something. Clicks and other interactions work on the location it should but the display is in wrong size.

    opened by thesct22 0
Owner
Nuitka Organization
Nuitka organization for shared maintenance of Nuitka
Nuitka Organization
executable archive format

XAR XAR lets you package many files into a single self-contained executable file. This makes it easy to distribute and install. A .xar file is a read-

Facebook Incubator 1.5k Dec 29, 2022
shiv is a command line utility for building fully self contained Python zipapps as outlined in PEP 441, but with all their dependencies included.

shiv shiv is a command line utility for building fully self-contained Python zipapps as outlined in PEP 441, but with all their dependencies included!

LinkedIn 1.5k Dec 28, 2022
A distutils extension to create standalone Windows programs from Python code

py2exe for Python 3 py2exe is a distutils extension which allows to build standalone Windows executable programs (32-bit and 64-bit) from Python scrip

py2exe 526 Jan 4, 2023
The Application can convert the .py file into exe for faster transformation and can result to build an app in a single click

PEXEY PEXEY Is a high robust py to exe app made top on pyinstaller this application is for the developer who constantly keep making py to exe apps IMP

Aaris Kazi 11 Dec 15, 2022
py2app is a Python setuptools command which will allow you to make standalone Mac OS X application bundles and plugins from Python scripts.

py2app is a Python setuptools command which will allow you to make standalone Mac OS X application bundles and plugins from Python scripts. py2app is

Ronald Oussoren 222 Dec 30, 2022
Psgcompiler A PySimpleGUI Application - Transform your Python programs in Windows, Mac, and Linux binary executables

psgcompiler A PySimpleGUI Application "Compile" your Python programs into an EXE for Windows, an APP for Mac, and a binary for Linux Installation Old-

PySimpleGUI 77 Jan 7, 2023
Auto locust load test config and worker distribution with Docker and GitHub Action

Auto locust load test config and worker distribution with Docker and GitHub Action Install Fork the repo and change the visibility option to private S

Márk Zsibók 1 Nov 24, 2021
Create standalone executables from Python scripts, with the same performance and is cross-platform.

About cx_Freeze cx_Freeze creates standalone executables from Python scripts, with the same performance, is cross-platform and should work on any plat

Marcelo Duarte 1k Jan 4, 2023
A modern Python application packaging and distribution tool

PyOxidizer PyOxidizer is a utility for producing binaries that embed Python. The over-arching goal of PyOxidizer is to make complex packaging and dist

Gregory Szorc 4.5k Jan 7, 2023
FreezeUI is a python package that creates applications using cx_freeze and GUI by converting .py to .exe .

FreezeUI is a python package use to create cx_Freeze setup files and run them to create applications and msi from python scripts (converts .py to .exe or .msi .

null 4 Aug 25, 2022
A library which implements low-level functions that relate to packaging and distribution of Python

What is it? Distlib is a library which implements low-level functions that relate to packaging and distribution of Python software. It is intended to

Python Packaging Authority 29 Oct 11, 2022
Anaconda is the OS installer used by Fedora, RHEL, CentOS and other Linux distributions.

Anaconda is the OS installer used by Fedora, RHEL, CentOS and other Linux distributions. Documentation Documentation for the Anaconda install

Red Hat Installer Engineering Team 454 Jan 8, 2023
Python virtualenvs in Debian packages

dh-virtualenv Contents Overview Presentations, Blogs & Other Resources Using dh-virtualenv How does it work? Running tests Building the package in a D

Spotify 1.5k Dec 16, 2022
A tool used to obfuscate python scripts, bind obfuscated scripts to fixed machine or expire obfuscated scripts.

PyArmor Homepage (中文版网站) Documentation(中文版) PyArmor is a command line tool used to obfuscate python scripts, bind obfuscated scripts to fixed machine

Dashingsoft 1.9k Jan 1, 2023
Freeze (package) Python programs into stand-alone executables

PyInstaller Overview PyInstaller bundles a Python application and all its dependencies into a single package. The user can run the packaged app withou

PyInstaller 9.9k Jan 8, 2023
Core utilities for Python packages

packaging Reusable core utilities for various Python Packaging interoperability specifications. This library provides utilities that implement the int

Python Packaging Authority 451 Jan 4, 2023
Build Windows installers for Python applications

Pynsist is a tool to build Windows installers for your Python applications. The installers bundle Python itself, so you can distribute your applicatio

Thomas Kluyver 818 Jan 5, 2023
Subpar is a utility for creating self-contained python executables. It is designed to work well with Bazel.

Subpar Subpar is a utility for creating self-contained python executables. It is designed to work well with Bazel. Status Subpar is currently owned by

Google 550 Dec 27, 2022
Python Wheel Obfuscator

pywhlobf obfuscates your wheel distribution by compiling python source file to shared library.

Hunt Zhan 79 Dec 22, 2022