Build Windows installers for Python applications

I created an installer using the code from https://github.com/siecje/qml-testing/tree/pynsist with the bitness=32 in installer.cfg

I installed it on the IE8 on Win7 machine from here

https://developer.microsoft.com/en-us/microsoft-edge/tools/vms/

When I launch the shortcut nothing happens. If I run it from Python

> Python\pythonw.exe QMLApplication.launch.pyw

In %appdata%\QMLApplication.launch.pyw

there is

Traceback (most recent call last):
  File "QMLApplication.launch.pyw", line 30, in <module>
    from qmlapp import Main
  File "pkgs\qmlapp\__init__.py", line 1, in <module>
    from .main import Main
  File "pkgs\qmlapp\main.py", line 5, in <module>
    from PyQt5 import QtCore, QtGui, QtQml
ImportError: DLL load failed: The specified module could not be found.

Python 3.5 will have a rebuilt Python installer, and it seems likely that we will need to pass some slightly different options to it. The details aren't finalised yet, I'll get back to this when they are.

http://stevedower.id.au/blog/the-python-3-5-installer/

So I continue to build my installer for certbot, and I would like to expose a certbot command on Windows command line.

The relevant part of installer.cfg is the following:

[Command certbot]
entry_point=certbot.main:main

Installer is built correctly, installation behaves correctly. However, from one build + install to another, sometimes the certbot command fails. And when it fails, it just returns failed to create process. Nothing more, and I did not find any way to trigger some debug mode and provide more logs.

So I understood that the certbot.exe executable, which is copied from the NSIS installer into %installdir%\bin\, is available to the user through an injection in PATH. Direct running of certbot.exe leads to the same inconsistencies.

I also understood that certbot.exe is in fact a copy of cli-{32|64}.exe from the project https://github.com/takluyver/win_cli_launchers, that is itself a 3 years old extract of the cli-{32|64}.exe from the well-known setuptools project. Tried to copy the executables from a fresh and recent copy of setuptools, same inconsistencies.

Finally, I understood that cli-{32|64}.exe is a VisualStudio binary that reads a python script in the same folder, with the name of type cli-{32|64}-script.py (or something-script.py if the executable is renamed to something.exe). It will then resolve the Python executable path from the environment or an optional shebang available at the beginning of the python script file (for instance #!C:\some\path\to\python).

I could not succeed in repairing this behavior from the existing executables files provided by setuptools or win_cli_launchers launchers. However, I succeeded in reproducing the expected behavior with a batch file. This one:

@echo off
setlocal ENABLEDELAYEDEXPANSION

set env_dir=%~dp0
set install_dir=%env_dir:\bin\=%
set python_path=%install_dir%\Python\python.exe
set script_path=%install_dir%\bin\%~n0-script.py
set error=false

if not exist "%python_path%" 2> nul (
    echo Error, python executable not found on path: %python_path%
    set error=true
)

if not exist "%script_path%" 2> nul (
    echo Error, python script file not found on path: %script_path%
    set error=true
)

if %error%==true (
    exit /b 1
)

"%python_path%" "%script_path%" %*

This batch file, assuming to be copied in the bin directory constructed by pynsist, and assuming to be named something.bat, will execute the python script file something-script.py, using the Python distribution prepared by pynsist, and passing all command lines arguments to the python script file.

It is everything needed, when the bin directory is in the user PATH, to have a working something command available in CLI. And this works everytime.

One question and one proposition

1. Question

Is there an explanation or a process to find it, about the inconsistencies I noticed with cli.exe ?

2. Proposition

We need to consider also that cli.exe comes from the easy_install approach of setuptools, but the setuptools maintainers advise to not use easy_install anymore, because there are better alternatives. One of them is that starting with Python 3.3 for Windows, the py launcher can start scripts with a correct interpretation of the shebang to find the correct Python environment.

But anyway the cli.exe executable here is used only to expose CLI commands. And because the Python environment is strictly controlled and isolated using a NSIS installer, one can have the expected behavior with a simple batch file as I shown before.

So why not replacing the cli.exe executable by a cli.bat batch script quite similar to what I wrote ?

I just tried to create an installer, but got the following error message:

Traceback (most recent call last):
  File "C:\Dropbox\Development\Pawlabeling\build\nsis\Pawlabeling.launch.py", line 21, in <module>
    from pawlabeling.widgets.mainwindow import main
  File "C:\Dropbox\Development\Pawlabeling\build\nsis\pkgs\pawlabeling\widgets\mainwindow.py", line 14, in <module>
    from ..models import model
  File "C:\Dropbox\Development\Pawlabeling\build\nsis\pkgs\pawlabeling\models\model.py", line 7, in <module>
    from ..models import table, subjectmodel, sessionmodel, measurementmodel, contactmodel, platemodel
  File "C:\Dropbox\Development\Pawlabeling\build\nsis\pkgs\pawlabeling\models\contactmodel.py", line 2, in <module>
    import cv2
ImportError: DLL load failed: The specified module could not be found.

It can't find OpenCV, even though I specified cv2 in my include list:

[Application]
name=Pawlabeling
version=0.2.1
entry_point=pawlabeling.widgets.mainwindow:main
icon=favicon.ico
console=true

[Python]
version=2.7.5
bitness=64

[Include]
# Importable packages that your application requires, one per line
packages = pawlabeling
    numpy
     PySide
     cv2
     scipy
     matplotlib
     pubsub
     tables
     pandas

# Other files and folders that should be installed
files = COPYING.txt
    readme.md
    requirements.txt
    docs/
    pawlabeling/

It seems to have forgotten to pick up the OpenCV dll's which is probably a side-effect of the rather quirky way OpenCV was installed (I used Gohlke's installer). All the dll's are in Lib/site-packages and are called something like opencv_*.dll instead of something resembling cv2.

I'm not sure how to make sure it get's included properly, I guess I could run a script that runs Miniconda to install it anyway or bundle the installer and have it run that. Any suggestions on how to fix this?

Distlib 0.3.4 has been released in December and there have been some changes in regards to the executables which pynsist uses to build the application executables.

The result is that the executables are broken when trying to run them from a non-command-line context (eg. a GUI application) and they immediately exit with code 3221225477 (0xC0000005). This does not happen when executing them from bash or powershell, for some reason. This was working perfectly fine previously when building pynsist installers with distlib==0.3.3 installed.

Pip 21.3.1 currently bundles distlib 0.3.3 and executables built via pip when installing wheels are therefore working. I am not sure though if this will change once pip upgrades. In this case, the issue is caused by distlib itself.

• https://github.com/pypa/distlib/blob/master/CHANGES.rst#034
• https://github.com/pypa/distlib/commits/0.3.4

This is how pynsist builds (concatenates) the wrappers: https://github.com/takluyver/pynsist/blob/master/nsist/commands.py

Original issue, in case more context is required to understand the issue: https://github.com/streamlink/streamlink/issues/4307

I successfully packaged a python application that relies on the Gooey for GUI portion. Below is the code I used inside the installer.cfg file.

[Application]
name=SAVI Tool
version=1.0a

# How to launch the app - this calls the 'main' function from the 'myapp' package:
entry_point=app.main:main
icon=img/program_icon.ico

[Python]
version=3.6.6
bitness=64
format=bundled

[Include]
# Packages from PyPI that your application requires, one per line
# These must have wheels on PyPI:
packages=app
local_wheels = wheels/*.whl
pypi_wheels = Gooey==1.0.1
    scikit-learn==0.19.0
    pandas==0.23.4
    numpy==1.15.1

# To bundle packages which don't publish wheels, or to include directly wheel files
# from a directory, see the docs on the config file.

# Other files and folders that should be installed
files = LICENSE
    sample_data/
    frag_models/

The resulting packaged folder is shown in the attached image.

If I double-click on the .exe file, the application installs inside the ./Program Files folder with name equal to the application name (in my case SAVI tool).

How can I now start the application and the GUI? The SAVI_Tool.launch.pyw file does not seem to launch/open anything. Am I doing something wrong?

As per #5

• Add 'optionparser' backport as a dependency when installing with Python 2
• Use Python 2.7.6 as default version when building with Python 2
• Include pylauncher in MSI when building with Python 2

I have implemented the configuration validation. I guess it's easy enough to maintain and understand it like this. I also wrote tests for it, which can be checked using nosetests.

Hi,

I'm currently programming an app using several libraries such as PyQt5, numpy, scipy, matplotlib,... I get no error while building with pynsist but when I try to launch the app after installation, this message shows up in the log file in %APPDATA%:

File "NameOfFile.py", line 1, in <module> 
     from PyQt5 import QtGui
ImportError: DLL load failed: The specified module could not be found.

I build on a Windows 10 64bit and try to install on a Windows 7 32bit and here is my installer.cfg file:

[Application]
name=Name of my app
version=1.0
publisher=Sirris
entry_point=mypackage.main:main
console=True
license_file=mypackage/resources/licence.txt

[Python]
version=3.6.3
format=bundled
bitness=32

[Include]
packages=mypackage
pypi_wheels=PyQt5==5.10.1
    numpy==1.14.1
    scipy==1.0.0
    numpy==1.14.1
    matplotlib==2.2.0rc1
files=mypackage/resources

[Build]
directory=Some directory

I found lots of threads on the internet talking about concrt140.dll and msvcp140.dll missing but in my case they are included with PyQt5 5.10.1. I tried everything I found, but nothing seems to help...

Do you have any idea of what could be causing this issue?

Thanks, Cirn3co

What needs the MSVC 2015 runtime? NSIS? Python? I wouldn't think it is needed to run the application since that is just Python and then the .pyw file.

Can't you just bundle the files that the Vista .msu installs? So when building the installer add the Vista 32-bit or 64-bit files to the installer?

Also both of the Vista links are 404. The other are fine.

https://cdn.rawgit.com/takluyver/pynsist/msvcrt-tag1/x64/Windows6-KB2999226-x64.msu
https://cdn.rawgit.com/takluyver/pynsist/msvcrt-tag1/x86/Windows6-KB2999226-x86.msu

I just tried pynsist on debian, running python 2.7.3.

The first thing I thought I would try was building the examples with examples/build_all_examples.py. It fails immediately.

The first crash happens immediately in configreader.py due to the use of the python3-only interpolation argument on ConfigParser.

Fixing this, the second crash is on the attempted use of __getitem__ with config[section_name]... also a python3-only thing.

Sorry - I have no time right now to fix this or diagnose further. I fear the python2 issues run deep since the top level example fails so immediately. I was hoping pynsist would work as it would be nice to stay in debian rather than having to jump over to windows all the time to make installers (currently doing this with cx_freeze). :(

This is in analogy to the pip option '--extra-index-urls'.

A custom package index must provide a PyPI interface, i.e. provide a json description such as https://pypi.org/pypi/numpy/json. This is a limitation of yarg.

A self signed certificate can be used with set REQUESTS_CA_BUNDLE=path/to/bundle.pem, see here.

The documentation fort target (and the related parameters) is currently a bit sparse.

It might be useful to explain, that unlike the other options to create link, target is not automatically relative to the installation directory. To link to a file in the files section, you therefore need to write target=$INSTDIR\filename.ext

Could this hint be added to the documentation, and or an example?

Hey, thanks for this great project!

I'm using the [Command my_command] section feature to accommodate multiple entry points to my application.

However, when invoking "\Program Files\my_app\bin\my_command.exe" [OPTIONS], error messages indicate that some standard built-in python modules are not available (No module named <module>), e.g., http, stringprep, or mimetypes.

Is there a simple way to bundle the built-in modules? It'd also help to find out which modules are used by my application, such that I don't have to add a module/package and test until I listed them all. Thanks!

Python has started offering embeddables and installers for Python 3.11 Windows ARM64. Is this something pynsist can include, if I want to distribute a native Windows ARM build?

https://www.python.org/downloads/release/python-3110a7/

You could probably add arm64 as an option for bitness in the config file.

I decided to ask about single sourcing dependencies in separate issue, since there might be slightly different approach than with version numbers.

Currently, I list my dependencies in setup.py:

import setuptools

setuptools.setup(
    name='mypackage',
    version=find_version(),
    author="Niko Pasanen",
    description="My package",
    packages=setuptools.find_packages(),
    install_requires=[
        'astropy>=3.2.1',  
        'click>=7.0',  
        'ipython>=7.4.0',  
        'geopy>=1.2.0', 
        'numba~=0.51.1', 
        'pandas~=1.1.2',
    ],
)

This is needed so that I can install the mypackage in editable state with

pip install -e <folder>

This is not, however the full list of packages, since all of the packages listed in install_requires might depend on other packages. Since I am using virtual environments, I can easily take a snapshot of everything I'm using with pip freeze. I could use this snapshotted list (with perhaps some manual editing, removing development time dependencies) then in the setup.py and installer.cfg. But if I modify anything, I have to remember modify two places. What would be the best way to single-source dependencies? I'm thinking that perhaps I could put all to installer.cfg and read them in setup.py. Would that be the way to go?