A library and tool for generating .pex (Python EXecutable) files

Related tags

Distribution pex
Overview

PEX

https://github.com/pantsbuild/pex/workflows/CI/badge.svg?branch=master

Overview

pex is a library for generating .pex (Python EXecutable) files which are executable Python environments in the spirit of virtualenvs. pex is an expansion upon the ideas outlined in PEP 441 and makes the deployment of Python applications as simple as cp. pex files may even include multiple platform-specific Python distributions, meaning that a single pex file can be portable across Linux and OS X.

pex files can be built using the pex tool. Build systems such as Pants, Buck, and {py}gradle also support building .pex files directly.

Still unsure about what pex does or how it works? Watch this quick lightning talk: WTF is PEX?.

pex is licensed under the Apache2 license.

Installation

To install pex, simply

$ pip install pex

You can also build pex in a git clone using tox:

$ tox -e package
$ cp dist/pex ~/bin

This builds a pex binary in dist/pex that can be copied onto your $PATH. The advantage to this approach is that it keeps your Python environment as empty as possible and is more in-line with what pex does philosophically.

Simple Examples

Launch an interpreter with requests, flask and psutil in the environment:

$ pex requests flask 'psutil>2,<3'

Or instead freeze your current virtualenv via requirements.txt and execute it anywhere:

$ pex $(pip freeze) -o my_virtualenv.pex
$ deactivate
$ ./my_virtualenv.pex

Run webserver.py in an environment containing flask as a quick way to experiment:

$ pex flask -- webserver.py

Launch Sphinx in an ephemeral pex environment using the Sphinx entry point sphinx:main:

$ pex sphinx -e sphinx:main -- --help

Build a standalone pex binary into pex.pex using the pex console_scripts entry point:

$ pex pex -c pex -o pex.pex

You can also build pex files that use a specific interpreter type:

$ pex pex -c pex --python=pypy -o pypy-pex.pex

Most pex options compose well with one another, so the above commands can be mixed and matched. For a full list of options, just type pex --help.

Integrating pex into your workflow

If you use tox (and you should!), a simple way to integrate pex into your workflow is to add a packaging test environment to your tox.ini:

[testenv:package]
deps = pex
commands = pex . -o dist/app.pex

Then tox -e package will produce a relocateable copy of your application that you can copy to staging or production environments.

Documentation

More documentation about Pex, building .pex files, and how .pex files work is available at https://pex.readthedocs.io.

Development

Pex uses tox for test and development automation. To run the test suite, just invoke tox:

$ tox

If you don't have tox, you can generate a pex of tox:

$ pex tox -c tox -o ~/bin/tox

Tox provides many useful commands and options, explained at https://tox.readthedocs.io/en/latest/. Below, we provide some of the most commonly used commands used when working on Pex, but the docs are worth acquainting yourself with to better understand how Tox works and how to do more advanced commmands.

To run a specific environment, identify the name of the environment you'd like to invoke by running tox --listenvs-all, then invoke like this:

$ tox -e format-run

To run MyPy:

$ tox -e typecheck

All of our tox test environments allow passthrough arguments, which can be helpful to run specific tests:

$ tox -e py37-integration -- -k test_reproducible_build

To run Pex from source, rather than through what is on your PATH, invoke via Python:

$ python -m pex

Contributing

To contribute, follow these instructions: https://www.pantsbuild.org/docs/contributor-overview

Comments
  • Add manylinux wheel support and fix a few bugs along the way

    Add manylinux wheel support and fix a few bugs along the way

    • Adds support for manylinux1_x86_64 and manylinux1_i686 platforms. I've treated them as platforms, because that's how PEP-513 describes them. This fixes #281.
    • Fixes a number of bugs related to the handling of wheels that have hyphens in the package names (e.g., cassandra-driver and msgpack-python).
    • Fixes a couple places where str.replace was being called without storing the result (thereby accomplishing nothing since str is immutable)
    opened by dan-blanchard 72
  • Introduce new `--no-compile` flag to not include .pyc in built Pex due to its non-determinism

    Introduce new `--no-compile` flag to not include .pyc in built Pex due to its non-determinism

    Problem

    Until Python 3.7 via https://www.python.org/dev/peps/pep-0552/, .pyc files were never reproducible because they included the timestamp. See the discussion at https://github.com/pantsbuild/pex/issues/716#issuecomment-487821354.

    We are aiming to ensure reproducible builds a la https://github.com/pantsbuild/pex/issues/716 so that we can safely remote PEXes in CI. Reproducible builds also play an important role in security, per https://reproducible-builds.org.

    Solution

    Allow users to toggle on and off .pyc files via the new --compile / --no-compile flag.

    We currently default to including .pyc, because this has been the precedent for 9 years. In a future release, e.g. 1.70, we will change the default to not include .pyc so that builds are reproducible by default, but people can still opt into including the files if they'd like via the flag.

    Alternative flag considered: --reproducible

    Originally, we were going to use --reproducible to toggle on multiple behaviors like not including .pyc and using a hardcoded timestamp instead of system time. However, after deciding that in a future release we are going to default to reproducible, it doesn't make sense for people to ever call --not-reproducible, but it does make sense to opt out of specific behaviors like wanting to use --include-pyc. So, we use behavior-specific flags instead of a universal flag.

    Alternative solution: ensure our own .pyc timestamp

    It may be possible to still include .pyc and have their timestamp be deterministic https://github.com/pantsbuild/pex/pull/718#issuecomment-488202220. This is not pursued for now because it is more complex than necessary, especially because currently we do not ship .pyc files if multiple platforms are used in the PEX. But it could be a followup PR if deemed worth it.

    Result

    When --no-compile is set, PEXes will no longer include .pyc. This results in all of the reproducible build acceptance tests passing the exploded pex portion of their tests, excluding the bdist_wheel test.

    Impact on performance

    Not using .pyc has a slight speedup of ~0.2 seconds to the creation of simple PEXes.

    • time python -m pex -o normal.pex --compile averaged around 0.49 seconds
    • time python -m pex -o repro.pex --no-compile averaged around 0.31 seconds

    Not using .pyc has a slight slowdown of ~0.06 seconds to the startup time of simple PEXes.

    • time ./normal.pex --version averaged around 0.26 seconds
    • time ./repro.pex --version averaged around 0.32 seconds

    python -m pex -m pydoc -o pydoc.pex likewise ran 0.07 seconds slower when not including .pyc.

    Note that the inclusion of .pyc only impacts startup time, not actual runtime performance. See https://stackoverflow.com/questions/2998215/if-python-is-interpreted-what-are-pyc-files for what .pyc files do / why they exist.

    opened by Eric-Arellano 27
  • Improve wheel support in pex.

    Improve wheel support in pex.

    Wheel handling in pex files is broken. The wheel standard says that wheel files are not designed to be importable archives, but pex treats them as if they are. This causes many standard compliant wheels, including things like tensorflow and opencv, to fail to import in pexes (and thus in pants).

    This change modifies wheel handling, so that when a wheel is added to a pex, it's installed in an importable form.

    opened by MarkChuCarroll 23
  • pexrc support

    pexrc support

    Hi!

    Big fan of pex. We are starting to face an issue with our deployments where setting custom PEX_ROOT and PEX_MODULE in some entry scripts. I'm not a huge fan of having to write a wrapper for our pex files and I'd rather drop a .pexrc with my deployment so the pex file itself can see these vars (rather than having to explicitly set them prior to invoking pex).

    This PR just adds the ability to drop a ~/.pexrc or .pexrc (relative to the pex itself, in that precedence). Variables() reads this file and updates the environment before looking in os.environ, so anything in .pexrc is overwritten by runtime environment variables.

    Let me know what you think, thanks for looking!

    opened by lorencarvalho 21
  • Introduce the spread app layout.

    Introduce the spread app layout.

    Beyond the top level pex and __main__.py files, the layout is an implementation detail, but the current structure sheds some light on the cache-friendly characteristics:

    Given loose sources:

    $ cat src/main.py
    from pex.version import __version__; print(__version__)
    

    A spread layout PEX looks like:

    $ python -mpex pex -Dsrc -emain -opex.spread.venv --spread \
      --seed verbose --venv prepend | jq .
    {
      "pex_root": "/home/jsirois/.pex",
      "python": "/usr/bin/python3.9",
      "pex": "/home/jsirois/.pex/venvs/d2f743b9c1ebb156f794419c01b3422653cbdb61/2d1d404c3de23b1810386195be7410700b1feb14/pex"
    }
    $ tree -a pex.spread.venv
    pex.spread.venv
    ├── .bootstrap
    ├── .deps
    │   └── pex-2.1.46-py2.py3-none-any.whl
    ├── __main__.py -> src/__main__.py
    ├── pex -> __main__.py
    ├── PEX-SPREAD-INFO
    └── src
        ├── __main__.py
        ├── main.py
        └── PEX-INFO
    

    And the runtime spreading is suggested by the new PEX-SPREAD-INFO manifest:

    {
      "sources": [
        "PEX-INFO",
        "__main__.py",
        "main.py"
      ],
      "spreads": [
        {
          "strip_zip_relpath": false,
          "unpack_relpath": "bootstraps/a54b6ae5e64e5b229388fdffc8adac141f3c416b",
          "zip_relpath": ".bootstrap"
        },
        {
          "strip_zip_relpath": true,
          "unpack_relpath": "installed_wheels/f627f0368a0e29be24aa8cadba74044b9ad990d7/pex-2.1.46-py2.py3-none-any.whl",
          "zip_relpath": ".deps/pex-2.1.46-py2.py3-none-any.whl"
        }
      ]
    }
    

    The layout adds new PEX_ROOT caches for the .bootstrap zip and installed wheel chroot zips such that neither bootstraps - which are tied to a version of Pex, nor installed wheel chroot zips, which are constant for a given distribution version, are created more than once.

    Closes #1424

    opened by jsirois 20
  • PEX_PATH transitivity

    PEX_PATH transitivity

    When building a graph of PEXes, each of which contains distributions for a single requirement (*mostly: disregard the pluggy_zipp_importlib-metadata_pytest_attrs cluster), it appears that the embedded pex_path in the PEX-INFO is not consumed. I'm wondering whether:

    1. these failures are expected
    2. providing a flattened transitive PEX_PATH at runtime instead of embedding the direct pex_path deps at build time is likely to be able to allow me to get the behavior I want

    As an example: a single requirement PEX containing humbug, with a pex_path containing its requirement PEXes might have PEX-INFO that looks like:

    {
      "always_write_cache": false,
      "build_properties": {
        "class": "CPython",
        "pex_version": "2.1.44",
        "platform": "macosx_10_16_x86_64",
        "version": [
          3,
          7,
          7
        ]
      },
      "code_hash": "da39a3ee5e6b4b0d3255bfef95601890afd80709",
      "distributions": {
        "humbug-0.2.6-py3-none-any.whl": "e772468167e997e051e8c20d71551a11bba19f26"
      },
      "emit_warnings": false,
      "ignore_errors": false,
      "includes_tools": false,
      "inherit_path": "false",
      "interpreter_constraints": [],
      "pex_hash": "53fc35393a34baa9b5665c00dc19f27a02a3ea07",
      "pex_path": "__reqs/requests.pex:__reqs/types-requests.pex:__reqs/setuptools.pex:__reqs/urllib3.pex:__reqs/certifi.pex:__reqs/idna.pex:__reqs/charset-normalizer.pex:__reqs/pluggy_zipp_importlib-metadata_pytest_attrs.pex:__reqs/typing-extensions.pex:__reqs/packaging.pex:__reqs/iniconfig.pex:__reqs/py.pex:__reqs/toml.pex:__reqs/six.pex:__reqs/pyparsing.pex",
      "requirements": [
        "humbug"
      ],
      "strip_pex_env": true,
      "unzip": false,
      "venv": false,
      "venv_bin_path": "false",
      "venv_copies": false,
      "zip_safe": false
    }
    

    But consuming it will fail to find the requests distribution embedded in the __reqs/requests.pex PEX:

    Below either PexBuilder.set_script or bin.pex.seed_cache:

      ...
      File ".deps/pex-2.1.44-py2.py3-none-any.whl/pex/pex.py", line 118, in resolve
        for dist in env.resolve():
      File ".deps/pex-2.1.44-py2.py3-none-any.whl/pex/environment.py", line 608, in resolve
        self._resolved_dists = self.resolve_dists(all_reqs)
      File ".deps/pex-2.1.44-py2.py3-none-any.whl/pex/environment.py", line 695, in resolve_dists
        "{items}".format(pex=self._pex, platform=self._platform, items="\n".join(items))
    pex.environment.ResolveError: Failed to resolve requirements from PEX environment @ /private/var/folders/bg/_r10hqp14kjcpv68yzdk5svc0000gn/T/process-executioneHVQxG/__reqs/humbug.pex.
    Needed macosx_10_16_x86_64-cp-37-cp37m compatible dependencies for:
     1: requests
        Required by:
          humbug 0.2.6
        But this pex had no 'requests' distributions.
    
    question answered 
    opened by stuhood 20
  • pex is inexplicably slow for a small project

    pex is inexplicably slow for a small project

    Reproduction

    1. Clone https://github.com/zmanji/git-squash
    2. Checkout 505cf06c598409db9d119a521480e43de3fa8908
    3. Run pex . -vv -r requirements.txt --no-build --wheel --no-transitive --inherit-path=false --output-file /Users/zmanji/code/git-squash/dist/git-squash-0.0.1.pex --script git-squash -o dist/out.pex

    With latest pex on macOS.

    Expectation

    Because of --no-build, --wheel, --no-transitive and a fully resolved requirements.txt from pip-compile building a pex should be very fast. The wheels can be fetched from pypi if they are not cached locally, the package it self a single file, so building the wheel should be very fast and then building the pex should be straight forward

    For reference building the wheel for git-squash via python ./setup.py bdist_wheel takes about 0.3s on my computer.

    Reality

    I see this output

    pex . -vv -r requirements.txt --no-build --wheel --no-transitive --inherit-path=false --output-file /Users/zmanji/code/git-squash/dist/git-squash-0.0.1.pex --script git-squash -o dist/out.pex
    pex: Building pex :: Resolving distributions (['.', 'requirements.txt']) :: Resolving requirements. :: Resolving for:
      DistributionTarget(interpreter=PythonInterpreter('/Users/zmanji/.pyenv/versions/3.9.4/bin/python3.9', PythonIdentity('/Users/zmanji/.pyenv/versions/3.9.4/bin/python3.9', 'cp39', 'cp39', 'macpex: Hashing pex                                                                                                                                                                                pex: Hashing pex: 67.4ms                                                                                                                            
    pex: Isolating pex: 0.1ms
    pex: Building pex :: Resolving distributions (['.', 'requirements.txt']) :: Resolving requirements. :: Building distributions for:
      BuildRequest(target=DistributionTarget(interpreter=PythonInterpreter('/Users/zmanji/.pyenv/versions/3.9.4/bin/python3.9', PythonIdentity('/Users/zmanji/.pyenv/versions/3.9.4/bin/python3.9', pex: Building /Users/zmanji/code/git-squash to /Users/zmanji/.pex/built_wheels/local_projects/git-squash/fa682a73c509acd8bf4e7e52f387f55f3d0b3e6e/cp39-cp39                                                                                                                                                                                                                                                                                                                                
    pex: Building pex :: Resolving distributions (['.', 'requirements.txt']) :: Resolving requirements. :: Calculating project names for direct requirements:
      LocalProjectRequirement(line=LogicalLine(raw_text='.', processed_text='.', source='<string>', start_line=1, end_line=1), path='/Users/zmanji/code/git-squash', extras=(), marker=None, editable=False)
      PyPIRequirement(line=LogicalLine(raw_text='gitdb==4.0.7\n', processed_text='gitdb==4.0.7', source='/Users/zmanji/code/git-squash/requirements.txt', start_line=7, end_line=7), requirement=Requirement.parse('gitdb==4.0.7'), editable=False)
      PyPIRequirement(line=LogicalLine(raw_text='gitpython==3.1.17\n', processed_text='gitpython==3.1.17', source='/Users/zmanji/code/git-squash/requirements.txt', start_line=9, end_line=9), requirement=Requirement.parse('gitpython==3.1.17'), editable=False)
      PyPIRequirement(line=LogicalLine(raw_text='smmap==4.0.0\n', processed_text='smmap==4.0.0', source='/Users/zmanji/code/git-squash/requirements.txt', start_line=11, end_line=11), requirement=Rpex: Building pex :: Resolving distributions (['.', 'requirements.txt']) :: Resolving requirements. :: Installing:
      InstallRequest(target=DistributionTarget(interpreter=PythonInterpreter('/Users/zmanji/.pyenv/versions/3.9.4/bin/python3.9', PythonIdentity('/Users/zmanji/.pyenv/versions/3.9.4/bin/python3.9', 'cp39', 'cp39', 'macosx_11_0_x86_64', (3, 9, 4)))), wheel_path='/private/var/folders/1j/9xbcby_57tx1kmn6s8zfsbww0000gn/T/tmpvty3ch2e/Users.zmanji..pyenv.versions.3.9.4.bin.python3.9/GitPython-3.1.17-py3-none-any.whl', fingerprint='59d63e79b53503f5403220d3159d8e409c46e107')
      InstallRequest(target=DistributionTarget(interpreter=PythonInterpreter('/Users/zmanji/.pyenv/versions/3.9.4/bin/python3.9', PythonIdentity('/Users/zmanji/.pyenv/versions/3.9.4/bin/python3.9', 'cp39', 'cp39', 'macosx_11_0_x86_64', (3, 9, 4)))), wheel_path='/private/var/folders/1j/9xbcby_57tx1kmn6s8zfsbww0000gn/T/tmpvty3ch2e/Users.zmanji..pyenv.versions.3.9.4.bin.python3.9/smmap-4.0.0-py2.py3-none-any.whl', fingerprint='5eb1d8910b7c93de92bd8db25d0a8293b2d1de6f')
      InstallRequest(target=DistributionTarget(interpreter=PythonInterpreter('/Users/zmanji/.pyenv/versions/3.9.4/bin/python3.9', PythonIdentity('/Users/zmanji/.pyenv/versions/3.9.4/bin/python3.9', 'cp39', 'cp39', 'macosx_11_0_x86_64', (3, 9, 4)))), wheel_path='/private/var/folders/1j/9xbcby_57tx1kmn6s8zfsbww0000gn/T/tmpvty3ch2e/Users.zmanji..pyenv.versions.3.9.4.bin.python3.9/gitdb-4.0.7-py3-none-any.whl', fingerprint='9dacbc989a247dabdad014d6397d8482fd55f26c')
      InstallRequest(target=DistributionTarget(interpreter=PythonInterpreter('/Users/zmanji/.pyenv/versions/3.9.4/bin/python3.9', PythonIdentity('/Users/zmanji/.pyenv/versions/3.9.4/bin/python3.9', 'cp39', 'cp39', 'macosx_11_0_x86_64', (3, 9, 4)))), wheel_path='/Users/zmanji/.pex/built_wheels/local_projects/git-squash/fa682a73c509acd8bf4e7e52f387f55f3d0b3e6e/cp39-cp39/git_squash-0.0.1-pex: Using cached installation of GitPython-3.1.17-py3-none-any.whl at /Users/zmanji/.pex/installed_wheels/59d63e79b53503f5403220d3159d8e409c46e107/GitPython-3.1.17-py3-none-any.whl                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              
    pex: Using cached installation of smmap-4.0.0-py2.py3-none-any.whl at /Users/zmanji/.pex/installed_wheels/5eb1d8910b7c93de92bd8db25d0a8293b2d1de6f/smmap-4.0.0-py2.py3-none-any.whl
    pex: Using cached installation of gitdb-4.0.7-py3-none-any.whl at /Users/zmanji/.pex/installed_wheels/9dacbc989a247dabdad014d6397d8482fd55f26c/gitdb-4.0.7-py3-none-any.whl
    pex: Installing /Users/zmanji/.pex/built_wheels/local_projects/git-squash/fa682a73c509acd8bf4e7e52f387f55f3d0b3e6e/cp39-cp39/git_squash-0.0.1-py3-none-any.whl in /Users/zmanji/.pex/installed_wheels/c355002f497d1f36a0bae44b05e55b4338da8a0f/git_squash-0.0.1-py3-none-any.whl
    pex: Set entrypoint to console_script 'git_squash.git_squash:main' in git-squash 0.0.1 (/Users/zmanji/.pex/installed_wheels/c355002f497d1f36a0bae44b05e55b4338da8a0f/git_squash-0.0.1-py3-none-any.whl)
    pex: Building pex: 13722.4ms
    pex:   Resolving distributions (['.', 'requirements.txt']): 13720.7ms
    pex:     Resolving requirements.: 13709.1ms
    pex:       Resolving for:
      DistributionTarget(interpreter=PythonInterpreter('/Users/zmanji/.pyenv/versions/3.9.4/bin/python3.9', PythonIdentity('/Users/zmanji/.pyenv/versions/3.9.4/bin/python3.9', 'cp39', 'cp39', 'macosx_11_0_x86_64', (3, 9, 4)))): 6592.5ms
    pex:       Building distributions for:
      BuildRequest(target=DistributionTarget(interpreter=PythonInterpreter('/Users/zmanji/.pyenv/versions/3.9.4/bin/python3.9', PythonIdentity('/Users/zmanji/.pyenv/versions/3.9.4/bin/python3.9', 'cp39', 'cp39', 'macosx_11_0_x86_64', (3, 9, 4)))), source_path='/Users/zmanji/code/git-squash', fingerprint='fa682a73c509acd8bf4e7e52f387f55f3d0b3e6e'): 6475.2ms
    pex:       Calculating project names for direct requirements:
      LocalProjectRequirement(line=LogicalLine(raw_text='.', processed_text='.', source='<string>', start_line=1, end_line=1), path='/Users/zmanji/code/git-squash', extras=(), marker=None, editable=False)
      PyPIRequirement(line=LogicalLine(raw_text='gitdb==4.0.7\n', processed_text='gitdb==4.0.7', source='/Users/zmanji/code/git-squash/requirements.txt', start_line=7, end_line=7), requirement=Requirement.parse('gitdb==4.0.7'), editable=False)
      PyPIRequirement(line=LogicalLine(raw_text='gitpython==3.1.17\n', processed_text='gitpython==3.1.17', source='/Users/zmanji/code/git-squash/requirements.txt', start_line=9, end_line=9), requirement=Requirement.parse('gitpython==3.1.17'), editable=False)
      PyPIRequirement(line=LogicalLine(raw_text='smmap==4.0.0\n', processed_text='smmap==4.0.0', source='/Users/zmanji/code/git-squash/requirements.txt', start_line=11, end_line=11), requirement=Requirement.parse('smmap==4.0.0'), editable=False): 4.3ms
    pex:       Installing:
      InstallRequest(target=DistributionTarget(interpreter=PythonInterpreter('/Users/zmanji/.pyenv/versions/3.9.4/bin/python3.9', PythonIdentity('/Users/zmanji/.pyenv/versions/3.9.4/bin/python3.9', 'cp39', 'cp39', 'macosx_11_0_x86_64', (3, 9, 4)))), wheel_path='/private/var/folders/1j/9xbcby_57tx1kmn6s8zfsbww0000gn/T/tmpvty3ch2e/Users.zmanji..pyenv.versions.3.9.4.bin.python3.9/GitPython-3.1.17-py3-none-any.whl', fingerprint='59d63e79b53503f5403220d3159d8e409c46e107')
      InstallRequest(target=DistributionTarget(interpreter=PythonInterpreter('/Users/zmanji/.pyenv/versions/3.9.4/bin/python3.9', PythonIdentity('/Users/zmanji/.pyenv/versions/3.9.4/bin/python3.9', 'cp39', 'cp39', 'macosx_11_0_x86_64', (3, 9, 4)))), wheel_path='/private/var/folders/1j/9xbcby_57tx1kmn6s8zfsbww0000gn/T/tmpvty3ch2e/Users.zmanji..pyenv.versions.3.9.4.bin.python3.9/smmap-4.0.0-py2.py3-none-any.whl', fingerprint='5eb1d8910b7c93de92bd8db25d0a8293b2d1de6f')
      InstallRequest(target=DistributionTarget(interpreter=PythonInterpreter('/Users/zmanji/.pyenv/versions/3.9.4/bin/python3.9', PythonIdentity('/Users/zmanji/.pyenv/versions/3.9.4/bin/python3.9', 'cp39', 'cp39', 'macosx_11_0_x86_64', (3, 9, 4)))), wheel_path='/private/var/folders/1j/9xbcby_57tx1kmn6s8zfsbww0000gn/T/tmpvty3ch2e/Users.zmanji..pyenv.versions.3.9.4.bin.python3.9/gitdb-4.0.7-py3-none-any.whl', fingerprint='9dacbc989a247dabdad014d6397d8482fd55f26c')
      InstallRequest(target=DistributionTarget(interpreter=PythonInterpreter('/Users/zmanji/.pyenv/versions/3.9.4/bin/python3.9', PythonIdentity('/Users/zmanji/.pyenv/versions/3.9.4/bin/python3.9', 'cp39', 'cp39', 'macosx_11_0_x86_64', (3, 9, 4)))), wheel_path='/Users/zmanji/.pex/built_wheels/local_projects/git-squash/fa682a73c509acd8bf4e7e52f387f55f3d0b3e6e/cp39-cp39/git_squash-0.0.1-py3-none-any.whl', fingerprint='c355002f497d1f36a0bae44b05e55b4338da8a0f'): 430.6ms
    Saving PEX file to dist/out.pex
    pex: Zipping PEX file.: 148.4ms
    pex . -vv -r requirements.txt --no-build --wheel --no-transitive    --script   9.65s user 4.27s system 96% cpu 14.348 total
    

    Notice:

    pex:     Resolving requirements.: 13709.1ms
    

    This is very unexpected and there is no information in the output as to why it takes so long. With all of the caching PEX uses, I would expect this to be very fast.

    answered 
    opened by zmanji 20
  • Use pip for resolving and building distributions

    Use pip for resolving and building distributions

    Now that we can vendor distributions, we might consider vendoring pip to get pip-compatible resolution and wheel building behavior. This would likely kill a fair amount of PEX code and open issues at the expense of a larger PEX tool binary and, without care, a larger PEX runtime. Since pip should be largely (completely?) un-needed at runtime, this might have little downside.

    enhancement resolver 
    opened by jsirois 20
  • Problem running PEX files with restricted permissions

    Problem running PEX files with restricted permissions

    When running PEX files, PexInfo determines the install_cache path based on the pex_root path, which in turn can be set in the PEX-INFO file - but isn't.

    That means it defaults to ~/.pex for writing the install cache - which fails when that path isn't writeable for the user. That can happen - is common practice, even - when securing a system against unwanted file system access. Writing to the home directory, for example, can modify environment variables in unwanted ways.

    There's a number of things that could be improved in the approach of defaulting to ~/.pex, but different requirements here would conflict with each other - so I'll leave that discussion to other people. The main point, though, is that it would be very, very useful if one could set the pex_root value that goes into the PEX-INFO file, so that you can effectively choose other paths for the install_cache. If additionally, install_cache could be overridden as well, that'd be excellent.

    enhancement 
    opened by jfinkhaeuser 20
  • Pex silently builds bad pexfile

    Pex silently builds bad pexfile

    This is on OSX against pex 1.4.2 using python 2.7.13. Same command works against pex 1.3.2

    Command:

    $ pex -r requirements.txt -o env.pex
    

    Contents of requirements.txt:

    # === Main Dependencies
    flask>=0.0
    pathlib2
    dulwich
    pandas
    codenamize
    pyOpenSSL>=16.2.0
    jupyter
    

    When attempting to run env.pex:

    $ ./env.pex
    Failed to execute PEX file, missing macosx_10_12_x86_64-cp-27-cp27m compatible dependencies for:
    ipython
    appnope
    

    Interestingly, changing the requirements.txt to include appnope and ipython==5.7 explicitly fixes it against pex 1.4.2.

    As mentioned, the same works either way against pex 1.3.2.

    bug 
    opened by kmannislands 20
  • "Resource deadlock avoided" when creating multiple `--venv` PEXs in parallel

    Testing out some change in Pants, I create some (>10 < 60) number of processes responsible for creating a PEX venv (--venv) using a PEX lockfile (--lock) specifying some requirement strings which among the processes are likely highly overlapping yet unique subset of my lockfile.

    The stacktrace is at the end of this issue. The issue is regularly reproducible in our work repo, but not reproducible in Pants' own repo with the relevant Pants code running (Pants is only relevant because that's how I'm opening the N processes).

    Let me know what else you need to help debug and solve.

    Stacktrace:

    Traceback (most recent call last):                                                                                                                                            File "/home/joshuacannon/.cache/pants/named_caches/pex_root/unzipped_pexes/15004f7f47d92f2a9c0b754a1c6d2997f332ea44/.bootstrap/pex/pex.py", line 504, in execute          
        exit_value = self._wrap_coverage(self._wrap_profiling, self._execute)                                                                                                     File "/home/joshuacannon/.cache/pants/named_caches/pex_root/unzipped_pexes/15004f7f47d92f2a9c0b754a1c6d2997f332ea44/.bootstrap/pex/pex.py", line 409, in _wrap_coverage   
        return runner(*args)                                                                                                                                                      File "/home/joshuacannon/.cache/pants/named_caches/pex_root/unzipped_pexes/15004f7f47d92f2a9c0b754a1c6d2997f332ea44/.bootstrap/pex/pex.py", line 440, in _wrap_profiling  
        return runner(*args)                                                                                                                                                      File "/home/joshuacannon/.cache/pants/named_caches/pex_root/unzipped_pexes/15004f7f47d92f2a9c0b754a1c6d2997f332ea44/.bootstrap/pex/pex.py", line 560, in _execute         
        return self.execute_entry(self._pex_info.entry_point)                                                                                                                   
      File "/home/joshuacannon/.cache/pants/named_caches/pex_root/unzipped_pexes/15004f7f47d92f2a9c0b754a1c6d2997f332ea44/.bootstrap/pex/pex.py", line 696, in execute_entry
        return self.execute_pkg_resources(entry_point)
      File "/home/joshuacannon/.cache/pants/named_caches/pex_root/unzipped_pexes/15004f7f47d92f2a9c0b754a1c6d2997f332ea44/.bootstrap/pex/pex.py", line 728, in execute_pkg_resources
        return runner()
      File "/home/joshuacannon/.cache/pants/named_caches/pex_root/installed_wheels/0457d0c3fb526f3f246a04ebe5fa67dfa4f0877fe617e106f23f3b083f7b1ab1/pex-2.1.73-py2.py3-none-any.whl/pex/bin/pex.py", line 724, in main
        do_main(
      File "/home/joshuacannon/.cache/pants/named_caches/pex_root/installed_wheels/0457d0c3fb526f3f246a04ebe5fa67dfa4f0877fe617e106f23f3b083f7b1ab1/pex-2.1.73-py2.py3-none-any.whl/pex/bin/pex.py", line 745, in do_main
        pex_builder = build_pex(
      File "/home/joshuacannon/.cache/pants/named_caches/pex_root/installed_wheels/0457d0c3fb526f3f246a04ebe5fa67dfa4f0877fe617e106f23f3b083f7b1ab1/pex-2.1.73-py2.py3-none-any.whl/pex/bin/pex.py", line 594, in build_pex
        resolve_from_lock(
      File "/home/joshuacannon/.cache/pants/named_caches/pex_root/installed_wheels/0457d0c3fb526f3f246a04ebe5fa67dfa4f0877fe617e106f23f3b083f7b1ab1/pex-2.1.73-py2.py3-none-any.whl/pex/resolve/lock_resolver.py", line 198, in resolve_from_lock
        pool.map(
      File "/usr/lib/python3.8/multiprocessing/pool.py", line 364, in map
        return self._map_async(func, iterable, mapstar, chunksize).get()
      File "/usr/lib/python3.8/multiprocessing/pool.py", line 771, in get
        raise self._value
      File "/usr/lib/python3.8/multiprocessing/pool.py", line 125, in worker
        result = (True, func(*args, **kwds))
      File "/usr/lib/python3.8/multiprocessing/pool.py", line 48, in mapstar
        return list(map(*args))
      File "/home/joshuacannon/.cache/pants/named_caches/pex_root/installed_wheels/0457d0c3fb526f3f246a04ebe5fa67dfa4f0877fe617e106f23f3b083f7b1ab1/pex-2.1.73-py2.py3-none-any.whl/pex/result.py", line 101, in catch
        return func(*args, **kwargs)
      File "/home/joshuacannon/.cache/pants/named_caches/pex_root/installed_wheels/0457d0c3fb526f3f246a04ebe5fa67dfa4f0877fe617e106f23f3b083f7b1ab1/pex-2.1.73-py2.py3-none-any.whl/pex/resolve/lock_resolver.py", line 46, in store_downloadable_artifact
        return self.store(artifact=downloadable_artifact.artifact)
      File "/home/joshuacannon/.cache/pants/named_caches/pex_root/installed_wheels/0457d0c3fb526f3f246a04ebe5fa67dfa4f0877fe617e106f23f3b083f7b1ab1/pex-2.1.73-py2.py3-none-any.whl/pex/resolve/lockfile/download_manager.py", line 40, in store
        with atomic_directory(download_dir, exclusive=True) as atomic_dir:
      File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__
        return next(self.gen)
      File "/home/joshuacannon/.cache/pants/named_caches/pex_root/installed_wheels/0457d0c3fb526f3f246a04ebe5fa67dfa4f0877fe617e106f23f3b083f7b1ab1/pex-2.1.73-py2.py3-none-any.whl/pex/common.py", line 470, in atomic_directory
        fcntl.lockf(lock_fd, fcntl.LOCK_EX)  # A blocking write lock.
    OSError: [Errno 35] Resource deadlock avoided
    
    bug resolver 
    opened by thejcannon 19
  • A loose layout, venv-with-symlink PEX creates brittle symlinks

    A loose layout, venv-with-symlink PEX creates brittle symlinks

    Pex version 2.1.113

    I found this through Pants, but ultimately seems like a PEX issue/fix.

    $ pex --no-venv-copies --layout=loose --venv prepend awscli --script aws -o  awscli.pex
    $ python awscli.pex help
    <help output>
    $ mv awscli.pex awscli2.pex/
    $ python awscli2.pex
    Traceback (most recent call last):
      File "/home/josh/.pex/venvs/ea7cb7998ed2db4f784178989a4034013cd10e01/ddab8011daaee380698ac2fb9701af18c90c03f6/bin/aws", line 19, in <module>
        import awscli.clidriver
    ModuleNotFoundError: No module named 'awscli
    

    See also:

    $ ls -l /home/josh/.pex/venvs/ea7cb7998ed2db4f784178989a4034013cd10e01/ddab8011daaee380698ac2fb9701af18c90c03f6/lib/python3.8/site-packages/awscli
    lrwxrwxrwx 1 josh josh 107 Jan  3 11:49 /home/josh/.pex/venvs/ea7cb7998ed2db4f784178989a4034013cd10e01/ddab8011daaee380698ac2fb9701af18c90c03f6/lib/python3.8/site-packages/awscli -> ../../../../../../../../../<CWD>/awscli.pex/.deps/awscli-1.27.41-py3-none-any.whl/awscli
    

    In Pants-land, this manifests as being a symlink to an ephemeral Pants sandbox which has since gone bye-bye.


    What I might expect is that on first run, the wheels get copied into the PEX root and then the venv symlinks to the extracted wheels in the pex root so moving/deleting the original pex doesn't invalidate the venv.

    bug 
    opened by thejcannon 7
  • Cannot use pex with statically linked python

    Cannot use pex with statically linked python

    due to https://github.com/pypa/pip/issues/6543 it's not possible for pex to resolve dependencies with a statically linked python.

    To reproduce in a clean chroot.

    $ wget https://github.com/indygreg/python-build-standalone/releases/download/20221220/cpython-3.10.9+20221220-x86_64_v3-unknown-linux-musl-install_only.tar.gz
    $ tar xvzf cpython-3.10.9+20221220-x86_64_v3-unknown-linux-musl-install_only.tar.gz 
    $ wget https://github.com/pantsbuild/pex/releases/download/v2.1.119/pex
    $ ./python/bin/python3 ./pex 'fortune' -o fortune.pex
    pid 31 -> /root/.pex/venvs/ffb7a52e5a35c86fd46ad0fdd44b7ac577115b21/58beba50ae31e0c54f07dbf739954fb14f7a9c36/bin/python -sE /root/.pex/venvs/ffb7a52e5a35c86fd46ad0fdd44b7ac577115b21/58beba50ae31e0c54f07dbf739954fb14f7a9c36/pex --disable-pip-version-check --no-python-version-warning --exists-action a --no-input --use-deprecated legacy-resolver --isolated -q --cache-dir /root/.pex/pip_cache download --dest /root/.pex/downloads/resolver_download.oodws27t/python.bin.python3.10 fortune --index-url https://pypi.org/simple --retries 5 --timeout 15 exited with 2 and STDERR:
    ERROR: Exception:
    Traceback (most recent call last):
      File "/root/.pex/venvs/ffb7a52e5a35c86fd46ad0fdd44b7ac577115b21/58beba50ae31e0c54f07dbf739954fb14f7a9c36/lib/python3.10/site-packages/pip/_internal/cli/base_command.py", line 223, in _main
        status = self.run(options, args)
      File "/root/.pex/venvs/ffb7a52e5a35c86fd46ad0fdd44b7ac577115b21/58beba50ae31e0c54f07dbf739954fb14f7a9c36/lib/python3.10/site-packages/pip/_internal/cli/req_command.py", line 180, in wrapper
        return func(self, options, args)
      File "/root/.pex/venvs/ffb7a52e5a35c86fd46ad0fdd44b7ac577115b21/58beba50ae31e0c54f07dbf739954fb14f7a9c36/lib/python3.10/site-packages/pip/_internal/commands/download.py", line 92, in run
        session = self.get_default_session(options)
      File "/root/.pex/venvs/ffb7a52e5a35c86fd46ad0fdd44b7ac577115b21/58beba50ae31e0c54f07dbf739954fb14f7a9c36/lib/python3.10/site-packages/pip/_internal/cli/req_command.py", line 78, in get_default_session
        self._session = self.enter_context(self._build_session(options))
      File "/root/.pex/venvs/ffb7a52e5a35c86fd46ad0fdd44b7ac577115b21/58beba50ae31e0c54f07dbf739954fb14f7a9c36/lib/python3.10/site-packages/pip/_internal/cli/req_command.py", line 88, in _build_session
        session = PipSession(
      File "/root/.pex/venvs/ffb7a52e5a35c86fd46ad0fdd44b7ac577115b21/58beba50ae31e0c54f07dbf739954fb14f7a9c36/lib/python3.10/site-packages/pip/_internal/network/session.py", line 248, in __init__
        self.headers["User-Agent"] = user_agent()
      File "/root/.pex/venvs/ffb7a52e5a35c86fd46ad0fdd44b7ac577115b21/58beba50ae31e0c54f07dbf739954fb14f7a9c36/lib/python3.10/site-packages/pip/_internal/network/session.py", line 135, in user_agent
        zip(["lib", "version"], libc_ver()),
      File "/root/.pex/venvs/ffb7a52e5a35c86fd46ad0fdd44b7ac577115b21/58beba50ae31e0c54f07dbf739954fb14f7a9c36/lib/python3.10/site-packages/pip/_internal/utils/glibc.py", line 94, in libc_ver
        glibc_version = glibc_version_string()
      File "/root/.pex/venvs/ffb7a52e5a35c86fd46ad0fdd44b7ac577115b21/58beba50ae31e0c54f07dbf739954fb14f7a9c36/lib/python3.10/site-packages/pip/_internal/utils/glibc.py", line 18, in glibc_version_string
        return glibc_version_string_confstr() or glibc_version_string_ctypes()
      File "/root/.pex/venvs/ffb7a52e5a35c86fd46ad0fdd44b7ac577115b21/58beba50ae31e0c54f07dbf739954fb14f7a9c36/lib/python3.10/site-packages/pip/_internal/utils/glibc.py", line 52, in glibc_version_string_ctypes
        process_namespace = ctypes.CDLL(None)
      File "/python/lib/python3.10/ctypes/__init__.py", line 374, in __init__
        self._handle = _dlopen(self._name, mode)
    OSError: Dynamic loading not supported
    

    Changing the pip version does not work because the vendored pip is used to fetch the newer pip. I see two possible solutions:

    • pex3 is released and vendors a newer pip.
    • two pips are vendored much like how two packaging versions are vendored, and when a statically linked python is detected we use the newer pip.
    opened by zmanji 2
  • Release 2.1.120

    Release 2.1.120

    Backlog:

    • [ ] Support REPL command history #2019
    • [ ] Allow system_site_packages=True in Virtualenv.create() #1973
    • [ ] Cleanup sys.path after pex is imported #1959
    • [ ] importing a pex leaks bootstrap vendored code #1954
    • [ ] The --compile option is only useful for Python 2.7 #1939
    • [ ] Universal lock creation failure for disjoint IC and Requires-Python is correct but the error message from Pip / Pex is confusing. #1927
    • [ ] The __pex__ import hook does not handle registering top level PEP-420 packages #1910
    • [ ] Expose setuptools and wheel version flags for use with non-vendored Pips. #1895
    • [ ] Plumb --pip-version to Platform tag calculation. #1894

    • [ ] PEX_INHERIT_PATH+default shebangs don't interact well with virtualenvs #1344
    • [ ] Pex interpreter cache is not upgrade / downgrade safe. #1176
    • [ ] Provide a better resolve error message for a Platform DistributionTarget when --resolve-local-platforms. #1070
    • [ ] Remove PEX_PYTHON_PATH from build time interpreter selection. #1075
    release 
    opened by jsirois 1
  • Specifying `--platform`in `lock create` impedes referring to sdists

    Specifying `--platform`in `lock create` impedes referring to sdists

    Both:

    • pex3 lock create --platform manylinux_2_31_x86_64-cp-38-cp38 timeout-decorator~=0.4
    • pex3 lock create --style sources --platform manylinux_2_31_x86_64-cp-38-cp38 timeout-decorator~=0.4

    result in an error because timeout-decorator is an sdist-only package.

    opened by thejcannon 0
  • Allow system_site_packages=True in Virtualenv.create()

    Allow system_site_packages=True in Virtualenv.create()

    Currently PEX could create virtualenv to populate bundled packages, and in PEX code it allows to create venv with "--system-site-packages", but there is no way to specify this parameter at runtime.

    We want to run with venv and system-site-packages enabled, can we have an extra environment variable to specify this behavior?

    enhancement feature request 
    opened by PhoenixHe-TSP 5
Releases(v2.1.119)
  • v2.1.119(Dec 15, 2022)


    2.1.119

    This release brings two new features. The venv pex tool now just warns when using --compile and there is a *.pyc compile error instead of failing to create the venv. Also, a new PEX_DISABLE_VARIABLES env var knob is added to turn off reading all PEX_* env vars from the environment.

    • Ignore compile error for PEX_TOOLS=1 (#2002)
    • Add PEX_DISABLE_VARIABLES to lock down a PEX run. (#2014)
    Source code(tar.gz)
    Source code(zip)
    pex(3.88 MB)
  • v2.1.118(Dec 9, 2022)


    2.1.118

    This is a very tardy hotfix release for a regression introduced in Pex 2.1.91 by #1785 that replaced sys.argv[0] with its fully resolved path. This prevented introspecting the actual file path used to launch the PEX which broke BusyBox-alike use cases.

    There is also a new --non-hermetic-scripts option accepted by the venv tool to allow running console scripts with PYTHONPATH adjustments to the sys.path.

    • Remove un-needed realpathing of sys.argv[0]. (#2007)
    • Add --non-hermetic-scripts option to venv tool. (#2010)
    Source code(tar.gz)
    Source code(zip)
    pex(3.88 MB)
  • v2.1.117(Nov 29, 2022)


    2.1.117

    This release fixes a bug introduced in Pex 2.1.109 where the released Pex PEX could not be executed by PyPy interpreters. More generally, any PEX created with interpreter constraints that did not specify the Python implementation, e.g.: ==3.8.*, were interpreted as being CPython specific, i.e.: CPython==3.8.*. This is now fixed, but if the intention of a constraint like ==3.8.* was in fact to restrict to CPython only, interpreter constraints need to say so now and use CPython==3.8.* explicitly.

    • Fix interpreter constraint parsing. (#1998)
    Source code(tar.gz)
    Source code(zip)
    pex(3.88 MB)
  • v2.1.116(Nov 25, 2022)

  • v2.1.115(Nov 24, 2022)


    2.1.115

    This release brings some attention to the pex3 lock export sub-command to make it more useful when inter-operating with pip-tools.

    • Sort requirements based on normalized project name when exporting (#1992)
    • Use raw version when exporting (#1990)
    Source code(tar.gz)
    Source code(zip)
    pex(3.87 MB)
  • v2.1.114(Nov 21, 2022)


    2.1.114

    This release brings two fixes for --venv mode PEXes.

    • Only insert "" to head of sys.path if a venv PEX runs in interpreter mode (#1984)
    • Map pex python path interpreter to realpath when creating venv dir hash. (#1972)
    Source code(tar.gz)
    Source code(zip)
    pex(3.87 MB)
  • v2.1.113(Nov 3, 2022)


    2.1.113

    This is a hotfix release that fixes errors installing wheels when there is high parallelism in execution of Pex processes. These issues were a regression introduced by #1961 included in the 2.1.112 release.

    • Restore AtomicDirectory non-locked good behavior. (#1974)
    Source code(tar.gz)
    Source code(zip)
    pex(3.87 MB)
  • v2.1.112(Oct 22, 2022)


    2.1.112

    This release brings support for the latest Pip release and includes some internal changes to help debug intermittent issues some users are seeing that implicate what may be file locking related bugs.

    • Add support for --pip-version 22.3. (#1953)
    Source code(tar.gz)
    Source code(zip)
    pex(3.87 MB)
  • v2.1.111(Oct 19, 2022)


    2.1.111

    This release fixes resolving requirements from a lock using arbitrary equality (===).

    In addition, you can now "inject" runtime environment variables and arguments into PEX files such that, when run, the PEX runtime ensures those environment variables and command line arguments are passed to the PEXed application. See PEX Recipes for more information.

    • Fix lock resolution to handle arbitrary equality. (#1951)
    • Support injecting args and env vars in a PEX. (#1948)
    Source code(tar.gz)
    Source code(zip)
    pex(3.87 MB)
  • v2.1.110(Oct 10, 2022)


    2.1.110

    This release fixes Pex runtime sys.path scrubbing for cases where Pex is not the main entry point. An important example of this is in Lambdex where the AWS Lambda Python runtime packages (boto3 and botocore) are leaked into the PEX runtime sys.path.

    • Fix sys.path scrubbing. (#1946)
    Source code(tar.gz)
    Source code(zip)
    pex(3.87 MB)
  • v2.1.109(Oct 6, 2022)


    2.1.109

    This release brings musllinux wheel support and a fix for a regression introduced in Pex 2.1.105 by #1902 that caused PEX_PATH= (an exported PEX_PATH with an empty string value) to raise an error in almost all use cases.

    • Vendor latest packaging; support musllinux wheels. (#1937)
    • Don't treat PEX_PATH= as . like other PATHS. (#1938)
    Source code(tar.gz)
    Source code(zip)
    pex(3.87 MB)
  • v2.1.108(Oct 4, 2022)

  • v2.1.107(Oct 1, 2022)

  • v2.1.106(Sep 30, 2022)

  • v2.1.105(Sep 20, 2022)


    2.1.105

    This is a fix release which addresses issues related to build time work_dir creation, virtualenv, and sh_boot support.

    In the unlikely event of a UUID collision in atomic workdir creation, pex could overwrite an existing directory and cause a corrupt state. When building a shell bootable --sh-boot pex the --runtime-pex-root was not always respected based on the condition of the build environment, and the value of the PEX_ROOT.

    • Fail on atomic_directory work_dir collision. (#1905)

    • Use raw_pex_root when constructing sh_boot pexes. (#1906)

    Docs.

    • Add support for offline downloads (#1898)
    Source code(tar.gz)
    Source code(zip)
    pex(3.65 MB)
  • v2.1.104(Aug 28, 2022)


    2.1.104

    This release brings a long-awaited upgrade of the Pip Pex uses, but behind a --pip-version 22.2.2 flag you must opt in to. Pex will then use that version of Pip if it can (your Pex operations target Python >=3.7) and warn and fall back to the older vendored Pip (20.3.4) if it can't. To turn the need to fallback to older Pip from a warning into a hard error you can also specify --no-allow-pip-version-fallback.

    The pex3 lock update command now gains the ability to update just the index and find links repos the lock's artifacts originate from by using a combination of --no-pypi, --index & --find-links along with --pin to ensure the project versions stay pinned as they are in the lockfile and just the repos they are downloaded from is altered. Consult the CLI --help for --fingerprint-mismatch {ignore,warn,error} to gain more control over repo migration behavior.

    There are several bug fixes as well dealing with somewhat esoteric corner cases involving changing a PEX --layout from one form to another and building artifacts using certain interpreters on macOS 11.0 (aka: 10.16).

    • Add support for Pip 22.2.2. (#1893)
    • Make lock update sensitive to artifacts. (#1887)
    • Ensure locally built wheel is consumable locally. (#1886)
    • Ensure --output always overwrites destination. (#1883)
    Source code(tar.gz)
    Source code(zip)
    pex(3.65 MB)
  • v2.1.103(Aug 5, 2022)


    2.1.103

    This release fixes things such that pex lockfiles can be created and updated using the Pex PEX when local projects are involved.

    • Fix pex3 lock ... when run from the Pex PEX. (#1874)
    Source code(tar.gz)
    Source code(zip)
    pex(3.63 MB)
  • v2.1.102(Jul 27, 2022)


    2.1.102

    This is a hotfix release that fixes a further corner missed by #1863 in the Pex 2.1.101 release whereby Pex would fail to install platform-specific packages on Red Hat based OSes.

    In addition, an old but only newly discovered bug in --inherit-path={prefer,fallback} handling is fixed. Previously only using PEX_INHERIT_PATH={prefer,fallback} at runtime worked properly.

    In the process of fixing the old --inherit-path={prefer,fallback} bug, also fix another old bug handling modern virtualenv venvs under Python 2.7 during zipapp execution mode PEX boots.

    • Fix wheel installs: account for purelib & platlib. (#1867)
    • Fix --inhert-path handling. (#1871)
    • Error using pex + virtualenv >=20.0.0 + python 2.7 (#992)
    Source code(tar.gz)
    Source code(zip)
    pex(3.63 MB)
  • v2.1.101(Jul 25, 2022)


    2.1.101

    This release fixes a corner-case revealed by python-certifi-win32 1.6.1 that was not previously handled when installing certain distributions.

    • Make wheel install site-packages detection robust. (#1863)
    Source code(tar.gz)
    Source code(zip)
    pex(3.63 MB)
  • v2.1.100(Jul 23, 2022)

  • v2.1.99(Jul 14, 2022)

  • v2.1.98(Jul 13, 2022)


    2.1.98

    This releases fixes regressions in foreign --platform handling and artifact downloading introduced by #1787 in Pex 2.1.91.

    In addition, PEXes can now be used as sys.path entries. Once on the sys.path, via PYTHONPATH or other means, the code in the PEX can be made importable by first importing __pex__ either as its own stand-alone import statement; e.g.: import __pex__; import psutil or as a prefix of the code to import from the PEX; e.g.: from __pex__ import psutil.

    • Tags should be patched for --platform. (#1846)
    • Add support for importing from PEXes. (#1845)
    • Fix artifact downloads for foreign platforms. #1851
    Source code(tar.gz)
    Source code(zip)
    pex(3.63 MB)
  • v2.1.97(Jul 12, 2022)


    2.1.97

    This release patches a hole left by #1828 in the Pex 2.1.95 release whereby, although you could run a PEX under a too-long PEX_ROOT you could not build a PEX under a tool-long PEX_ROOT.

    • Avoid ENOEXEC for Pex internal --venvs. (#1843)
    Source code(tar.gz)
    Source code(zip)
    pex(3.62 MB)
  • v2.1.96(Jul 12, 2022)


    2.1.96

    This is a hotfix release that fixes --venv mode PEX_EXTRA_SYS_PATH propogation introduced in Pex 2.1.95 to only apply to sys.executable and not other Pythons.

    • Fix --venv PEX PEX_EXTRA_SYS_PATH propagation. (#1837)
    Source code(tar.gz)
    Source code(zip)
    pex(3.62 MB)
  • v2.1.95(Jul 9, 2022)


    2.1.95

    This release brings two new pex3 lock features for --style universal locks.

    By default, universal locks are created to target all operating systems. This can cause problems when you only target a subset of operating systems and a lock transitive dependency that is conditional on an OS you do not target is not lockable. The new --target-system {linux,mac,windows} option allows you to restrict the set of targeted OSes to work around this sort of issue. Since PEX files currently only support running on Linux and Mac, specifying --target-system linux --target-system mac is a safe way to pre-emptively avoid these sorts of locking issues when creating a universal lock.

    Previously you could not specify the --platform\s or --complete-platform\s you would be using later to build PEXes with when creating a universal lock. You now can, and Pex will verify the universal lock can support all the specified platforms.

    As is usual there are also several bug fixes including properly propagating PEX_EXTRA_SYS_PATH additions to forked Python processes, fixing pex3 lock export to only attempt to export for the selected target and avoiding too long shebang errors for --venv mode PEXes in a robust way.

    • Fix PEX_EXTRA_SYS_PATH propagation. (#1832)
    • Fix pex3 lock export: re-use --lock resolver. (#1831)
    • Avoid ENOEXEC for --venv shebangs. (#1828)
    • Check lock can resolve platforms at creation time. (#1824)
    • Support restricting universal lock target os. (#1823)
    Source code(tar.gz)
    Source code(zip)
    pex(3.62 MB)
  • v2.1.94(Jul 1, 2022)

  • v2.1.93(Jun 21, 2022)


    2.1.93

    This release brings several new features in addition to bug fixes.

    When creating a PEX the entry point can now be any local python script by passing --exe path/to/python-script.

    The pex3 lock update command now supports a -dry-dun check mode that exits non-zero to indicate that a lock needs updating and the -p / --project targeted update arguments can now be new projects to attempt to add to the lock.

    On the bug fix front, traditional zipapp mode PEX files now properly scrub sys.displayhook and sys.excepthook and their teardown sequence has now been simplified fixing logging to stderr late in teardown.

    Finally, pex3 lock create now logs when requirement resolution is taking a long time to provide some sense of progress and suggest generic remedies and pex --lock now properly handles authentication.

    • Support adding new requirements in a lock update. (#1797)
    • Add pex3 lock update --dry-run check mode. (#1799)
    • Universal locks no longer record a platform_tag. (#1800)
    • Support python script file executable. (#1807)
    • Fix PEX scrubbing to account for sys.excepthook. (#1810)
    • Simplify PEX teardown / leave stderr in tact. (#1813)
    • Surface pip download logging. (#1808)
    • Use pip download instead or URLFetcher. (#1811)
    Source code(tar.gz)
    Source code(zip)
    pex(3.62 MB)
  • v2.1.92(May 31, 2022)

  • v2.1.91(May 30, 2022)


    2.1.91

    This release fixes --sh-boot mode PEXes to have an argv0 and exported PEX environment variable consistent with standard Python boot PEXes; namely the absolute path of the originally invoked PEX.

    • Fix --sh-boot argv0. (#1785)
    Source code(tar.gz)
    Source code(zip)
    pex(3.59 MB)
  • v2.1.90(May 25, 2022)


    2.1.90

    This release fixes Pex handling of sdists to be atomic and also fixes lock files to be emitted ending with a newline. In addition, many typos in Pex documentation were fixed in a contribution by Kian-Meng Ang.

    • Ensure Pip cache operations are atomic. (#1778)
    • Ensure that lockfiles end in newlines. (#1774)
    • Fix typos (#1773)
    Source code(tar.gz)
    Source code(zip)
    pex(3.58 MB)
Owner
Pants Build
Pants Build System
Pants Build
executable archive format

XAR XAR lets you package many files into a single self-contained executable file. This makes it easy to distribute and install. A .xar file is a read-

Facebook Incubator 1.5k Dec 29, 2022
debinstaller - A tool to install .deb files in any distro.

debinstaller A tool to install .deb files in any distro. Installation for debinstaller

Manoj Paramsetti 6 Nov 6, 2022
A modern Python application packaging and distribution tool

PyOxidizer PyOxidizer is a utility for producing binaries that embed Python. The over-arching goal of PyOxidizer is to make complex packaging and dist

Gregory Szorc 4.5k Jan 7, 2023
A library which implements low-level functions that relate to packaging and distribution of Python

What is it? Distlib is a library which implements low-level functions that relate to packaging and distribution of Python software. It is intended to

Python Packaging Authority 29 Oct 11, 2022
A tool used to obfuscate python scripts, bind obfuscated scripts to fixed machine or expire obfuscated scripts.

PyArmor Homepage (中文版网站) Documentation(中文版) PyArmor is a command line tool used to obfuscate python scripts, bind obfuscated scripts to fixed machine

Dashingsoft 1.9k Jan 1, 2023
tool for creating installers from conda packages

(conda) Constructor Description Constructor is a tool which allows constructing an installer for a collection of conda packages. It solves needed pack

Conda 386 Jan 4, 2023
py2app is a Python setuptools command which will allow you to make standalone Mac OS X application bundles and plugins from Python scripts.

py2app is a Python setuptools command which will allow you to make standalone Mac OS X application bundles and plugins from Python scripts. py2app is

Ronald Oussoren 222 Dec 30, 2022
Create standalone executables from Python scripts, with the same performance and is cross-platform.

About cx_Freeze cx_Freeze creates standalone executables from Python scripts, with the same performance, is cross-platform and should work on any plat

Marcelo Duarte 1k Jan 4, 2023
FreezeUI is a python package that creates applications using cx_freeze and GUI by converting .py to .exe .

FreezeUI is a python package use to create cx_Freeze setup files and run them to create applications and msi from python scripts (converts .py to .exe or .msi .

null 4 Aug 25, 2022
Psgcompiler A PySimpleGUI Application - Transform your Python programs in Windows, Mac, and Linux binary executables

psgcompiler A PySimpleGUI Application "Compile" your Python programs into an EXE for Windows, an APP for Mac, and a binary for Linux Installation Old-

PySimpleGUI 77 Jan 7, 2023
Auto locust load test config and worker distribution with Docker and GitHub Action

Auto locust load test config and worker distribution with Docker and GitHub Action Install Fork the repo and change the visibility option to private S

Márk Zsibók 1 Nov 24, 2021
Python virtualenvs in Debian packages

dh-virtualenv Contents Overview Presentations, Blogs & Other Resources Using dh-virtualenv How does it work? Running tests Building the package in a D

Spotify 1.5k Dec 16, 2022
Freeze (package) Python programs into stand-alone executables

PyInstaller Overview PyInstaller bundles a Python application and all its dependencies into a single package. The user can run the packaged app withou

PyInstaller 9.9k Jan 8, 2023
shiv is a command line utility for building fully self contained Python zipapps as outlined in PEP 441, but with all their dependencies included.

shiv shiv is a command line utility for building fully self-contained Python zipapps as outlined in PEP 441, but with all their dependencies included!

LinkedIn 1.5k Dec 28, 2022
Core utilities for Python packages

packaging Reusable core utilities for various Python Packaging interoperability specifications. This library provides utilities that implement the int

Python Packaging Authority 451 Jan 4, 2023
A distutils extension to create standalone Windows programs from Python code

py2exe for Python 3 py2exe is a distutils extension which allows to build standalone Windows executable programs (32-bit and 64-bit) from Python scrip

py2exe 526 Jan 4, 2023
Build Windows installers for Python applications

Pynsist is a tool to build Windows installers for your Python applications. The installers bundle Python itself, so you can distribute your applicatio

Thomas Kluyver 818 Jan 5, 2023
Subpar is a utility for creating self-contained python executables. It is designed to work well with Bazel.

Subpar Subpar is a utility for creating self-contained python executables. It is designed to work well with Bazel. Status Subpar is currently owned by

Google 550 Dec 27, 2022
Python Wheel Obfuscator

pywhlobf obfuscates your wheel distribution by compiling python source file to shared library.

Hunt Zhan 79 Dec 22, 2022