/!\ Breaks compatibility

overrides_dh_virtualenv:
    dh_virtualenv (options)

become

overrides_dh_virtualenv_build:
    dh_virtualenv_build (options for build)

overrides_dh_virtualenv_install:
    dh_virtualenv_install (options for install)

overrides_dh_virtualenv_clean:
    dh_virtualenv_clean (there is no option ;))

On a typical run:

• dh_virtualenv_build create a virtualenv in build_dir (default to: debian/dh_virtualenv) and a debian/dh_virtualenv.debhelper file to log build_dir path.
• dh_virtualenv_install copy file from build_dir to correct place in package
• dh_virtualenv_clean clean build_dir and log file.

Additional option: We can set destination dir in dh_virtualenv_install to install venv in another path in final package (default remain in /usr/share/python/PACKAGE)

Layout changes in the lib of virtualenvs (v16.4.0+) damage system files because symlinks are followed.

Right now, this is usually only triggered if, as part of your build process, you update virtualenv to latest (e.g. when building in Docker). See https://github.com/1and1/debianized-jupyterhub/blob/67faf583d0f20215c7d0e639efd399668659d513/Dockerfile.build#L85 for an example.

Symptoms: On package install, all symlinked folders in the stdlib get their contents replaced by symlinks to self, i.e.

…venv/…/importlib -> /usr/lib/python3.6/importlib/
/usr/lib/python3.6/importlib/abc.py -> /usr/lib/python3.6/importlib/abc.py  ← self-link

This happens on mere unpacking a package with new layout over an installed one with the old layout.

Hello,

Packaged python scripts get shebang interpreter #!python.

Invocation like this:

override_dh_virtualenv:
    dh_virtualenv --extra-pip-arg '--find-links=$(WHEELDIR)' \
                                --no-test

It seems that dh-virtualenv is not available on Ubuntu 20.04 LTS (focal)

https://packages.ubuntu.com/search?suite=default&section=all&arch=any&keywords=dh-virtualenv&searchon=names

Do you know why and if there is any alternative?

Thank you.

We're also running into the exact issue in #82.

Wondering if you'd be amenable to a patch which does something like this:

Use /usr/share/$exename for the install location -- that is if I'm using no arguments it'll still use /usr/share/python, but if I select (for example --python=/usr/bin/python3.5) it'll install into /usr/share/python3.5

As explained in this documentation: https://wiki.debian.org/IntroDebianPackaging:

"Architecture:"
tells which computer architectures the binary package is expected to work on: i386 for 32-bit Intel CPUs, amd64 for 64-bit, armel for ARM processors, and so on. Debian works on about a dozen computer architectures in total, so this architecture support is crucial. The "Architecture:" field can contain names of particular architectures, but usually it contains one of two special values.
"any"
(which we see in the example) means that the package can be built for any architecture. In other words, the code has been written portably, so it does not make too many assumptions about the hardware. However, the binary package will still need to be built for each architecture separately.
"all"
means that the same binary package will work on all architectures, without having to be built separately for each. For example, a package consisting only of shell scripts would be "all". Shell scripts work the same everywhere and not need to be compiled.

Hi, I started using github actions to build debian packages for flask applications. Previously we used to do this on jenkins. The problem is, every time the package is built, all the scripts that are located in /opt/venvs//bin now contain the line below as a shebang

#!bin/sh
'''exec' /home/runner/work/<package-name>/<package-name>/debian/<package-name>/opt/venvs/<package-name>/bin/python "$0" "$@"' 

eg. Alembic now contains the alembic script now looks like

#!/bin/sh
'''exec' /home/runner/work/<package-name>/<package-name>/debian/<packag-name>/opt/venvs/<package-name>/bin/python "$0" "$@"
' '''
# -*- coding: utf-8 -*-
import re
import sys
from alembic.config import main
if __name__ == '__main__':
    sys.argv[0] = re.sub(r'(-script\.pyw|\.exe)?$', '', sys.argv[0])
    sys.exit(main())

Where it previously looked like

#!/opt/venvs/<package-name>/bin/python
# -*- coding: utf-8 -*-
import re
import sys
from alembic.config import main
if __name__ == '__main__':
    sys.argv[0] = re.sub(r'(-script\.pyw|\.exe)?$', '', sys.argv[0])
    sys.exit(main())

Basically, the shebang appended to all scripts packaged in /opt/venvs/package-name/bin now contain the default path where the repository is cloned when running any github workflow. link to documentation about $GITHUB_WORKSPACE

Is there something i'm missing?

When trying to run dh-virtualenv version 1.0 in a clean pbuilder environment, it fails when trying to create a virtualenv, saying that the build directory is unavailable. I've checked to make sure that the directory exists, and it is there.

Here's the full error:

I: Extracting source
dpkg-source: warning: extracting unsigned source package (mycroft-core_0.7.20.dsc)
dpkg-source: info: extracting mycroft-core in mycroft-core-0.7.20
dpkg-source: info: unpacking mycroft-core_0.7.20.tar.gz
I: Building the package
I: Running cd /build/*/ && env PATH="/usr/sbin:/usr/bin:/sbin:/bin" HOME="/nonexistent" dpkg-buildpackage -us -uc -rfakeroot
dpkg-buildpackage: source package mycroft-core
dpkg-buildpackage: source version 0.7.20
dpkg-buildpackage: source distribution xenial
dpkg-buildpackage: source changed by Mycroft Devs <[email protected]>
 dpkg-source --before-build mycroft-core-0.7.20
dpkg-buildpackage: host architecture amd64
 fakeroot debian/rules clean
dh clean --buildsystem=dh_virtualenv --python=python2.7 --setuptools
   dh_testdir -O--buildsystem=dh_virtualenv -O--python=python2.7 -O--setuptools
   dh_auto_clean -O--buildsystem=dh_virtualenv -O--python=python2.7 -O--setuptools
   dh_clean -O--buildsystem=dh_virtualenv -O--python=python2.7 -O--setuptools
	rm -f debian/debhelper-build-stamp
	rm -f debian/mycroft-core.substvars
	rm -f debian/mycroft-core.*.debhelper
	rm -rf debian/mycroft-core/
	rm -rf debian/.debhelper/
	rm -f debian/*.debhelper.log
	rm -f debian/files
	find .  \( \( \
		\( -path .\*/.git -o -path .\*/.svn -o -path .\*/.bzr -o -path .\*/.hg -o -path .\*/CVS \) -prune -o -type f -a \
	        \( -name '#*#' -o -name '.*~' -o -name '*~' -o -name DEADJOE \
		 -o -name '*.orig' -o -name '*.rej' -o -name '*.bak' \
		 -o -name '.*.orig' -o -name .*.rej -o -name '.SUMS' \
		 -o -name TAGS -o \( -path '*/.deps/*' -a -name '*.P' \) \
		\) -exec rm -f {} + \) -o \
		\( -type d -a -name autom4te.cache -prune -exec rm -rf {} + \) \)
	rm -f *-stamp
 dpkg-source -b mycroft-core-0.7.20
dpkg-source: warning: no source format specified in debian/source/format, see dpkg-source(1)
dpkg-source: info: using source format '1.0'
dpkg-source: info: building mycroft-core in mycroft-core_0.7.20.tar.gz
dpkg-source: info: building mycroft-core in mycroft-core_0.7.20.dsc
 debian/rules build
dh build --buildsystem=dh_virtualenv --python=python2.7 --setuptools
   dh_testdir -O--buildsystem=dh_virtualenv -O--python=python2.7 -O--setuptools
   dh_update_autotools_config -O--buildsystem=dh_virtualenv -O--python=python2.7 -O--setuptools
   dh_auto_configure -O--buildsystem=dh_virtualenv -O--python=python2.7 -O--setuptools
	mkdir -p build/usr/lib/mycroft-core
   dh_auto_build -O--buildsystem=dh_virtualenv -O--python=python2.7 -O--setuptools
	cd build
	virtualenv --always-copy --no-download --extra-search-dir=/build/mycroft-core-\*/debian/pypi /build/mycroft-core-0.7.20/build/usr/lib/mycroft-core
dh_auto_build: virtualenv --always-copy --no-download --extra-search-dir=/build/mycroft-core-*/debian/pypi /build/mycroft-core-0.7.20/build/usr/lib/mycroft-core failed to to execute: No such file or directory
	cd /build/mycroft-core-0.7.20
debian/rules:11: recipe for target 'build' failed
make: *** [build] Error 2
dpkg-buildpackage: error: debian/rules build gave error exit status 2

The rules file I'm using is as follows:

#!/usr/bin/make -f

export DH_VERBOSE=1
#export PYBUILD_NAME=mycroft-core
export DH_VIRTUALENV_ARGUMENTS=--always-copy --no-download --extra-search-dir=/build/mycroft-core-*/debian/pypi
export DH_PIP_EXTRA_ARGS=--no-index --find-links=/build/mycroft-core-*/debian/pypi
#export DEB_BUILD_OPTIONS=nostrip
export DH_VIRTUALENV_INSTALL_ROOT=/usr/lib

%:
        dh $@ --buildsystem=dh_virtualenv --python=python2.7 --setuptools

override_dh_installinit:
        dh_installinit --name=mycroft-messagebus
        dh_installinit --name=mycroft-skills
        dh_installinit --name=mycroft-speech-client
        dh_installinit --name=mycroft-enclosure-client

override_dh_auto_install:
        dh_auto_install
        find debian/ -name '*.py[co]' -delete

override_dh_strip:
        dh_strip --exclude=_cffi_backend

And the control file is the following:

Source: mycroft-core
Section: misc
Priority: optional
Maintainer: Arron Atchison <[email protected]>
Build-Depends: debhelper (>=9), python, dh-virtualenv, python-dev, python-setuptools, libffi-dev, libssl-dev, pulseaudio, portaudio19-dev, swig
Standards-Version: 3.9.7

Package: mycroft-core
Priority: optional
Architecture: any
Depends: portaudio19-dev, libglib2.0-0, flac, espeak, mpg123, mimic, python, ${python:Depends}, ${shlibs:Depends}
Description: <description snipped>

I'm attempting to follow your tutorial with little success. So I wasn't entirely sure what's the best way to install dh-virtualenv first, so I've just installed it into my home directory:

┌─[~]
└─▪ pip install --user git+https://github.com/spotify/dh-virtualenv
Downloading/unpacking git+https://github.com/spotify/dh-virtualenv
  Cloning https://github.com/spotify/dh-virtualenv to /tmp/pip-i1waBN-build
  Running setup.py egg_info for package from git+https://github.com/spotify/dh-virtualenv

Installing collected packages: dh-virtualenv
  Running setup.py install for dh-virtualenv
    changing mode of build/scripts-2.7/dh_virtualenv from 664 to 775

    changing mode of /home/dgillies/.local/bin/dh_virtualenv to 775
Successfully installed dh-virtualenv
Cleaning up...

So then I create a directory and add my requirements.txt the contents of:

sentry

A debian directory in it with some basics:

─[python-virtualenv-sentry]
└─▪ ls debian/
changelog  compat  control  rules  source

debian/rules looks like as per the docs:

#!/usr/bin/make -f
%:
        dh $@ --with python-virtualenv

Attempt to build, but then get:

┌─[python-virtualenv-sentry]
└─▪ dpkg-buildpackage -uc -us
dpkg-buildpackage: export CFLAGS from dpkg-buildflags (origin: vendor): -g -O2 -fstack-protector --param=ssp-buffer-size=4 -Wformat -Wformat-security
dpkg-buildpackage: export CPPFLAGS from dpkg-buildflags (origin: vendor): -D_FORTIFY_SOURCE=2
dpkg-buildpackage: export CXXFLAGS from dpkg-buildflags (origin: vendor): -g -O2 -fstack-protector --param=ssp-buffer-size=4 -Wformat -Wformat-security
dpkg-buildpackage: export FFLAGS from dpkg-buildflags (origin: vendor): -g -O2
dpkg-buildpackage: export LDFLAGS from dpkg-buildflags (origin: vendor): -Wl,-Bsymbolic-functions -Wl,-z,relro
dpkg-buildpackage: source package python-virtualenv-sentry
dpkg-buildpackage: source version 6.3.0-1
dpkg-buildpackage: source changed by Dave <dg@foobarbash>
dpkg-buildpackage: host architecture amd64
 dpkg-source --before-build python-virtualenv-sentry
 fakeroot debian/rules clean
dh clean --with python-virtualenv
dh: unable to load addon python-virtualenv: Can't locate Debian/Debhelper/Sequence/python_virtualenv.pm in @INC (@INC contains: /etc/perl /usr/local/lib/perl/5.14.2 /usr/local/share/perl/5.14.2 /usr/lib/perl5 /usr/share/perl5 /usr/lib/perl/5.14 /usr/share/perl/5.14 /usr/local/lib/site_perl .) at (eval 3) line 2.
BEGIN failed--compilation aborted at (eval 3) line 2.

make: *** [clean] Error 2
dpkg-buildpackage: error: fakeroot debian/rules clean gave error exit status 2

I am trying to debianize a python package, containing a few non pure-python libraries. I was excluding them in the debian/rules:

override_dh_shlibdeps:
	dh_shlibdeps -X/x86/ -X/numpy/.libs -X/cv2/.libs -X/PIL/.libs

up until dpkg-buildpackage was able to create a .deb file.

However, after installation, calling the package's entry point results in an error:

Traceback (most recent call last):
  [ ... ]
    import cv2
  File "/opt/venvs/py3software/lib/python3.6/site-packages/cv2/__init__.py", line 3, in <module>
    from .cv2 import *
ImportError: /opt/venvs/py3software/lib/python3.6/site-packages/cv2/cv2.cpython-36m-x86_64-linux-gnu.so: ELF load command address/offset not properly aligned.

I have read through this numpy issue mentioning similar problem, but I am still very lost.

Am I doing it the wrong way?

We'd love to be able to leverage these tools when building dh-virtualenv packages:

• pip-faster
• pip-custom-platform

We currently have no way of doing so with dh-virtualenv.

Ideally (?) we could do something like this:

override_dh_virtualenv:
    dh_virtualenv --preinstall venv-update==1.1 --pip-tool 'pip-faster --upgrade'

CC @chriskuehl

According to the documentation, extra arguments to pip can be specified using DH_PIP_EXTRA_ARGS or --extra-pip-arg. DH_PIP_EXTRA_ARGS doesn't seem to be working. Reading the code I see DH_PIP_EXTRA_ARGS being used in the perl code, but it is the python code that calls pip and that only supports --extra-pip-arg.

Hi,

when calling dpkg-buildpackage -us -uc -b, it fails with

...
Successfully built ad-tools
sed: can't read debian/mypackage/opt/venvs/mypackage/bin/rst2xetex.py: No such file or directory
Traceback (most recent call last):
  File "/usr/bin/dh_virtualenv", line 111, in <module>
    sys.exit(main() or 0)
  File "/usr/bin/dh_virtualenv", line 103, in main
    deploy.fix_shebangs()
  File "/usr/lib/python3/dist-packages/dh_virtualenv/deployment.py", line 234, in fix_shebangs
    subprocess.check_call(['sed', '-i', regex, f])
  File "/usr/lib/python3.8/subprocess.py", line 364, in check_call
    raise CalledProcessError(retcode, cmd)

here's the content of my rules

#!/usr/bin/make -f

%:
	dh $@ --with python-virtualenv

and control

Source: mypackage
Section: python
Priority: extra
Maintainer: None <None>
Build-Depends: debhelper (>= 9), python, dh-virtualenv (>= 0.7)
Standards-Version: 3.9.5

Package: mypackage
Architecture: any
Pre-Depends: dpkg (>= 1.16.1), python2.7-minimal | python2.6-minimal, ${misc:Pre-Depends}
Depends: ${python:Depends}, ${misc:Depends}
Description: mydescr
 mydescr

Environment:

Ubuntu 20.04 Debian dpkg-buildpackage version 1.19.7. dh_virtualenv 1.2.2

Using dh-virtualenv to install a packages, command line entrypoint are properly created into the virtual env as /opt/venv/bin/myscript. For a user, it's possible to call then directly. But when doing so, the executable get started without the appropriate modification of PATH and PYTHONHOME. This can cause a problem for some project that expect to call executable. Someone would expect the executable to be loaded from the same virtualenv. But since PATH is not properly define to use the virtualenv, either the executable is not found or the wrong version is loaded from the operating system.

Expected behaviour. In addition to shebang update, I would expect dh-virtualenv to update the script to define the PATH.

I would recommand replacing the shebang by something similar to

env PATH=/opt/venv/bin:$PATH python

Allow to specify dh-virtualenv options per binary package like so (debian/control):

Package: foo
Conflicts: foo-http
[...]
Package: foo-http
Conflicts: foo
X-Dh-Virtualenv-Extras: http
X-Dh-Virtualenv-Install-Suffix: foo

With the following in setup.cfg:

[options.extras_require]
http = aiohttp ~= 3.8

I wanted to build two debian packages with different extras and did not find an easier way. If this method seems reasonable and gets accepted, I will write and push documentation and some tests.

(cherry picked from commit a43d163901aeeb70940755867056ff6a2a4d0eab)

Raising this as an issue since it prevent any useful usage of dh-virtualenv.

dh-virtualenv pacakge is available in Debian buster and bullsyeye, but is not available in Debian Bookworm.

https://packages.debian.org/source/bullseye/dh-virtualenv

Seams to be related to this bug: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1017046

When I try to use the newest version of Celery celery==5.2.3 I have compilation errors, like:

Installing collected packages: vine, amqp, appdirs, six, python-dateutil, arrow, asgiref, async-timeout, attrs, pytz, babel, soupsieve, beautifulsoup4, billiard, urllib3, jmespath, botocore, s3transfer, boto3, brotli, cached-property, click, click-didyoumean, wcwidth, prompt-toolkit, click-repl, click-plugins, kombu, setuptools, celery, certifi, pycparser, cffi, chardet, webencodings, tinycss2, cssselect2, defusedxml, wrapt, deprecated, sqlparse, django, django-admin-sortable2, django-appconf, django-celery-email, django-dbbackup, django-debug-toolbar, django-extensions, django-flatpickr, django-formtools, hashids, django-hashid-field, markdown, pillow, django-markdownx, django-modeltranslation, py-moneyed, django-money, python-monkey-business, django-nested-admin, pyparsing, packaging, typing-extensions, redis, django-redis, django-s3direct, django-storages, text-unidecode, python-slugify, django-uuslug, django-widget-tweaks, zopfli, fonttools, gunicorn, html5lib, idna, isodate, lxml, psycopg2, pydyf, pygments, pyphen, python-stdnum, requests, requests-toolbelt, zeep, regonapi, sorl-thumbnail, unidecode, weasyprint
  Attempting uninstall: setuptools
    Found existing installation: setuptools 44.0.0
    Uninstalling setuptools-44.0.0:
      Successfully uninstalled setuptools-44.0.0
ERROR: Could not install packages due to an EnvironmentError: [Errno 39] Directory not empty: 'extern'

Traceback (most recent call last):
  File "/usr/bin/dh_virtualenv", line 114, in <module>
    sys.exit(main() or 0)
  File "/usr/bin/dh_virtualenv", line 94, in main
    deploy.install_dependencies()
  File "/usr/lib/python2.7/dist-packages/dh_virtualenv/deployment.py", line 202, in install_dependencies
    subprocess.check_call(self.pip('-r', requirements_path))
  File "/usr/lib/python2.7/subprocess.py", line 190, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['/bin/python', '/bin/pip', 'install', '--log=/tmp/tmp0OTAte', '-r', './requirements.txt']' returned non-zero exit status 1
make[1]: *** [debian/rules:22: override_dh_virtualenv] Error 1
make[1]: Leaving directory '/srv/'
make: *** [debian/rules:9: binary] Error 2
dpkg-buildpackage: error: fakeroot debian/rules binary subprocess returned exit status 2

Any chance to fix this? My dh-virtualenv is 1.2.