Official project repository for the Setuptools build system

Overview
tests Code style: Black https://img.shields.io/codecov/c/github/pypa/setuptools/master.svg?logo=codecov&logoColor=white https://tidelift.com/badges/github/pypa/setuptools?style=flat

See the Installation Instructions in the Python Packaging User's Guide for instructions on installing, upgrading, and uninstalling Setuptools.

Questions and comments should be directed to the distutils-sig mailing list. Bug reports and especially tested patches may be submitted directly to the bug tracker.

Code of Conduct

Everyone interacting in the setuptools project's codebases, issue trackers, chat rooms, and mailing lists is expected to follow the PSF Code of Conduct.

For Enterprise

Available as part of the Tidelift Subscription.

Setuptools and the maintainers of thousands of other packages are working with Tidelift to deliver one enterprise subscription that covers all of the open source you use.

Learn more.

Security Contact

To report a security vulnerability, please use the Tidelift security contact. Tidelift will coordinate the fix and disclosure.

Comments
  • Avoid full path enumeration on import of setuptools or pkg_resources?

    Avoid full path enumeration on import of setuptools or pkg_resources?

    Originally reported by: konstantint (Bitbucket: konstantint, GitHub: konstantint)


    At the moment on my machine, it takes about 1.28 seconds to do a bare import pkg_resources, 1.47 seconds to do a bare import setuptools, 1.36 seconds to do a bare from pkg_resources import load_entry_point and 1.25 seconds to do a bare from pkg_resources import load_entry_point.

    This obviously affects all of the python scripts that are installed as console entry points, because each and every one of them starts with a line like that. In code which does not rely on entry points this may be a problem whenever I want to use resource_filename to consistently access static data.

    I believe this problem is decently common, yet I did not find any issue or discussion, hence I'm creating one, hoping I'm not repeating what has been said already elsewhere unnecessarily.

    I am using Anaconda Python, which comes along with a fairly large package, alongside several of my own packages, which I commonly add my path via setup.py develop, however I do not believe this setup is anything out of the ordinary. There are 37 items on my sys.path at the moment. Profiling import pkg_resources shows that this leads to 76 calls to workingset.add_entry (timing at about a second), of which most of the time is spent in 466 calls to Distribution.from_location.

    Obviously, the reason for the problem lies in the two _call_aside methods at the end of pkg_resources which lead to a full scan of the python path at the moment when the package is imported, and the only way to alleviate it would be to somehow avoid or delay the need for this scan as much as possible.

    I see two straightforward remedies: a) Make the scanning lazy. After all, if all one needs is to find a particular package, the scan could stop as soon as the corresponding package is located. At the very least this would allow me to "fix" my ipython loading problem by moving it up in the path. This might break some import rules which do not respect the precedence of the path, which I'm not aware. b) Cache a precomputed index and update it lazily. Yes, this might requre some ad-hoc rules for resolving inconsistencies, and this may lead to ugly conflicts with external tools that attempt to install multiple versions of a package, but this will basically avoid the current startup delay in 99.99% of cases and solve so much of my problems, that I'd be willing to pay the price.

    Although both options might seem somewhat controversial, the problem itself seems to be serious enough to deserve at least some fix eventually (for example, I've recently discovered I'm reluctant to start ipython for short calculations because of its startup delay which I've now tracked back to this same issue).

    I'm contemplating making a separate utility, e.g. fast_pkg_resources, which would implement the strategy b) by simply caching calls to pkg_resources in an external file, yet I thought of raising the issue here to figure out whether someone has already addressed it, whether there are plans to do something about it in the setuptools core codebase, or perhaps I'm missing something obvious.


    • Bitbucket: https://bitbucket.org/pypa/setuptools/issue/510
    enhancement wontfix 
    opened by ghost 99
  • find_packages() doesn't find PEP 420 packages

    find_packages() doesn't find PEP 420 packages

    Originally reported by: gwideman (Bitbucket: gwideman, GitHub: gwideman)


    #!python
    
    setup(...
        packages=find_packages(..)
        )
    

    on the developer machine will fail to find packages that lack a __init__.py file, as is allowed in Python 3.3. However, such packages listed explicitly: packages=['mypkg'] do appear to get included and later installed.

    Note: When testing this, before each test be sure to delete all generated metadata, including that which setup may previously have placed in the original source directory, as it seems that setup may use metadata created on a previous run in order to include files.

    This is part of a general problem reported in issue #83, but I've logged it separately as it's specifically about setuptools.


    • Bitbucket: https://bitbucket.org/pypa/setuptools/issue/97
    bug help wanted major 
    opened by ghost 95
  • How to bootstrap setuptools from source

    How to bootstrap setuptools from source

    Hi, I'm a developer for the Spack package manager. We've had a setuptools package that has worked fine for a while now, and the vast majority of our Python packages depend on it. However, I went to update to the latest version of setuptools and noticed that it now depends on six, packaging, and appdirs. Unfortunately all 3 of these packages also depend on setuptools. six and packaging can fall back on distutils.core if necessary, but the setup.py for appdirs has a hard setuptools import.

    It seems to me like it is no longer possible to build setuptools from source. Is this correct??

    opened by adamjstewart 73
  • _NamespacePath object has no attribute sort (31.0.0)

    _NamespacePath object has no attribute sort (31.0.0)

    When pip installing a package that shares a namespace it fails out on 31.0.0 but installs fine on 28.8.0

        Traceback (most recent call last):
          File "<string>", line 1, in <module>
          File "/usr/lib/python3.5/site-packages/setuptools/__init__.py", line 10, in <module>
            from setuptools.extern.six.moves import filter, filterfalse, map
          File "/usr/lib/python3.5/site-packages/setuptools/extern/__init__.py", line 1, in <module>
            from pkg_resources.extern import VendorImporter
          File "/usr/lib/python3.5/site-packages/pkg_resources/__init__.py", line 3015, in <module>
            @_call_aside
          File "/usr/lib/python3.5/site-packages/pkg_resources/__init__.py", line 2999, in _call_aside
            f(*args, **kwargs)
          File "/usr/lib/python3.5/site-packages/pkg_resources/__init__.py", line 3043, in _initialize_master_working_set
            for dist in working_set
          File "/usr/lib/python3.5/site-packages/pkg_resources/__init__.py", line 3043, in <genexpr>
            for dist in working_set
          File "/usr/lib/python3.5/site-packages/pkg_resources/__init__.py", line 2573, in activate
            declare_namespace(pkg)
          File "/usr/lib/python3.5/site-packages/pkg_resources/__init__.py", line 2147, in declare_namespace
            _handle_ns(packageName, path_item)
          File "/usr/lib/python3.5/site-packages/pkg_resources/__init__.py", line 2087, in _handle_ns
            _rebuild_mod_path(path, packageName, module)
          File "/usr/lib/python3.5/site-packages/pkg_resources/__init__.py", line 2116, in _rebuild_mod_path
            orig_path.sort(key=position_in_sys_path)
        AttributeError: '_NamespacePath' object has no attribute 'sort'
    
    opened by jkbbwr 73
  • Discussion: support for pyproject.toml configuration

    Discussion: support for pyproject.toml configuration

    Over the last few months, I've been using setup.cfg more, and I find the syntax and types and such to be super counter-intuitive. I don't find TOML to be a breeze per se, but it's at least a lot clearer to me how to specify a string, a list, etc. We also hear a lot of cries to move towards supporting pyproject.toml as the "one file" for setup configuration, in favor of setup.cfg. I'm not terribly opposed to this, but I want to open up the discussion for a few options:

    1. Add pyproject.toml as the one true way to do declarative builds, moving all setuptools configuration over there and deprecating setup.cfg (partially what is discussed in #1160, and if we decide on that option we can move over to that issue).
    2. Add pyproject.toml as a second supported way to do a declarative configuration, alongside setup.cfg.
    3. Stay the course with setup.cfg indefinitely.

    I am deeply concerned with the large amount of churn that we're introducing into the Python packaging space, so I don't want to make a lot of changes for the sake of changes, but I do think that at least having the option to use a pyproject.toml would be a real benefit, so I'm basically in favor of option 2.

    I think the biggest downside of option 2 is that it means maintaining, testing and documenting three separate locations for all of our options - how to do it in setup.cfg, how to do it in pyproject.toml, and how to do it in setup.py. I think the hard part is going from 1 way to do everything to 2 ways. Our documentation is a mess anyway, but my beautiful vision for the future is that all our examples have "tabs" that allow you to toggle them between setup.cfg and setup.py. Adding a pyproject.toml in there would probably not be the worst thing.

    enhancement help wanted major Needs Implementation 
    opened by pganssle 64
  • Invalid or non-existent authentication information

    Invalid or non-existent authentication information

    When I run python setup.py sdist upload

    I always get:

    Submitting dist/xxxxxxx-0.1.4.tar.gz to https://upload.pypi.org/legacy/
    Upload failed (403): Invalid or non-existent authentication information.
    error: Upload failed (403): Invalid or non-existent authentication information.
    

    It never asks me to enter my username but only password. So I assume the username is the same as when I run "register".

    I am pretty sure that password I entered is correct because I can login https://pypi.python.org/pypi

    Could you please help? I am not new to pypi. I published several packages before. But now I am confused. What has changed recently?

    opened by tylerlong 50
  • 20.5.0+ breaks existing setup.py files - UndefinedEnvironmentName

    20.5.0+ breaks existing setup.py files - UndefinedEnvironmentName

    The latest 20.6.4 PyPI is breaking our setup.py files with the following exception:

    Traceback (most recent call last):
      File "setup.py", line 163, in <module>
        cmdclass={'test': PyTest},
      File "/opt/python/2.7.9/lib/python2.7/distutils/core.py", line 151, in setup
        dist.run_commands()
      File "/opt/python/2.7.9/lib/python2.7/distutils/dist.py", line 953, in run_commands
        self.run_command(cmd)
      File "/opt/python/2.7.9/lib/python2.7/distutils/dist.py", line 972, in run_command
        cmd_obj.run()
      File "/home/travis/virtualenv/python2.7.9/lib/python2.7/site-packages/setuptools/command/test.py", line 152, in run
        self.distribution.fetch_build_eggs(self.distribution.tests_require)
      File "/home/travis/virtualenv/python2.7.9/lib/python2.7/site-packages/setuptools/dist.py", line 313, in fetch_build_eggs
        replace_conflicting=True,
      File "/home/travis/virtualenv/python2.7.9/lib/python2.7/site-packages/pkg_resources/__init__.py", line 814, in resolve
        if req.marker and not req.marker.evaluate():
      File "/home/travis/virtualenv/python2.7.9/lib/python2.7/site-packages/pkg_resources/_vendor/packaging/markers.py", line 273, in evaluate
        return _evaluate_markers(self._markers, current_environment)
      File "/home/travis/virtualenv/python2.7.9/lib/python2.7/site-packages/pkg_resources/_vendor/packaging/markers.py", line 198, in _evaluate_markers
        lhs_value = _get_env(environment, lhs.value)
      File "/home/travis/virtualenv/python2.7.9/lib/python2.7/site-packages/pkg_resources/_vendor/packaging/markers.py", line 180, in _get_env
        "{0!r} does not exist in evaluation environment.".format(name)
    pkg_resources._vendor.packaging.markers.UndefinedEnvironmentName: 'extra' does not exist in evaluation environment.
    
    bug 
    opened by lnielsen 50
  • Newlines in the `description` field produce a malformed PKG-INFO

    Newlines in the `description` field produce a malformed PKG-INFO

    We discovered this accidentally by way of https://github.com/zopefoundation/zc.relation/issues/4#issuecomment-397532224: if you pass a string containing newlines to the description argument of setup(), setuptools will generate a malformed PKG-INFO.

    To reproduce:

    # setup.py
    from setuptools import setup
    setup(
        name='test-package',
        version='0.1',
        author='Blah Blah',
        author_email='[email protected]',
        description='description\n\n',
        py_modules=['blah'],
    )
    

    (The contents of blah.py do not matter, but the file should exist.)

    Run python setup.py sdist and the inspect test_package.egg-info/PKG-INFO. For me, with setuptools 39.1.0, it looks like this:

    Metadata-Version: 1.0
    Name: test-package
    Version: 0.1
    Summary: description
    
    
    Home-page: UNKNOWN
    Author: Blah Blah
    Author-email: [email protected]
    License: UNKNOWN
    Description: UNKNOWN
    Platform: UNKNOWN
    

    The extra newlines lead tools to treat the rest of the PKG-INFO as a long_description.

    I would expect setuptools to complain about the newlines in the description field, or at least escape them properly (i.e. prepend whitespace, like it does for the long_description field).

    bug help wanted Needs Implementation 
    opened by mgedmin 42
  • Bootstrap script unable to locate releases following 20.10.0 release

    Bootstrap script unable to locate releases following 20.10.0 release

    How to test:

    Download

    https://bootstrap.pypa.io/bootstrap-buildout.py

    At the time of this ticket, __version__ of this file is 2015-07-01.

    Run

    python bootstrap-buildout.py

    You get

    Downloading https://pypi.python.org/packages/source/s/setuptools/setuptools-20.10.1.zip
    Extracting in /tmp/tmpIKldPD
    Traceback (most recent call last):
      File "bootstrap-buildout.py", line 117, in <module>
        ez['use_setuptools'](**setup_args)
      File "<string>", line 173, in use_setuptools
      File "<string>", line 124, in _do_download
      File "<string>", line 64, in _build_egg
      File "/usr/lib/python2.7/contextlib.py", line 17, in __enter__
        return self.gen.next()
      File "<string>", line 103, in archive_context
      File "/usr/lib/python2.7/zipfile.py", line 714, in __init__
        self._GetContents()
      File "/usr/lib/python2.7/zipfile.py", line 748, in _GetContents
        self._RealGetContents()
      File "/usr/lib/python2.7/zipfile.py", line 763, in _RealGetContents
        raise BadZipfile, "File is not a zip file"
    zipfile.BadZipfile: File is not a zip file
    

    You get the same error with 20.10.0.

    python bootstrap-buildout.py --setuptools-version=20.10.0

    Downloading https://pypi.python.org/packages/source/s/setuptools/setuptools-20.10.0.zip
    Extracting in /tmp/tmp6G6ZVT
    Traceback (most recent call last):
      File "bootstrap-buildout.py", line 117, in <module>
        ez['use_setuptools'](**setup_args)
      File "<string>", line 173, in use_setuptools
      File "<string>", line 124, in _do_download
      File "<string>", line 64, in _build_egg
      File "/usr/lib/python2.7/contextlib.py", line 17, in __enter__
        return self.gen.next()
      File "<string>", line 103, in archive_context
      File "/usr/lib/python2.7/zipfile.py", line 714, in __init__
        self._GetContents()
      File "/usr/lib/python2.7/zipfile.py", line 748, in _GetContents
        self._RealGetContents()
      File "/usr/lib/python2.7/zipfile.py", line 763, in _RealGetContents
        raise BadZipfile, "File is not a zip file"
    zipfile.BadZipfile: File is not a zip file
    

    20.9.0 works fine, so this bug was introduced in 20.10.x.

    python bootstrap-buildout.py --setuptools-version=20.9.0

    Downloading https://pypi.python.org/packages/source/s/setuptools/setuptools-20.9.0.zip
    Extracting in /tmp/tmpXrDIjV
    Now working in /tmp/tmpXrDIjV/setuptools-20.9.0
    Building a Setuptools egg in /tmp/bootstrap-OVHpkN
    warning: no files found matching '*' under directory 'setuptools/_vendor'
    warning: no files found matching '*.txt'
    /tmp/bootstrap-OVHpkN/setuptools-20.9.0-py2.7.egg
    
    opened by idgserpro 42
  • Support pyproject.toml-style configuration (PEP 621) - Round 2

    Support pyproject.toml-style configuration (PEP 621) - Round 2

    Summary of changes

    Based on the experience acquired in #2924, I created this PR as a simplification (please check the discussion there for details). There are many individual steps, but the following image illustrates how they are organised in 4 main groups of stacked changes:

    a) https://github.com/abravalheri/setuptools/compare/1a1397a4...4a986086: Update build_meta to use dist objects instead of simply running setup.py b) https://github.com/abravalheri/setuptools/compare/4a986086...7fe7a419: Separate setup.cfg parsing and extract common post-processing functions

    (new vendorised packages)

    c) https://github.com/abravalheri/setuptools/compare/fab3b790...56665563: Add means to apply configuration from pyproject.toml files d) https://github.com/abravalheri/setuptools/compare/7fe7a419...d34794d7: Integrate pyproject.toml configuration into existing classes

    Finally there is a completely non-required group of changes that only adds some type hints to improve (my) level of confidence in the implementation of the setuptools.config sub-package: e) https://github.com/abravalheri/setuptools/compare/d34794d7...0aff4ea4: Add some type hints to the setuptools.config subpackage

    image

    Unfortunately GitHub does not provide tools to facilitate the review of stacked changes/PRs, but I am open to suggestions on how to facilitate the review. I hope that this organisation in groups makes sense and help in the process.

    I also have another branch with a proof-of-concept where I remove setuptools' own setup.cfg and add all the existing configurations in the pyproject.toml.

    Closes #1688, #2671 (mostly... see discussion bellow).

    This PR adds 2 vendorised dependencies: tomli and validate-pyproject.

    Pull Request Checklist

    opened by abravalheri 41
  • setuptools pkg_resources fails to detect version properly

    setuptools pkg_resources fails to detect version properly

    Originally reported by: Eric89GXL (Bitbucket: Eric89GXL, GitHub: Eric89GXL)


    When installing a package:

      File "/home/larsoner/.local/lib/python2.7/site-packages/setuptools/dist.py", line 268, in __init__
        self.fetch_build_eggs(attrs['setup_requires'])
      File "/home/larsoner/.local/lib/python2.7/site-packages/setuptools/dist.py", line 313, in fetch_build_eggs
        replace_conflicting=True,
      File "/home/larsoner/.local/lib/python2.7/site-packages/pkg_resources/__init__.py", line 836, in resolve
        dist = best[req.key] = env.best_match(req, ws, installer)
      File "/home/larsoner/.local/lib/python2.7/site-packages/pkg_resources/__init__.py", line 1074, in best_match
        dist = working_set.find(req)
      File "/home/larsoner/.local/lib/python2.7/site-packages/pkg_resources/__init__.py", line 711, in find
        raise VersionConflict(dist, req)
    pkg_resources.VersionConflict: (numpy 1.11.0.dev0-2329eae (/home/larsoner/.local/lib/python2.7/site-packages), Requirement.parse('numpy>=1.6.1'))
    

    In other words, my numpy 1.11.0.dev0 is not seen as satisfying a >= 1.6.1 version requirement. It seems like using disutils.version.LooseVersion or so should solve this. I'm happy to look into making a PR if this seems like the right solution.


    • Bitbucket: https://bitbucket.org/pypa/setuptools/issue/419
    bug major 
    opened by ghost 41
  • [BUG] If you are seeing this error, please report to

    [BUG] If you are seeing this error, please report to

    setuptools version

    65.3.0

    Python version

    3.10.6

    OS

    M1 Mac

    Additional environment information

    No response

    Description

    Tried installing onnx using https://github.com/onnx/onnx#mac.

    Ran into an error asking to report the issue here. See full output below.

    SETUPTOOLS_ENABLE_FEATURES="legacy-editable" works.

    Let me know if you anything else.

    Expected behavior

    How to Reproduce

    see above

    Output

          Consolidate compiler generated dependencies of target onnx_cpp2py_export
          [100%] Built target onnx_cpp2py_export
          running build_ext
          copying somewhere/.setuptools-cmake-build/onnx_cpp2py_export.cpython-310-darwin.so -> /private/var/folders/yz/5_6_nfl928v85_5lk2gqpctm0000gn/T/tmpmzrmzw7l.build-lib/onnx
          Traceback (most recent call last):
            File "/private/var/folders/yz/5_6_nfl928v85_5lk2gqpctm0000gn/T/pip-build-env-kv9wckuz/overlay/lib/python3.10/site-packages/setuptools/_distutils/file_util.py", line 40, in _copy_file_contents
              fdst = open(dst, 'wb')
          FileNotFoundError: [Errno 2] No such file or directory: '/private/var/folders/yz/5_6_nfl928v85_5lk2gqpctm0000gn/T/tmpmzrmzw7l.build-lib/onnx/onnx_cpp2py_export.cpython-310-darwin.so'
          
          During handling of the above exception, another exception occurred:
          
          Traceback (most recent call last):
            File "/private/var/folders/yz/5_6_nfl928v85_5lk2gqpctm0000gn/T/pip-build-env-kv9wckuz/overlay/lib/python3.10/site-packages/setuptools/command/editable_wheel.py", line 140, in run
              self._create_wheel_file(bdist_wheel)
            File "/private/var/folders/yz/5_6_nfl928v85_5lk2gqpctm0000gn/T/pip-build-env-kv9wckuz/overlay/lib/python3.10/site-packages/setuptools/command/editable_wheel.py", line 330, in _create_wheel_file
              files, mapping = self._run_build_commands(dist_name, unpacked, lib, tmp)
            File "/private/var/folders/yz/5_6_nfl928v85_5lk2gqpctm0000gn/T/pip-build-env-kv9wckuz/overlay/lib/python3.10/site-packages/setuptools/command/editable_wheel.py", line 261, in _run_build_commands
              self._run_build_subcommands()
            File "/private/var/folders/yz/5_6_nfl928v85_5lk2gqpctm0000gn/T/pip-build-env-kv9wckuz/overlay/lib/python3.10/site-packages/setuptools/command/editable_wheel.py", line 288, in _run_build_subcommands
              self.run_command(name)
            File "/private/var/folders/yz/5_6_nfl928v85_5lk2gqpctm0000gn/T/pip-build-env-kv9wckuz/overlay/lib/python3.10/site-packages/setuptools/_distutils/cmd.py", line 319, in run_command
              self.distribution.run_command(command)
            File "/private/var/folders/yz/5_6_nfl928v85_5lk2gqpctm0000gn/T/pip-build-env-kv9wckuz/overlay/lib/python3.10/site-packages/setuptools/dist.py", line 1217, in run_command
              super().run_command(command)
            File "/private/var/folders/yz/5_6_nfl928v85_5lk2gqpctm0000gn/T/pip-build-env-kv9wckuz/overlay/lib/python3.10/site-packages/setuptools/_distutils/dist.py", line 992, in run_command
              cmd_obj.run()
            File "<string>", line 255, in run
            File "/private/var/folders/yz/5_6_nfl928v85_5lk2gqpctm0000gn/T/pip-build-env-kv9wckuz/overlay/lib/python3.10/site-packages/setuptools/command/build_ext.py", line 84, in run
              _build_ext.run(self)
            File "/private/var/folders/yz/5_6_nfl928v85_5lk2gqpctm0000gn/T/pip-build-env-kv9wckuz/overlay/lib/python3.10/site-packages/setuptools/_distutils/command/build_ext.py", line 346, in run
              self.build_extensions()
            File "<string>", line 272, in build_extensions
            File "/private/var/folders/yz/5_6_nfl928v85_5lk2gqpctm0000gn/T/pip-build-env-kv9wckuz/overlay/lib/python3.10/site-packages/setuptools/_distutils/cmd.py", line 351, in copy_file
              return file_util.copy_file(
            File "/private/var/folders/yz/5_6_nfl928v85_5lk2gqpctm0000gn/T/pip-build-env-kv9wckuz/overlay/lib/python3.10/site-packages/setuptools/_distutils/file_util.py", line 163, in copy_file
              _copy_file_contents(src, dst)
            File "/private/var/folders/yz/5_6_nfl928v85_5lk2gqpctm0000gn/T/pip-build-env-kv9wckuz/overlay/lib/python3.10/site-packages/setuptools/_distutils/file_util.py", line 42, in _copy_file_contents
              raise DistutilsFileError(
          distutils.errors.DistutilsFileError: could not create '/private/var/folders/yz/5_6_nfl928v85_5lk2gqpctm0000gn/T/tmpmzrmzw7l.build-lib/onnx/onnx_cpp2py_export.cpython-310-darwin.so': No such file or directory
          error: Support for editable installs via PEP 660 was recently introduced
          in `setuptools`. If you are seeing this error, please report to:
          
          https://github.com/pypa/setuptools/issues
          
          Meanwhile you can try the legacy behavior by setting an
          environment variable and trying to install again:
          
          SETUPTOOLS_ENABLE_FEATURES="legacy-editable"
          [end of output]
      
      note: This error originates from a subprocess, and is likely not a problem with pip.
      ERROR: Failed building editable for onnx
    Failed to build onnx
    ERROR: Could not build wheels for onnx, which is required to install pyproject.toml-based projects
    
    bug Needs Triage 
    opened by rickwierenga 0
  • [BUG] Editable install fails after introducing pyproject.toml

    [BUG] Editable install fails after introducing pyproject.toml

    setuptools version

    65.3.0

    Python version

    3.9

    OS

    Windows

    Additional environment information

    No response

    Description

    Thank you for maintaining setuptools! Recently we saw pip install -e . started to fail after adding pyproject.toml in the package. Not sure whether it is a bug or we need to set something else properly in pyproject.toml. Provide some findings here:

    • pip install -e . will fail due to missing .pyd file (FileNotFoundError: [Errno 2] No such file or directory: 'C:\Users\jacky\AppData\Local\Temp\tmpba9gls20.build-lib\onnx\onnx_cpp2py_export.cp39-win_amd64.pyd')
    • set SETUPTOOLS_ENABLE_FEATURES="legacy-editable" and then pip install -e . works fine
    • pip install -e . without added pyproject.toml works fine

    I tried to find any related existing issues, but I didn't find a solution except adding SETUPTOOLS_ENABLE_FEATURES="legacy-editable". I am not sure whether this issue is related: https://github.com/pypa/setuptools/issues/3557. Thus, I would like to confirm if it is a known issue that will be fixed from setuptools side, or something we should fix in our package. Thank you for looking into it.

    Related issue: https://github.com/onnx/onnx/issues/4539.

    Expected behavior

    pip install -e . should build without errors.

    How to Reproduce

    Build onnx from source with editable mode:

    conda install -y -c conda-forge libprotobuf=3.16.0  # protoc (Protobuf compiler) is required to build onnx from source
    git clone https://github.com/onnx/onnx.git
    cd onnx
    git submodule update --init --recursive
    set CMAKE_ARGS=-DONNX_USE_PROTOBUF_SHARED_LIBS=ON  # use export if POSIX
    pip install -e .
    

    Output

          -- ******** Summary ********                                                                                                                                                                     
          --   CMake version             : 3.19.0-rc3                                                                                                                                                      
          --   CMake command             : C:/Program Files/CMake/bin/cmake.exe                                                                                                                            
          --   System                    : Windows                                                                                                                                                         
          --   C++ compiler              : D:/Program Files (x86)/Microsoft Visual Studio/2019/Community/VC/Tools/MSVC/14.29.30133/bin/Hostx64/x64/cl.exe                                                  
          --   C++ compiler version      : 19.29.30141.0                                                                                                                                                   
          --   CXX flags                 : /DWIN32 /D_WINDOWS /W3 /GR /EHsc /std:c++17 /EHsc /wd26812                                                                                                      
          --   Build type                : Debug                                                                                                                                                           
          --   Compile definitions       : __STDC_FORMAT_MACROS                                                                                                                                            
          --   CMAKE_PREFIX_PATH         :                                                                                                                                                                 
          --   CMAKE_INSTALL_PREFIX      : C:/Program Files/onnx                                                                                                                                           
          --   CMAKE_MODULE_PATH         :                                                                                                                                                                 
          --                                                                                                                                                                                               
          --   ONNX version              : 1.12.0                                                                                                                                                          
          --   ONNX NAMESPACE            : onnx                                                                                                                                                            
          --   ONNX_USE_LITE_PROTO       : OFF                                                                                                                                                             
          --   USE_PROTOBUF_SHARED_LIBS  : OFF                                                                                                                                                             
          --   Protobuf_USE_STATIC_LIBS  : ON                                                                                                                                                              
          --   ONNX_DISABLE_EXCEPTIONS   : OFF                                                                                                                                                             
          --   ONNX_WERROR               : OFF                                                                                                                                                             
          --   ONNX_BUILD_TESTS          : OFF                                                                                                                                                             
          --   ONNX_BUILD_BENCHMARKS     : OFF                                                                                                                                                             
          --                                                                                                                                                                                               
          --   Protobuf compiler         : D:/protobuf/install/bin/protoc.exe                                                                                                                              
          --   Protobuf includes         : D:/protobuf/install/include                                                                                                                                     
          --   Protobuf libraries        : D:/protobuf/install/lib/libprotobufd.lib                                                                                                                        
          --   BUILD_ONNX_PYTHON         : ON                                                                                                                                                              
          --     Python version        : 3.9                                                                                                                                                               
          --     Python executable     : C:/Users/jacky/AppData/Local/Programs/Python/Python39/python.exe                                                                                                  
          --     Python includes       : C:/Users/jacky/AppData/Local/Programs/Python/Python39/include                                                                                                     
          -- Configuring done                                                                                                                                                                              
          -- Generating done                                                                                                                                                                               
          -- Build files have been written to: D:/github/onnx/.setuptools-cmake-build                                                                                                                      
          Microsoft (R) Build Engine version 16.11.2+f32259642 for .NET Framework                                                                                                                          
          Copyright (C) Microsoft Corporation. All rights reserved.                                                                                                                                        
                                                                                                                                                                                                           
            onnx_proto.vcxproj -> D:\github\onnx\.setuptools-cmake-build\Debug\onnx_proto.lib                                                                                                              
            onnx.vcxproj -> D:\github\onnx\.setuptools-cmake-build\Debug\onnx.lib                                                                                                                          
            onnx_cpp2py_export.vcxproj -> D:\github\onnx\.setuptools-cmake-build\Debug\onnx_cpp2py_export.cp39-win_amd64.pyd                                                                               
          running build_ext                                                                                                                                                                                
          copying D:\github\onnx\.setuptools-cmake-build\Debug\onnx_cpp2py_export.cp39-win_amd64.pyd -> C:\Users\jacky\AppData\Local\Temp\tmpba9gls20.build-lib\onnx                                       
          Traceback (most recent call last):                                                                                                                                                               
            File "C:\Users\jacky\AppData\Local\Temp\pip-build-env-onoe7scg\overlay\Lib\site-packages\setuptools\_distutils\file_util.py", line 40, in _copy_file_contents                                  
              fdst = open(dst, 'wb')                                                                                                                                                                       
          FileNotFoundError: [Errno 2] No such file or directory: 'C:\\Users\\jacky\\AppData\\Local\\Temp\\tmpba9gls20.build-lib\\onnx\\onnx_cpp2py_export.cp39-win_amd64.pyd'                             
                                                                                                                                                                                                           
          During handling of the above exception, another exception occurred:                                                                                                                              
                                                                                                                                                                                                           
          Traceback (most recent call last):                                                                                                                                                               
            File "C:\Users\jacky\AppData\Local\Temp\pip-build-env-onoe7scg\overlay\Lib\site-packages\setuptools\command\editable_wheel.py", line 140, in run                                               
              self._create_wheel_file(bdist_wheel)                                                                                                                                                         
            File "C:\Users\jacky\AppData\Local\Temp\pip-build-env-onoe7scg\overlay\Lib\site-packages\setuptools\command\editable_wheel.py", line 330, in _create_wheel_file                                
              files, mapping = self._run_build_commands(dist_name, unpacked, lib, tmp)                                                                                                                     
            File "C:\Users\jacky\AppData\Local\Temp\pip-build-env-onoe7scg\overlay\Lib\site-packages\setuptools\command\editable_wheel.py", line 261, in _run_build_commands                               
              self._run_build_subcommands()                                                                                                                                                                
            File "C:\Users\jacky\AppData\Local\Temp\pip-build-env-onoe7scg\overlay\Lib\site-packages\setuptools\command\editable_wheel.py", line 288, in _run_build_subcommands                            
              self.run_command(name)                                                                                                                                                                       
            File "C:\Users\jacky\AppData\Local\Temp\pip-build-env-onoe7scg\overlay\Lib\site-packages\setuptools\_distutils\cmd.py", line 319, in run_command                                               
              self.distribution.run_command(command)                                                                                                                                                       
            File "C:\Users\jacky\AppData\Local\Temp\pip-build-env-onoe7scg\overlay\Lib\site-packages\setuptools\dist.py", line 1217, in run_command                                                        
              super().run_command(command)                                                                                                                                                                 
            File "C:\Users\jacky\AppData\Local\Temp\pip-build-env-onoe7scg\overlay\Lib\site-packages\setuptools\_distutils\dist.py", line 992, in run_command                                              
              cmd_obj.run()                                                                                                                                                                                
            File "<string>", line 255, in run                                                                                                                                                              
            File "C:\Users\jacky\AppData\Local\Temp\pip-build-env-onoe7scg\overlay\Lib\site-packages\setuptools\command\build_ext.py", line 84, in run                                                     
              _build_ext.run(self)                                                                                                                                                                         
            File "C:\Users\jacky\AppData\Local\Temp\pip-build-env-onoe7scg\overlay\Lib\site-packages\setuptools\_distutils\command\build_ext.py", line 346, in run                                         
              self.build_extensions()                                                                                                                                                                      
            File "<string>", line 272, in build_extensions                                                                                                                                                 
            File "C:\Users\jacky\AppData\Local\Temp\pip-build-env-onoe7scg\overlay\Lib\site-packages\setuptools\_distutils\cmd.py", line 351, in copy_file                                                 
              return file_util.copy_file(                                                                                                                                                                  
            File "C:\Users\jacky\AppData\Local\Temp\pip-build-env-onoe7scg\overlay\Lib\site-packages\setuptools\_distutils\file_util.py", line 163, in copy_file                                           
              _copy_file_contents(src, dst)                                                                                                                                                                
            File "C:\Users\jacky\AppData\Local\Temp\pip-build-env-onoe7scg\overlay\Lib\site-packages\setuptools\_distutils\file_util.py", line 42, in _copy_file_contents                                  
              raise DistutilsFileError(                                                                                                                                                                    
          distutils.errors.DistutilsFileError: could not create 'C:\Users\jacky\AppData\Local\Temp\tmpba9gls20.build-lib\onnx\onnx_cpp2py_export.cp39-win_amd64.pyd': No such file or directory            
          error: Support for editable installs via PEP 660 was recently introduced                                                                                                                         
          in `setuptools`. If you are seeing this error, please report to:                                                                                                                                 
                                                                                                                                                                                                           
          https://github.com/pypa/setuptools/issues                                                                                                                                                        
                                                                                                                                                                                                           
          Meanwhile you can try the legacy behavior by setting an                                                                                                                                          
          environment variable and trying to install again:                                                                                                                                                
                                                                                                                                                                                                           
          SETUPTOOLS_ENABLE_FEATURES="legacy-editable"                                                                                                                                                     
          [end of output]                                                                                                                                                                                  
                                                                                                                                                                                                           
      note: This error originates from a subprocess, and is likely not a problem with pip.                                                                                                                 
      ERROR: Failed building editable for onnx                                                                                                                                                             
    Failed to build onnx                                                                                                                                                                                   
    ERROR: Could not build wheels for onnx, which is required to install pyproject.toml-based projects                                                                                                     
    
    bug Needs Triage 
    opened by jcwchen 0
  • [BUG] Difference among distutils.sysconfig.get_python_inc()

    [BUG] Difference among distutils.sysconfig.get_python_inc()

    setuptools version

    65.3.0

    Python version

    3.9.2

    OS

    Linux

    Additional environment information

    My env is in a docker.

    Description

    Hi there, i find that setuptools 65.3.0 hack the distutils. I want to get the include dir of python And i found that

    before install setuptools

    python -c 'import distutils.sysconfig; print(distutils.sysconfig.get_python_inc())'
    

    and get /opt/aaa/python/python-3.9.2/include/python3.9

    after install that

    python -c 'import distutils.sysconfig; print(distutils.sysconfig.get_python_inc())'
    

    and get /devcloud/ws/sirFh/workspace/j_VWBQ8FGA/aaa_python/third_build/python/include/python3.9

    /opt/aaa/python/python-3.9.2/include/python3.9 is the right anwser. And /devcloud/ws/sirFh/workspace/j_VWBQ8FGA/aaa_python/third_build/python/include/python3.9 does not exist in my docker or even my machine

    Expected behavior

    Get the right include dir

    How to Reproduce

    import distutils.sysconfig; print(distutils.sysconfig.get_python_inc())

    Output

    python -c 'import distutils.sysconfig; print(distutils.sysconfig.get_python_inc())'

    bug Needs Repro 
    opened by wqh17101 8
  • [FR] option for making unknown arguments to setup into a hard error

    [FR] option for making unknown arguments to setup into a hard error

    What's the problem this feature will solve?

    Sometimes users attempt to install build dependencies themselves (such as cffi). When they mess this up, and then run setup.py, the unknown options passed to setup.py become a warning, which users often miss in the large output.

    The ability to make these into an exception would ensure users catch their errors early.

    Describe the solution you'd like

    A setup() argument (setup.cfg as well, I suppose) like error_on_unknown=True that changes https://github.com/pypa/setuptools/blob/ba3995e5705a22e13bb5d2231ac22c77e4417747/setuptools/_distutils/dist.py#L261-L262 into an exception, instead of a warning.

    Alternative Solutions

    No response

    Additional context

    No response

    Code of Conduct

    • [X] I agree to follow the PSF Code of Conduct
    enhancement Needs Triage 
    opened by alex 4
  • use filename of RTD configuration

    use filename of RTD configuration

    Summary of changes

    the RTD documentation indicates a specific filename .readthedocs.yaml: https://docs.readthedocs.io/en/stable/config-file/v2.html

    Pull Request Checklist

    opened by zacharyburnett 0
  • [BUG] `license_files` not showing up on wheel if not set on `setup.cfg` (`setup.py` & `pyproject.toml`).

    [BUG] `license_files` not showing up on wheel if not set on `setup.cfg` (`setup.py` & `pyproject.toml`).

    setuptools version

    setuptools==65.2.0

    Python version

    Python 3.10.7

    OS

    Arch Linux

    Additional environment information

    No response

    Description

    A setuptools based distribtuion does not include the files specified in license-files under the tool.setuptools table in the wheel files dist-info (by a setup.py/setup.cfg) based project does).

    Expected behavior

    It works for sdist, and I see appropriate lines about "adding license-A.txt" but it doesn't show up in the wheel when pyproject.toml is present

    How to Reproduce

    With pyproject.toml

    mkdir -p licenses
    date > licenses/license-A.txt
    cat > pyproject.toml << EOF
    [build-system]
    requires = ['setuptools==65.2.0']
    build-backend = "setuptools.build_meta"
    
    [project]
    name = "repro-case"
    version = '2.5.0dev0'
    requires-python = "~=3.7"
    license = { text = "Apache License 2.0" }
    authors = [
        { name = "Ash", email = "[email protected]" },
    ]
    [tool.setuptools]
    license-files = [
        'LICENSE',
        'licenses/license-A.txt',
    ]
    EOF
    rm -r repro_case.egg-info/ dist
    python -m build
    unzip -l dist/*.whl
    

    With out pyproject.toml:

    mkdir -p licenses
    date > licenses/license-A.txt
    cat > setup.cfg << EOF
    [metadata]
    name = repro-case
    version = 2.5.0dev0
    requires_python = ~=3.7
    author = Apache Software Foundation
    author_email = [email protected]
    license_files =
        LICENSE
        licenses/license-A.txt
    EOF
    cat > setup.py <<EOF
    import setuptools
    setuptools.setup()
    EOF
    rm -r repro_case.egg-info/ dist
    python -m build
    unzip -l dist/*.whl
    

    Output

    Expected output (and this is what I get with setup.cfg includes repro_case-2.5.0.dev0.dist-info/license-A.txt

    $ unzip -l dist/*.whl 
    Archive:  dist/repro_case-2.5.0.dev0-py3-none-any.whl
      Length      Date    Time    Name
    ---------  ---------- -----   ----
          169  2022-09-20 16:04   repro_case-2.5.0.dev0.dist-info/METADATA
           92  2022-09-20 16:04   repro_case-2.5.0.dev0.dist-info/WHEEL
           32  2022-09-20 16:04   repro_case-2.5.0.dev0.dist-info/license-A.txt
            9  2022-09-20 16:04   repro_case-2.5.0.dev0.dist-info/top_level.txt
          428  2022-09-20 16:04   repro_case-2.5.0.dev0.dist-info/RECORD
    ---------                     -------
          730                     5 files
    

    Output with pyproject.toml does not include extra file

     unzip -l dist/*.whl                                                                                                                                                                                  [17:22] ✓ !1161
    Archive:  dist/repro_case-2.5.0.dev0-py3-none-any.whl
      Length      Date    Time    Name
    ---------  ---------- -----   ----
          205  2022-09-20 16:22   repro_case-2.5.0.dev0.dist-info/METADATA
           92  2022-09-20 16:22   repro_case-2.5.0.dev0.dist-info/WHEEL
            1  2022-09-20 16:22   repro_case-2.5.0.dev0.dist-info/top_level.txt
          328  2022-09-20 16:22   repro_case-2.5.0.dev0.dist-info/RECORD
    ---------                     -------
          626                     4 files
    
    bug 
    opened by ashb 5
Releases(v65.3.0)
Owner
Python Packaging Authority
Python Packaging Authority
The Pants Build System

Pants Build System Pants is a scalable build system for monorepos: codebases containing multiple projects, often using multiple programming languages

Pants Build 2.2k Sep 30, 2022
Clang-based cross platform build system written in Python

Clang-build Find the full documentation at https://clang-build.readthedocs.io First steps Customisations Multiple targets Multiple projects Defaults M

Trick 17 9 Jun 29, 2022
Software build automation tool for Python.

PyBuilder — an easy-to-use build automation tool for Python PyBuilder is a software build tool written in 100% pure Python, mainly targeting Python ap

PyBuilder 1.4k Sep 24, 2022
A pynt of Python build.

A pynt of Python build. Raghunandan Rao Features Easy to learn. Build tasks are just python funtions. Manages dependencies between tasks. Automaticall

Raghunandan Rao 150 Jul 14, 2022
🔨🐍Make-like build automation tool for Python projects with extensive DSL features.

Pyke (WIP, Beta Release) Make-like build automation tool for Python projects with extensive DSL features. Features: Users can specify tasks, subtasks,

Ire 17 Jul 5, 2022
A Star Trek Online build tool in Python

SETS - STO Equipment and Trait Selector A Star Trek Online build tool in Python Description Pre-alpha version of build tool for STO Getting Started De

Star Trek Online Community Developers 6 Mar 17, 2022
The official binary distribution format for Python

wheel This library is the reference implementation of the Python wheel packaging standard, as defined in PEP 427. It has two different roles: A setupt

Python Packaging Authority 339 Sep 5, 2022
Python-based project scripting.

Paver - Easy Scripting for Software Projects Web: https://pythonhosted.org/Paver/ Download: https://pypi.python.org/pypi/Paver/ Source: https://github

Paver community 449 Jul 27, 2022
py2app is a Python setuptools command which will allow you to make standalone Mac OS X application bundles and plugins from Python scripts.

py2app is a Python setuptools command which will allow you to make standalone Mac OS X application bundles and plugins from Python scripts. py2app is

Ronald Oussoren 212 Sep 28, 2022
Ubuntu env build; Nginx build; DB build;

Deploy 介绍 Deploy related scripts bitnami Dependencies Ubuntu openssl envsubst docker v18.06.3 docker-compose init base env upload https://gitlab-runn

Colin(liuji) 10 Dec 1, 2021
:package: :fire: Python project management. Manage packages: convert between formats, lock, install, resolve, isolate, test, build graph, show outdated, audit. Manage venvs, build package, bump version.

THE PROJECT IS ARCHIVED Forks: https://github.com/orsinium/forks DepHell -- project management for Python. Why it is better than all other tools: Form

DepHell 1.7k Sep 22, 2022
An easy way to build PyTorch datasets. Modularly build datasets and automatically cache processed results

EasyDatas An easy way to build PyTorch datasets. Modularly build datasets and automatically cache processed results Installation pip install git+https

Ximing Yang 4 Dec 14, 2021
The Meson Build System

Meson® is a project to create the best possible next-generation build system. Status Dependencies Python (version 3.6 or newer) Ninja (version 1.8.2 o

The Meson Build System 4.2k Sep 23, 2022
The Pants Build System

Pants Build System Pants is a scalable build system for monorepos: codebases containing multiple projects, often using multiple programming languages

Pants Build 2.2k Sep 30, 2022
open-information-extraction-system, build open-knowledge-graph(SPO, subject-predicate-object) by pyltp(version==3.4.0)

中文开放信息抽取系统, open-information-extraction-system, build open-knowledge-graph(SPO, subject-predicate-object) by pyltp(version==3.4.0)

null 6 Sep 23, 2021
Ninja is a small build system with a focus on speed.

Ninja Python Distributions Ninja is a small build system with a focus on speed. The latest Ninja python wheels provide ninja 1.10.2.g51db2.kitware.job

null 25 Aug 12, 2022
Clang-based cross platform build system written in Python

Clang-build Find the full documentation at https://clang-build.readthedocs.io First steps Customisations Multiple targets Multiple projects Defaults M

Trick 17 9 Jun 29, 2022
Build AGNOS, the operating system for your comma three

agnos-builder This is the tool to build AGNOS, our Ubuntu based OS. AGNOS runs on the comma three devkit. NOTE: the edk2_tici and agnos-firmare submod

comma.ai 17 Sep 26, 2022
A dynamic multi-STL, multi-process OpenSCAD build system with autoplating support

scad-build This is a multi-STL OpenSCAD build system based around GNU make. It supports dynamic build targets, intelligent previews with user-defined

Jordan Mulcahey 1 Dec 21, 2021
Build a medical knowledge graph based on Unified Language Medical System (UMLS)

UMLS-Graph Build a medical knowledge graph based on Unified Language Medical System (UMLS) Requisite Install MySQL Server 5.6 and import UMLS data int

Donghua Chen 4 May 2, 2022