A modern Python package manager with PEP 582 support.

Python Development Master(PDM)

Last update: Jan 5, 2023

Related tags

Overview

PDM - Python Development Master

A modern Python package manager with PEP 582 support. 中文版本说明

What is PDM?

PDM is meant to be a next generation Python package management tool. It was originally built for personal use. If you feel you are going well with Pipenv or Poetry and don't want to introduce another package manager, just stick to it. But if you are missing something that is not present in those tools, you can probably find some goodness in pdm.

PEP 582 proposes a project structure as below:

foo
    __pypackages__
        3.8
            lib
                bottle
    myscript.py

There is a __pypackages__ directory in the project root to hold all dependent libraries, just like what npm does. Read more about the specification here.

Highlights of features

PEP 582 local package installer and runner, no virtualenv involved at all.
Simple and relatively fast dependency resolver, mainly for large binary distributions.
A PEP 517 build backend.
A full-featured plug-in system.
PEP 621 project metadata format.

Why not virtualenv?

The majority of Python packaging tools also act as virtualenv managers to gain the ability to isolate project environments. But things get tricky when it comes to nested venvs: One installs the virtualenv manager using a venv encapsulated Python, and create more venvs using the tool which is based on an encapsulated Python. One day a minor release of Python is released and one has to check all those venvs and upgrade them if required.

PEP 582, on the other hand, introduces a way to decouple the Python interpreter from project environments. It is a relative new proposal and there are not many tools supporting it (one that does is pyflow, but it is written with Rust and thus can't get much help from the big Python community and for the same reason it can't act as a PEP 517 backend).

Installation

PDM requires python version 3.7 or higher.

Like Pip, PDM provides an installation script that will install PDM into an isolated environment.

For Linux/Mac

curl -sSL https://raw.githubusercontent.com/pdm-project/pdm/main/install-pdm.py | python -

For Windows

(Invoke-WebRequest -Uri https://raw.githubusercontent.com/pdm-project/pdm/main/install-pdm.py -UseBasicParsing).Content | python -

The installer will install PDM into the user site and the location depends on the system:

$HOME/.local/bin for Unix
%APPDATA%\Python\Scripts on Windows

You can pass additional options to the script to control how PDM is installed:

usage: install-pdm.py [-h] [-v VERSION] [--prerelease] [--remove] [-p PATH] [-d DEP]

optional arguments:
  -h, --help            show this help message and exit
  -v VERSION, --version VERSION | envvar: PDM_VERSION
                        Specify the version to be installed, or HEAD to install from the main branch
  --prerelease | envvar: PDM_PRERELEASE    Allow prereleases to be installed
  --remove | envvar: PDM_REMOVE            Remove the PDM installation
  -p PATH, --path PATH | envvar: PDM_HOME  Specify the location to install PDM
  -d DEP, --dep DEP | envvar: PDM_DEPS     Specify additional dependencies, can be given multiple times

You can either pass the options after the script or set the env var value.

Alternative Installation Methods

If you are on MacOS and using homebrew, install it by:

$ brew install pdm

If you are on Windows and using Scoop, install it by:

PS> scoop bucket add frostming https://github.com/frostming/scoop-frostming.git
PS> scoop install pdm

Otherwise, it is recommended to install pdm in an isolated environment with pipx:

$ pipx install pdm

Or you can install it under a user site:

$ pip install --user pdm

Quickstart

Initialize a new PDM project

$ pdm init

Answer the questions following the guide, and a PDM project with a pyproject.toml file will be ready to use.

Install dependencies into the __pypackages__ directory

$ pdm add requests flask

You can add multiple dependencies in the same command. After a while, check the pdm.lock file to see what is locked for each package.

Run your script with PEP 582 support

Suppose you have a script app.py placed next to the __pypackages__ directory with the following content(taken from Flask's website):

from flask import Flask
app = Flask(__name__)

@app.route('/')
def hello_world():
    return 'Hello World!'

if __name__ == '__main__':
    app.run()

If you are a Bash user, set the environment variable by eval "$(pdm --pep582)". Now you can run the app directly with your familiar Python interpreter:

$ python /home/frostming/workspace/flask_app/app.py
 * Serving Flask app "app" (lazy loading)
 ...
 * Running on http://127.0.0.1:5000/ (Press CTRL+C to quit)

Ta-da! You are running an app with its dependencies installed in an isolated place, while no virtualenv is involved.

For Windows users, please refer to the doc about how to make it work.

If you are curious about how this works, check this doc section for some explanation.

Docker image

$ docker pull frostming/pdm

Badges

Tell people you are using PDM in your project by including the markdown code in README.md:

[![pdm-managed](https://img.shields.io/badge/pdm-managed-blueviolet)](https://pdm.fming.dev)

PDM Eco-system

Awesome PDM is a curated list of awesome PDM plugins and resources.

FAQ

1. What is put in `pypackages`?

PEP 582 is a draft proposal which still needs a lot of polishing. For instance, it doesn't mention how to manage CLI executables. PDM makes the decision to put bin and include together with lib under __pypackages__/X.Y.

2. How do I run CLI scripts in the local package directory?

The recommended way is to prefix your command with pdm run. It is also possible to run CLI scripts directly from the outside. PDM's installer has already injected the package path to the sys.path in the entry script file.

3. What site-packages will be loaded when using PDM?

Packages in the local __pypackages__ directory will be loaded before the system-level site-packages for isolation.

4. Can I relocate or move the `pypackages` folder for deployment?

You'd better not. The packages installed inside __pypackages__ are OS dependent. Instead, you should keep pdm.lock in VCS and do pdm sync on the target environment to deploy.

5. Can I use `pdm` to manage a Python 2.7 project?

Sure. The pdm itself can be installed under Python 3.7+ only, but it doesn't restrict the Python used by the project.

Credits

This project is strongly inspired by pyflow and poetry.

License

This project is open sourced under MIT license, see the LICENSE file for more details.

Comments

Multiple pypi.url

Is your feature request related to a problem? Please describe.

I need to define multiple pypi mirrors. As far as I understand it, using pdm.config pypi.url, I can set ONE mirror.

Describe the solution you'd like

There should be a way to define multiple pypi mirrors.

Alternatively, when adding a package, there should be a parameter available that specifies the mirror that will be used to look for the package.
enhancement

opened by andreas-vester 24
Local dependency and packaging
Is your feature request related to a problem? Please describe.

Hi Dear community

I'm struggling to build and package an app depending on a local dependency using PDM.

So maybe it's because I'm quite new to python (long time I didn't code in that language), or because I did not understand some concepts related to PEP 582, so please excuse me if my question is obvious (or stupid).

So I want to bundle an AWS Lambda function (in Python) as a Docker image (let's name it A) following the documentation provided.

That function depends on a library (local dependency named B) which itself can bring some external dependencies (C, D, E).

So basically the dependency tree is something like:

├── A (local) │ └── B. (local) │ ├── C (external) │ ├── D (external) │ └── E (external)

Add B as a local dependency into A project

When I add to project A the local dependency B running pdm add -e ../relative/path/to/B I have:

an absolute path in the pyproject.toml :
dependencies = [ "-e file:///Absolute/path/to/B#egg=B", ]

(This looks an issue to me because the pyproject.toml is not portable anymore)

a relative path in the pdm.lock :

name = "B" sections = ["default"] version = "0.0.1" editable = true path = "../relative/path/to/B"

a directory __pypackages__ loooking like that:

├── __pypackages__ │ └── 3.8 │ ├── bin │ ├── include │ └── lib │ ├── B.egg-link │ ├── easy-install.pth │ └── A.egg-link

Install A

When I run pdm install I have:

[CHANGE IN 1.5.0]: dev-dependencies are included by default and can be excluded with `--prod` option Synchronizing working set with lock file: 0 to add, 1 to update, 0 to remove ✔ Update B 0.0.1 -> 0.0.1 successful Installing the project as an editable package... ✔ Update A 0.0.1 -> 0.0.1 successful 🎉 All complete!

Describe the solution you'd like

What I expect when I run pdm install in the A project is to resolve the local and external dependencies then install them to the __pypackages__ directory.

When I build the Docker image like that, I just need to copy the __pypackages__/3.8/lib directory.

So how do we do that ?
enhancement
opened by nhuray 23
Install for production

Is your feature request related to a problem? Please describe.

Most dependency managers have a --production (or similar) flag to only install for production. A normal install without that flag install all dependencies (production + development).

Looking at https://pdm.fming.dev/usage/dependency/#install-the-packages-pinned-in-lock-file this is confusing.

Describe the solution you'd like

I would like it to mimic how other tools does this and believe this tool incorrectly took the approach of pipenv. See https://github.com/pypa/pipenv/issues/3093#issuecomment-432718562 for more reference of tools (the entire thread is interesting).

The reason is that developers are bound to want to install all (production + dev) dependencies in their daily work. A pdm install is thus quicker to write. It also mimics most (all?) other package managers out there.
enhancement

opened by thernstig 22
Feature: support groups in dev-dependencies
Pull Request Check List

[ ] A news fragment is added in news/ describing what is new.

[x] Test cases added for changed code.

Describe what you have changed in this PR.

[x] Move out dev-dependencies, includes, excludes, package-dir from [project] table to [tool.pdm], close #345

[x] Update pdm-pep517 to support both formats (transition solution)

[x] Auto-migrate the pyproject.toml to the new format with a message, the change should not break anything.

[x] Change the dev-dependencies to a group of dependencies

[x] The dev-dependencies of the old format will be transferred to the new table under dev group by the migration. This may require possible relock as the lock file hash changes.

[x] Adjust add, sync, install, remove, and update commands to support the new behavior, in a backward-compatible way.
opened by frostming 20

Issue when building Calibre -- "-c: Unable to find file 'QtWidgets/QtWidgetsmod.sip'"

[x] I have searched the issue tracker and believe that this is not a duplicate.

Make sure you run commands with -v flag before pasting the output.

Steps to reproduce

Start a new project, install pyqt5, sip, and pyqt-builder. Clone the calibre source code and run pdm run python setup.py build

Actual behavior

Failure with -c: Unable to find file 'QtWidgets/QtWidgetsmod.sip'

Expected behavior

Succeeding

Environment Information

# Paste the output of `pdm info && pdm info --env` below:
PDM version:        1.12.6
Python Interpreter: /usr/bin/python (3.9)
Project Root:       /home/system/Downloads/calibre
Project Packages:   /home/system/Downloads/calibre/__pypackages__/3.9
{
  "implementation_name": "cpython",
  "implementation_version": "3.9.9",
  "os_name": "posix",
  "platform_machine": "aarch64",
  "platform_release": "5.15.5-1-ARCH",
  "platform_system": "Linux",
  "platform_version": "#1 SMP Wed Dec 1 01:26:36 UTC 2021",
  "python_full_version": "3.9.9",
  "platform_python_implementation": "CPython",
  "python_version": "3.9",
  "sys_platform": "linux"
}

bug

opened by DUOLabs333 19

1.15.0: pytest is failing
I'm trying to package your module as an rpm package. So I'm using the typical PEP517 based build, install and test cycle used on building packages from non-root account.

python3 -sBm build -w --no-isolation

because I'm calling build with --no-isolation I'm using during all processes only locally installed modules

install .whl file in </install/prefix>

run pytest with PYTHONPATH pointing to sitearch and sitelib inside </install/prefix>

bug
opened by kloczek 18

Broken editable install of the project itself?

[x] I have searched the issue tracker and believe that this is not a duplicate.

There's something going on. Yesterday I was working on a project, and everything worked, and moments later, I couldn't run my CLI tool anymore. I'm investigating today again.

This used to work:

% python -m griffe
/usr/bin/python: No module named griffe.__main__; 'griffe' is a package and cannot be directly executed

So I'm like, OK, lets try with pdm:

% pdm run griffe
Traceback (most recent call last):
  File "/media/data/dev/griffe/__pypackages__/3.9/bin/griffe", line 5, in <module>
    from griffe.cli import main
ModuleNotFoundError: No module named 'griffe.cli'

What's happening? Lets list the contents of __pypackages__.

% ll __pypackages__/3.9/lib
total 716K
drwxr-xr-x 2 pawamoy users 4.0K Sep 11 14:49 griffe
drwxr-xr-x 2 pawamoy users 4.0K Sep 11 14:49 griffe-0.1.0+d20210911.dist-info
-rw-r--r-- 1 pawamoy users   14 Sep 11 14:49 griffe.pth
-rw-r--r-- 1 pawamoy users  139 Sep 11 14:49 _griffe.py

# relevant contents only

OK, what's in all this?

% ll __pypackages__/3.9/lib/griffe
total 0
-rw-r--r-- 1 pawamoy users 0 Sep 11 14:53 py.typed

That explains the ModuleNotFoundErrors. Nothing in this directory.

% cat __pypackages__/3.9/lib/griffe.pth
import _griffe

% cat __pypackages__/3.9/lib/_griffe.py
from editables.redirector import RedirectingFinder as F
F.install()
F.map_module('griffe', '/media/data/dev/griffe/src/griffe/__init__.py')

Does this come from PDM?

% cat __pypackages__/3.9/lib/griffe-0.1.0+d20210911.dist-info/RECORD 
../bin/griffe,sha256=c1929c0a3f0745abc2a583a6942b2de8516ba10510fc57666bca58b6f47e87aa,211
griffe/py.typed,sha256=e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855,0
griffe.pth,sha256=eb6dc9570253f46b83cbce7e9ed8be76563b9874dd3b14a20600854cd3f2c367,14
_griffe.py,sha256=e4d75e89e7f2c219d8b2628d65c665f3cb39b23d13e423e09d3a3ae6287a5e10,139
griffe-0.1.0+d20210911.dist-info/entry_points.txt,sha256=81028a1dd270a1f2cb9966cb7c5756b0d83ce176ac5a5106812012ea06c0fc45,42
griffe-0.1.0+d20210911.dist-info/WHEEL,sha256=a23c91d75e2282fd898cebe66304e98ede8608a4da60f79176b2881e5a853bad,87
griffe-0.1.0+d20210911.dist-info/METADATA,sha256=bb17b48ccc7093b255c0816ed563342c770d55e2c303c6fb224fb9888ef4992c,3043
griffe-0.1.0+d20210911.dist-info/LICENSE,sha256=2466f8a5d3c433c4d38e386bfae3423bba1792256bc06e4fcf425a99c8cd17fb,754
griffe-0.1.0+d20210911.dist-info/INSTALLER,sha256=2581b4c142e7c10ed0b1975de1e2b9f3db44b4be4964ee99881f6ca6b362c986,15
griffe-0.1.0+d20210911.dist-info/direct_url.json,sha256=56736bedb20e23873455a9325c8468360c726ba9938114630b561e9607206445,86
griffe-0.1.0+d20210911.dist-info/RECORD,,

At this point I tried everything: reinstalling PDM, switching the feature.install_cache on and off, deleting PDM's cache, reinstalling all dependencies, trying with PDM 1.8.2, trying with all Python versions (3.6, 3.7, 3.8, 3.9, 3.10), etc. Nothing fixes it.

So I tried in another project. There I get this:

% python -m duty -h
Error processing line 1 of /media/data/dev/duty/__pypackages__/3.9/lib/duty.pth:

  Traceback (most recent call last):
    File "/usr/lib/python3.9/site.py", line 169, in addpackage
      exec(line)
    File "<string>", line 1, in <module>
    File "/media/data/dev/duty/__pypackages__/3.9/lib/_duty.py", line 1, in <module>
      from editables.redirector import RedirectingFinder as F
  ModuleNotFoundError: No module named 'editables'

Remainder of file ignored
/usr/bin/python: No module named duty.__main__; 'duty' is a package and cannot be directly executed

% pdm run duty -h
Error processing line 1 of /media/data/dev/duty/__pypackages__/3.9/lib/duty.pth:

  Traceback (most recent call last):
    File "/usr/lib/python3.9/site.py", line 169, in addpackage
      exec(line)
    File "<string>", line 1, in <module>
    File "/media/data/dev/duty/__pypackages__/3.9/lib/_duty.py", line 1, in <module>
      from editables.redirector import RedirectingFinder as F
  ModuleNotFoundError: No module named 'editables'

Remainder of file ignored
Error processing line 1 of /media/data/dev/duty/__pypackages__/3.9/lib/duty.pth:

  Traceback (most recent call last):
    File "/usr/lib/python3.9/site.py", line 169, in addpackage
      exec(line)
    File "<string>", line 1, in <module>
    File "/media/data/dev/duty/__pypackages__/3.9/lib/_duty.py", line 1, in <module>
      from editables.redirector import RedirectingFinder as F
  ModuleNotFoundError: No module named 'editables'

Remainder of file ignored
Traceback (most recent call last):
  File "/media/data/dev/duty/__pypackages__/3.9/bin/duty", line 5, in <module>
    from duty.cli import main
ModuleNotFoundError: No module named 'duty.cli'

OK so it seems to have something to do with the editables package?

Steps to reproduce

git clone https://github.com/pawamoy/griffe
cd griffe
pdm install
python -m griffe
pdm run griffe

Actual behavior

Problem with editable installation of the project.

Expected behavior

Environment Information

% pdm info && pdm info --env
PDM version:        1.8.3                                     
Python Interpreter: /usr/bin/python3.10 (3.10)                
Project Root:       /media/data/dev/griffe                    
Project Packages:   /media/data/dev/griffe/__pypackages__/3.10
{
  "implementation_name": "cpython",
  "implementation_version": "3.10.0c2",
  "os_name": "posix",
  "platform_machine": "x86_64",
  "platform_release": "5.14.2-arch1-2",
  "platform_system": "Linux",
  "platform_version": "#1 SMP PREEMPT Thu, 09 Sep 2021 09:42:35 +0000",
  "python_full_version": "3.10.0rc2",
  "platform_python_implementation": "CPython",
  "python_version": "3.10",
  "sys_platform": "linux"
}

bug

opened by pawamoy 18

package py code and dependency to pyz

I noticed there say “ we can publish a single file .pyz so that people can download and run without installing any dependencies”。How do you do that？for example： pyproject.toml

[tool.poetry]
name = "test"
version = "0.1.0"
description = ""
authors = ["qbit"]

[tool.poetry.dependencies]
python = "^3.8"
requests = "~2.25.1"

[tool.poetry.dev-dependencies]

[build-system]
requires = ["poetry-core>=1.0.0"]
build-backend = "poetry.core.masonry.api"

test.py

import requests

def test():
    r = requests.get(r'https://www.httpbin.org/get')
    print(r)

if __name__ == '__main__':
    test()

It is best to compile to pyc before packaging ：）

question

opened by qbit-git 18

Towards release 2.0
Planned changes in version 2.0

[x] ~PEP 665 support~ Rejected

[x] Switch the UI framework to rich, drop click. #1066 #1091

[x] Remove support of legacy pyproject.toml format: #1157

[x] Drop the dependency of pip #1096

[x] Decouple PDM from any build backend, making it a pure fronend

[x] #1150

[x] #593

[x] Restructure the documentation about PEP 621 metadata, tool settings and backend settings

[ ] (Possible) #46 Experiment on sub-resolvers to address the multi specifications for one dependency

~~It won't happen shortly, at least before PEP 665's finalization.~~

See the Release 2.0 milestone for the latest progress.
meta
opened by frostming 17
Add option for `~/.local` directory

Is your feature request related to a problem? Please describe.

pdm has a -g option, to edit the "global" Python environment, eg. the one in /usr. However, there's no option for the local python environment, the one in ~/.local

Describe the solution you'd like

Maybe an option like --local or --user to act on that directory?
enhancement

opened by DUOLabs333 16

how to install with version annotation and wheel? (e.g. pytorch cpu version)

Screenshot from 2021-10-22 03-00-13

Pytorch's official page guides user to install the cpu-only mode of package by the following.

pip3 install torch==1.10.0+cpu \ 
  torchvision==0.11.0+cpu \ 
  torchaudio==0.10.0+cpu \
  -f https://download.pytorch.org/whl/cpu/torch_stable.html

If I run

pdm add torch~=1.10.0+cpu -f https://download.pytorch.org/whl/cpu/torch_stable.html

then an error occurs.

Usage: pdm [OPTIONS] [COMMANDS] ...
pdm: error: unrecognized arguments: -f https://download.pytorch.org/whl/cpu/torch_stable.html

If I run

pdm add torch~=1.10.0+cpu

then another error occurs.

[RequirementError]: The local path torch=1.10.0+cpu does not exist.
Add '-v' to see the detailed traceback

How can I do?

Thanks.

Environment Information

# Paste the output of `pdm info && pdm info --env` below:
➜  pdm (☸ |k3d-ateam:local) pdm info && pdm info --env
PDM version:        1.9.0                                                 
Python Interpreter: /home/jjangga/.pyenv/versions/3.9.7/bin/python3 (3.9) 
Project Root:       /media/jjangga/SHARE/experiment/pdm                   
Project Packages:   /media/jjangga/SHARE/experiment/pdm/__pypackages__/3.9
{
  "implementation_name": "cpython",
  "implementation_version": "3.9.7",
  "os_name": "posix",
  "platform_machine": "x86_64",
  "platform_release": "5.11.0-37-generic",
  "platform_system": "Linux",
  "platform_version": "#41~20.04.2-Ubuntu SMP Fri Sep 24 09:06:38 UTC 2021",
  "python_full_version": "3.9.7",
  "platform_python_implementation": "CPython",
  "python_version": "3.9",
  "sys_platform": "linux"
}

bug

opened by jjangga0214 16

pdm run don't exclude system site-packages if python has a different name

[x ] have searched the issue tracker and believe that this is not a duplicate.

Steps to reproduce

I have installed python 3.11 as C:\Python\python.exe. Also I made hardlink C:\Python\python.exe -> C:\Python\p.exe.

 > pdm init
 ...
 > pdm run -v python -c "import sys; print(sys.path)
['', 'z:\\test', 'C:\\Python\\python311.zip', 'C:\\Python\\Lib',
'C:\\Python\\DLLs', 'z:\\test\\__pypackages__\\3.11\\lib']

pdm run -v p -c "import sys; print(sys.path)"
['', 'z:\\test', 'C:\\Python\\python311.zip', 'C:\\Python\\Lib', 'C:\\Python\\DLLs', 
'z:\\test\\__pypackages__\\3.11\\lib', 'C:\\Python', 'C:\\Python\\Lib\\site-packages']

Actual behavior

When using a shortcut, there is no isolation form system packages

Expected behavior

Expect the same result in both cases

Environment Information

pdm info && pdm info --env
PDM version:
  2.3.4
Python Interpreter:
  C:\Python\python.EXE (3.11)
Project Root:
  z:/test
Project Packages:
  z:\test\__pypackages__\3.11
{
  "implementation_name": "cpython",
  "implementation_version": "3.11.1",
  "os_name": "nt",
  "platform_machine": "AMD64",
  "platform_release": "10",
  "platform_system": "Windows",
  "platform_version": "10.0.19044",
  "python_full_version": "3.11.1",
  "platform_python_implementation": "CPython",
  "python_version": "3.11",
  "sys_platform": "win32"
}

doc say:

 "The executable is from PATH but not inside the __pypackages__ folder."

But the executable file p.exe is located in the same place as python.exe

bug

opened by vitidev 0

After successful installation, pdm install fails if certain dependencies installed

[X] I have searched the issue tracker and believe that this is not a duplicate.

Make sure you run commands with -v flag before pasting the output.

Steps to reproduce

pdm add accelerate then pdm install afterwards

Actual behavior

pdm.termui: Error occurs
Traceback (most recent call last):
  File "/home/_/.local/share/pdm/venv/lib/python3.10/site-packages/pdm/termui.py", line 227, in logging
    yield logger
  File "/home/_/.local/share/pdm/venv/lib/python3.10/site-packages/pdm/cli/actions.py", line 139, in resolve_candidates_from_lockfile
    mapping, *_ = resolve(
  File "/home/_/.local/share/pdm/venv/lib/python3.10/site-packages/pdm/resolver/core.py", line 35, in resolve
    result = resolver.resolve(requirements, max_rounds)
  File "/home/_/.local/share/pdm/venv/lib/python3.10/site-packages/resolvelib/resolvers.py", line 521, in resolve
    state = resolution.resolve(requirements, max_rounds=max_rounds)
  File "/home/_/.local/share/pdm/venv/lib/python3.10/site-packages/resolvelib/resolvers.py", line 372, in resolve
    self._add_to_criteria(self.state.criteria, r, parent=None)
  File "/home/_/.local/share/pdm/venv/lib/python3.10/site-packages/resolvelib/resolvers.py", line 172, in _add_to_criteria
    if not criterion.candidates:
  File "/home/_/.local/share/pdm/venv/lib/python3.10/site-packages/resolvelib/structs.py", line 127, in __bool__
    next(iter(self))
  File "/home/_/.local/share/pdm/venv/lib/python3.10/site-packages/pdm/resolver/providers.py", line 151, in <genexpr>
    return (
  File "/home/_/.local/share/pdm/venv/lib/python3.10/site-packages/pdm/models/repositories.py", line 493, in find_candidates
    for key in self._matching_keys(requirement):
  File "/home/_/.local/share/pdm/venv/lib/python3.10/site-packages/pdm/models/repositories.py", line 477, in _matching_keys
    if can_req.path != getattr(requirement, "path", None):  # type: ignore
AttributeError: 'NamedRequirement' object has no attribute 'path'

Expected behavior

All packages are synced to date, nothing to do.

🎉 All complete!

Environment Information

# Paste the output of `pdm info && pdm info --env` below:

PDM version:
  2.3.4
Python Interpreter:
  /usr/bin/python3.10 (3.10)
Project Root:
  /home/_/Documents/<project>
Project Packages:
  /home/_/Documents/<project>/__pypackages__/3.10
{
  "implementation_name": "cpython",
  "implementation_version": "3.10.7",
  "os_name": "posix",
  "platform_machine": "x86_64",
  "platform_release": "5.15.79.1-microsoft-standard-WSL2",
  "platform_system": "Linux",
  "platform_version": "#1 SMP Wed Nov 23 01:01:46 UTC 2022",
  "python_full_version": "3.10.7",
  "platform_python_implementation": "CPython",
  "python_version": "3.10",
  "sys_platform": "linux"
}

bug

opened by edible-programs 2

feat(shell): add basic completion for `pdm venv` in Fish

related: discord thread

Describe what you have changed in this PR.

Add pdm venv subcommands suggestions: create, list, remove, activate, purge

Pull Request Check List

~- [ ] A news fragment is added in news/ describing what is new.~ ~- [ ] Test cases added for changed code.~

opened by edouard-lopez 0
pdm import does not check for duplicates
Running pdm import when you already have dependencies defined (maybe because you want to unify multiple requirements) or just by running it for the same file multiple times will result in duplicate entries.

[x] I have searched the issue tracker and believe that this is not a duplicate.

(“Make sure you run commands with -v flag before pasting the output.”; -v does not seem to change the output of pdm import.)

Steps to reproduce

Have a initialized pdm project with pyproject.toml and a simple requirements.txt file (echo sqlalchemy > requirements.txt).

Run pdm import requirements.txt -f requirements multiple times.

Actual behavior

You get duplicate dependencies:

[project] ... dependencies = ["sqlalchemy", "sqlalchemy"]

Expected behavior

Entries already existing should not be added again.

[project] ... dependencies = ["sqlalchemy"]

Environment Information

$ pdm info && pdm info --env PDM version: 2.3.3 Python Interpreter: /path/to/project/.venv/bin/python (3.10) Project Root: /path/to/project Project Packages: None { "implementation_name": "cpython", "implementation_version": "3.10.9", "os_name": "posix", "platform_machine": "arm64", "platform_release": "22.2.0", "platform_system": "Darwin", "platform_version": "Darwin Kernel Version 22.2.0: Fri Nov 11 02:06:26 PST 2022; root:xnu-8792.61.2~4/RELEASE_ARM64_T8112", "python_full_version": "3.10.9", "platform_python_implementation": "CPython", "python_version": "3.10", "sys_platform": "darwin" }
bug contribution welcome good first issue
opened by quassy 1

pdm lock resolves packages but does not write lock file

pdm lock seems to resolve all dependencies but does not write a lock file to disk and does not exit. Other smaller sets of requirements work. Simple dependency resolution with pip of the requirements in question works.

[x] I have searched the issue tracker and believe that this is not a duplicate.

Steps to reproduce

To migrate to pdm, I imported my requirements with pdm import requirements.in -f requirements to get this section in pyproject.toml:

[tool.pdm]

[project]
# PEP 621 project metadata
# See https://www.python.org/dev/peps/pep-0621/
dependencies = [
    "wheel",
    "GeoAlchemy2",
    "Pillow",
    "albumentations",
    "docopt",
    "gcsfs",
    "geopandas",
    "google-cloud-bigquery",
    "google-cloud-bigquery-storage",
    "jinja2",
    "joblib",
    "mlflow",
    "numpy",
    "orjson",
    "pandas",
    "pandas-gbq",
    "prefect",
    "prefect-dask",
    "prefect-shell",
    "psycopg2-binary",
    "rasterio",
    "requests",
    "segmentation-models-pytorch==0.2.1",
    "shapely",
    "sqlalchemy",
    "tenacity",
    "tqdm",
    "xmltodict",
    "zipfile-deflate64",
]
requires-python = ">=3.10"

Then running pdm lock quite quickly gets to pycparser but then nothing happens, even after hours.

The output of pdm lock -v (first lines omitted):

...
pdm.termui: ======== Ending round 181 ========
pdm.termui: ======== Starting round 182 ========
pdm.termui: Pinning: heapdict 1.0.1
pdm.termui: ======== Ending round 182 ========
pdm.termui: ======== Starting round 183 ========
pdm.termui: Pinning: pycparser 2.21
pdm.termui: ======== Ending round 183 ========
pdm.termui: ======== Starting round 184 ========
pdm.termui: ======== Resolution Result ========
pdm.termui: Stable pins:
pdm.termui:                          python None
pdm.termui:     segmentation-models-pytorch 0.2.1
pdm.termui:                  albumentations 1.3.0
pdm.termui:                          docopt 0.6.2
pdm.termui:                           gcsfs 2022.11.0
pdm.termui:                     geoalchemy2 0.12.5
pdm.termui:                      sqlalchemy 1.4.45
pdm.termui:                       geopandas 0.12.2
pdm.termui:                          pandas 1.5.2
pdm.termui:                         shapely 2.0.0
pdm.termui:           google-cloud-bigquery 3.4.1
pdm.termui:                        requests 2.28.1
pdm.termui:   google-cloud-bigquery-storage 2.17.0
pdm.termui:                          jinja2 3.1.2
pdm.termui:                          joblib 1.2.0
pdm.termui:                          mlflow 2.1.1
pdm.termui:                          orjson 3.8.3
pdm.termui:                      pandas-gbq 0.18.1
pdm.termui:                          pillow 9.3.0
pdm.termui:                         prefect 2.7.4
pdm.termui:                    prefect-dask 0.2.2
pdm.termui:                   prefect-shell 0.1.3
pdm.termui:                 psycopg2-binary 2.9.5
pdm.termui:                        rasterio 1.3.4
pdm.termui:                        tenacity 8.1.0
pdm.termui:                            tqdm 4.64.1
pdm.termui:                           wheel 0.38.4
pdm.termui:                       xmltodict 0.13.0
pdm.termui:               zipfile-deflate64 0.2.0
pdm.termui:                          fsspec 2022.11.0
pdm.termui:            efficientnet-pytorch 0.6.3
pdm.termui:                pretrainedmodels 0.7.4
pdm.termui:                            timm 0.4.12
pdm.termui:                        protobuf 4.21.12
pdm.termui:           google-api-core[grpc] 2.11.0
pdm.termui:                 google-api-core 2.11.0
pdm.termui:                          grpcio 1.51.1
pdm.termui:                           click 8.1.3
pdm.termui:                     google-auth 2.15.0
pdm.termui:                       packaging 21.3
pdm.termui:                      proto-plus 1.22.1
pdm.termui:                          docker 6.0.1
pdm.termui:              importlib-metadata 5.2.0
pdm.termui:                         pyarrow 10.0.1
pdm.termui:                 python-dateutil 2.8.2
pdm.termui:                            pytz 2022.7
pdm.termui:                          pyyaml 6.0
pdm.termui:                         urllib3 1.26.13
pdm.termui:                         aiohttp 3.8.3
pdm.termui:              charset-normalizer 2.1.1
pdm.termui:                         alembic 1.9.1
pdm.termui:                     cloudpickle 2.2.0
pdm.termui:                  databricks-cli 0.17.4
pdm.termui:                       db-dtypes 1.0.5
pdm.termui:                       gitpython 3.1.29
pdm.termui:               google-cloud-core 2.3.2
pdm.termui:          google-resumable-media 2.4.0
pdm.termui:                            idna 3.4
pdm.termui:                        markdown 3.4.1
pdm.termui:                            shap 0.41.0
pdm.termui:                          slicer 0.0.7
pdm.termui:             sqlalchemy[asyncio] 1.4.45
pdm.termui:                        greenlet 2.0.1
pdm.termui:                        sqlparse 0.4.3
pdm.termui:                       aiosqlite 0.18.0
pdm.termui:                           anyio 3.6.2
pdm.termui:                         apprise 1.2.0
pdm.termui:                   asgi-lifespan 2.0.0
pdm.termui:                         asyncpg 0.27.0
pdm.termui:                           attrs 22.2.0
pdm.termui:                         certifi 2022.12.7
pdm.termui:                           cligj 0.7.2
pdm.termui:                        coolname 2.1.0
pdm.termui:                        croniter 1.3.8
pdm.termui:                    cryptography 38.0.4
pdm.termui:                       decorator 5.1.1
pdm.termui:                     distributed 2022.12.1
pdm.termui:                            dask 2022.12.1
pdm.termui:                     entrypoints 0.4
pdm.termui:                         fastapi 0.88.0
pdm.termui:                       starlette 0.22.0
pdm.termui:                        pydantic 1.10.2
pdm.termui:               typing-extensions 4.4.0
pdm.termui:                           fiona 1.8.22
pdm.termui:                   click-plugins 1.1.1
pdm.termui:                           flask 2.2.2
pdm.termui:            google-auth-oauthlib 0.8.0
pdm.termui:                          griffe 0.25.2
pdm.termui:                        colorama 0.4.6
pdm.termui:                        gunicorn 20.1.0
pdm.termui:                    httpx[http2] 0.23.1
pdm.termui:                           httpx 0.23.1
pdm.termui:                       jsonpatch 1.32
pdm.termui:                      kubernetes 25.3.0
pdm.termui:                      setuptools 65.6.3
pdm.termui:                      markupsafe 2.1.1
pdm.termui:                      matplotlib 3.6.2
pdm.termui:          opencv-python-headless 4.6.0.66
pdm.termui:                        pathspec 0.10.3
pdm.termui:                        pendulum 2.1.2
pdm.termui:              pydata-google-auth 1.4.0
pdm.termui:                          pyproj 3.4.1
pdm.termui:                  python-slugify 7.0.0
pdm.termui:                          qudida 0.0.4
pdm.termui:                    scikit-learn 1.2.0
pdm.termui:                           scipy 1.9.3
pdm.termui:              querystring-parser 1.2.4
pdm.termui:                        readchar 4.0.3
pdm.termui:                            rich 12.6.0
pdm.termui:                    scikit-image 0.19.3
pdm.termui:                          snuggs 1.4.7
pdm.termui:                            toml 0.10.2
pdm.termui:                     torchvision 0.14.1
pdm.termui:                           typer 0.7.0
pdm.termui:                         uvicorn 0.20.0
pdm.termui:                        waitress 2.1.2
pdm.termui:                          affine 2.3.1
pdm.termui:            google-cloud-storage 2.7.0
pdm.termui:                             six 1.16.0
pdm.termui:                websocket-client 1.4.2
pdm.termui:                   grpcio-status 1.51.1
pdm.termui:                       pyparsing 3.0.9
pdm.termui:        googleapis-common-protos 1.57.0
pdm.termui:                   async-timeout 4.0.2
pdm.termui:                      cachetools 5.2.0
pdm.termui:                      commonmark 0.9.1
pdm.termui:                           gitdb 4.0.10
pdm.termui:                   google-crc32c 1.5.0
pdm.termui:                              h2 4.1.0
pdm.termui:                       multidict 6.0.4
pdm.termui:                        pygments 2.13.0
pdm.termui:                             rsa 4.9
pdm.termui:                sortedcontainers 2.4.0
pdm.termui:                           toolz 0.12.0
pdm.termui:                            yarl 1.8.2
pdm.termui:                       aiosignal 1.3.1
pdm.termui:                      frozenlist 1.3.3
pdm.termui:                            cffi 1.15.1
pdm.termui:                       contourpy 1.0.6
pdm.termui:                          cycler 0.11.0
pdm.termui:                       fonttools 4.38.0
pdm.termui:                             h11 0.14.0
pdm.termui:                         imageio 2.23.0
pdm.termui:                    itsdangerous 2.1.2
pdm.termui:                     jsonpointer 2.3
pdm.termui:                      kiwisolver 1.4.4
pdm.termui:                          locket 1.0.0
pdm.termui:                         msgpack 1.0.4
pdm.termui:                        networkx 2.8.8
pdm.termui:                        oauthlib 3.2.2
pdm.termui:                          psutil 5.9.4
pdm.termui:                  pyasn1-modules 0.2.8
pdm.termui:                           pyjwt 2.6.0
pdm.termui:                        pytzdata 2020.1
pdm.termui:                      pywavelets 1.4.1
pdm.termui:                         pywin32 305
pdm.termui:               requests-oauthlib 1.3.1
pdm.termui:                         sniffio 1.3.0
pdm.termui:                        tabulate 0.9.0
pdm.termui:                           tblib 1.7.0
pdm.termui:                  text-unidecode 1.3
pdm.termui:                   threadpoolctl 3.1.0
pdm.termui:                        tifffile 2022.10.10
pdm.termui:                           torch 1.13.1
pdm.termui:                         tornado 6.2
pdm.termui:                        werkzeug 2.2.2
pdm.termui:                            zict 2.2.0
pdm.termui:                            zipp 3.11.0
pdm.termui:                            mako 1.2.4
pdm.termui:                           munch 2.5.0
pdm.termui:                           numba 0.56.4
pdm.termui:                           numpy 1.23.5
pdm.termui:                          pyasn1 0.4.8
pdm.termui:                           hpack 4.0.0
pdm.termui:                        httpcore 0.16.3
pdm.termui:                      hyperframe 6.0.1
pdm.termui:                        llvmlite 0.39.1
pdm.termui:               rfc3986[idna2008] 1.5.0
pdm.termui:                         rfc3986 1.5.0
pdm.termui:                           smmap 5.0.0
pdm.termui:                           partd 1.3.0
pdm.termui:                        heapdict 1.0.1
pdm.termui:                       pycparser 2.21

Last empty line is intentional: It does not exit and does not return to shell prompt.

Actual behavior

pdm lock seems to resolve all dependencies but does not write a lockfile to disk.

Expected behavior

pdm lock should write the lockfile pdm.lock to disk after resolving all dependencies.

Environment Information

$ pdm info && pdm info --env
PDM version:
  2.3.3
Python Interpreter:
  /path/to/project/.venv/bin/python (3.10)
Project Root:
  /path/to/project
Project Packages:
  None
{
  "implementation_name": "cpython",
  "implementation_version": "3.10.9",
  "os_name": "posix",
  "platform_machine": "arm64",
  "platform_release": "22.2.0",
  "platform_system": "Darwin",
  "platform_version": "Darwin Kernel Version 22.2.0: Fri Nov 11 02:06:26 PST 2022; root:xnu-8792.61.2~4/RELEASE_ARM64_T8112",
  "python_full_version": "3.10.9",
  "platform_python_implementation": "CPython",
  "python_version": "3.10",
  "sys_platform": "darwin"
}

pdm and python are installed via brew. I'm using an existing 3.10 venv created with the python venv module (not virtualenv).

bug

opened by quassy 2

Dummy dependency problem during install of ansible-lint
[x] I have searched the issue tracker and this is definitely not a duplicate.

Make sure you run commands with -v flag before pasting the output.

Context

This is a follow up to the following issues:

https://github.com/ansible/ansible-lint/pull/2712 the thing that started it all

But most recent release v6.10.0 https://github.com/ansible/ansible-lint/pull/2807 drops setup.cfg (finally...), but the problem remains.

https://github.com/ansible/ansible-lint/issues/2816 the "pip works, we don't care about anything else".

https://github.com/python-poetry/poetry/issues/7112#issuecomment-1329385572 the poetry workaround

In short, ansible-lint decided to use a dummy/fake package (will-not-work-on-windows-try-from-wsl-instead; platform_system == "Windows") to stop installation on that unsupported platform.

edit. Feels related to https://github.com/pdm-project/pdm/issues/1535

Steps to reproduce

$ pdm --version PDM, version 2.3.3 $ pdm init ... $ pdm add -d 'ansible-lint' Adding packages to dev dev-dependencies: ansible-lint See /tmp/pdm-lock-7z2owtju.log for detailed debug log. [CandidateNotFound]: Unable to find candidates for will-not-work-on-windows-try-from-wsl-instead. There may exist some issues with the package name or network condition. Add '-v' to see the detailed traceback

Also trying the poetry workaround by adding the platform_system

#pyproject.toml ... [tool.pdm] [tool.pdm.dev-dependencies] dev = [ "ansible-lint<7.0.0,>=6.0.0; platform_system != \"Windows\"", ] ...

then running pdm install yields the same results.

Actual behavior

Package ansible-lint does not install.

Expected behavior

It unfortunately should, even with their horrendous dependency...

Environment Information

$ pdm info && pdm info --env PDM version: 2.3.3 Python Interpreter: /home/****/ansible-lint-test/.venv/bin/python (3.11) Project Root: /home/****/ansible-lint-test Project Packages: None { "implementation_name": "cpython", "implementation_version": "3.11.1", "os_name": "posix", "platform_machine": "x86_64", "platform_release": "6.0.12-arch1-1", "platform_system": "Linux", "platform_version": "#1 SMP PREEMPT_DYNAMIC Thu, 08 Dec 2022 11:03:38 +0000", "python_full_version": "3.11.1", "platform_python_implementation": "CPython", "python_version": "3.11", "sys_platform": "linux" }
bug
opened by la-magra 0

Releases(2.3.4)

2.3.4(Dec 27, 2022)
Features & Improvements

Detect PDM inside a zipapp and disable some functions. #1578

Bug Fixes

Don't write sitecustomize to the home directory if it exists in the filesystem(not packed in a zipapp). #1572

Fix a bug that a directory is incorrectly marked as to be deleted when it contains symlinks. #1580

Source code(tar.gz)
Source code(zip)
2.3.3(Dec 15, 2022)
Bug Fixes

Allow relative paths in build-system.requires, since build and hatch both support it. Be aware it is not allowed in the standard. #1560

Strip the local part when building a specifier for comparison with the package version. This is not permitted by PEP 508 as implemented by packaging 22.0. #1562

Update the version for check_update after self update #1563

Fix the matching problem of packages in the lockfile. #1569

Dependencies

Exclude package==22.0 from the dependencies to avoid some breakages to the end users. #1568

Source code(tar.gz)
Source code(zip)
2.3.2(Dec 8, 2022)
Bug Fixes

Fix an installation failure when the RECORD file contains commas in the file path. #1010

Fallback to pdm.pep517 as the metadata transformer for unknown custom build backends. #1546

Fix a bug that Ctrl + C kills the python interactive session instead of clearing the current line. #1547

Fix a bug with egg segment for local dependency #1552

Dependencies

Update installer to 0.6.0. #1550

Update minimum version of unearth to 0.6.3 and test against packaging==22.0. #1555

Source code(tar.gz)
Source code(zip)
2.3.1(Dec 5, 2022)
Bug Fixes

Fix a resolution loop issue when the current project depends on itself and it uses the dynamic version from SCM. #1541

Don't give duplicate results when specifying a relative path for pdm use. #1542

Source code(tar.gz)
Source code(zip)
2.3.0(Dec 2, 2022)
Features & Improvements

Beautify the error message of build errors. Default to showing the last 10 lines of the build output. #1491

Rename the tool.pdm.overrides table to tool.pdm.resolution.overrides. The old name is deprecated at the same time. #1503

Add backend selection and --backend option to pdm init command, users can choose a favorite backend from setuptools, flit, hatchling and pdm-pep517(default), since they all support PEP 621 standards. #1504

Allows specifying the insertion position of user provided arguments in scripts with the {args[:default]} placeholder. #1507

Bug Fixes

The local package is now treated specially during installation and locking. This means it will no longer be included in the lockfile, and should never be installed twice even when using nested extras. This will ensure the lockdown stays relevant when the version changes. #1481

Fix the version diff algorithm of installed packages to consider local versions as compatible. #1497

Fix the confusing message when detecting a Python interpreter under python.use_venv=False #1508

Fix the test failure with the latest findpython installed. #1516

Fix the module missing error of pywin32 in a virtualenv with install.cache set to true and caching method is pth. #863

Dependencies

Drop the dependency pdm-pep517. #1504

Replace pep517 with pyproject-hooks because of the rename. #1528

Removals and Deprecations

Remove the support for exporting the project file to a setup.py format, users are encouraged to migrate to the PEP 621 metadata. #1504

Source code(tar.gz)
Source code(zip)
2.2.1(Nov 3, 2022)
Features & Improvements

Make sitecustomize.py respect the PDM_PROJECT_MAX_DEPTH environment variable #1471

Bug Fixes

Fix the comparison of python_version in the environment marker. When the version contains only one digit, the result was incorrect. #1484

Source code(tar.gz)
Source code(zip)
2.2.0(Oct 31, 2022)
Features & Improvements

Add venv.prompt configuration to allow customizing prompt when a virtualenv is activated #1332

Allow the use of custom CA certificates per publish repository using ca_certs or from the command line via pdm publish --ca-certs <path> .... #1392

Rename the plugin command to self, and it can not only manage plugins but also all dependencies. Add a subcommand self update to update PDM itself. #1406

Allow pdm init to receive a Python path or version via --python option. #1412

Add a default value for requires-python when importing from other formats. #1426

Use pdm instead of pip to resolve and install build requirements. So that PDM configurations can control the process. #1429

Customizable color theme via pdm config command. #1450

A new pdm lock --check flag to validate whether the lock is up to date. #1459

Add both option and config item to ship pip when creating a new venv. #1463

Issue warning and skip the requirement if it has the same name as the current project. #1466

Enhance the pdm list command with new formats: --csv,--markdown and add options --fields,--sort to control the output contents. Users can also include licenses in the --fields option to display the package licenses. #1469

A new pre-commit hook to run pdm lock --check in pre-commit. #1471

Bug Fixes

Fix the issue that relative paths don't work well with --project argument. #1220

It is now possible to refer to a package from outside the project with relative paths in dependencies. #1381

Ensure pypi.[ca,client]_cert[s] config items are passed to distribution builder install steps to allow for custom PyPI index sources with self signed certificates. #1396

Fix a crash issue when depending on editable packages with extras. #1401

Do not save the python path when using non-interactive mode in pdm init. #1410

Fix the matching of python* command in pdm run. #1414

Show the Python path, instead of the real executable, in the Python selection menu. #1418

Fix the HTTP client of package publishment to prompt for password and read PDM configurations correctly. #1430

Ignore the unknown fields when constructing a requirement object. #1445

Fix a bug of unrelated candidates being fetched if the requirement is matching wildcard versions(e.g. ==1.*). #1465

Use importlib-metadata from PyPI for Python < 3.10. #1467

Documentation

Clarify the difference between a library and an application. Update the guide of multi-stage docker build. #1371

Removals and Deprecations

Remove all top-level imports, users should import from the submodules instead. #1404

Remove the usages of old config names deprecated since 2.0. #1422

Remove the deprecated color functions, use rich's console markup instead. #1452

Source code(tar.gz)
Source code(zip)
2.1.5(Oct 5, 2022)
Bug Fixes

Ensure pypi.[ca,client]_cert[s] config items are passed to distribution builder install steps to allow for custom PyPI index sources with self signed certificates. #1396

Fix a crash issue when depending on editable packages with extras. #1401

Do not save the python path when using non-interactive mode in pdm init. #1410

Restrict importlib-metadata (<5.0.0) for Python <3.8 #1411

Source code(tar.gz)
Source code(zip)
2.1.4(Sep 17, 2022)
Bug Fixes

Fix a lock failure when depending on self with URL requirements. #1347

Ensure list to concatenate args for composite scripts. #1359

Fix an error in pdm lock --refresh if some packages has URLs. #1361

Fix unnecessary package downloads and VCS clones for certain commands. #1370

Fix a conversion error when converting a list of conditional dependencies from a Poetry format. #1383

Documentation

Adds a section to the docs on how to correctly work with PDM and version control systems. #1364

Source code(tar.gz)
Source code(zip)
2.1.3(Aug 30, 2022)
Features & Improvements

When adding a package to (or removing from) a group, enhance the formatting of the group name in the printed message. #1329

Bug Fixes

Fix a bug of missing hashes for packages with file:// links the first time they are added. #1325

Ignore invalid values of data-requires-python when parsing package links. #1334

Leave an incomplete project metadata if PDM fails to parse the project files, but emit a warning. #1337

Fix the bug that editables package isn't installed for self package. #1344

Fix a decoding error for non-ASCII characters in package description when publishing it. #1345

Documentation

Clarify documentation explaining setup-script, run-setuptools, and is-purelib. #1327

Source code(tar.gz)
Source code(zip)
2.1.2(Aug 15, 2022)
Bug Fixes

Fix a bug that dependencies from different versions of the same package override each other. #1307

Forward SIGTERM to child processes in pdm run. #1312

Fix errors when running on FIPS 140-2 enabled systems using Python 3.9 and newer. #1313

Fix the build failure when the subprocess outputs with non-UTF8 characters. #1319

Delay the trigger of post_lock for add and update operations, to ensure the pyproject.toml is updated before the hook is run. #1320

Source code(tar.gz)
Source code(zip)
2.1.1(Aug 5, 2022)
Features & Improvements

Add a env_file.override option that allows the user to specify that the env_file should override any existing environment variables. This is not the default as the environment the code runs it should take precedence. #1299

Bug Fixes

Fix a bug that unnamed requirements can't override the old ones in either add or update command. #1287

Support mutual TLS to private repositories via pypi.client_cert and pypi.client_key config options. #1290

Set a minimum version for the packaging dependency to ensure that packaging.utils.parse_wheel_filename is available. #1293

Fix a bug that checking for PDM update creates a venv. #1301

Prefer compatible packages when fetching metadata. #1302

Source code(tar.gz)
Source code(zip)
2.1.0(Jul 29, 2022)
Features & Improvements

Allow the use of custom CA certificates using the pypi.ca_certs config entry. #1240

Add pdm export to available pre-commit hooks. #1279

Bug Fixes

Skip incompatible requirements when installing build dependencies. #1264

Fix a crash when pdm tries to publish a package with non-ASCII characters in the metadata. #1270

Try to read the lock file even if the lock version is incompatible. #1273

For packages that are only available as source distribution, the summary field in pdm.lock contains the description from the package's pyproject.toml. #1274

Do not crash when calling pdm show for a package that is only available as source distribution. #1276

Fix a bug that completion scripts are interpreted as rich markups. #1283

Dependencies

Remove the dependency of pip. #1268

Removals and Deprecations

Deprecate the top-level imports from pdm module, it will be removed in the future. #1282

Source code(tar.gz)
Source code(zip)
1.15.5(Jul 23, 2022)
Dependencies

1.15.x: set upper bound for pip dependency to <22.2. #1261

Source code(tar.gz)
Source code(zip)
2.0.3(Jul 22, 2022)
Bug Fixes

Support Conda environments when detecting the project environment. #1253

Fix the interpreter resolution to first try python executable in the PATH. #1255

Stabilize sorting of URLs in metadata.files in pdm.lock. #1256

Don't expand credentials in the file URLs in the [metada.files] table of the lock file. #1259

Source code(tar.gz)
Source code(zip)
2.0.2(Jul 20, 2022)
Features & Improvements

env_file variables no longer override existing environment variables. #1235

Support referencing other optional groups in optional-dependencies with <this_package_name>[group1, group2] #1241

Bug Fixes

Respect requires-python when creating the default venv. #1237

Source code(tar.gz)
Source code(zip)
2.0.1(Jul 17, 2022)
Bug Fixes

Write lockfile before calling 'post_lock' hook #1224

Suppress errors when cache dir isn't accessible. #1226

Don't save python path for venv commands. #1230

Source code(tar.gz)
Source code(zip)
2.0.0(Jul 15, 2022)
Bug Fixes

Fix a bug that the running env overrides the PEP 582 PYTHONPATH. #1211

Add pwsh as an alias of powershell for shell completion. #1216

Fixed a bug with zsh completion regarding --pep582 flag. #1218

Fix a bug of requirement checking under non-isolated mode. #1219

Fix a bug when removing packages, TOML document might become invalid. #1221

Source code(tar.gz)
Source code(zip)
2.0.0b2(Jul 8, 2022)
Install and test

pipx:

pipx install --pip-args=--pre pdm

install script:

curl -sSL https://raw.githubusercontent.com/pdm-project/pdm/main/install-pdm.py | python3 - --prerelease # Or Windows powershell: (Invoke-WebRequest -Uri https://raw.githubusercontent.com/pdm-project/pdm/main/install-pdm.py -UseBasicParsing).Content | python - --prerelease

setup-pdm action:

- uses: pdm-project/setup-pdm@main with: prerelease: true

Breaking Changes

Store file URLs instead of filenames in the lock file, bump lock version to 4.0. #1203

Features & Improvements

Read site-wide configuration, which serves as the lowest-priority layer. This layer will be read-only in the CLI. #1200

Get package links from the urls stored in the lock file. #1204

Bug Fixes

Fix a bug that the host pip(installed with pdm) may not be compatible with the project python. #1196

Update unearth to fix a bug that install links with weak hashes are skipped. This often happens on self-hosted PyPI servers. #1202

Source code(tar.gz)
Source code(zip)
2.0.0b1(Jul 2, 2022)
Install and test

pipx:

pipx install --pip-args=--pre pdm

install script:

curl -sSL https://raw.githubusercontent.com/pdm-project/pdm/main/install-pdm.py | python3 - --prerelease # Or Windows powershell: (Invoke-WebRequest -Uri https://raw.githubusercontent.com/pdm-project/pdm/main/install-pdm.py -UseBasicParsing).Content | python - --prerelease

setup-pdm action:

- uses: pdm-project/setup-pdm@main with: prerelease: true

What's Changed

Features & Improvements

Integrate pdm venv commands into the main program. Make PEP 582 an opt-in feature. #1162

Add config global_project.fallback_verbose defaulting to True. When set to False disables message Project is not found, fallback to the global project #1188

Add --only-keep option to pdm sync to keep only selected packages. Originally requested at #398. #1191

Bug Fixes

Fix a bug that requirement extras and underlying are resolved to the different version #1173

Update unearth to 0.4.1 to skip the wheels with invalid version parts. #1178

Fix reading PDM_RESOLVE_MAX_ROUNDS environment variable (was spelled …ROUDNS before). #1180

Deduplicate the list of found Python versions. #1182

Use the normal stream handler for logging, to fix some display issues under non-tty environments. #1184

Removals and Deprecations

Remove the useless --no-clean option from pdm sync command. #1191

Source code(tar.gz)
Source code(zip)
2.0.0a1(Jun 29, 2022)
Install and test

pipx:

pipx install --pip-args=--pre pdm

install script:

curl -sSL https://raw.githubusercontent.com/pdm-project/pdm/main/install-pdm.py | python3 - --prerelease # Or Windows powershell: (Invoke-WebRequest -Uri https://raw.githubusercontent.com/pdm-project/pdm/main/install-pdm.py -UseBasicParsing).Content | python - --prerelease

setup-pdm action:

- uses: pdm-project/setup-pdm@main with: prerelease: true

What's Changed

Breaking Changes

Editable dependencies in the [project] table is not allowed, according to PEP 621. They are however still allowed in the [tool.pdm.dev-dependencies] table. PDM will emit a warning when it finds editable dependencies in the [project] table, or will abort when you try to add them into the [project] table via CLI. #1083

Now the paths to the global configurations and global project are calculated according to platform standards. #1161

Features & Improvements

Add support for importing from a setup.py project. #1062

Switch the UI backend to rich. #1091

Improved the terminal UI and logging. Disable live progress under verbose mode. The logger levels can be controlled by the -v option. #1096

Use unearth to replace pip's PackageFinder and related data models. PDM no longer relies on pip internals, which are unstable across updates. #1096

Lazily load the candidates returned by find_matches() to speed up the resolution. #1098

Add a new command publish to PDM since it is required for so many people and it will make the workflow easier. #1107

Add a composite script kind allowing to run multiple defined scripts in a single command as well as reusing scripts but overriding env or env_file. #1117

Add a new execution option --skip to opt-out some scripts and hooks from any execution (both scripts and PDM commands). #1127

Add the pre/post_publish, pre/post_run and pre/post_script hooks as well as an extensive lifecycle and hooks documentation. #1147

Shorter scripts listing, especially for multilines and composite scripts. #1151

Build configurations have been moved to [tool.pdm.build], according to pdm-pep517 1.0.0. At the same time, warnings will be shown against old usages. #1153

Improve the lock speed by parallelizing the hash fetching. #1154

Retrieve the candidate metadata by parsing the pyproject.toml rather than building it. #1156

Update the format converters to support the new [tool.pdm.build] table. #1157

Scripts are now available as root command if they don't conflict with any builtin or plugin-contributed command. #1159

Add a post_use hook triggered after succesfully switching Python version. #1163

Add project configuration respect-source-order under [tool.pdm.resolution] to respect the source order in the pyproject.toml file. Packages will be returned by source earlier in the order or later ones if not found. #593

Bug Fixes

Fix a bug that candidates with local part in the version can't be found and installed correctly. #1093

Dependencies

Prefer tomllib on Python 3.11 #1072

Drop the vendored libraries click, halo, colorama and log_symbols. PDM has no vendors now. #1091

Update dependency version pdm-pep517 to 1.0.0. #1153

Removals and Deprecations

PDM legacy metadata format(from pdm 0.x) is no longer supported. #1157

Miscellany

Provide a tox.ini file for easier local testing against all Python versions. #1160

Source code(tar.gz)
Source code(zip)
1.15.4(Jun 28, 2022)
Bug Fixes

Revert #1106: Do not use venv scheme for prefix kind install scheme. #1158

Fix a bug when updating a package with extra requirements, the package version doesn't get updated correctly. #1166

Miscellany

Add additional installation option via asdf-pdm. Add skip-add-to-path option to installer in order to prevent changing PATH. Replace bin variable name with bin_dir. #1145

Source code(tar.gz)
Source code(zip)
1.15.3(Jun 14, 2022)
Bug Fixes

Fix a defect in the resolution preferences that causes an infinite resolution loop. #1119

Update the poetry importer to support the new [tool.poetry.build] config table. #1131

Improved Documentation

Add support for multiple versions of documentations. #1126

Source code(tar.gz)
Source code(zip)
1.15.2(Jun 6, 2022)
Bug Fixes

Fix bug where SIGINT is sent to the main pdm process and not to the process actually being run. #1095

Fix a bug due to the build backend fallback, which causes different versions of the same requirement to exist in the build environment, making the building unstable depending on which version being used. #1099

Don't include the version in the cache key of the locked candidates if they are from a URL requirement. #1099

Fix a bug where dependencies with requires-python pre-release versions caused pdm update to fail with InvalidPyVersion. #1111

Source code(tar.gz)
Source code(zip)
1.15.1(Jun 1, 2022)
Bug Fixes

Fix a bug that dependencies are missing from the dep graph when they are depended by a requirement with extras. #1097

Give a default version if the version is dynamic in setup.cfg or setup.py. #1101

Fix a bug that the hashes for file URLs are not included in the lock file. #1103

Fix a bug that package versions are updated even when they are excluded by pdm update command. #1104

Prefer venv install scheme when available. This scheme is more stable than posix_prefix scheme since the latter is often patched by distributions. #1106

Miscellany

Move the test artifacts to a submodule. It will make it easier to package this project. #1084

Source code(tar.gz)
Source code(zip)
1.15.0(May 16, 2022)
Features & Improvements

Allow specifying lockfile other than pdm.lock by --lockfile option or PDM_LOCKFILE env var. #1038

Bug Fixes

Replace the editable entry in pyproject.toml when running pdm add --no-editable <package>. #1050

Ensure the pip module inside venv in installation script. #1053

Fix the py2 compatiblity issue in the in-process get_sysconfig_path.py script. #1056

Fix a bug that file paths in URLs are not correctly unquoted. #1073

Fix a bug on Python 3.11 that overriding an existing command from plugins raises an error. #1075

Replace the ${PROJECT_ROOT} variable in the result of export command. #1079

Removals and Deprecations

Show a warning if Python 2 interpreter is being used and remove the support on 2.0. #1082

Source code(tar.gz)
Source code(zip)
test-artifacts.tar.gz(2.90 MB)
1.14.1(Apr 21, 2022)
Features & Improvements

Ask for description when doing pdm init and create default README for libraries. #1041

Bug Fixes

Fix a bug of missing subdirectory fragment when importing from a requirements.txt. #1036

Fix use_cache.json with corrupted python causes pdm use error. #1039

Ignore the optional key when converting from Poetry's dependency entries. #1042

Improved Documentation

Clarify documentation on enabling PEP582 globally. #1033

Source code(tar.gz)
Source code(zip)
1.14.0(Apr 8, 2022)
Features & Improvements

Editable installations won't be overridden unless --no-editable is passed. pdm add --no-editable will now override the editable mode of the given packages. #1011

Re-calculate the file hashes when running pdm lock --refresh. #1019

Bug Fixes

Fix a bug that requirement with extras isn't resolved to the version as specified by the range. #1001

Replace the ${PROJECT_ROOT} in the output of pdm list. #1004

Further fix the python path issue of MacOS system installed Python. #1023

Fix the install path issue on Python 3.10 installed from homebrew. #996

Improved Documentation

Document how to install PDM inside a project with Pyprojectx. #1004

Dependencies

Support installer 0.5.x. #1002

Source code(tar.gz)
Source code(zip)
1.13.6(Mar 28, 2022)
Bug Fixes

Default the optional license field to "None". #991

Don't create project files in pdm search command. #993

Fix a bug that the env vars in source urls in exported result are not expanded. #997

Source code(tar.gz)
Source code(zip)
1.13.5(Mar 23, 2022)
Features & Improvements

Users can change the install destination of global project to the user site(~/.local) with global_project.user_site config. #885

Make the path to the global project configurable. Rename the configuration auto_global to global_project.fallback and deprecate the old name. #986

Bug Fixes

Fix the compatibility when fetching license information in show command. #966

Don't follow symlinks for the paths in the requirement strings. #976

Use the default install scheme when installing build requirements. #983

Fix a bug that _.site_packages is overriden by default option value. #985

Source code(tar.gz)
Source code(zip)

A modern Python package manager with PEP 582 support.

Related tags

Overview

PDM - Python Development Master

What is PDM?

Highlights of features

Why not virtualenv?

Installation

Alternative Installation Methods

Quickstart

Docker image

Badges

PDM Eco-system

FAQ

1. What is put in __pypackages__?

2. How do I run CLI scripts in the local package directory?

3. What site-packages will be loaded when using PDM?

4. Can I relocate or move the __pypackages__ folder for deployment?

5. Can I use pdm to manage a Python 2.7 project?

Credits

License

Comments

Is your feature request related to a problem? Please describe.

Describe the solution you'd like

Is your feature request related to a problem? Please describe.

Add B as a local dependency into A project

Install A

Describe the solution you'd like

Is your feature request related to a problem? Please describe.

Describe the solution you'd like

Pull Request Check List

Describe what you have changed in this PR.

Steps to reproduce

Actual behavior

Expected behavior

Environment Information

Steps to reproduce

Actual behavior

Expected behavior

Environment Information

Planned changes in version 2.0

Is your feature request related to a problem? Please describe.

Describe the solution you'd like

Environment Information

Steps to reproduce

Actual behavior

Expected behavior

Environment Information

Steps to reproduce

Actual behavior

Expected behavior

Environment Information

Describe what you have changed in this PR.

Pull Request Check List

Steps to reproduce

Actual behavior

Expected behavior

Environment Information

Steps to reproduce

Actual behavior

Expected behavior

Environment Information

Context

Steps to reproduce

Actual behavior

Expected behavior

Environment Information

Releases(2.3.4)

2.3.4(Dec 27, 2022)

Features & Improvements

Bug Fixes

2.3.3(Dec 15, 2022)

Bug Fixes

Dependencies

2.3.2(Dec 8, 2022)

Bug Fixes

Dependencies

2.3.1(Dec 5, 2022)

Bug Fixes

2.3.0(Dec 2, 2022)

1. What is put in `pypackages`?

4. Can I relocate or move the `pypackages` folder for deployment?

5. Can I use `pdm` to manage a Python 2.7 project?