Library and command-line utility for rendering projects templates.

Last update: Jan 4, 2023

Related tags

Overview

A library for rendering project templates.

Works with local paths and git URLs.
Your project can include any file and Copier can dynamically replace values in any kind of text file.
It generates a beautiful output and takes care of not overwrite existing files unless instructed to do so.

Installation

Install Python 3.6.1 or newer (3.8 or newer if you're on Windows).
Install Git 2.24 or newer.
To use as a CLI app: pipx install copier
To use as a library: pip install copier

Quick usage

Use it in your Python code:

from copier import copy

# Create a project from a local path
copy("path/to/project/template", "path/to/destination")

# Or from a git URL.
copy("https://github.com/copier-org/copier.git", "path/to/destination")

# You can also use "gh:" as a shortcut of "https://github.com/"
copy("gh:copier-org/copier.git", "path/to/destination")

# Or "gl:" as a shortcut of "https://gitlab.com/"
copy("gl:copier-org/copier.git", "path/to/destination")

Or as a command-line tool:

copier path/to/project/template path/to/destination

Browse or tag public templates

You can browse public copier templates in GitHub using the copier-template topic. Use them as inspiration!

If you want your template to appear in that list, just add the topic to it! 🏷

Credits

Special thanks go to jpscaletti for originally creating Copier. This project would not be a thing without him.

Many thanks to pykong who took over maintainership on the project, promoted it, and laid out the bases of what the project is today.

Big thanks also go to Yajo for his relentless zest for improving Copier even further.

Thanks a lot, pawamoy for polishing very important rough edges and improving the documentation and UX a lot.

Comments

Flexible conflict resolution
Fixes #613

Based on #627, but:

Supports both old (.rej files) and new (inline markers) conflict behavior

Organizes the code to reduce likelihood of merge conflicts (while this PR is in development/review)
opened by barrywhart 22
Add the ability to customise how questions are asked

Discussed in https://github.com/copier-org/copier/discussions/673

^{Originally posted by pfmoore May 23, 2022} The prompt currently appears to be a 🎤emoji, followed by the help (if specified), then the item name and a format prompt. I'd like to be able to change the prompt as I find it difficult to read (in particular, the format prompt is very distracting, and I'd like to omit it). Is that possible?

Having checked the source code, it seems that this isn't possible. But the questionary library looks very flexible, so it would be possible to do this if there was a way to describe the format in the template configuration. Would such a change be welcome? I'd be more than happy to work on a PR for this, but I'm very new to the project (I'm currently trying to migrate my existing cookiecutter templates over to copier) so I don't know if that's something that would be considered a reasonable feature for the project.

I'd imagine doing this via settings in the copier.yml file, so the format would be decided by the template, rather than by the user. Would that make sense?
enhancement good first issue help wanted

opened by pfmoore 22

bug: _templates_suffix: "" gives UnicodeDecodeError on binary files ( .ico or .png for example )

Describe the problem

I want to use _templates_suffix: "" The problem is if I use it, then even image files will be processed by copier. And it will give error:

UnicodeDecodeError: 'utf-8' codec can't decode byte 0x93 in position 14: invalid start byte

Environment

OS: Ubuntu on WSL2 on Win10
Copier version: copier 6.0.0a6.post43.dev0+2fe3072
Python version: 3.9.6
Installation method: pipx+git

Additional context

    create  .
    create  .dependabot
    create  .dependabot/config.yml
    create  .en v copy
    create  .env
    create  .gitignore
    create  LICENSE
    create  README.md
    create  Banana
    create  Banana/.prettierignore
    create  Banana/backend
    create  Banana/backend/alembic.ini
    create  Banana/backend/app
    create  Banana/backend/app/alembic
    create  Banana/backend/app/alembic/env.py
    create  Banana/backend/app/alembic/README
    create  Banana/backend/app/alembic/script.py.mako
    create  Banana/backend/app/alembic/versions
    create  Banana/backend/app/alembic/versions/91979b40eb38_create_users_table.py
    create  Banana/backend/app/alembic/__init__.py
    create  Banana/backend/app/alembic.ini
    create  Banana/backend/app/api
    create  Banana/backend/app/api/api_v1
    create  Banana/backend/app/api/api_v1/routers
    create  Banana/backend/app/api/api_v1/routers/auth.py
    create  Banana/backend/app/api/api_v1/routers/tests
    create  Banana/backend/app/api/api_v1/routers/tests/test_auth.py
    create  Banana/backend/app/api/api_v1/routers/tests/test_users.py
    create  Banana/backend/app/api/api_v1/routers/tests/__init__.py
    create  Banana/backend/app/api/api_v1/routers/users.py
    create  Banana/backend/app/api/api_v1/routers/__init__.py
    create  Banana/backend/app/api/api_v1/__init__.py
    create  Banana/backend/app/api/dependencies
    create  Banana/backend/app/api/dependencies/__init__.py
    create  Banana/backend/app/api/__init__.py
    create  Banana/backend/app/core
    create  Banana/backend/app/core/auth.py
    create  Banana/backend/app/core/celery_app.py
    create  Banana/backend/app/core/config.py
    create  Banana/backend/app/core/security.py
    create  Banana/backend/app/core/__init__.py
    create  Banana/backend/app/db
    create  Banana/backend/app/db/crud.py
    create  Banana/backend/app/db/models.py
    create  Banana/backend/app/db/schemas.py
    create  Banana/backend/app/db/session.py
    create  Banana/backend/app/db/__init__.py
    create  Banana/backend/app/initial_data.py
    create  Banana/backend/app/main.py
    create  Banana/backend/app/tasks.py
    create  Banana/backend/app/tests
    create  Banana/backend/app/tests/test_main.py
    create  Banana/backend/app/tests/test_tasks.py
    create  Banana/backend/app/tests/__init__.py
    create  Banana/backend/app/__init__.py
    create  Banana/backend/conftest.py
    create  Banana/backend/Dockerfile
    create  Banana/backend/pyproject.toml
    create  Banana/backend/requirements.txt
    create  Banana/docker-compose.yml
    create  Banana/frontend
    create  Banana/frontend/.dockerignore
    create  Banana/frontend/.eslintrc.js
    create  Banana/frontend/.prettierrc.js
    create  Banana/frontend/Dockerfile
    create  Banana/frontend/package.json
    create  Banana/frontend/public
Traceback (most recent call last):
  File "/home/orange/.local/bin/copier", line 8, in <module>
    sys.exit(CopierApp.run())
  File "/home/orange/.local/pipx/venvs/copier/lib/python3.8/site-packages/plumbum/cli/application.py", line 614, in run
    inst, retcode = subapp.run(argv, exit=False)
  File "/home/orange/.local/pipx/venvs/copier/lib/python3.8/site-packages/plumbum/cli/application.py", line 609, in run
    retcode = inst.main(*tailargs)
  File "/home/orange/.local/pipx/venvs/copier/lib/python3.8/site-packages/copier/cli.py", line 70, in _wrapper
    return method(*args, **kwargs)
  File "/home/orange/.local/pipx/venvs/copier/lib/python3.8/site-packages/copier/cli.py", line 293, in main
    self.parent._worker(
  File "/home/orange/.local/pipx/venvs/copier/lib/python3.8/site-packages/copier/main.py", line 575, in run_copy
    self._render_folder(src_abspath)
  File "/home/orange/.local/pipx/venvs/copier/lib/python3.8/site-packages/copier/main.py", line 467, in _render_folder
    self._render_folder(file)
  File "/home/orange/.local/pipx/venvs/copier/lib/python3.8/site-packages/copier/main.py", line 467, in _render_folder
    self._render_folder(file)
  File "/home/orange/.local/pipx/venvs/copier/lib/python3.8/site-packages/copier/main.py", line 467, in _render_folder
    self._render_folder(file)
  File "/home/orange/.local/pipx/venvs/copier/lib/python3.8/site-packages/copier/main.py", line 469, in _render_folder
    self._render_file(file)
  File "/home/orange/.local/pipx/venvs/copier/lib/python3.8/site-packages/copier/main.py", line 429, in _render_file
    tpl = self.jinja_env.get_template(str(src_relpath))
  File "/home/orange/.local/pipx/venvs/copier/lib/python3.8/site-packages/jinja2/environment.py", line 883, in get_template
    return self._load_template(name, self.make_globals(globals))
  File "/home/orange/.local/pipx/venvs/copier/lib/python3.8/site-packages/jinja2/environment.py", line 857, in _load_template
    template = self.loader.load(self, name, globals)
  File "/home/orange/.local/pipx/venvs/copier/lib/python3.8/site-packages/jinja2/loaders.py", line 115, in load
    source, filename, uptodate = self.get_source(environment, name)
  File "/home/orange/.local/pipx/venvs/copier/lib/python3.8/site-packages/jinja2/loaders.py", line 184, in get_source
    contents = f.read().decode(self.encoding)
UnicodeDecodeError: 'utf-8' codec can't decode byte 0x93 in position 14: invalid start byte

bug good first issue help wanted

opened by karolzlot 21

feat: questionnaire validation
I'd love a possibility to validate questionnaire inputs. Being able to validate str against regex would already be a huge help, e.g. by specifying a field appropriately in copier.yml:

project_name: type: str help: Your project name validate: ^[a-zA-Z][_a-zA-Z0-9]+$
enhancement
opened by RoyalTS 16
Add a slugify filter
I cannot think of a template I wrote that didn't need such a filter 🙂

I usually write a very naive one myself:

repository_name: type: str help: Your repository name default: "[[ project_name|lower|replace('_', '-')|replace(' ', '-') ]]"

But it will break in so many cases.

As an alternative, if we don't want to support more and more filters in copier, maybe we could think of allowing custom filters to be used. Take a look at https://github.com/mattrobenolt/jinja2-cli/pull/86, it allows this use of jinja2-cli:

jinja2 -f ansible.plugins.filter.ipaddr.ipaddr test.txt

In copier's case, we would not be able to load the filter using its Python dotted-path, so we would have to think of another way to add it to the Jinja environment.
opened by pawamoy 16
Autodoc
Issue #173

We use mkdocs to write and build the documentation, with the material theme for mkdocs, as well as the mkdocstrings plugin for auto-documentation of the source code.

I had to remove the doctoc pre-commit hook because I couldn't get its generated tables of contents to be rendered correctly in the documentation (they are a mix of HTML and Markdown). Note that this is not a great loss, as the material theme for mkdocs adds its own table of contents in the right sidebar for every page. We lose the ToC on GitHub only (and it was not really usable either because it's not sticky).

To try it, clone my branch and run poetry run mkdocs serve, then go to localhost:8000.

I could use early feedback on the following TODO points:

TODO:

[x] Improve the code documentation: some modules have no docstrings at all, so they appear empty in the docs. The other docstrings could really be detailed a bit more. We can also benefit from the cross-reference ability of mkdocstrings, linking objects between them to improve navigation in the reference.

[x] Customize the docs? Version 5 of mkdocs-material makes it easier to customize the docs color and icons. I'd be more inclined to let the authors/maintainers of copier do so :slightly_smiling_face:

[x] Split the README (aka overview in the docs) into multiple files? Create a template, generate a project, update a project, etc..., then clean it up (remove API section maybe).

enhancement maintenance
opened by pawamoy 15
Show diff on conflict before deciding if overwrite or skip interactively

When the user is asked interactively if he would want to overwrite or skip a conflicting file, he has no way to know what's the good answer.

This relies on the expectation that the subproject will be git-tracked and thus the user is able to inspect the diff afterwards. But, if that's not the case (for example a git-ignored file with secrets within), he cannot safely decide.

In the case where it is git-tracked, anyway being able to inspect the diff interactively still seems to be a good UX enhancement.
enhancement

opened by yajo 14
Refactoring of configuration
This PR cleans and hardens the code base via the following refactorizations:

Untangling the creation of the configuration object from copy_local.

Introducing basic data validation and default handling via pydantic.

Various minor improvements like making flags a class for explicit member access via dot syntax.

TODO

[x] Make tests green.

[x] Add pydantic to dependencies.

[x] Fix type annotations.

[x] Consider moving user_data into sub module to together with conf

[x] General cleanup and removal of dead code.

[x] Add new tests.

[x] Test merging precedence.

Preceded by: #54

Closes: #49
opened by pykong 14

`pip install copier` fails on windows

Describe the problem Running pip install copier with python 3.10 fails on Windows when the Microsoft C++ build tools package is not installed. I would prefer not install the heavy (7GB) c++ compiler installed just to use copier.

This seems to be caused by the iteration_utilities package lacking a python 3.10 wheel file in pypi and it doesn't look like they are planning to generate wheel files for python 3.10 any time soon.

To Reproduce

Run pip install copier on Windows with python 3.10 and without visual c++ compiler installed

Logs

Running pip install copier

PS C:\Users\wilson> pip install copier
Collecting copier
  Using cached copier-7.0.1-py3-none-any.whl (34 kB)
Collecting iteration_utilities>=0.11.0
  Using cached iteration_utilities-0.11.0.tar.gz (124 kB)
  Preparing metadata (setup.py) ... done
Collecting pygments>=2.7.1
  Downloading Pygments-2.13.0-py3-none-any.whl (1.1 MB)
     ---------------------------------------- 1.1/1.1 MB 6.3 MB/s eta 0:00:00
Requirement already satisfied: plumbum>=1.6.9 in c:\users\wilson\appdata\local\programs\python\python310\lib\site-packages (from copier) (1.8.0)
Collecting pyyaml-include>=1.2
  Downloading pyyaml_include-1.3-py3-none-any.whl (20 kB)
Collecting questionary>=1.8.1
  Downloading questionary-1.10.0-py3-none-any.whl (31 kB)
Collecting jinja2-ansible-filters>=1.3.1
  Downloading jinja2_ansible_filters-1.3.2-py3-none-any.whl (18 kB)
Requirement already satisfied: packaging>=21.0 in c:\users\wilson\appdata\local\programs\python\python310\lib\site-packages (from copier) (21.3)
Collecting jinja2>=3.1.1
  Downloading Jinja2-3.1.2-py3-none-any.whl (133 kB)
     ---------------------------------------- 133.1/133.1 kB 4.0 MB/s eta 0:00:00
Collecting pydantic>=1.10.2
  Downloading pydantic-1.10.2-cp310-cp310-win_amd64.whl (2.1 MB)
     ---------------------------------------- 2.1/2.1 MB 6.3 MB/s eta 0:00:00
Collecting dunamai>=1.7.0
  Downloading dunamai-1.15.0-py3-none-any.whl (23 kB)
Collecting pyyaml>=5.3.1
  Using cached PyYAML-6.0-cp310-cp310-win_amd64.whl (151 kB)
Requirement already satisfied: colorama>=0.4.3 in c:\users\wilson\appdata\local\programs\python\python310\lib\site-packages (from copier) (0.4.5)
Requirement already satisfied: pathspec>=0.9.0 in c:\users\wilson\appdata\local\programs\python\python310\lib\site-packages (from copier) (0.10.2)
Collecting MarkupSafe>=2.0
  Downloading MarkupSafe-2.1.1-cp310-cp310-win_amd64.whl (17 kB)
Requirement already satisfied: pyparsing!=3.0.5,>=2.0.2 in c:\users\wilson\appdata\local\programs\python\python310\lib\site-packages (from packaging>=21.0->copier) (3.0.9)
Requirement already satisfied: pywin32 in c:\users\wilson\appdata\local\programs\python\python310\lib\site-packages (from plumbum>=1.6.9->copier) (305)
Requirement already satisfied: typing-extensions>=4.1.0 in c:\users\wilson\appdata\local\programs\python\python310\lib\site-packages (from pydantic>=1.10.2->copier) (4.4.0)
Collecting prompt_toolkit<4.0,>=2.0
  Downloading prompt_toolkit-3.0.33-py3-none-any.whl (383 kB)
     ---------------------------------------- 383.6/383.6 kB 5.9 MB/s eta 0:00:00
Collecting wcwidth
  Downloading wcwidth-0.2.5-py2.py3-none-any.whl (30 kB)
Building wheels for collected packages: iteration_utilities
  Building wheel for iteration_utilities (setup.py) ... error
  error: subprocess-exited-with-error

  × python setup.py bdist_wheel did not run successfully.
  │ exit code: 1
  ╰─> [15 lines of output]
      running bdist_wheel
      running build
      running build_py
      creating build
      creating build\lib.win-amd64-cpython-310
      creating build\lib.win-amd64-cpython-310\iteration_utilities
      copying src\iteration_utilities\_additional_recipes.py -> build\lib.win-amd64-cpython-310\iteration_utilities
      copying src\iteration_utilities\_classes.py -> build\lib.win-amd64-cpython-310\iteration_utilities
      copying src\iteration_utilities\_convenience.py -> build\lib.win-amd64-cpython-310\iteration_utilities
      copying src\iteration_utilities\_recipes.py -> build\lib.win-amd64-cpython-310\iteration_utilities
      copying src\iteration_utilities\_utils.py -> build\lib.win-amd64-cpython-310\iteration_utilities
      copying src\iteration_utilities\__init__.py -> build\lib.win-amd64-cpython-310\iteration_utilities
      running build_ext
      building 'iteration_utilities._iteration_utilities' extension
      error: Microsoft Visual C++ 14.0 or greater is required. Get it with "Microsoft C++ Build Tools": https://visualstudio.microsoft.com/visual-cpp-build-tools/
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
  ERROR: Failed building wheel for iteration_utilities
  Running setup.py clean for iteration_utilities
Failed to build iteration_utilities
Installing collected packages: wcwidth, pyyaml, pygments, pydantic, prompt_toolkit, MarkupSafe, iteration_utilities, questionary, pyyaml-include, jinja2, dunamai, jinja2-ansible-filters, copier
  Running setup.py install for iteration_utilities ... error
  error: subprocess-exited-with-error

  × Running setup.py install for iteration_utilities did not run successfully.
  │ exit code: 1
  ╰─> [17 lines of output]
      running install
      C:\Users\wilson\AppData\Local\Programs\Python\Python310\lib\site-packages\setuptools\command\install.py:34: SetuptoolsDeprecationWarning: setup.py install is deprecated. Use build and pip and other standards-based tools.
        warnings.warn(
      running build
      running build_py
      creating build
      creating build\lib.win-amd64-cpython-310
      creating build\lib.win-amd64-cpython-310\iteration_utilities
      copying src\iteration_utilities\_additional_recipes.py -> build\lib.win-amd64-cpython-310\iteration_utilities
      copying src\iteration_utilities\_classes.py -> build\lib.win-amd64-cpython-310\iteration_utilities
      copying src\iteration_utilities\_convenience.py -> build\lib.win-amd64-cpython-310\iteration_utilities
      copying src\iteration_utilities\_recipes.py -> build\lib.win-amd64-cpython-310\iteration_utilities
      copying src\iteration_utilities\_utils.py -> build\lib.win-amd64-cpython-310\iteration_utilities
      copying src\iteration_utilities\__init__.py -> build\lib.win-amd64-cpython-310\iteration_utilities
      running build_ext
      building 'iteration_utilities._iteration_utilities' extension
      error: Microsoft Visual C++ 14.0 or greater is required. Get it with "Microsoft C++ Build Tools": https://visualstudio.microsoft.com/visual-cpp-build-tools/
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
error: legacy-install-failure

× Encountered error while trying to install package.
╰─> iteration_utilities

note: This is an issue with the package mentioned above, not pip.
hint: See above for output from the failure.

[notice] A new release of pip available: 22.2.1 -> 22.3.1
[notice] To update, run: python.exe -m pip install --upgrade pip

Likewise when running poetry install as part of the copier contribution dev guidelines


PS D:\repositories\MakerX\copier> poetry config virtualenvs.in-project true --local
PS D:\repositories\MakerX\copier> poetry install --with docs
Creating virtualenv copier in D:\repositories\MakerX\copier\.venv
Installing dependencies from lock file

Package operations: 83 installs, 1 update, 0 removals

  • Installing six (1.16.0)
  • Installing colorama (0.4.6)
  • Installing markupsafe (2.1.1)
  • Installing pyparsing (3.0.9)
  • Installing python-dateutil (2.8.2)
  • Installing pyyaml (6.0)
  • Installing zipp (3.8.1)
  • Installing attrs (22.1.0)
  • Installing click (8.1.3)
  • Installing ghp-import (2.1.0)
  • Installing jinja2 (3.1.2)
  • Installing mergedeep (1.3.4)
  • Installing markdown (3.3.5)
  • Installing packaging (21.3)
  • Installing pluggy (1.0.0)
  • Installing importlib-metadata (4.2.0)
  • Installing pyyaml-env-tag (0.1)
  • Installing py (1.11.0)
  • Installing iniconfig (1.1.1)
Connection pool is full, discarding connection: pypi.org. Connection pool size: 10
Connection pool is full, discarding connection: pypi.org. Connection pool size: 10
  • Installing tomli (2.0.1)
  • Installing watchdog (2.1.9)
  • Installing wcwidth (0.2.5)
  • Installing distlib (0.3.6)
  • Installing filelock (3.8.0)
  • Installing mkdocs (1.2.4)
  • Installing mccabe (0.7.0)
  • Installing griffe (0.22.1)
  • Installing pyflakes (2.5.0)
  • Installing pytest (7.1.3)
  • Installing pycodestyle (2.9.1)
  • Installing platformdirs (2.5.2)
  • Installing mkdocstrings (0.19.0)
  • Installing prompt-toolkit (3.0.31)
  • Updating setuptools (65.5.1 -> 65.3.0)
Connection pool is full, discarding connection: pypi.org. Connection pool size: 10
Connection pool is full, discarding connection: pypi.org. Connection pool size: 10
  • Installing argcomplete (1.12.3)
  • Installing astor (0.8.1)
  • Installing cfgv (3.3.1)
  • Installing decli (0.5.2)
  • Installing coverage (6.4.4)
  • Installing mkdocs-autorefs (0.4.1)
  • Installing mkdocstrings-python (0.7.1)
  • Installing identify (2.5.5)
  • Installing execnet (1.9.0)
  • Installing charset-normalizer (2.1.1)
  • Installing flake8 (5.0.4)
  • Installing mkdocs-material-extensions (1.0.3)
Connection pool is full, discarding connection: pypi.org. Connection pool size: 10
  • Installing mypy-extensions (0.4.3)
  • Installing nodeenv (1.7.0)
  • Installing pastel (0.2.1)
  • Installing pathspec (0.10.1)
  • Installing ptyprocess (0.7.0)
  • Installing pygments (2.13.0)
  • Installing pymdown-extensions (9.5)
  • Installing pytest-forked (1.4.0)
  • Installing pywin32 (304)
  • Installing questionary (1.10.0)
  • Installing termcolor (1.1.0)
  • Installing toml (0.10.2)
  • Installing tomlkit (0.11.4)
  • Installing typing-extensions (4.4.0)
  • Installing virtualenv (20.16.2)
  • Installing autoflake (1.6.1)
  • Installing black (22.8.0)
  • Installing flake8-bugbear (22.9.23)
  • Installing dunamai (1.14.1)
  • Installing commitizen (2.35.0)
  • Installing iteration-utilities (0.11.0)
  • Installing jinja2-ansible-filters (1.3.2)
  • Installing flake8-simplify (0.19.3)
  • Installing mkdocs-material (8.2.6)
  • Installing flake8-comprehensions (3.10.0)
  • Installing isort (5.10.1)
  • Installing flake8-debugger (4.1.2)
Connection pool is full, discarding connection: pypi.org. Connection pool size: 10
Connection pool is full, discarding connection: pypi.org. Connection pool size: 10
  • Installing mypy (0.981)
  • Installing pexpect (4.8.0)
  • Installing plumbum (1.8.0)
  • Installing poethepoet (0.16.2)
  • Installing pre-commit (2.20.0)
  • Installing pydantic (1.10.2)
  • Installing pytest-cov (4.0.0)
  • Installing pytest-xdist (2.5.0)
  • Installing pyyaml-include (1.3)
  • Installing types-backports (0.1.3)
  • Installing types-pyyaml (6.0.12)

  CalledProcessError

  Command 'D:\repositories\MakerX\copier\.venv\Scripts\python.exe -m pip install --use-pep517 --disable-pip-version-check --prefix D:\repositories\MakerX\copier\.venv --no-deps C:\Users\wilson\AppData\Local\pypoetry\Cache\artifacts\13\fe\58\dd3a5413393eab8ca2c5b884ff40af30da952dbdc4598df1d7b6cf61b0\iteration_utilities-0.11.0.tar.gz' returned non-zero exit status 1.

  at ~\AppData\Local\Programs\Python\Python310\lib\subprocess.py:524 in run
       520│             # We don't call process.wait() as .__exit__ does that for us.
       521│             raise
       522│         retcode = process.poll()
       523│         if check and retcode:
    →  524│             raise CalledProcessError(retcode, process.args,
       525│                                      output=stdout, stderr=stderr)
       526│     return CompletedProcess(process.args, retcode, stdout, stderr)
       527│
       528│

The following error occurred when trying to handle this error:


  EnvCommandError

  Command D:\repositories\MakerX\copier\.venv\Scripts\python.exe -m pip install --use-pep517 --disable-pip-version-check --prefix D:\repositories\MakerX\copier\.venv --no-deps C:\Users\wilson\AppData\Local\pypoetry\Cache\artifacts\13\fe\58\dd3a5413393eab8ca2c5b884ff40af30da952dbdc4598df1d7b6cf61b0\iteration_utilities-0.11.0.tar.gz errored with the following return code 1, and output:
  Processing c:\users\wilson\appdata\local\pypoetry\cache\artifacts\13\fe\58\dd3a5413393eab8ca2c5b884ff40af30da952dbdc4598df1d7b6cf61b0\iteration_utilities-0.11.0.tar.gz
    Installing build dependencies: started
    Installing build dependencies: finished with status 'done'
    Getting requirements to build wheel: started
    Getting requirements to build wheel: finished with status 'done'
    Preparing metadata (pyproject.toml): started
    Preparing metadata (pyproject.toml): finished with status 'done'
  Building wheels for collected packages: iteration-utilities
    Building wheel for iteration-utilities (pyproject.toml): started
    Building wheel for iteration-utilities (pyproject.toml): finished with status 'error'
    error: subprocess-exited-with-error

    Building wheel for iteration-utilities (pyproject.toml) did not run successfully.
    exit code: 1

    [15 lines of output]
    running bdist_wheel
    running build
    running build_py
    creating build
    creating build\lib.win-amd64-cpython-310
    creating build\lib.win-amd64-cpython-310\iteration_utilities
    copying src\iteration_utilities\_additional_recipes.py -> build\lib.win-amd64-cpython-310\iteration_utilities
    copying src\iteration_utilities\_classes.py -> build\lib.win-amd64-cpython-310\iteration_utilities
    copying src\iteration_utilities\_convenience.py -> build\lib.win-amd64-cpython-310\iteration_utilities
    copying src\iteration_utilities\_recipes.py -> build\lib.win-amd64-cpython-310\iteration_utilities
    copying src\iteration_utilities\_utils.py -> build\lib.win-amd64-cpython-310\iteration_utilities
    copying src\iteration_utilities\__init__.py -> build\lib.win-amd64-cpython-310\iteration_utilities
    running build_ext
    building 'iteration_utilities._iteration_utilities' extension
    error: Microsoft Visual C++ 14.0 or greater is required. Get it with "Microsoft C++ Build Tools": https://visualstudio.microsoft.com/visual-cpp-build-tools/
    [end of output]

    note: This error originates from a subprocess, and is likely not a problem with pip.
    ERROR: Failed building wheel for iteration-utilities
  Failed to build iteration-utilities
  ERROR: Could not build wheels for iteration-utilities, which is required to install pyproject.toml-based projects


  at ~\AppData\Local\Programs\Python\Python310\lib\site-packages\poetry\utils\env.py:1476 in _run
      1472│                 output = subprocess.check_output(
      1473│                     command, stderr=subprocess.STDOUT, env=env, **kwargs
      1474│                 )
      1475│         except CalledProcessError as e:
    → 1476│             raise EnvCommandError(e, input=input_)
      1477│
      1478│         return decode(output)
      1479│
      1480│     def execute(self, bin: str, *args: str, **kwargs: Any) -> int:

The following error occurred when trying to handle this error:


  PoetryException

  Failed to install C:/Users/wilson/AppData/Local/pypoetry/Cache/artifacts/13/fe/58/dd3a5413393eab8ca2c5b884ff40af30da952dbdc4598df1d7b6cf61b0/iteration_utilities-0.11.0.tar.gz

  at ~\AppData\Local\Programs\Python\Python310\lib\site-packages\poetry\utils\pip.py:51 in pip_install
       47│
       48│     try:
       49│         return environment.run_pip(*args)
       50│     except EnvCommandError as e:
    →  51│         raise PoetryException(f"Failed to install {path.as_posix()}") from e
       52│

Warning: The file chosen for install of markdown 3.3.5 (Markdown-3.3.5-py3-none-any.whl) is yanked. Reason for being yanked: Invalid dependency specified. Fixed in 3.3.6.

Expected behavior pip install copier should install copier without error

Environment

OS: Windows 10
Copier version: 7.0.1
Python version: 3.10
Installation method: pip+pypi

enhancement good first issue documentation help wanted

opened by wilsonwaters 13

Internal keys / non-prompted values

Is your feature request related to a problem? Please describe.

Trying to build a template for Terraform structure, which means a lot of the file tree structure depends on one or more of the inputs. Using a "choice" type here makes a lot more sense from a usability standpoint than using multiple competing boolean keys. For example:

Example copier.yaml

flavor:
  type: str
  choices:
    - Docker
    - Instances
    - Kubernetes
    - None

But that means having to do string comparison if-statements in directory names to create conditional directories. And everywhere else when this value is evaluated. If multiple answers are correct for some conditional directories then that results in even longer if-and statements:

Example filetree

.
├── {% if flavor == 'docker' %}ecs{% endif %}
├── {% if flavor == 'kubernetes' %}eks{% endif %}
├── {% if flavor == 'instance' %}bastion{% endif %}
├── {% if flavor == 'docker' or flavor == 'kubernetes' %}portainer{% endif %}
└── {% if flavor != 'none' %}vpc{% endif %}

This can make the directory names quite long and makes everything poorly readable. It also results in a lot of Jinja repetition across the template.

Describe the solution you'd like

I was hoping to create keys derived from the choice key behind the scenes, e.g. without prompting the user for input. In my scenario these would be booleans, but it could also be useful for concatenating strings or other post-input processing purposes that deduplicates a lot of code from the template files.

Example of how I'd love for this to work for my use case:

Future copier.yaml

flavor:
  type: str
  choices:
    - Docker
    - Instances
    - Kubernetes
    - None

# Internal variables
isContainer:
  type: bool
  value: "{% if flavor == 'docker' or flavor == 'kubernetes' %}true{% else %}false{% endif %}"

isDocker:
  type: bool
  value: "{% if flavor == 'docker' %}true{% else %}false{% endif %}"

isInstance:
  type: bool
  value: "{% if flavor == 'instance' %}true{% else %}false{% endif %}"

isKubernetes:
  type: bool
  value: "{% if flavor == 'kubernetes' %}true{% else %}false{% endif %}"

isLite:
  type: bool
  value: "{% if flavor == 'none' %}true{% else %}false{% endif %}"

Future filetree

.
├── {% if isDocker %}ecs{% endif %}
├── {% if isKubernetes %}eks{% endif %}
├── {% if isInstance %}bastion{% endif %}
├── {% if isContainer %}portainer{% endif %}
└── {% if isLite %}vpc{% endif %}

For this to work the value: key would have to be added and if this key is set then prompting the user for input would always be skipped. Alternatively prompt: false could be used in combination with setting the value using default: similar to what was suggested in https://github.com/copier-org/copier/issues/229 before.

If preferred these "internal variables" could also be moved one layer down under an identifier:

flavor:
  type: str
  choices:
    - Docker
    - Instances
    - Kubernetes
    - None

_derived_keys:
  isContainer:
    type: bool
    value: "{% if flavor == 'docker' or flavor == 'kubernetes' %}true{% else %}false{% endif %}"
  isDocker:
    type: bool
    value: "{% if flavor == 'docker' %}true{% else %}false{% endif %}"
  isInstance:
    type: bool
    value: "{% if flavor == 'instance' %}true{% else %}false{% endif %}"
  isKubernetes:
    type: bool
    value: "{% if flavor == 'kubernetes' %}true{% else %}false{% endif %}"
  isLite:
    type: bool
    value: "{% if flavor == 'none' %}true{% else %}false{% endif %}"

Describe alternatives you've considered

I've tried creating the variable like this, but setting when: false doesn't set the variable at all, resulting in the boolean always resolving to true:

isLite:
  type: bool
  when: false
  default: "{% if flavor == 'none' %}true{% else %}false{% endif %}"

Additional context

enhancement

opened by verdaatt 13

feat: include local dirty changes by default. fixes #184
A quick attempt at implementing the changes described in this comment.

if a template is vcs-tracked and dirty, the changes are committed automatically with a "wip" commit and propagated to the subproject

user is warned in this scenario

tests for copy and update on dirty templates

I made this the default behavior, but would be happy to add a flag for this behavior and maintain the current default.

Also, think this will have to rebase off of #519 to pass CI
opened by sabard 13
add --no-verify to 2nd "dumb commit"

Closes https://github.com/copier-org/copier/issues/871

-> disable pre-commit-like checks to ensure that there won't be any more conflicts concerning this commit.
bug

opened by NicMar01 2
Memory leak when running with --conflict inline
Describe the problem

Run copier --conflict inline -f update. Get a memory leak.

Template

N/A.

Logs

Expected behavior

Memory happy.

Screenshots/screencasts/logs

Environment

OS: Fedora 37

Copier version:  7.0.1.dev20221202132857+git569d8c0

Python version: Installed from nix package produced by #696

Installation method: nix

Additional context

The feature is experimental since #807, but this is very important to be fixed before the next release.
bug triage
opened by yajo 0
Answers file templating

Closes #866

This adds support for templating in the _answers_file variable, allowing multiple answers files to be generated depending on answers given in copier.yml.

Was unsure whether to put tests in a new file or tests/test_answersfile.py. Went ahead and made a new file, but let me know if I should put them elsewhere.

opened by EthanBar 2
uncommited changes only used in template repos without tags
Describe the problem According to this thread: https://github.com/copier-org/copier/discussions/743 copier is supposed to use uncommited changes by default. But it only does so when no tags exist in the template repository.

To Reproduce

mkdir template cd template echo "Hi" > test.txt git init git add . git commit -a -m "1" git tag v0.0.1 echo "Hi2" > test.txt copier copy . /tmp/foo cat /tmp/foo/test.txt

The result is Hi

Expected behavior Hi2

Environment

OS: Linux

Copier version: 7.0.1

Python version: 3.11.0

Installation method: pipx

bug
opened by FlorianLudwig 3

Fix conflict with commitizen (commit message format)

Problem description

Copier initializes a git repository in the current directory according to github
- While using commitizen as pre-commit-hook, this ~~can be~~ is problematic, since the commit message isn't properly formatted
- Issue gets pointed out in the following code snippet:

# (main.py) 
# _git_initialize_repo(self):
# (...)
# 1st commit could fail if any pre-commit hook reformats code
git("commit", "--allow-empty", "-am", "dumb commit 1", retcode=None)
git("commit", "--allow-empty", "-am", "dumb commit 2")

REQUEST

Add commitizen supported / formatted commit message for initialization

EDIT:

This is what the conflict looks like:

| commitizen check.........................................................Failed
| - hook id: commitizen
| - exit code: 14
|
| commit validation: failed!
| please enter a commit message in the commitizen format.
| commit "": "dumb commit 2"
| pattern: (build|ci|docs|feat|fix|perf|refactor|style|test|chore|revert|bump)(\(\S+\))?!?:(\s.*)

bug good first issue

opened by NicMar01 1

Releases(v7.0.1)

v7.0.1(Oct 14, 2022)
Fix

remove deprecated code scheduled for removal in Copier v7 (#843)

Source code(tar.gz)
Source code(zip)
v7.0.0(Oct 12, 2022)
Feat

expand tilde in template source path (#835)

Fix

delete temporary clones after execution automatically (#802)

typing: remove invalid migration task stage "task"

Refactor

typing: use abstract container types where possible (#832)

use dict constructor as default factory

typing: remove unused types

remove unreachable code (#826)

model a task to execute using a dataclass

reduce module imports

Source code(tar.gz)
Source code(zip)
v6.2.0(Sep 18, 2022)
Feat

add validator field to Question (#719)

support passing github or gitlab urls without the .git suffix (#677)

Fix

compatibility with pydantic 1.10

git bundle support breaks with relative paths

prevent name collision for question var name "value"

Refactor

add "flake8-simplify" plugin and simplify code

Source code(tar.gz)
Source code(zip)
v6.1.0(Jun 13, 2022)
Feat

support getting template commit hash with {{ _copier_conf.vcs_ref_hash }}

simplify the format of the question prompt (#689)

Fix

ignore non-PEP-440-compliant tags (#676)

Source code(tar.gz)
Source code(zip)
v6.0.0(May 15, 2022)
All changes here. Summary:

Added

Allow using additional Jinja 2 extensions.

Major version mismatch warning. If your Copier version is too new, you'll be warned.

Specific exceptions, which will help on error detection for API usages.

Multiline questions.

Conditional questions.

Placeholders.

Interactive TUI for questionaries. Prompts are way cooler now. 😎

Python 3.9 support.

Python 3.10 support.

Support empty templates suffix, telling Copier to render every file.

Added --defaults flag to use default answers to questions, which might be null if not specified.

Added --overwrite flag to overwrite files that already exist, without asking.

In migration scripts, we have the new environment variables $VERSION_PEP440_FROM, $VERSION_PEP440_CURRENT and $VERSION_PEP440_TO, which will always get a valid PEP440 version identifier, without the v prefix, allowing your migration scripts to have a valid standard where to base their logic.

Raise a CopierAnswersInterrupt instead of a bare KeyboardInterrupt to provide callers with additional context - such as the partially completed AnswersMap.

Support for user_defaults, which take precedence over template defaults.

Copy dirty changes from a git-tracked template to the project by default, to make testing easier.

Advertise clearly which version is being copied or updated in the CLI.

Add jinja variable _copier_python to provide python sys.executable.

Changed

Fully refactored core.

Running copier copy on a preexisting project now recopies the project instead of updating it. That means that it respects old answers, but ignores history diff.

We use Jinja 2 defaults now. {{ }} instead of [[ ]] and similar.

We keep trailing newlines by default for Jinja 2 templates.

Copier will never ask for overwriting the answers file.

Multi-typed choices follow the same type-casting logic as any other question, so it's easier to reason about them. However, if you were using this feature, you might be surprised about its side effects if you don't specify the type explicitly. Just add type: yaml to make it behave mostly as before. Or just don't use that, it's complicated anyway (warn added to docs).

Changed --force to be the same as --defaults --overwrite.

Copied files will reflect permissions on the same files in the template.

Copier now uses git clone --filter=blob:none when cloning, to be faster.

Removing files from templates will remove them too from the subprojects when they get updated.

Deprecated

Deprecated now and make_secret functions. If your template used those, Copier will emit warnings leading you on how to upgrade it.

Templates marked with _min_copier_version below 6 will still default to use bracket-based Jinja defaults, but that will disappear soon. If you want your template to work on Copier 5 and 6, make sure to declare _envops explicitly in your copier.yaml.

copier.copy() is confusing, now that actually copying and updating are 2 completely different actions (before, you were actually always updating if possible). Its direct equivalent is now copier.run_auto(), and copier.copy() will disappear in the future.

Removed

Minimal supported Python version is now 3.7 (dropped Python 3.6 support).

Removed the json method on _copier_conf. Where you would previously use _copier_conf.json() in your templates, please now use _copier_conf|to_json instead.

--subdirectory flag, which was confusing... and probably useless.

Lots of dead code.

Fixed

A directory that gets an empty name works as expected: not copied (nor its contents).

When comparing versions to update, PEP 440 is always used now. This way, we avoid fake ordering when git commit descriptions happen to be ordered in a non-predictable way.

Answers file will only remember answers to questions specified in the questionary.

Source code(tar.gz)
Source code(zip)
v5.1.0(Aug 17, 2020)
All changes here. Summary:

Forbid downgrades.

Print all logs to STDERR.

Source code(tar.gz)
Source code(zip)
v5.0.0(Aug 13, 2020)
All changes here. Summary:

Add --prerelease flag, which will be False by default. This is a behavioral change and that's basically why I'm doing a new major release. All other changes are minor here.

Better docs.

Source code(tar.gz)
Source code(zip)
v4.1.0(Aug 10, 2020)
All changes here. Summary:

Make copier work fine with Git 2.28.

We have docs!

Polish docs a little bit.

We now run tests on macOS and Windows!

Source code(tar.gz)
Source code(zip)
v4.0.2(Jul 21, 2020)
All changes here. Summary:

Fix a bug where questions with templated defaults would not be asked to the user.

Source code(tar.gz)
Source code(zip)
v4.0.1(Jun 23, 2020)
All changes here. Summary:

Fix a bug where the default value prompted to the user didn't match the reality from his last answer.

Source code(tar.gz)
Source code(zip)
v4.0.0(Jun 18, 2020)
All changes here. Summary:

Remove semver to avoid having 2 different versioning systems. We stick to PEP 440 now.

Remember where an answer comes from.

Do not re-ask to the user if already answer via --data.

Support pre-migration scripts that modify the answers file.

Source code(tar.gz)
Source code(zip)
v3.2.0(Jun 3, 2020)

Details: https://github.com/pykong/copier/milestone/8?closed=1
Source code(tar.gz)
Source code(zip)
v3.1.0(May 11, 2020)
All changes here. Summary:

Assert minimum copier version.

Prettier prompts.

Prompt self-templating.

Better README.

Source code(tar.gz)
Source code(zip)
v3.0.6(Apr 13, 2020)

Changes: https://github.com/pykong/copier/milestone/6?closed=1
Source code(tar.gz)
Source code(zip)
v3.0.5(Apr 8, 2020)

Changes: https://github.com/pykong/copier/milestone/5?closed=1
Source code(tar.gz)
Source code(zip)
v3.0.4(Mar 27, 2020)

Source code(tar.gz)
Source code(zip)
v3.0.3(Mar 26, 2020)
Little bugfix release:

Fix shortcode without .git suffix #167

Source code(tar.gz)
Source code(zip)
v3.0.2(Mar 25, 2020)

Source code(tar.gz)
Source code(zip)
v3.0.1(Mar 24, 2020)

This is an attempt to work around failing to upload to Test PyPI due to a version 3.0 being already present there.
Source code(tar.gz)
Source code(zip)
v3.0.0(Mar 24, 2020)

Source code(tar.gz)
Source code(zip)
v3.0.0-beta1(Mar 18, 2020)

Source code(tar.gz)
Source code(zip)
v3.0.0-alpha8(Mar 16, 2020)

Source code(tar.gz)
Source code(zip)
v3.0.0-alpha7(Mar 15, 2020)

Source code(tar.gz)
Source code(zip)
3.0.0-alpha6(Mar 4, 2020)

Source code(tar.gz)
Source code(zip)
3.0.0-alpha5(Mar 3, 2020)

Source code(tar.gz)
Source code(zip)
v3.0.0-alpha4(Mar 3, 2020)

Source code(tar.gz)
Source code(zip)
v3.0.0-alpha3(Feb 27, 2020)

Source code(tar.gz)
Source code(zip)
v3.0.0-alpha2(Feb 27, 2020)

Source code(tar.gz)
Source code(zip)
v3.0.0-alpha1(Feb 27, 2020)

Source code(tar.gz)
Source code(zip)