Synthetic structured data generators

Last update: Jan 7, 2023

Overview

Join us on

What is Synthetic Data?

Synthetic data is artificially generated data that is not collected from real world events. It replicates the statistical components of real data without containing any identifiable information, ensuring individuals' privacy.

Why Synthetic Data?

Synthetic data can be used for many applications:

Privacy
Remove bias
Balance datasets
Augment datasets

ydata-synthetic

This repository contains material related with Generative Adversarial Networks for synthetic data generation, in particular regular tabular data and time-series. It consists a set of different GANs architectures developed using Tensorflow 2.0. Several example Jupyter Notebooks and Python scripts are included, to show how to use the different architectures.

Quickstart

The source code is currently hosted on GitHub at: https://github.com/ydataai/ydata-synthetic

Binary installers for the latest released version are available at the Python Package Index (PyPI).

pip install ydata-synthetic

Examples

Here you can find usage examples of the package and models to synthesize tabular data.

Synthesizing the minority class with VanillaGAN on credit fraud dataset
Time Series synthetic data generation with TimeGAN on stock dataset
More examples are continously added and can be found in /examples directory.

Project Resources

In this repository you can find the several GAN architectures that are used to create synthesizers:

Tabular data

Sequential data

TimeGAN

Contributing

We are open to collaboration! If you want to start contributing you only need to:

Search for an issue in which you would like to work. Issues for newcomers are labeled with good first issue.
Create a PR solving the issue.
We would review every PRs and either accept or ask for revisions.

Support

For support in using this library, please join the #help Slack channel. The Slack community is very friendly and great about quickly answering questions about the use and development of the library. Click here to join our Slack community!

License

GNU General Public License v3.0

Comments

Generating long sequences - GAN mode collapse

Hi,

Thank you for uploading this code!

I've been trying to use your tutorial for time series generation to generate raw EEG signals. In this domain the sequence legnth has to be much larger than the proposed 24, otherwise, the data is not informative.

I've been working with data of shape (N, 2, 1500) and the output seems to be in mode collapse, i.e. all outputs look the same. After some hyper-parameter optimization, and modifying the generator loss to Wasserstein loss, I've managed to make the two channels differ from one another. Nevertheless, the outputs are almost all the same, meaning that my model is still in mode collapse.

Is there anything you recommend doing? @fabclmnt your post indicates that it is possible to generate EEG signals. Do you have any ready implantation?

Thank you :)
help wanted

opened by Rane90 6
chore(actions): Bump actions/cache from 3.0.2 to 3.0.11
Bumps actions/cache from 3.0.2 to 3.0.11.

Release notes

Sourced from actions/cache's releases.

v3.0.11

What's Changed

Call out cache not saved on hit by @Phantsure in actions/cache#946

Update @actions/core to 1.10.0 by @rentziass in actions/cache#950

Update cache to use @actions/core@^1.10.0 by @pdotl in actions/cache#956

New Contributors

@rentziass made their first contribution in actions/cache#950

Full Changelog: https://github.com/actions/cache/compare/v3...v3.0.11

v3.0.10

Fix a bug with sorting inputs.

Update definition for restore-keys in README.md

v3.0.9

Enhanced the warning message for cache unavailability in case of GHES.

v3.0.8

What's Changed

Fix zstd not working for windows on gnu tar in issues.

Allow users to provide a custom timeout as input for aborting cache segment download using the environment variable SEGMENT_DOWNLOAD_TIMEOUT_MIN. Default is 60 minutes.

v3.0.7

What's Changed

Fix for the download stuck problem has been added in actions/cache for users who were intermittently facing the issue. As part of this fix, new timeout has been introduced in the download step to stop the download if it doesn't complete within an hour and run the rest of the workflow without erroring out.

v3.0.6

What's Changed

Add example for clojure lein project dependencies by @shivamarora1 in PR actions/cache#835

Update toolkit's cache npm module to latest. Bump cache version to v3.0.6 by @pdotl in PR actions/cache#887

Fix issue #809 where cache save/restore was failing for Amazon Linux 2 runners due to older tar version

Fix issue #833 where cache save was not working for caching github workspace directory

New Contributors

@shivamarora1 made their first contribution in actions/cache#835

@pdotl made their first contribution in actions/cache#887

Full Changelog: https://github.com/actions/cache/compare/v3...v3.0.6

v3.0.5

Removed error handling by consuming actions/cache 3.0 toolkit, Now cache server error handling will be done by toolkit.

v3.0.4

In this release, we have fixed the tar creation error while trying to create it with path as ~/ home folder on ubuntu-latest.

v3.0.3

Fixed avoiding empty cache save when no files are available for caching. (actions/cache#624)

Changelog

Sourced from actions/cache's changelog.

3.0.2

Added support for dynamic cache size cap on GHES.

3.0.3

Fixed avoiding empty cache save when no files are available for caching. (issue)

3.0.4

Fixed tar creation error while trying to create tar with path as ~/ home folder on ubuntu-latest. (issue)

3.0.5

Removed error handling by consuming actions/cache 3.0 toolkit, Now cache server error handling will be done by toolkit. (PR)

3.0.6

Fixed #809 - zstd -d: no such file or directory error

Fixed #833 - cache doesn't work with github workspace directory

3.0.7

Fixed #810 - download stuck issue. A new timeout is introduced in the download process to abort the download if it gets stuck and doesn't finish within an hour.

3.0.8

Fix zstd not working for windows on gnu tar in issues #888 and #891.

Allowing users to provide a custom timeout as input for aborting download of a cache segment using an environment variable SEGMENT_DOWNLOAD_TIMEOUT_MIN. Default is 60 minutes.

3.0.9

Enhanced the warning message for cache unavailablity in case of GHES.

3.0.10

Fix a bug with sorting inputs.

Update definition for restore-keys in README.md

3.0.11

Update toolkit version to 3.0.5 to include @actions/core@^1.10.0

Update @actions/cache to use updated saveState and setOutput functions from @actions/core@^1.10.0

Commits

9b0c1fc Merge pull request #956 from actions/pdotl-version-bump

18103f6 Fix licensed status error

3e383cd Update RELEASES

43428ea toolkit versioon update and version bump for cache

1c73980 3.0.11

a3f5edc Merge pull request #950 from rentziass/rentziass/update-actions-core

831ee69 Update licenses

b9c8bfe Update @actions/core to 1.10.0

0f20846 Merge pull request #946 from actions/Phantsure-patch-2

862fc14 Update README.md

Additional commits viewable in compare view

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

Dependabot commands and options

You can trigger Dependabot actions by commenting on this PR:

@dependabot rebase will rebase this PR

@dependabot recreate will recreate this PR, overwriting any edits that have been made to it

@dependabot merge will merge this PR after your CI passes on it

@dependabot squash and merge will squash and merge this PR after your CI passes on it

@dependabot cancel merge will cancel a previously requested merge and block automerging

@dependabot reopen will reopen this PR if it is closed

@dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually

@dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)

@dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)

@dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)

dependencies github_actions
opened by dependabot[bot] 4

[BUG] 'gumbel_softmax_activation_5/******/' is not a valid scope name

Describe the bug `ValueError: in user code:

/usr/local/lib/python3.7/dist-packages/ydata_synthetic/utils/gumbel_softmax.py:77 call  *
    cat_cols = [GumbelSoftmaxLayer(name=name)(col)[0] for name, col in             zip(self.cat_feats.feat_names_in, cat_cols)]
/usr/local/lib/python3.7/dist-packages/tensorflow/python/keras/engine/base_layer.py:1006 __call__  **
    with ops.name_scope_v2(name_scope):
/usr/local/lib/python3.7/dist-packages/tensorflow/python/framework/ops.py:6650 __enter__
    scope_name = scope.__enter__()
/usr/lib/python3.7/contextlib.py:112 __enter__
    return next(self.gen)
/usr/local/lib/python3.7/dist-packages/tensorflow/python/framework/ops.py:4241 name_scope
    raise ValueError("'%s' is not a valid scope name" % name)

ValueError: 'gumbel_softmax_activation_5/*********/' is not a valid scope name`

******* is the cat_cols that we are using in our dataset We can only use num_cols by specifying cat_cols=[]. Any cat_cols with values will cause this error.

Expected behavior How to run VanillaGan and WGAN_GP with categorical features included. Currently, we followed the example in https://github.com/ydataai/ydata-synthetic/blob/dev/examples/regular/adult_wgangp.py

Desktop (please complete the following information): We are working in google colab.

bug

opened by INSACY98 4

feat: Add new regular data model synthesizer CWGANGP
Adds a Conditional Wasserstein GAN with Gradient Penalty, together with an example file.

This PR also:

Fixes the column order in the sample method output of CGAN (label column was not being inserted in correct location)

Simplifies the logic of label_col property setting (use a validation method instead of property setter with tuple args)
opened by jfsantos-ds 3
[BUG] Cannot load saved model when using GumbelSoftmaxActivation SD-73
Describe the bug Hi all, I encountered this error when trying to load a trained model using WGAN_GP.

ValueError: Unknown layer: Synthetic Data>GumbelSoftmaxActivation. Please ensure this object is passed to the custom_objects argument. See https://www.tensorflow.org/guide/keras/save_and_serialize#registering_the_custom_object for details.

To Reproduce Steps to reproduce the behavior:

Run the example adult_wgangp.py

Expected behavior It should load the saved model successfully.

Desktop (please complete the following information):

OS: Ubuntu Desktop 20.04

Cuda version: 11.3

tensorflow version: 2.7.0

keras version: 2.7.0

bug
opened by buihaduong 3
[Question] TimeGAN postprocessing of generated data

A question about TimeGAN postprocessing. I'm using TimeGAN to experiment with generation of stock returns of a handful of stocks simultaneously, and I'm using a sequence length of 24 following the seminal paper by Yoon 2019.

After successfully running TimeGAN on the input data I end up with a multi-dimensional array of shape (5000, 24, 10) where 5000 is the generated length, 24 is the sequence length and 10 is the number of stocks.

Now I want to take the generated sequences and produce a matrix (x, 10) where x is the resulting matrix length, so that I can use it in my subsequent experiments. How do I convert (5000, 24, 10) to (x,10). Do I just reshape the array or is there a better way?
question

opened by pbezz1 3
[BUG] AttributeError: 'TimeGAN' object has no attribute '__MODEL__'

If you execute this notebook https://github.com/ydataai/ydata-synthetic/blob/master/examples/timeseries/TimeGAN_Synthetic_stock_data.ipynb

At the line synth.save() you'll get this: AttributeError: 'TimeGAN' object has no attribute 'MODEL'
bug

opened by RodrigoAbrao 3
[BUG] Incorrect computation of the supervised loss in train_supervisor of TimeGAN.

Incorrect computation of the supervised loss in train_supervisor of TimeGAN.

I believe there is a mistake in the way the supervised loss g_loss_s is computed inside train_supervisor. You compute: g_loss_s = self._mse(h[:, 1:, :], h_hat_supervised[:, 1:, :]) . This aligns with earlier commits of the original paper code. However, this step would merely learn an identity mapping.

Based on the paper and the new commits of the authors, I believe the loss should be computed as follows: g_loss_s = self._mse(h[:, 1:, :], h_hat_supervised[:, :-1, :]) . This should help learning the temporal dependencies between time steps and help guiding the generator.

Finally, I have a question regarding the gradients of the generator for the supervised training. In the second step of training TGAN, the supervisor and generator are optimised on g_loss_s. However, the generator is not contributing to the computations and thus has no gradients. Do you have an explanation for this?
bug

opened by gmbss0 3
[BUG]
First of all, thank you for the work.

Describe the bug I am trying to train a TimeGAN on my own dataset. I used the preprocessing function, i.e, real_data_loading() func, from preprocessing.utils to prepare the data. And all my parameter settings are the same as what you explained in the this jupyter notebook, except for changing the number of sequences which is equal to the number of the columns of my dataset.

To Reproduce Steps to reproduce the behavior:

Go to 'https://github.com/Sorooshi/SuperOX'

Click on 'u06-TGAN.ipynb'

Scroll down to 'cell [19]'

See error

Expected behavior The expected dimension should be (69, 1, 10) where 69 is batch_size.

Desktop (please complete the following information):

OS: Ubuntu 18.

Additional context Add any other context about the problem here.
bug
opened by Sorooshi 3
chore(actions): Bump actions/setup-python from 2 to 4.3.0
Bumps actions/setup-python from 2 to 4.3.0.

Release notes

Sourced from actions/setup-python's releases.

v4.3.0

Update @actions/core to 1.10.0 version #517

Update @actions/cache to 3.0.4 version #499

Only use github.token on github.com #443

Improvement of documentation #477 #479 #491 #492

Add check-latest input and bug fixes

In scope of this release we add the check-latest input. If check-latest is set to true, the action first checks if the cached version is the latest one. If the locally cached version is not the most up-to-date, the version will then be downloaded from python-versions repository. By default check-latest is set to false. For PyPy it will to try to reach https://downloads.python.org/pypy/versions.json

Example of usage:

steps: - uses: actions/checkout@v3 - uses: actions/setup-python@v4 with: python-version: '3.9' check-latest: true - run: python --version

Besides, it includes such changes as

Resolved logs for python-version and file inputs: actions/setup-python#465

Added linux os release info to primary key: actions/setup-python#467

Added fix to change Python versions for poetry: actions/setup-python#445

Fix Tool Path handling for self-hosted runners: actions/setup-python#466

v4.1.0

In scope of this pull request we updated actions/cache package as the new version contains fixes for caching error handling. Moreover, we added a new input update-environment. This option allows to specify if the action shall update environment variables (default) or not.

Update-environment input

- name: setup-python 3.9 uses: actions/setup-python@v4 with: python-version: 3.9 update-environment: false

Besides, we added such changes as:

Allow python-version-file to be a relative path: actions/setup-python#431

Added new environment variables for Cmake: actions/setup-python#440

Updated error message for resolveVersion: actions/setup-python#450

Assign default value of AGENT_TOOLSDIRECTORY if not set: actions/setup-python#394

v4.0.0

What's Changed

Support for python-version-file input: #336

... (truncated)

Commits

13ae5bb Merge pull request #517 from rentziass/rentziass/update-actions-core

0c4d7b8 Update @actions/core to 1.10.0

13a464f Fix typo (#503)

b4fe97e upgrade @actions/cache so it respects SEGMENT_DOWNLOAD_TIMEOUT_MINS (#499)

434aeab Bump @actions/core from 1.7.0 to 1.9.1 (#495)

98c991d Only use github.token on github.com (#443)

397a35f Merge pull request #492 from al-cheb/al-cheb/update-runner-link

48a0f00 Update runner links

978fd06 Merge pull request #491 from lkfortuna/patch-2

050e616 Update README.md

Additional commits viewable in compare view

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

Dependabot commands and options

You can trigger Dependabot actions by commenting on this PR:

@dependabot rebase will rebase this PR

@dependabot recreate will recreate this PR, overwriting any edits that have been made to it

@dependabot merge will merge this PR after your CI passes on it

@dependabot squash and merge will squash and merge this PR after your CI passes on it

@dependabot cancel merge will cancel a previously requested merge and block automerging

@dependabot reopen will reopen this PR if it is closed

@dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually

@dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)

@dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)

@dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)

dependencies github_actions
opened by dependabot[bot] 2
chore(actions): Bump pypa/gh-action-pypi-publish from 1.5.0 to 1.5.1
Bumps pypa/gh-action-pypi-publish from 1.5.0 to 1.5.1.

Release notes

Sourced from pypa/gh-action-pypi-publish's releases.

v1.5.1

What's Changed

Fixed printing out the dist hashes when packages_dir is a wildcard value. — by @meowmeowmeowcat in pypa/gh-action-pypi-publish#91

Full Changelog: https://github.com/pypa/gh-action-pypi-publish/compare/v1.5.0...v1.5.1

Commits

37f50c2 Merge PR #91

9f0421c Add #StandWithUkraine banner to README

c3fbd68 Remove quotes

See full diff in compare view

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

Dependabot commands and options

You can trigger Dependabot actions by commenting on this PR:

@dependabot rebase will rebase this PR

@dependabot recreate will recreate this PR, overwriting any edits that have been made to it

@dependabot merge will merge this PR after your CI passes on it

@dependabot squash and merge will squash and merge this PR after your CI passes on it

@dependabot cancel merge will cancel a previously requested merge and block automerging

@dependabot reopen will reopen this PR if it is closed

@dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually

@dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)

@dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)

@dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)

dependencies github_actions
opened by dependabot[bot] 2
[BUG]
Describe the bug Once I found a google colab to work I found that TimeGAN doesn't supply or reference the location of the data.

FileNotFoundError: [Errno 2] No such file or directory: 'sample_data/stock_data.csv'

To Reproduce Steps to reproduce the behavior:

Go to 'TIME'

Run code to the seventh cell, it generates the error shown below

Screenshots

Desktop (please complete the following information):

MacOS: 11.3.1

Browser safari

bug
opened by datatalking 0
[BUG] Google Colab link on README.md links to a broken page.
Describe the bug I clicked the link for VanillaGAN on credit fraud dataset which is an invalid link. Instead I'm given a screen that has a few dozen options and not one of them says or has keywords that match including fraud, Vanilla, Gan, credit.

So where should that link point to?

To Reproduce Steps to reproduce the behavior:

Go to 'ydata-synthetic repo README.md file'

Under Examples in 'README.md' click the google colab icon after 'Synthesizing the minority class with VanillaGAN on credit fraud dataset'... the icond https://colab.research.google.com/github/ydataai/ydata-synthetic/blob/master/examples/regular/gan_example.ipynb....'

See screenshot

Expected behavior To be able to run a google colab for VanillaGAN

Screenshots Included

Desktop (please complete the following information):

MacOS: 11.3.1

Browser Safari

Version 14.1

Additional context Should I redirect the URL or where is the VanillaGAN notebook and I'll submit a PR
bug
opened by datatalking 0
[BUG]
Describe the bug A clear and concise description of what the bug is.

To Reproduce Steps to reproduce the behavior:

Go to '...'

Click on ' pip install pmlb pip install ydata-synthetic'

Scroll down to ' from pmlb import fetch_data

from ydata_synthetic.synthesizers.regular import WGAN_GP from ydata_synthetic.synthesizers import ModelParameters, TrainParameters

model = WGAN_GP' 5. See error

Error ImportError Traceback (most recent call last) in 1 from pmlb import fetch_data 2 ----> 3 from ydata_synthetic.synthesizers.regular import WGAN_GP 4 from ydata_synthetic.synthesizers import ModelParameters, TrainParameters 5

ImportError: cannot import name 'WGAN_GP' from 'ydata_synthetic.synthesizers.regular' (/usr/local/lib/python3.8/dist-packages/ydata_synthetic/synthesizers/regular/init.py)

NOTE: If your import is failing due to a missing package, you can manually install dependencies using either !pip or !apt.

To view examples of installing some common dependencies, click the "Open Examples" button below.

Screenshots If applicable, add screenshots to help explain your problem.

Desktop (please complete the following information):

Google Colab

Additional context Add any other context about the problem here.
bug
opened by oky4nd4y 1
chore(deps): update dependency scikit-learn to ==1.2.*
This PR contains the following updates:

| Package | Change | Age | Adoption | Passing | Confidence | |---|---|---|---|---|---| | scikit-learn (source) | ==1.1.* -> ==1.2.* | | | | |

Release Notes

scikit-learn/scikit-learn

v1.2.0: Scikit-learn 1.2.0

Compare Source

We're happy to announce the 1.2.0 release.

You can read the release highlights under https://scikit-learn.org/stable/auto_examples/release_highlights/plot_release_highlights_1_2_0.html and the long version of the change log under https://scikit-learn.org/stable/whats_new/v1.2.html

This version supports Python versions 3.8 to 3.11.

Configuration

📅 Schedule: Branch creation - "at 10am every weekday" (UTC), Automerge - At any time (no schedule defined).

🚦 Automerge: Disabled by config. Please merge this manually once you are satisfied.

♻ Rebasing: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.

🔕 Ignore: Close this PR and you won't be reminded about this update again.

[ ] If you want to rebase/retry this PR, check this box

This PR has been generated by Mend Renovate. View repository job log here.
opened by renovate[bot] 0
chore(deps): update dependency pytest to v7
This PR contains the following updates:

| Package | Change | Age | Adoption | Passing | Confidence | |---|---|---|---|---|---| | pytest (source, changelog) | ==6.2.* -> ==7.2.* | | | | |

Release Notes

pytest-dev/pytest

v7.2.0

Compare Source

pytest 7.2.0 (2022-10-23)

Deprecations

#10012: Update pytest.PytestUnhandledCoroutineWarning{.interpreted-text role="class"} to a deprecation; it will raise an error in pytest 8.

#10396: pytest no longer depends on the py library. pytest provides a vendored copy of py.error and py.path modules but will use the py library if it is installed. If you need other py.* modules, continue to install the deprecated py library separately, otherwise it can usually be removed as a dependency.

#4562: Deprecate configuring hook specs/impls using attributes/marks.

Instead use :pypytest.hookimpl{.interpreted-text role="func"} and :pypytest.hookspec{.interpreted-text role="func"}. For more details, see the docs <legacy-path-hooks-deprecated>{.interpreted-text role="ref"}.

#9886: The functionality for running tests written for nose has been officially deprecated.

This includes:

Plain setup and teardown functions and methods: this might catch users by surprise, as setup() and teardown() are not pytest idioms, but part of the nose support.

Setup/teardown using the @with_setup decorator.

For more details, consult the deprecation docs <nose-deprecation>{.interpreted-text role="ref"}.

Features

#9897: Added shell-style wildcard support to testpaths.

Improvements

#10218: @pytest.mark.parametrize() (and similar functions) now accepts any Sequence[str] for the argument names, instead of just list[str] and tuple[str, ...].

(Note that str, which is itself a Sequence[str], is still treated as a comma-delimited name list, as before).

#10381: The --no-showlocals flag has been added. This can be passed directly to tests to override --showlocals declared through addopts.

#3426: Assertion failures with strings in NFC and NFD forms that normalize to the same string now have a dedicated error message detailing the issue, and their utf-8 representation is expresed instead.

#7337: A warning is now emitted if a test function returns something other than [None]{.title-ref}. This prevents a common mistake among beginners that expect that returning a [bool]{.title-ref} (for example [return foo(a, b) == result]{.title-ref}) would cause a test to pass or fail, instead of using [assert]{.title-ref}.

#8508: Introduce multiline display for warning matching via :pypytest.warns{.interpreted-text role="func"} and enhance match comparison for :py_pytest._code.ExceptionInfo.match{.interpreted-text role="func"} as returned by :pypytest.raises{.interpreted-text role="func"}.

#8646: Improve :pypytest.raises{.interpreted-text role="func"}. Previously passing an empty tuple would give a confusing error. We now raise immediately with a more helpful message.

#9741: On Python 3.11, use the standard library's tomllib{.interpreted-text role="mod"} to parse TOML.

tomli{.interpreted-text role="mod"}` is no longer a dependency on Python 3.11.

#9742: Display assertion message without escaped newline characters with -vv.

#9823: Improved error message that is shown when no collector is found for a given file.

#9873: Some coloring has been added to the short test summary.

#9883: Normalize the help description of all command-line options.

#9920: Display full crash messages in short test summary info, when runng in a CI environment.

#9987: Added support for hidden configuration file by allowing .pytest.ini as an alternative to pytest.ini.

Bug Fixes

#10150: sys.stdin{.interpreted-text role="data"} now contains all expected methods of a file-like object when capture is enabled.

#10382: Do not break into pdb when raise unittest.SkipTest() appears top-level in a file.

#7792: Marks are now inherited according to the full MRO in test classes. Previously, if a test class inherited from two or more classes, only marks from the first super-class would apply.

When inheriting marks from super-classes, marks from the sub-classes are now ordered before marks from the super-classes, in MRO order. Previously it was the reverse.

When inheriting marks from super-classes, the [pytestmark]{.title-ref} attribute of the sub-class now only contains the marks directly applied to it. Previously, it also contained marks from its super-classes. Please note that this attribute should not normally be accessed directly; use pytest.Node.iter_markers{.interpreted-text role="func"} instead.

#9159: Showing inner exceptions by forcing native display in ExceptionGroups even when using display options other than --tb=native. A temporary step before full implementation of pytest-native display for inner exceptions in ExceptionGroups.

#9877: Ensure caplog.get_records(when) returns current/correct data after invoking caplog.clear().

Improved Documentation

#10344: Update information on writing plugins to use pyproject.toml instead of setup.py.

#9248: The documentation is now built using Sphinx 5.x (up from 3.x previously).

#9291: Update documentation on how pytest.warns{.interpreted-text role="func"} affects DeprecationWarning{.interpreted-text role="class"}.

Trivial/Internal Changes

#10196: ~pytest.PytestReturnNotNoneWarning{.interpreted-text role="class"} is now a subclass of ~pytest.PytestRemovedIn8Warning{.interpreted-text role="class"}: the plan is to make returning non-None from tests an error in the future.

#10313: Made _pytest.doctest.DoctestItem export pytest.DoctestItem for type check and runtime purposes. Made [_pytest.doctest]{.title-ref} use internal APIs to avoid circular imports.

#9906: Made _pytest.compat re-export importlib_metadata in the eyes of type checkers.

#9910: Fix default encoding warning (EncodingWarning) in cacheprovider

#9984: Improve the error message when we attempt to access a fixture that has been torn down. Add an additional sentence to the docstring explaining when it's not a good idea to call getfixturevalue.

v7.1.3

Compare Source

pytest 7.1.3 (2022-08-31)

Bug Fixes

#10060: When running with --pdb, TestCase.tearDown is no longer called for tests when the class has been skipped via unittest.skip or pytest.mark.skip.

#10190: Invalid XML characters in setup or teardown error messages are now properly escaped for JUnit XML reports.

#10230: Ignore .py files created by pyproject.toml-based editable builds introduced in pip 21.3.

#3396: Doctests now respect the --import-mode flag.

#9514: Type-annotate FixtureRequest.param as Any as a stop gap measure until 8073{.interpreted-text role="issue"} is fixed.

#9791: Fixed a path handling code in rewrite.py that seems to work fine, but was incorrect and fails in some systems.

#9917: Fixed string representation for pytest.approx{.interpreted-text role="func"} when used to compare tuples.

Improved Documentation

#9937: Explicit note that tmpdir{.interpreted-text role="fixture"} fixture is discouraged in favour of tmp_path{.interpreted-text role="fixture"}.

Trivial/Internal Changes

#10114: Replace atomicwrites dependency on windows with [os.replace]{.title-ref}.

v7.1.2

Compare Source

pytest 7.1.2 (2022-04-23)

Bug Fixes

#9726: An unnecessary numpy import inside pytest.approx{.interpreted-text role="func"} was removed.

#9820: Fix comparison of dataclasses with InitVar.

#9869: Increase stacklevel for the NODE_CTOR_FSPATH_ARG deprecation to point to the user's code, not pytest.

#9871: Fix a bizarre (and fortunately rare) bug where the [temp_path]{.title-ref} fixture could raise an internal error while attempting to get the current user's username.

v7.1.1

Compare Source

pytest 7.1.1 (2022-03-17)

Bug Fixes

#9767: Fixed a regression in pytest 7.1.0 where some conftest.py files outside of the source tree (e.g. in the [site-packages]{.title-ref} directory) were not picked up.

v7.1.0

Compare Source

pytest 7.1.0 (2022-03-13)

Breaking Changes

#8838: As per our policy, the following features have been deprecated in the 6.X series and are now removed:

pytest._fillfuncargs function.

pytest_warning_captured hook - use pytest_warning_recorded instead.

-k -foobar syntax - use -k 'not foobar' instead.

-k foobar: syntax.

pytest.collect module - import from pytest directly.

For more information consult Deprecations and Removals in the docs.

#9437: Dropped support for Python 3.6, which reached end-of-life at 2021-12-23.

Improvements

#5192: Fixed test output for some data types where -v would show less information.

Also, when showing diffs for sequences, -q would produce full diffs instead of the expected diff.

#9362: pytest now avoids specialized assert formatting when it is detected that the default __eq__ is overridden in attrs or dataclasses.

#9536: When -vv is given on command line, show skipping and xfail reasons in full instead of truncating them to fit the terminal width.

#9644: More information about the location of resources that led Python to raise ResourceWarning{.interpreted-text role="class"} can now be obtained by enabling tracemalloc{.interpreted-text role="mod"}.

See resource-warnings{.interpreted-text role="ref"} for more information.

#9678: More types are now accepted in the ids argument to @pytest.mark.parametrize. Previously only [str]{.title-ref}, [float]{.title-ref}, [int]{.title-ref} and [bool]{.title-ref} were accepted; now [bytes]{.title-ref}, [complex]{.title-ref}, [re.Pattern]{.title-ref}, [Enum]{.title-ref} and anything with a [__name__]{.title-ref} are also accepted.

#9692: pytest.approx{.interpreted-text role="func"} now raises a TypeError{.interpreted-text role="class"} when given an unordered sequence (such as set{.interpreted-text role="class"}).

Note that this implies that custom classes which only implement __iter__ and __len__ are no longer supported as they don't guarantee order.

Bug Fixes

#8242: The deprecation of raising unittest.SkipTest{.interpreted-text role="class"} to skip collection of tests during the pytest collection phase is reverted - this is now a supported feature again.

#9493: Symbolic link components are no longer resolved in conftest paths. This means that if a conftest appears twice in collection tree, using symlinks, it will be executed twice. For example, given

tests/real/conftest.py tests/real/test_it.py tests/link -> tests/real

running pytest tests now imports the conftest twice, once as tests/real/conftest.py and once as tests/link/conftest.py. This is a fix to match a similar change made to test collection itself in pytest 6.0 (see 6523{.interpreted-text role="pull"} for details).

#9626: Fixed count of selected tests on terminal collection summary when there were errors or skipped modules.

If there were errors or skipped modules on collection, pytest would mistakenly subtract those from the selected count.

#9645: Fixed regression where --import-mode=importlib used together with PYTHONPATH{.interpreted-text role="envvar"} or pythonpath{.interpreted-text role="confval"} would cause import errors in test suites.

#9708: pytester{.interpreted-text role="fixture"} now requests a monkeypatch{.interpreted-text role="fixture"} fixture instead of creating one internally. This solves some issues with tests that involve pytest environment variables.

#9730: Malformed pyproject.toml files now produce a clearer error message.

v7.0.1

Compare Source

pytest 7.0.1 (2022-02-11)

Bug Fixes

#9608: Fix invalid importing of importlib.readers in Python 3.9.

#9610: Restore [UnitTestFunction.obj]{.title-ref} to return unbound rather than bound method. Fixes a crash during a failed teardown in unittest TestCases with non-default [__init__]{.title-ref}. Regressed in pytest 7.0.0.

#9636: The pythonpath plugin was renamed to python_path. This avoids a conflict with the pytest-pythonpath plugin.

#9642: Fix running tests by id with :: in the parametrize portion.

#9643: Delay issuing a ~pytest.PytestWarning{.interpreted-text role="class"} about diamond inheritance involving ~pytest.Item{.interpreted-text role="class"} and ~pytest.Collector{.interpreted-text role="class"} so it can be filtered using standard warning filters <warnings>{.interpreted-text role="ref"}.

v7.0.0

Compare Source

pytest 7.0.0 (2022-02-03)

(Please see the full set of changes for this release also in the 7.0.0rc1 notes below)

Deprecations

#9488: If custom subclasses of nodes like pytest.Item{.interpreted-text role="class"} override the __init__ method, they should take **kwargs. See uncooperative-constructors-deprecated{.interpreted-text role="ref"} for details.

Note that a deprection warning is only emitted when there is a conflict in the arguments pytest expected to pass. This deprecation was already part of pytest 7.0.0rc1 but wasn't documented.

Bug Fixes

#9355: Fixed error message prints function decorators when using assert in Python 3.8 and above.

#9396: Ensure pytest.Config.inifile{.interpreted-text role="attr"} is available during the pytest_cmdline_main <_pytest.hookspec.pytest_cmdline_main>{.interpreted-text role="func"} hook (regression during 7.0.0rc1).

Improved Documentation

#9404: Added extra documentation on alternatives to common misuses of [pytest.warns(None)]{.title-ref} ahead of its deprecation.

#9505: Clarify where the configuration files are located. To avoid confusions documentation mentions that configuration file is located in the root of the repository.

Trivial/Internal Changes

#9521: Add test coverage to assertion rewrite path.

pytest 7.0.0rc1 (2021-12-06)

Breaking Changes

#7259: The Node.reportinfo() <non-python tests>{.interpreted-text role="ref"} function first return value type has been expanded from [py.path.local | str]{.title-ref} to [os.PathLike[str] | str]{.title-ref}.

Most plugins which refer to [reportinfo()]{.title-ref} only define it as part of a custom pytest.Item{.interpreted-text role="class"} implementation. Since [py.path.local]{.title-ref} is a [os.PathLike[str]]{.title-ref}, these plugins are unaffacted.

Plugins and users which call [reportinfo()]{.title-ref}, use the first return value and interact with it as a [py.path.local]{.title-ref}, would need to adjust by calling [py.path.local(fspath)]{.title-ref}. Although preferably, avoid the legacy [py.path.local]{.title-ref} and use [pathlib.Path]{.title-ref}, or use [item.location]{.title-ref} or [item.path]{.title-ref}, instead.

Note: pytest was not able to provide a deprecation period for this change.

#8246: --version now writes version information to stdout rather than stderr.

#8733: Drop a workaround for pyreadline that made it work with --pdb.

The workaround was introduced in #1281 in 2015, however since then pyreadline seems to have gone unmaintained, is generating warnings, and will stop working on Python 3.10.

#9061: Using pytest.approx{.interpreted-text role="func"} in a boolean context now raises an error hinting at the proper usage.

It is apparently common for users to mistakenly use pytest.approx like this:

assert pytest.approx(actual, expected)

While the correct usage is:

assert actual == pytest.approx(expected)

The new error message helps catch those mistakes.

#9277: The pytest.Instance collector type has been removed. Importing pytest.Instance or _pytest.python.Instance returns a dummy type and emits a deprecation warning. See instance-collector-deprecation{.interpreted-text role="ref"} for details.

#9308: PytestRemovedIn7Warning deprecation warnings are now errors by default.

Following our plan to remove deprecated features with as little disruption as possible, all warnings of type PytestRemovedIn7Warning now generate errors instead of warning messages by default.

The affected features will be effectively removed in pytest 7.1, so please consult the deprecations{.interpreted-text role="ref"} section in the docs for directions on how to update existing code.

In the pytest 7.0.X series, it is possible to change the errors back into warnings as a stopgap measure by adding this to your pytest.ini file:

[pytest] filterwarnings = ignore::pytest.PytestRemovedIn7Warning

But this will stop working when pytest 7.1 is released.

If you have concerns about the removal of a specific feature, please add a comment to 9308{.interpreted-text role="issue"}.

Deprecations

#7259: py.path.local arguments for hooks have been deprecated. See the deprecation note <legacy-path-hooks-deprecated>{.interpreted-text role="ref"} for full details.

py.path.local arguments to Node constructors have been deprecated. See the deprecation note <node-ctor-fspath-deprecation>{.interpreted-text role="ref"} for full details.

::: {.note} ::: {.admonition-title} Note :::

The name of the ~_pytest.nodes.Node{.interpreted-text role="class"} arguments and attributes (the new attribute being path) is the opposite of the situation for hooks (the old argument being path).

This is an unfortunate artifact due to historical reasons, which should be resolved in future versions as we slowly get rid of the py{.interpreted-text role="pypi"} dependency (see 9283{.interpreted-text role="issue"} for a longer discussion). :::

#7469: Directly constructing the following classes is now deprecated:

_pytest.mark.structures.Mark

_pytest.mark.structures.MarkDecorator

_pytest.mark.structures.MarkGenerator

_pytest.python.Metafunc

_pytest.runner.CallInfo

_pytest._code.ExceptionInfo

_pytest.config.argparsing.Parser

_pytest.config.argparsing.OptionGroup

_pytest.pytester.HookRecorder

These constructors have always been considered private, but now issue a deprecation warning, which may become a hard error in pytest 8.

#8242: Raising unittest.SkipTest{.interpreted-text role="class"} to skip collection of tests during the pytest collection phase is deprecated. Use pytest.skip{.interpreted-text role="func"} instead.

Note: This deprecation only relates to using unittest.SkipTest{.interpreted-text role="class"} during test collection. You are probably not doing that. Ordinary usage of unittest.SkipTest{.interpreted-text role="class"} / unittest.TestCase.skipTest{.interpreted-text role="meth"} / unittest.skip{.interpreted-text role="func"} in unittest test cases is fully supported.

#8315: Several behaviors of Parser.addoption <pytest.Parser.addoption>{.interpreted-text role="meth"} are now scheduled for removal in pytest 8 (deprecated since pytest 2.4.0):

parser.addoption(..., help=".. %default ..") - use %(default)s instead.

parser.addoption(..., type="int/string/float/complex") - use type=int etc. instead.

#8447: Defining a custom pytest node type which is both an pytest.Item <Item>{.interpreted-text role="class"} and a pytest.Collector <Collector>{.interpreted-text role="class"} (e.g. pytest.File <File>{.interpreted-text role="class"}) now issues a warning. It was never sanely supported and triggers hard to debug errors.

See the deprecation note <diamond-inheritance-deprecated>{.interpreted-text role="ref"} for full details.

#8592: pytest_cmdline_preparse{.interpreted-text role="hook"} has been officially deprecated. It will be removed in a future release. Use pytest_load_initial_conftests{.interpreted-text role="hook"} instead.

See the deprecation note <cmdline-preparse-deprecated>{.interpreted-text role="ref"} for full details.

#8645: pytest.warns(None) <pytest.warns>{.interpreted-text role="func"} is now deprecated because many people used it to mean "this code does not emit warnings", but it actually had the effect of checking that the code emits at least one warning of any type - like pytest.warns() or pytest.warns(Warning).

#8948: pytest.skip(msg=...) <pytest.skip>{.interpreted-text role="func"}, pytest.fail(msg=...) <pytest.fail>{.interpreted-text role="func"} and pytest.exit(msg=...) <pytest.exit>{.interpreted-text role="func"} signatures now accept a reason argument instead of msg. Using msg still works, but is deprecated and will be removed in a future release.

This was changed for consistency with pytest.mark.skip <pytest.mark.skip>{.interpreted-text role="func"} and pytest.mark.xfail <pytest.mark.xfail>{.interpreted-text role="func"} which both accept reason as an argument.

#8174: The following changes have been made to types reachable through pytest.ExceptionInfo.traceback{.interpreted-text role="attr"}:

The path property of _pytest.code.Code returns Path instead of py.path.local.

The path property of _pytest.code.TracebackEntry returns Path instead of py.path.local.

There was no deprecation period for this change (sorry!).

Features

#5196: Tests are now ordered by definition order in more cases.

In a class hierarchy, tests from base classes are now consistently ordered before tests defined on their subclasses (reverse MRO order).

#7132: Added two environment variables PYTEST_THEME{.interpreted-text role="envvar"} and PYTEST_THEME_MODE{.interpreted-text role="envvar"} to let the users customize the pygments theme used.

#7259: Added cache.mkdir() <pytest.Cache.mkdir>{.interpreted-text role="meth"}, which is similar to the existing cache.makedir() <pytest.Cache.makedir>{.interpreted-text role="meth"}, but returns a pathlib.Path{.interpreted-text role="class"} instead of a legacy py.path.local.

Added a paths type to parser.addini() <pytest.Parser.addini>{.interpreted-text role="meth"}, as in parser.addini("mypaths", "my paths", type="paths"), which is similar to the existing pathlist, but returns a list of pathlib.Path{.interpreted-text role="class"} instead of legacy py.path.local.

#7469: The types of objects used in pytest's API are now exported so they may be used in type annotations.

The newly-exported types are:

pytest.Config for Config <pytest.Config>{.interpreted-text role="class"}.

pytest.Mark for marks <pytest.Mark>{.interpreted-text role="class"}.

pytest.MarkDecorator for mark decorators <pytest.MarkDecorator>{.interpreted-text role="class"}.

pytest.MarkGenerator for the pytest.mark <pytest.MarkGenerator>{.interpreted-text role="class"} singleton.

pytest.Metafunc for the metafunc <pytest.MarkGenerator>{.interpreted-text role="class"} argument to the pytest_generate_tests{.interpreted-text role="hook"} hook.

pytest.CallInfo for the CallInfo <pytest.CallInfo>{.interpreted-text role="class"} type passed to various hooks.

pytest.PytestPluginManager for PytestPluginManager <pytest.PytestPluginManager>{.interpreted-text role="class"}.

pytest.ExceptionInfo for the ExceptionInfo <pytest.ExceptionInfo>{.interpreted-text role="class"} type returned from pytest.raises{.interpreted-text role="func"} and passed to various hooks.

pytest.Parser for the Parser <pytest.Parser>{.interpreted-text role="class"} type passed to the pytest_addoption{.interpreted-text role="hook"} hook.

pytest.OptionGroup for the OptionGroup <pytest.OptionGroup>{.interpreted-text role="class"} type returned from the parser.addgroup <pytest.Parser.getgroup>{.interpreted-text role="func"} method.

pytest.HookRecorder for the HookRecorder <pytest.HookRecorder>{.interpreted-text role="class"} type returned from ~pytest.Pytester{.interpreted-text role="class"}.

pytest.RecordedHookCall for the RecordedHookCall <pytest.HookRecorder>{.interpreted-text role="class"} type returned from ~pytest.HookRecorder{.interpreted-text role="class"}.

pytest.RunResult for the RunResult <pytest.RunResult>{.interpreted-text role="class"} type returned from ~pytest.Pytester{.interpreted-text role="class"}.

pytest.LineMatcher for the LineMatcher <pytest.RunResult>{.interpreted-text role="class"} type used in ~pytest.RunResult{.interpreted-text role="class"} and others.

pytest.TestReport for the TestReport <pytest.TestReport>{.interpreted-text role="class"} type used in various hooks.

pytest.CollectReport for the CollectReport <pytest.CollectReport>{.interpreted-text role="class"} type used in various hooks.

Constructing most of them directly is not supported; they are only meant for use in type annotations. Doing so will emit a deprecation warning, and may become a hard-error in pytest 8.0.

Subclassing them is also not supported. This is not currently enforced at runtime, but is detected by type-checkers such as mypy.

#7856: --import-mode=importlib <import-modes>{.interpreted-text role="ref"} now works with features that depend on modules being on :pysys.modules{.interpreted-text role="data"}, such as pickle{.interpreted-text role="mod"} and dataclasses{.interpreted-text role="mod"}.

#8144: The following hooks now receive an additional pathlib.Path argument, equivalent to an existing py.path.local argument:

pytest_ignore_collect{.interpreted-text role="hook"} - The collection_path parameter (equivalent to existing path parameter).

pytest_collect_file{.interpreted-text role="hook"} - The file_path parameter (equivalent to existing path parameter).

pytest_pycollect_makemodule{.interpreted-text role="hook"} - The module_path parameter (equivalent to existing path parameter).

pytest_report_header{.interpreted-text role="hook"} - The start_path parameter (equivalent to existing startdir parameter).

pytest_report_collectionfinish{.interpreted-text role="hook"} - The start_path parameter (equivalent to existing startdir parameter).

::: {.note} ::: {.admonition-title} Note :::

The name of the ~_pytest.nodes.Node{.interpreted-text role="class"} arguments and attributes (the new attribute being path) is the opposite of the situation for hooks (the old argument being path).

This is an unfortunate artifact due to historical reasons, which should be resolved in future versions as we slowly get rid of the py{.interpreted-text role="pypi"} dependency (see 9283{.interpreted-text role="issue"} for a longer discussion). :::

#8251: Implement Node.path as a pathlib.Path. Both the old fspath and this new attribute gets set no matter whether path or fspath (deprecated) is passed to the constructor. It is a replacement for the fspath attribute (which represents the same path as py.path.local). While fspath is not deprecated yet due to the ongoing migration of methods like ~_pytest.Item.reportinfo{.interpreted-text role="meth"}, we expect to deprecate it in a future release.

::: {.note} ::: {.admonition-title} Note :::

The name of the ~_pytest.nodes.Node{.interpreted-text role="class"} arguments and attributes (the new attribute being path) is the opposite of the situation for hooks (the old argument being path).

This is an unfortunate artifact due to historical reasons, which should be resolved in future versions as we slowly get rid of the py{.interpreted-text role="pypi"} dependency (see 9283{.interpreted-text role="issue"} for a longer discussion). :::

#8421: pytest.approx{.interpreted-text role="func"} now works on ~decimal.Decimal{.interpreted-text role="class"} within mappings/dicts and sequences/lists.

#8606: pytest invocations with --fixtures-per-test and --fixtures have been enriched with:

Fixture location path printed with the fixture name.

First section of the fixture's docstring printed under the fixture name.

Whole of fixture's docstring printed under the fixture name using --verbose option.

#8761: New version-tuple{.interpreted-text role="ref"} attribute, which makes it simpler for users to do something depending on the pytest version (such as declaring hooks which are introduced in later versions).

#8789: Switch TOML parser from toml to tomli for TOML v1.0.0 support in pyproject.toml.

#8920: Added pytest.Stash{.interpreted-text role="class"}, a facility for plugins to store their data on ~pytest.Config{.interpreted-text role="class"} and ~_pytest.nodes.Node{.interpreted-text role="class"}s in a type-safe and conflict-free manner. See plugin-stash{.interpreted-text role="ref"} for details.

#8953: RunResult <_pytest.pytester.RunResult>{.interpreted-text role="class"} method assert_outcomes <_pytest.pytester.RunResult.assert_outcomes>{.interpreted-text role="meth"} now accepts a warnings argument to assert the total number of warnings captured.

#8954: --debug flag now accepts a str{.interpreted-text role="class"} file to route debug logs into, remains defaulted to [pytestdebug.log]{.title-ref}.

#9023: Full diffs are now always shown for equality assertions of iterables when [CI]{.title-ref} or BUILD_NUMBER is found in the environment, even when -v isn't used.

#9113: RunResult <_pytest.pytester.RunResult>{.interpreted-text role="class"} method assert_outcomes <_pytest.pytester.RunResult.assert_outcomes>{.interpreted-text role="meth"} now accepts a deselected argument to assert the total number of deselected tests.

#9114: Added pythonpath{.interpreted-text role="confval"} setting that adds listed paths to sys.path{.interpreted-text role="data"} for the duration of the test session. If you currently use the pytest-pythonpath or pytest-srcpaths plugins, you should be able to replace them with built-in [pythonpath]{.title-ref} setting.

Improvements

#7480: A deprecation scheduled to be removed in a major version X (e.g. pytest 7, 8, 9, ...) now uses warning category [PytestRemovedInXWarning]{.title-ref}, a subclass of ~pytest.PytestDeprecationWarning{.interpreted-text role="class"}, instead of PytestDeprecationWarning{.interpreted-text role="class"} directly.

See backwards-compatibility{.interpreted-text role="ref"} for more details.

#7864: Improved error messages when parsing warning filters.

Previously pytest would show an internal traceback, which besides being ugly sometimes would hide the cause of the problem (for example an ImportError while importing a specific warning type).

#8335: Improved pytest.approx{.interpreted-text role="func"} assertion messages for sequences of numbers.

The assertion messages now dumps a table with the index and the error of each diff. Example:

> assert [1, 2, 3, 4] == pytest.approx([1, 3, 3, 5]) E assert comparison failed for 2 values: E Index | Obtained | Expected E 1 | 2 | 3 +- 3.0e-06 E 3 | 4 | 5 +- 5.0e-06

#8403: By default, pytest will truncate long strings in assert errors so they don't clutter the output too much, currently at 240 characters by default.

However, in some cases the longer output helps, or is even crucial, to diagnose a failure. Using -v will now increase the truncation threshold to 2400 characters, and -vv or higher will disable truncation entirely.

#8509: Fixed issue where unittest.TestCase.setUpClass{.interpreted-text role="meth"} is not called when a test has [/]{.title-ref} in its name since pytest 6.2.0.

This refers to the path part in pytest node IDs, e.g. TestClass::test_it in the node ID tests/test_file.py::TestClass::test_it.

Now, instead of assuming that the test name does not contain /, it is assumed that test path does not contain ::. We plan to hopefully make both of these work in the future.

#8803: It is now possible to add colors to custom log levels on cli log.

By using add_color_level <_pytest.logging.add_color_level>{.interpreted-text role="func"} from a pytest_configure hook, colors can be added:

logging_plugin = config.pluginmanager.get_plugin('logging-plugin') logging_plugin.log_cli_handler.formatter.add_color_level(logging.INFO, 'cyan') logging_plugin.log_cli_handler.formatter.add_color_level(logging.SPAM, 'blue')

See log_colors{.interpreted-text role="ref"} for more information.

#8822: When showing fixture paths in [--fixtures]{.title-ref} or [--fixtures-by-test]{.title-ref}, fixtures coming from pytest itself now display an elided path, rather than the full path to the file in the [site-packages]{.title-ref} directory.

#8898: Complex numbers are now treated like floats and integers when generating parameterization IDs.

#9062: --stepwise-skip now implicitly enables --stepwise and can be used on its own.

#9205: pytest.Cache.set{.interpreted-text role="meth"} now preserves key order when saving dicts.

Bug Fixes

#7124: Fixed an issue where __main__.py would raise an ImportError when --doctest-modules was provided.

#8061: Fixed failing staticmethod test cases if they are inherited from a parent test class.

#8192: testdir.makefile now silently accepts values which don't start with . to maintain backward compatibility with older pytest versions.

pytester.makefile now issues a clearer error if the . is missing in the ext argument.

#8258: Fixed issue where pytest's faulthandler support would not dump traceback on crashes if the faulthandler{.interpreted-text role="mod"} module was already enabled during pytest startup (using python -X dev -m pytest for example).

#8317: Fixed an issue where illegal directory characters derived from getpass.getuser() raised an OSError.

#8367: Fix Class.from_parent so it forwards extra keyword arguments to the constructor.

#8377: The test selection options pytest -k and pytest -m now support matching names containing forward slash (/) characters.

#8384: The @pytest.mark.skip decorator now correctly handles its arguments. When the reason argument is accidentally given both positional and as a keyword (e.g. because it was confused with skipif), a TypeError now occurs. Before, such tests were silently skipped, and the positional argument ignored. Additionally, reason is now documented correctly as positional or keyword (rather than keyword-only).

#8394: Use private names for internal fixtures that handle classic setup/teardown so that they don't show up with the default --fixtures invocation (but they still show up with --fixtures -v).

#8456: The required_plugins{.interpreted-text role="confval"} config option now works correctly when pre-releases of plugins are installed, rather than falsely claiming that those plugins aren't installed at all.

#8464: -c <config file> now also properly defines rootdir as the directory that contains <config file>.

#8503: pytest.MonkeyPatch.syspath_prepend{.interpreted-text role="meth"} no longer fails when setuptools is not installed. It now only calls pkg_resources.fixup_namespace_packages{.interpreted-text role="func"} if pkg_resources was previously imported, because it is not needed otherwise.

#8548: Introduce fix to handle precision width in log-cli-format in turn to fix output coloring for certain formats.

#8796: Fixed internal error when skipping doctests.

#8983: The test selection options pytest -k and pytest -m now support matching names containing backslash ([\]{.title-ref}) characters. Backslashes are treated literally, not as escape characters (the values being matched against are already escaped).

#8990: Fix [pytest -vv]{.title-ref} crashing with an internal exception [AttributeError: 'str' object has no attribute 'relative_to']{.title-ref} in some cases.

#9077: Fixed confusing error message when request.fspath / request.path was accessed from a session-scoped fixture.

#9131: Fixed the URL used by --pastebin to use bpa.st.

#9163: The end line number and end column offset are now properly set for rewritten assert statements.

#9169: Support for the files API from importlib.resources within rewritten files.

#9272: The nose compatibility module-level fixtures [setup()]{.title-ref} and [teardown()]{.title-ref} are now only called once per module, instead of for each test function. They are now called even if object-level [setup]{.title-ref}/[teardown]{.title-ref} is defined.

Improved Documentation

#4320: Improved docs for [pytester.copy_example]{.title-ref}.

#5105: Add automatically generated plugin-list{.interpreted-text role="ref"}. The list is updated on a periodic schedule.

#8337: Recommend numpy.testing module on pytest.approx{.interpreted-text role="func"} documentation.

#8655: Help text for --pdbcls more accurately reflects the option's behavior.

#9210: Remove incorrect docs about confcutdir being a configuration option: it can only be set through the --confcutdir command-line option.

#9242: Upgrade readthedocs configuration to use a newer Ubuntu version` with better unicode support for PDF docs.

#9341: Various methods commonly used for non-python tests{.interpreted-text role="ref"} are now correctly documented in the reference docs. They were undocumented previously.

Trivial/Internal Changes

#8133: Migrate to setuptools_scm 6.x to use SETUPTOOLS_SCM_PRETEND_VERSION_FOR_PYTEST for more robust release tooling.

#8174: The following changes have been made to internal pytest types/functions:

The _pytest.code.getfslineno() function returns Path instead of py.path.local.

The _pytest.python.path_matches_patterns() function takes Path instead of py.path.local.

The _pytest._code.Traceback.cut() function accepts any os.PathLike[str], not just py.path.local.

#8248: Internal Restructure: let python.PyObjMixin inherit from nodes.Node to carry over typing information.

#8432: Improve error message when pytest.skip{.interpreted-text role="func"} is used at module level without passing [allow_module_level=True]{.title-ref}.

#8818: Ensure regendoc opts out of TOX_ENV cachedir selection to ensure independent example test runs.

#8913: The private CallSpec2._arg2scopenum attribute has been removed after an internal refactoring.

#8967: pytest_assertion_pass{.interpreted-text role="hook"} is no longer considered experimental and future changes to it will be considered more carefully.

#9202: Add github action to upload coverage report to codecov instead of bash uploader.

#9225: Changed the command used to create sdist and wheel artifacts: using the build package instead of setup.py.

#9351: Correct minor typos in doc/en/example/special.rst.

Configuration

📅 Schedule: Branch creation - "at 10am every weekday" (UTC), Automerge - At any time (no schedule defined).

🚦 Automerge: Disabled by config. Please merge this manually once you are satisfied.

♻ Rebasing: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.

🔕 Ignore: Close this PR and you won't be reminded about this update again.

[ ] If you want to rebase/retry this PR, check this box

This PR has been generated by Mend Renovate. View repository job log here.
opened by renovate[bot] 0
chore(deps): update dependency pandas to ==1.5.*
This PR contains the following updates:

| Package | Change | Age | Adoption | Passing | Confidence | |---|---|---|---|---|---| | pandas (source) | ==1.4.* -> ==1.5.* | | | | |

Release Notes

pandas-dev/pandas

v1.5.2: Pandas 1.5.2

Compare Source

This is a patch release in the 1.5.x series and includes some regression and bug fixes. We recommend that all users upgrade to this version.

See the full whatsnew for a list of all the changes.

The release will be available on the defaults and conda-forge channels:

conda install pandas

Or via PyPI:

python3 -m pip install --upgrade pandas

Please report any issues with the release on the pandas issue tracker.

Thanks to all the contributors who made this release possible.

v1.5.1: Pandas 1.5.1

Compare Source

This is a patch release in the 1.5.x series and includes some regression and bug fixes. We recommend that all users upgrade to this version.

See the full whatsnew for a list of all the changes.

The release will be available on the defaults and conda-forge channels:

conda install pandas

Or via PyPI:

python3 -m pip install --upgrade pandas

Please report any issues with the release on the pandas issue tracker.

Thanks to all the contributors who made this release possible.

v1.5.0: Pandas 1.5.0

Compare Source

This release includes some new features, bug fixes, and performance improvements. We recommend that all users upgrade to this version.

See the full whatsnew for a list of all the changes. pandas 1.5.0 supports Python 3.8 and higher.

The release will be available on the defaults and conda-forge channels:

conda install -c conda-forge pandas

Or via PyPI:

python3 -m pip install --upgrade pandas

Please report any issues with the release on the pandas issue tracker.

Configuration

📅 Schedule: Branch creation - "at 10am every weekday" (UTC), Automerge - At any time (no schedule defined).

🚦 Automerge: Disabled by config. Please merge this manually once you are satisfied.

♻ Rebasing: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.

🔕 Ignore: Close this PR and you won't be reminded about this update again.

[ ] If you want to rebase/retry this PR, check this box

This PR has been generated by Mend Renovate. View repository job log here.
opened by renovate[bot] 0