yt is an open-source, permissively-licensed Python library for analyzing and visualizing volumetric data.

The yt project

Last update: Dec 25, 2022

Related tags

Data Analysis visualization python data analysis astronomy data-visualization geophysics scientific-visualization scientific-computing astrophysics finite-element-analysis nuclear-engineering

Overview

The yt Project

yt is an open-source, permissively-licensed Python library for analyzing and visualizing volumetric data.

yt supports structured, variable-resolution meshes, unstructured meshes, and discrete or sampled data such as particles. Focused on driving physically-meaningful inquiry, yt has been applied in domains such as astrophysics, seismology, nuclear engineering, molecular dynamics, and oceanography. Composed of a friendly community of users and developers, we want to make it easy to use and develop - we'd love it if you got involved!

We've written a method paper you may be interested in; if you use yt in the preparation of a publication, please consider citing it.

Code of Conduct

yt abides by a code of conduct partially modified from the PSF code of conduct, and is found in our contributing guide.

Installation

You can install the most recent stable version of yt either with conda from conda-forge:

conda install -c conda-forge yt

or with pip:

python -m pip install yt

More information on the various ways to install yt, and in particular to install from source, can be found on the project's website.

Getting Started

yt is designed to provide meaningful analysis of data. We have some Quickstart example notebooks in the repository:

If you'd like to try these online, you can visit our yt Hub and run a notebook next to some of our example data.

Contributing

We love contributions! yt is open source, built on open source, and we'd love to have you hang out in our community.

We have developed some guidelines for contributing to yt.

Imposter syndrome disclaimer: We want your help. No, really.

There may be a little voice inside your head that is telling you that you're not ready to be an open source contributor; that your skills aren't nearly good enough to contribute. What could you possibly offer a project like this one?

We assure you - the little voice in your head is wrong. If you can write code at all, you can contribute code to open source. Contributing to open source projects is a fantastic way to advance one's coding skills. Writing perfect code isn't the measure of a good developer (that would disqualify all of us!); it's trying to create something, making mistakes, and learning from those mistakes. That's how we all improve, and we are happy to help others learn.

Being an open source contributor doesn't just mean writing code, either. You can help out by writing documentation, tests, or even giving feedback about the project (and yes - that includes giving feedback about the contribution process). Some of these contributions may be the most valuable to the project as a whole, because you're coming to the project with fresh eyes, so you can see the errors and assumptions that seasoned contributors have glossed over.

(This disclaimer was originally written by Adrienne Lowe for a PyCon talk, and was adapted by yt based on its use in the README file for the MetPy project)

Resources

We have some community and documentation resources available.

Our latest documentation is always at http://yt-project.org/docs/dev/ and it includes recipes, tutorials, and API documentation
The discussion mailing list should be your first stop for general questions
The development mailing list is better suited for more development issues
You can also join us on Slack at yt-project.slack.com (request an invite)

Is your code compatible with yt ? Great ! Please consider giving us a shoutout as a shiny badge in your README

markdown

[![yt-project](https://img.shields.io/static/v1?label="works%20with"&message="yt"&color="blueviolet")](https://yt-project.org)

|yt-project|

.. |yt-project| image:: https://img.shields.io/static/v1?label="works%20with"&message="yt"&color="blueviolet"
   :target: https://yt-project.org

Powered by NumFOCUS

yt is a fiscally sponsored project of NumFOCUS. If you're interested in supporting the active maintenance and development of this project, consider donating to the project.

Comments

merging yt-4 into master

This is a reopening of #2172 on my repository. I'm removing WIP because, although it's still a work in progress, it's getting close enough that I want others to check in with it once in a while.
demeshening infrastructure yt-4.0

opened by matthewturk 92
ENH: Adding cfradial frontend.
PR Summary

This pull request contains the commits from @ngoldbaum , @MatthewTurk and I for the cfradial frontend. The new front ends allows for a radar file to be loaded using yt. Unit tests have been written for the frontend. I had to do a revert because I forgot I had the plot callback in this branch, so I reverted the commit, and have the plot callback code in a separate branch for a future pull request.

PR Checklist

[x] Code passes flake8 checker

[x] New features are documented, with docstrings and narrative docs

[x] Adds a test for any bugs fixed. Adds tests for new features.

code frontends new feature domain: meteorology
opened by zssherman 56
ENH: swap_axis kwarg for PlotWindow
This PR adds callbacks to adjust axes orientations (in support of #3631), inroducing the swap_axes, flip_horizontal and flip_vertical callbacks.

Example usage:

import yt ds = yt.load_sample("IsolatedGalaxy") slc = yt.SlicePlot(ds, "z", ("enzo", "Density"), width=(0.03, 0.01)) slc.save("/tmp/swap_axes_normal.png") slc.swap_axes() slc.save("/tmp/swap_axes_swapped.png") slc = yt.SlicePlot(ds, "z", ("enzo", "Density"), width=(0.03, 0.01), swap_axes=True) slc.save("/tmp/swap_axes_swapped_2.png")

Results in:

The unswapped reference (swap_axes_normal.png):

The two images with swapped axes produced by the above snippet are identical:

Works with projections as well, works with axis aligned and off-axis plots (though it's probably not super useful or intuitive to use it with off-axis plots).

The flip_horizontal and flip_vertical are essentially aliases to matplotlib's invert_xaxis and invert_yaxis.

Additionally, the callbacks have been adjusted to work with the new callbacks.

Note that there is potential for some user confusion since the operations apply to image axes and not dataset coordinates and the swap always has to happen first, a related issue #3890 was started to track this so it is resolved before next release.
enhancement viz: 2D new feature deprecation
opened by chrishavlin 47
ENH: Chimera Frontend
PR Summary

This pull request includes a new frontend for use with the CHIMERA code's data format. This code is a grid-based, non-AMR MHD code with spherical geometry and a logarithmic radial coordinate and is utilized for simulations of Type-II core-collapse supernovae in 2 and 3 dimensions. The data required for the answer tests included in the frontend was previously submitted via curl. Development was performed from the basis of the _skeleton frontend, along with advice from John ZuHone and Matt Turk. If this request should be submitted to a different branch, or if there are any other issues, please let me know.

PR Checklist

[ ] New features are documented, with docstrings and narrative docs

[ ] Adds a test for any bugs fixed. Adds tests for new features.

code frontends new feature
opened by Ronan-Hix 47
BUG: Projection of periodic boxes with specified width smaller than boxsize.
I've seemingly found an issue with periodic box wrapping in ProjectionPlot. Below is a 25 Mpc/h IllustrisTNG periodic volume, and I kept getting maps zoomed into halos cut off at the periodic boundary. This was seemingly confounding because when I run a whole box, the wrapper works perfectly. After a lot of tinkering, I think it has to do with the width argument being not equal to the domain. I made images of 23000, 24000, 25000, and 26000 kpc/h centered at (22000, 22000, 22000), which I think serves as a good diagnostic of the problem. A loss gap opens up at the periodic boundary (i.e. 25000) and grows as the width becomes smaller (23000 is upper left, 24000 upper right, and so on). Therefore for a ProjectionPlot zoomed on a specific halo near the periodic boundary, you often incorrectly miss the halo. Here's the code snippet (I used FITSProjection which is a wrapper for ProjectionPlot, which has same issue):

my_slice = ds.box(left_edge, right_edge) Pos = [22000,22000,22000] Width = 23000 proj_fits_slice = yt.FITSProjection(ds, "z", ("gas","O_p6_number_density"), data_source=my_slice, image_res=npix,center=Pos,width=Width)

Note this is in all types of periodic simulations- EAGLE, Simba too. Maybe I'm doing something wrong, and if there is a quick fix, let me know.

I've added a ProjectionPlot of oxygen (instead of O^6+) to more clearly show the 23000 case.

Version Information python 3.9.5 yt 4.1.dev0

Note that this issue has been seen in earlier version of yt going back a couple years. i.e. before yt 4.0

bug viz: 2D
opened by benopp99 46
RFC: a more robust plot norm/colorbar API
PR Summary

This is a deep refactor of how colorbar and plot norm data are stored and utilized internally. It allows for a much more robust API where chaining operations like set_zlim and set_unit doesn't ruin the whole viz (fix #2538). It also makes room to implement stuff like custom norm (fix #3840), and helps resolve a documented quirky behaviour (fix #3852).

It unifies the various strategies scattered across plot classes to determine what norm can be used in a given situation (symlog, log, linear ?), which is a area where we've seen many bugs in the recent pastn, basically on every matplotlib feature release.

I've been iterating and testing on this for a couple weeks locally, I'm opening the PR while it's still hot to see what bug existing tests might reveal. Documentation is already included, but for now I haven't committed my test files because they are not adapated to yt's image comparison testing framework yet.

Note that this includes the changes from #2504, which changes some phase plots with respect to their gold standard, so CI should fail at the very least there, but no other failures are desired.

edit: I'm bumping matplotlib's minimal required version to 3.1 because 2.2.3 has some incompatibilities with the refactor and 3.1 is the oldest testable version for macOS. FTR Matplotlib 3.1 was released in May 2019 and was only distributed for Python 3.6 and 3.7, so it's still a pretty conservative requirement now.

Note to reviewers

I opened a "companion" PR at #3900, which contains only the new tests, not the patch, to help demonstrate what's working here and not on the main branch.

I also tried to keep the commit history somewhat tidy and readable, so reading commits by chronological order is one possible angle to review this in steps.

After a lot of iterations, I bumped/added new answers both on Jenkins and the answer store. The supporting PR is https://github.com/yt-project/answer-store/pull/31

A Summary of user-facing changes

A lot of symlog-related code is now centralised, making it reusable to all PlotContainer subclasses (not just ImagePlotContainer).

api changes

updated ImagePlotContainer.set_zlim to :

allow passing zmin or zmax only (previously they had to be both specified somehow)

allow passing unit-aware values (either as unyt_quantity objects or parsable tuples e.g. (10, "g/cm**2"))

updated PlotContainer.set_log so that the second argument (log) isn't necessary when passing linthresh. Previously users had to provide a value to the log argument but it was not actually used. Now a warning is emitted if a log value is received but discarded (this is however not deprecated).

added a PlotContainer.set_norm method to allow using matplotlib norms other than lin, log and symlog

deprecations

passing zmin=None (resp zmax=None) to set_zlim explicitly is deprecated (users can leave the parameter unset or pass zmin="min" (resp zmax="max") explicitly instead)

PlotContainer.set_log's symlog_auto parameter is deprecated (the same behaviour can be obtained by passing linthresh="auto" instead)

dependencies

bumped minimal supported version of MPL from 2.2.3 to 3.1 (this represents less than a 1yr jump)

added a dependency to typing_extensions (which are backports from the standard lib's typing module), exclusively for Python 3.7 (we can rely fully on the std lib in 3.8 and beyond)

PR Checklist

[x] New features are documented, with docstrings and narrative docs

[x] cleanup remaining todos in the code (TODO(clm))

[x] Adds a test for any bug fixed. Adds tests for new features.

[x] cleanup history to facilitate reviews

[x] check that #3901 is (or stays) fixed

[x] add a test for it

The following PRs should be handled before this one:

[x] #2504

[x] #3818

[x] #3888

bug viz: 2D new feature api-consistency refactor deprecation
opened by neutrinoceros 44
ENH: Enable variable cell widths for grid datasets
PR Summary

I'm starting the process of adding variable dds attributes to grid objects, so that we can potentially support arbitrarily spaced grids without requiring that we use the semi structured mesh work.

I'd like this to be in a constant state of workability, so it won't show up as "mergeable" but I do hope the GHA tests run.

PR Checklist

[ ] pass black --check yt/

[ ] pass isort . --check --diff

[ ] pass flake8 yt/

[ ] pass flynt yt/ --fail-on-change --dry-run -e yt/extern

[ ] New features are documented, with docstrings and narrative docs

[ ] Adds a test for any bugs fixed. Adds tests for new features.

new feature generalised orthogonal grids
opened by matthewturk 44
document issue with old setuptools or distribute
Originally reported by: chummels (Bitbucket: chummels, GitHub: chummels)

I just pulled from the tip of yt dev (be66247e2a8a), and ran python setup.py develop in the yt-hg directory only to receive this error:

http://paste.yt-project.org/show/6228/

On the advice of @ngoldbaum , I ran pip install -U setuptools to update setuptools and tried rerunning python setup.py develop, which resulted in a new error:

http://paste.yt-project.org/show/6229/

Running clean.sh did not help with the problem, and I continue to receive the same error as above when I attempt to python setup.py develop. I'm not sure what to do. For reference, when I run pip freeze, I get the following versions:

http://pastebin.com/Kh8wih9D

Not sure what to do. But I had a working version of ~week-old yt prior to pulling the updates from last weeks' triage. For reference, my yt install was originally built from the install_script on Mac OS X 10.8.5.

Bitbucket: https://bitbucket.org/yt_analysis/yt/issue/1169

bug
opened by yt-fido 41
Enabling PhasePlots to have log axis ranges by default Fixes #2212
PR Summary

Enables PhasePlots to have log-based axis ranges by default, just like grid-based datasets. Fixes #2212.

Now when you run the script from that issue:

import yt ds = yt.load('FIRE_M12i_ref11/snapshot_600.hdf5') yt.PhasePlot(ds, ('gas', 'density'), ('gas', 'temperature'), ('gas', 'mass'), weight_field=('gas', 'mass')).save() ds = yt.load('IsolatedGalaxy/galaxy0030/galaxy0030') yt.PhasePlot(ds, ('gas', 'density'), ('gas', 'temperature'), ('gas', 'mass'), weight_field=('gas', 'mass')).save()

PR Checklist

[ ] Code passes flake8 checker

[ ] New features are documented, with docstrings and narrative docs

[ ] Adds a test for any bugs fixed. Adds tests for new features.

bug demeshening backwards incompatible yt-4.0
opened by chummels 40
ENH: Stop preallocating particles

This is a reasonably self-contained pull request that removes all preallocation of particles.

With just this change, the example code in #2383 goes from 20s per iteration to 5 seconds per iteration.

See discussion in #2412 .
enhancement performance demeshening

opened by matthewturk 38
Add support for AMRVAC native data files
PR Summary

We propose a new frontend for yt to support MPI-AMRVAC (http://amrvac.org/index.html) Most functionalities are hopefully ready for production, however a few issues are identified and should be adressed in this PR or later as separate issues. (see last section)

Testing

data samples can be fetched here https://www.dropbox.com/s/45sbkq57neacbu8/datasets_amrvac.tar.gz?dl=0

and should be registered to https://github.com/yt-project/website/blob/master/data/datafiles.json

tests are located in yt/frontends/amrvac/tests/test_unit.py and currently refer to a local (not bundled in) data directory (tmpdir). This obvious needs some cleanup before final validation.

PR Checklist

[x] Code passes flake8 checker

[ ] New features are documented, with docstrings and narrative docs

[x] Adds a test for any bugs fixed. Adds tests for new features.

note : I do not know if the code is documented enough. Given that it is subclassing existing class from yt, it should be clear. If not, let us know.

Known issues

plot.annotate_velocities() breaks (crash)

~default units displayed for figure axes should be "code units" instead of cgs.~ SOLVED ✅

~ProjectionPlot breaks : all patches are drawn in a corner of the figure frame while the grids seem correctly parsed~ SOLVED ✅

# example with a cartesian dataset ds = yt.load("bw_3d0000.dat") projplot = yt.ProjectionPlot(ds, "z", "rho")
opened by neutrinoceros 38
BUG: Use the correct axes for particle positions
In lower-than-three dimension datasets, we still attempt to utilize x, y, z for all particle fields.

One specific place this can cause issues is in the setup_particle_fields function in field_info_container.py. Here, lines like:

[f"particle_position_{ax}" for ax in "xyz"],

always assume that we have all three axes. In some cases, these do exist! We often will mock up the particle z for lower-dimension fields. But, in some cases, they do not. We should be more careful about this.
opened by matthewturk 0
BUG: yt.load("~...") doesn't work anymore
Bug report

Bug summary

up until yt 4.0.5, this used to work

import yt yt.load("~/path/to/my/data")

in yt 4.1.x it doesn't (because the ~ is not expanded).

Without running a bisection it seems pretty clear that the breaking change was in #3525, and I can say it wasn't intentional. Luckily the fix is trivial.
bug
opened by neutrinoceros 0

BLD: deprecation warnings from setuptools (auto-included packages)

Building a wheel with --verbose

python -m pip wheel --verbose --progress-bar off --no-deps --wheel-dir build .

Yields the following warnings

SetuptoolsDeprecationWarning:     Installing 'yt.frontends.amrvac.tests.sample_parfiles' as data is deprecated, please list it in `packages`.
      !!


      ############################
      # Package would be ignored #
      ############################
      Python recognizes 'yt.frontends.amrvac.tests.sample_parfiles' as an importable package,
      but it is not listed in the `packages` configuration of setuptools.

      'yt.frontends.amrvac.tests.sample_parfiles' has been automatically added to the distribution only
      because it may contain data files, but this behavior is likely to change
      in future versions of setuptools (and therefore is considered deprecated).

      Please make sure that 'yt.frontends.amrvac.tests.sample_parfiles' is included as a package by using
      the `packages` configuration field or the proper discovery methods
      (for example by using `find_namespace_packages(...)`/`find_namespace:`
      instead of `find_packages(...)`/`find:`).

      You can read more about "package discovery" and "data files" on setuptools
      documentation page.

7 directories are affected

yt.frontends.amrvac.tests.sample_parfiles
yt.frontends.artio.artio_headers
yt.geometry._selection_routines
yt.utilities.grid_data_format.docs
yt.utilities.grid_data_format.scripts
yt.utilities.lib.cykdtree.windows
yt.utilities.lib.ewahboolarray (soon to be removed in #2711)

It should be easily fixable, but it's likely that we don't actually want to include everything there (I suspect some of these files are actually not used anymore)

bug infrastructure

opened by neutrinoceros 6

BLD: Fails to build: internal_error

Bug report

Bug summary

When building the package as part of the packaging yt for openSUSE, the fails often (not always, something between 50 % and 80 %) it fails with an error like this:

[  338s]   gcc -Wno-unused-result -Wsign-compare -DNDEBUG -O2 -Wall -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=3 -fstack-protector-strong -funwind-tables -fasynchronous-unwind-tables -fstack-clash-protection -Werror=r
eturn-type -g -DOPENSSL_LOAD_CONF -fwrapv -fno-semantic-interposition -O2 -Wall -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=3 -fstack-protector-strong -funwind-tables -fasynchronous-unwind-tables -fstack-clash-protecti
on -Werror=return-type -g -IVendor/ -O2 -Wall -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=3 -fstack-protector-strong -funwind-tables -fasynchronous-unwind-tables -fstack-clash-protection -Werror=return-type -g -IVendor
/ -O2 -Wall -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=3 -fstack-protector-strong -funwind-tables -fasynchronous-unwind-tables -fstack-clash-protection -Werror=return-type -flto=auto -freport-bug -fPIC -Iyt/utilities/
lib/ -Iyt/utilities/lib -I./yt/utilities/lib -I/usr/include/python3.10 -I/usr/lib64/python3.10/site-packages/numpy/core/include -c yt/utilities/lib/marching_cubes.cpp -o build/temp.linux-x86_64-cpython-310/yt/u
tilities/lib/marching_cubes.o
[  338s]   lto1: internal compiler error: resolution sub id 0xadc1906a211a636c not in object file
[  338s]   0x102f76b internal_error(char const*, ...)
[  338s]         ???:0
[  338s]   0x15cfb50 lto_main()
[  338s]         ???:0
[  338s]   Please submit a full bug report, with preprocessed source.
[  338s]   Please include the complete backtrace with any bug report.
[  338s]   See <https://bugs.opensuse.org/> for instructions.
[  338s]   lto-wrapper: fatal error: /usr/bin/g++ returned 1 exit status
[  338s]   compilation terminated.
[  338s]   /usr/lib64/gcc/x86_64-suse-linux/12/../../../../x86_64-suse-linux/bin/ld: error: lto-wrapper failed
[  338s]   collect2: error: ld returned 1 exit status

Code for reproduction

It gets triggered while building the package, so no obvious code snippet.

Complete build log with all packages used and steps taken to achieve this problem.

Actual outcome

See the build error above.

Expected outcome

Clean build.

Version Information

Operating System: Linux/openSUSE/Tumbleweed (rolling distro) as of 2023-01-02
Python Version: this one is for Python 3.10.9, but I don’t think it is interpreter driven
yt version: 4.1.2 from the PyPI tarball
Other Libraries (if applicable): see above log for the complete list

All packages from openSUSE packages.

When I have consulted this with our GCC expert (hi, @marxin!) I got this suggestion:

Hi, I'm almost sure that the problem is that there are some wrong dependencies and some .o file is used before its producer completes it. I have seen it many times, it manifests itself in different voices and the fact that it is not deterministic:

[ 10s] lto1: internal compiler error: resolution sub id 0xb2bc78268f7df7a0 not in object file [ 10s] 0x102f987 internal_error(char const*, ...) [ 10s] ???:0 [ 10s] 0x15d02f0 lto_main() [ 10s] ???:0

or that message of yours with lto1: internal compiler error: original not compressed with zstd

bug infrastructure

opened by mcepl 4

UX: add information about missing optional dependencies as warnings and error messages in yt.load
PR Summary

Fix #4274 For now I've only tested this manually but it should be relatively straighfoward to add tests.

TODO:

[x] tests

[x] clean up redundant checks and warnings (warn_h5py and the likes) from _is_valid methods

[x] implement early returns in _is_valid methods when requirements are missing

[x] rebase

enhancement deprecation UX
opened by neutrinoceros 0

Releases(yt-4.1.3)

yt-4.1.3(Dec 20, 2022)
Summary

This is the third bugfix release in the yt 4.1.x series. It contains fixes to bugs discovered since the 4.1.2 release. Most importantly, it reverts a unstable patch that was accidentally included in yt 4.1.2 (https://github.com/yt-project/yt/pull/4198). EWAH files should now behave as in yt 4.1.1, and we're aiming at re-releasing the new behaviour (with fixes) in yt 4.2.0

There are no new features or deprecations in this version, all users are encouraged to upgrade.

Python versions supported via our main distribution channels

| Python version | PyPI | conda-forge | |----------------|:-----:|:------------:| | CPython 3.7 | ✅ | ❌ | | CPython 3.8 | ✅ | ✅ | | CPython 3.9 | ✅ | ✅ | | CPython 3.10 | ✅ | ✅ | | CPython 3.11 | ✅ | ✅ |

notes

CPython 3.7 has been dropped by conda-forge itself

How to upgrade

To upgrade from PyPI, run

python -m pip install --upgrade yt

or, with conda

conda update --channel conda-forge yt

🐛 Bug fixes

#4231 Fix pixel size calculation for density option in particle projection plots and FITS images, by @jzuhone #4239 Fix a broken docs example, by @neutrinoceros

🧩 Build and testing

#4247 ignore a deprecation warning triggered from cartopy, by @neutrinoceros #4250 Revert "Backport PR #4198 on branch yt-4.1.x (Automatically find EWAH files with increased index_order2)", by @neutrinoceros #4258 on branch yt-4.1.x, by @neutrinoceros

Full Changelog: https://github.com/yt-project/yt/compare/yt-4.1.2...yt-4.1.3
Source code(tar.gz)
Source code(zip)
yt-4.1.2(Nov 19, 2022)
Summary

This is the second bugfix release in the yt 4.1.x series. It contains fixes to bugs discovered since the 4.1.1 release. There are no new features or deprecations in this version, all users are encouraged to upgrade.

Python versions supported via our main distribution channels

| Python version | PyPI | conda-forge | |----------------|:-----:|:------------:| | CPython 3.7 | ✅ | ❌ | | CPython 3.8 | ✅ | ✅ | | CPython 3.9 | ✅ | ✅ | | CPython 3.10 | ✅ | ✅ | | CPython 3.11 | ✅ | ✅ |

notes

CPython 3.7 has been dropped by conda-forge itself

update December 14, 2022: This release is now also available on conda-forge for CPython 3.11

How to upgrade

To upgrade from PyPI, run

python -m pip install --upgrade yt

or, with conda

conda update --channel conda-forge yt

🐛 Bug fixes

#4176 Make a MPL failure non-fatal for set_norm with old version of MPL, by @neutrinoceros #4198 Automatically find EWAH files with increased index_order2, by @jzuhone #4216 Fix compatibility between eps_writer and yt 4.1, by @neutrinoceros #4211 Fix double counting of H nuclei in the Arepo frontend, by @jzuhone #4221 Fix security issue (CVE-2007-4559), by @TrellixVulnTeam

🧩 Build and testing

#4186 Migrate OSX CI from osxfuse to macfuse, by @neutrinoceros #4189 Resolve compatibility with pytest 7.2, by @neutrinoceros (issue #4188) #4200 Fix a missing type declaration in Cython file, by @neutrinoceros #4191 Add missing __init__.py files in Cholla and Chimera test folders, by @neutrinoceros (issue #4190 by @olebole) #4206 Add libm to build for some Cython files, by @matthewturk (issue #4204 by @papoteur-mga)

Full Changelog: https://github.com/yt-project/yt/compare/yt-4.1.1...yt-4.1.2
Source code(tar.gz)
Source code(zip)
yt-4.1.1(Oct 17, 2022)
Summary

This is the first bugfix release in the yt 4.1.x series. It contains a couple of critical fixes to bugs discovered since the 4.1.0 release. Most importantly, it resolves a memory leak that was affecting octree-based frontends (RAMSES and ART). There are no new features or deprecations in this version, all users are encouraged to upgrade.

How to upgrade

To upgrade from PyPI, run

python -m pip install --upgrade yt

or, with conda

conda update --channel conda-forge yt

Note that binaries for Python 3.11 are only available via pip.

🐛 Bug fixes

#4160 hotfix iterative plot annotations, by @neutrinoceros (issue #4159) #4164 Memory leak: free C++ memory before deallocing Python object, by @cphyc #4156 (RAMSES) catch assertion errors with ill-written namelist files, by @cphyc

🧩 Requirements

#4155 ipywidget>=8.0.0 is now required (issue #4154) #4163 and #4166: unyt>=3.0 is now forbidden (this is a temporary, and pre-emptive measure before unyt 3.0 is actually released)

Python 3.7 to 3.11 is supported
Source code(tar.gz)
Source code(zip)
yt-4.1.0(Oct 7, 2022)
Summary

This is the first feature release in the yt 4.x series. It contains about a year of work by many contributors, with many new features and improvements to the code base. It is also the first release to officially supported Python 3.11 Some deprecations from yt < 4 are now expired, some minor backwards incompatibilities are expected. Anything broken on purpose in this version should correspond to a warning in yt 4.0.x, anything beyond that should be considered a regression; please report any unexpected breaking change.

How to upgrade

To upgrade from PyPI, run

python -m pip install --upgrade yt

or, with conda

conda update --channel conda-forge yt

Note that binaries for Python 3.11 are only available via pip.

🌟 Highlights

Newly supported data formats

yt 4.1 ships with 3 brand new frontends !

#1990 CFRadial, by @zssherman and @chrishavlin

#3638 Chimera, by @Ronan-Hix

#3663 Cholla, by @chummels , @evaneschneider , and @matthewturk

Improved loading for "ambiguous" data formats

yt.load sometimes raises YTAmbiguousDataType when more than one data format appears to be valid. Ideally this should never happen, but in some cases it is very frequent, in particular with BoxLib data, for which we support many hard-to-distinguish flavours. In order to mitigate this issue, yt.load now accepts a hint keyword argument. For instance

yt.load("det_x_plt00000", hint="castro")

Stretched grid support

yt now has experimental support for "stretched" grid schemes (grids with arbitrarily varying cell width). It is currently not compatible with AMR.

The following contains a non-exhaustive list of changes

🐛 Bug fixes

#2538 Combining set_zlim and set_unit methods doesn't work as expected (fixed in #3849, by @neutrinoceros ) #3901 Cannot plot data containing both finite and infinite values and use set_zlim (fixed in #3849, by @neutrinoceros) #3986 Restore and fix flexible linear threshold guessing in symmetric logarithmic plots, by @chrishavlin (issue #3944) #4021 Fix multiple issues with magnetic field derived fields definitions, by @neutrinoceros #4075 Fix scaling in off-axis projections of filtered gas particles, by @jzuhone #4098 Correcting deceptive typo in an error message, by @chummels #4149 Fix a bug in the computation of metallicity for the generation of X-ray emission fields, by @geoyuqiu

🚀 New features

#3346 Allow to set units from config file, by @cphyc #3421 Allow stream frontends to accept callables, by @matthewturk #3443 Directly open compressed files[^1], by @cphyc #3640 Add swap_axes, flip_horizontal and flip_vertical plot modifier methods, by @chrishavlin #3666 Implement hint keyword argument for yt.load to help lifting ambiguities in data formats, by @neutrinoceros (issues #3510 by @mameehan5 and #3005 by @zingale) #3686 Allow 'factor' argument in plot annotation methods to accept 2-tuples to distinguish x/y resolutions, by @neutrinoceros #3756 Expose FRB filtering API, by @cphyc and @neutrinoceros #3812 Allow for different normalizations of magnetic fields in CGS[^2], by @jzuhone (issue #3471 by @forrestglines) #3822 Add support for varying colors in quiver annotations, by @neutrinoceros #3940 Add "min" method to ProjectionPlot, by @rjfarber #3489 yt.ProjectionPlot can now perform off-axis projections[^3], by @neutrinoceros (also #3723 and #3737) #3817 Add support for VelocityCallback and MagFieldCallBack in spherical coordinates, by @neutrinoceros #3989 Add support for projecting a field's standard deviation, by @jzuhone #4063 Add data_source argument to covering grid, by @chrishavlin #4119 Allow passing a dataset as a data source to ParticlePhasePlot, by @neutrinoceros #4121 Add public API render methods to PlotContainer and FixedResolutionBuffer, by @neutrinoceros #4142 ENH: add API to switch raytracing engine at runtime, by @neutrinoceros

experimental features

#2567 First cut at showing multiple transfer functions on VR, by @zingale #2998 Enable variable cell widths for grid datasets #2998, by @matthewturk #3849 Add ability to use colorbar norms other than linear, log and symlog, by @neutrinoceros (issue #3840)

[^1]: this feature requires ratarmount as an optional dependency to yt, currently only Linux is supported, see #3749 [^2]: at the moment this is only used in Athena and Athena++ frontends [^3]: yt.SlicePlot is now a class (previously it was a function). Instantiating OffAxisProjectionPlot (resp OffAxisSlicePlot) directly is now unnecessary (but still supported)

♻️ Optimisations and refactors

Performance optimisations

#2416 Stop preallocating particles, by @matthewturk and @chrishavlin, issue #2383 by @zhafen #3439 Cache _determine_fields for a given dataset by @Xarthisius #3469 Optimise memory layout for particle oct container, by @themousepotato #4029 Only compute field suggestions if YTFieldNotFound isn't caught, by @neutrinoceros #3935 Import optional dependencies lazily[^4], by @neutrinoceros #4025 Optimise startup time, by @neutrinoceros #4037 First steps toward utilizing the octree infrastructure as an engine for block-structured data, by @BolunThompson #4071 Optimise some grid selection operations by skipping unnecessary symbolic unit computations, by @yipihey #4079 Optimise radius calculation, by @yipihey (+ #4091 by @brittonsmith)

[^4]: with the exception of netCDF4, which is still imported at startup time if available

User experience and developer experience improvements

#3065 Make Scene.save less verbose, by @neutrinoceros #3436 Improve clarity of error message for region slicing with 2D dataset, by @neutrinoceros #3495 Warn in case a Dataset subclass is redefined, by @neutrinoceros #3519 Improve user input validation for PLViewerMPL._setup_origin, by @neutrinoceros #3525 Allow direct dataset instantiation using the user path '~' glyph, consistently with yt.load, by @neutrinoceros #3527 Improve formatting for YTUnidentifiedDataType exception, by @neutrinoceros #3621 Improve verbosity of yt.enable_parallelism in case of error, by @neutrinoceros

Quality improvements and refactors

#3526 Particle reader cleanup, by @matthewturk #3546 Setup basic type checking and fix existing type errors, by @neutrinoceros #3617 Factorise coordinate handler, by @cphyc #3624 Factorise code in halo containers, by @cphyc #3626 Decouple yt.config from initialisation[^5], by @neutrinoceros #3772 Avoid storing relative paths in Dataset attributes, by @neutrinoceros #3920 Improve formatting consistency in YTException error messages, by @neutrinoceros #3957 Refactor plot callbacks registration, by @neutrinoceros (issue #3945) #3998 Update to fields used in Enzo answer testing, by @jwise77 #4116 Avoid subclassing builtin dict, by @neutrinoceros #4057 Cleanup dead attribute Dataset.file_style, by @neutrinoceros #4125 Stop using deprecated np.int, by @neutrinoceros #4143 Updating Firefly-dependent areas to match API changes and PyPI packaging, by @mtryan83

[^5]: a consequence of this patch is that yt will no longer create configuration files on startup if it can't find one

📚 Documentation

#3395 and #3412 Avoid using pylab in docs, by @neutrinoceros #3608 Document how Dataset._is_valid works, by @neutrinoceros (issue #1018 by @ngoldbaum) #3627 Remove a false statement from yt.funcs.rootonly's docstring, by @neutrinoceros (issue #2054 by @DTHaydon) #3642 Fix a broken example (change of api in the Athena frontend), by @neutrinoceros (issue #3641) #3813 Update contribution guidelines, by @neutrinoceros (issue #2554 by @munkm) #3926 Update broken link to re-licensing blog post, by @tracykteal #3982 Fix broken annotate_particles example, by @chrishavlin (issue #3968 by @neutrinoceros) #4080 Document plot callbacks extensibility, by @neutrinoceros (issue #4072) #4124 Update instructions and redirections for installing cartopy, by @neutrinoceros #4148 Add more info to magnetic fields documentation, by @jzuhone

🤖 Frontends

Frontend-specific improvements and bug fixes

[AMRVAC] #3508 Optimise reading speed, by @neutrinoceros

[AREPO] #3815 Add three new derived fields, by @jzuhone #3258 New fields for Arepo and Gadget frontends, by @jzuhone #4120 Add Arepo stellar photometric fields , by @jzuhone

[BoxLib] #3515 Add a missing close statement, by @mameehan5 #4112 allow reading of 1D particle data, by @dpgrote

[Enzo-E] #3914 Update Enzo-E particle mass/density distinction, by @brittonsmith #4097 Store the Enzo-E version number in parameters attribute of EnzoEDataset, by @mabruzzo #4099 Improved Enzo-E frontend's determination of fluid properties, by @mabruzzo

[Gadget] #3258 New fields for Arepo and Gadget frontends, by @jzuhone #3455 Make sure we add the Parameters group to hvals if it's present, by @jzuhone

[GAMER] #4070 Use lookback time in cosmological simulation, by @koarakawaii

[RAMSES] #3332 Allow user customization of RAMSES field loading, by @cphyc

💣 Deprecations

Expired deprecations and known-breaking changes

#3240 Deprecation cycle (4.0 -> 4.1), by @neutrinoceros #3957 Deprecate dict argument plot_args in annotate_* methods where a **kwargs is sufficient, by @neutrinoceros #3970 Turn ambiguous fields warnings into errors[^8], by @neutrinoceros (issues #3416, and #3235 by @cphyc)

#4142 changed the default ray tracing engine from "embree" to "yt" (builtin engine). Previous versions of yt would switch to embree at startup if pyembree was installed. This version requires that embree be explicitly required, either with configuration file:

[yt] ray_tracing_engine = "embree"

or at runtime

from yt.visualization.volume_rendering.api import set_raytracing_engine set_raytracing_engine("embree")

New deprecations

#3278 Deprecate yt.mods, by @neutrinoceros (issue #3090) #3418 Deprecate iyt, by @neutrinoceros #3567 Deprecate annotate_halos[^6], by @neutrinoceros (issue #3566 by @brittonsmith) #3773 Deprecate yt.testing.run_nose and wrap it in the yt namespace[^7], by @neutrinoceros (issue #3771 by @brittonsmith) #4021 Deprecate some ambiguous field aliases, by @neutrinoceros

[^6]: this method is now being defined in the yt_astro_analysis extension package, which was required already, so it should continue to work seamlessly with sufficiently recent versions of the extension [^7]: a side effect of this patch is that importing yt doesn't load the yt.testing module by default, this change may break downstream code using yt.testing without an explicit import

Prolonged deprecations

#3836 Complete existing deprecation warnings' metadata, by @neutrinoceros

[^8]: this is nuanced by #3948 and #3949 (both PRs by @neutrinoceros)

🧩 Requirements

Python 3.7 or newer is now required

numpy 1.14.5 or newer is now required

matplotlib 3.1 or newer is now required

typing-extensions is a new dependency, required only on Python 3.7

importlib-metadata is a new dependency, required only on Python 3.7

tomli is now only required for Python 3.10 and older

Notes for downstream packagers and developers

We've started adding type hints to yt ! yt now ships a py.typed marker file, enableing type checking code which uses yt as a dependency via mypy. This initiative has a limited scope for now (we're far from having a completely typed code base), and there may be rough edges, feedback from downstream is extremely welcome !

We've also added a yt.version_info named tuple (#3761 by @neutrinoceros). This tuple is analogous to sys.version_info, and can be used to check yt's version at runtime, and write version-dependent code like so

import yt if yt.version_info >= (4, 1): ... else: ...

Of course for now only yt 4.1.0 has this tuple, so it is not going to be useful right away, but you can rely on it when you drop support for versions older than 4.1.0

Previous versions also incorrectly prevented compilation with Python 3.12 and above, this limitation has been lifted.

#3526 by @matthewturk -> Although considered a refactor, this is a breaking change for any third party particle frontends. We do not know of one, please let us know if you're developing such a frontend.

🏅 special thanks to

@anchwr and @drannarosen and @chummels for early reports of regressions on the dev branch (#3809, #4078, #4090)

@Xarthisius who's made a lot of crucial maintenance contributions, which are otherwise invisible in this summary, primarily focused on user-facing change

footnotes
Source code(tar.gz)
Source code(zip)
yt-4.0.5(Aug 11, 2022)
Summary

This is the fifth bugfix release in the 4.0.x series. This release fixes a critical bug with SPH particle selection in bounding boxes.

This release contains no new deprecation or removal and should be fully backward compatible, we recommend all users to upgrade.

How to upgrade

To upgrade from PyPI, run

python -m pip install --upgrade yt

or, with conda

conda update --channel conda-forge yt

Changes

BUG: Broadcast missing attribute #3973 by @brittonsmith

BUG: Fix broken annotate_particles example #3982 by @chrishavlin (issue #3968 by @neutrinoceros)

BUG: volume rendering: sigma_clip normalization #3984 by @chrishavlin (issue #3979 by @zingale)

BUG: fix a ResourceWarning (file is never closed) #3997 by @neutrinoceros

BUG: fix for composite volume rendering with octree sources #4011 by @chrishavlin

BUG: fix a typo in warning message #4017 by @neutrinoceros

BUG: setting GeoAxes extent #4040 by @chrishavlin (issue #4039)

BUG: Fix flux calculation on surface when there is no fluxing_field #4042 by @jzuhone

BUG: Fix intersections between SPH particles and regions when the boundary is periodic #4050 by @jzuhone (issue https://github.com/yt-project/yt/issues/3916 by @benopp99)

Requirements

Python 3.6 to 3.10 is supported. The next feature release, yt 4.1, will require Python 3.7 or newer.
Source code(tar.gz)
Source code(zip)
yt-4.0.4(May 27, 2022)
Summary

This is the fourth bugfix release in the 4.0.x series. This release fixes a critical incompatibility with the upcoming release of Matplotlib (3.6). This should have no immediate effect but allow yt to be installed alongside Matplotlib 3.6. It is our first release to be distributed for Mac ARM (M1) on PyPI.

This release contains no new deprecation or removal and should be fully backward compatible, we recommend all users to upgrade.

How to upgrade

To upgrade from PyPI, run

python -m pip install --upgrade yt

or, with conda

conda update --channel conda-forge yt

Changes

BUG: fix compatibility with GADGET2 files using '.gad.' template #3899 by @neutrinoceros

BUG: fix an UnboundLocalError #3898 by @neutrinoceros

BUG: fix an upcoming incompatibility with matplotlib 3.6 #3912 by @neutrinoceros

BLD: fix a build error on Windows #3925 by @neutrinoceros

Requirements

Python 3.6 to 3.10 is supported. The next feature release, yt 4.1, will require Python 3.7 or newer.
Source code(tar.gz)
Source code(zip)
yt-4.0.3(Apr 17, 2022)
Summary

This is the third bugfix release in the 4.0.x series. This release fixes a critical bug in particle indexing, as well as many small 2D visualisation bugs. It also contains a couple frontend specific bugfixes, most notably for RAMSES. It is our first release to be distributed for Mac ARM (M1) on conda-forge.

This release contains no new deprecation or removal and should be fully backward compatible, we recommend all users to upgrade.

How to upgrade

To upgrade from PyPI, run

python -m pip install --upgrade yt

or, with conda

conda update --channel conda-forge yt

Changes are categorised below.

Particle Indexing

Fix a bug in bitmap index for particle datasets #3788 by @langmm . This was the result of some checks of the number of refined zones in the bitmap index being set to higher than it should be.

This bug was responsible for some particle data files not being selected correctly, which manifested e.g. as "holes" in SlicePlots, and were first identified with AREPO data (see issue #3672 by @jzuhone)

2D Plotting

Symlog colorbars (used for plotting data with negative values)

We made some adjustments in how yt internally chooses to use symlog over log norms for 2D plots.

BUG: add missing MPL version check to a temporary hack #3754 by @neutrinoceros (issue #3170 by @chrishavlin)

BUG: fix heuristic to switch from log scale to symlog scale #3793 by @neutrinoceros (issue #3791)

BUG: fix a bug where automatically computed symlog linear threshold would sometimes cause a crash in matplotlib #3859 by @neutrinoceros (issue #3858, original report by @benwibking)

BUG: fix a bug where setting zmin/zmax on a 2D plot with symlog norm would cause an undesired change in linthresh #3888 by @neutrinoceros (issue #3887 by @vdbma)

Plotting non-cartesian data

BUG: fix pixelize_cylinder #3782 by @neutrinoceros (issue #3781)

BUG: fix a bug in pixelize_cylinder where buffer shape was read backwards #3795 by @neutrinoceros (issue #3794)

BUG: don't colorize pixels that are not completely within the data domain in pixelize_cylinder routine #3818 by @neutrinoceros

BUG: fix plot bounds for slices in cylindrical coordinates #3820 by @neutrinoceros (issue #3819)

BUG: fix a bug in aitoff-hammer pixelizer where some pixels were skipped on the central meridian and the equator #3824 by @neutrinoceros (issue #3823)

BUG: fix a bug where PlotWindow origin parameter wasn't handled correctly in non-cartesian geometries #3863 by @neutrinoceros (issue #3862 , original report by @vdbma)

Misc

Phase plots are now properly masked #2504 by @matthewturk

Format error string #3780 by @Yash-10

BUG: fix a bug in masking invalid values in line integral convolution callback #3739 by @neutrinoceros (issue #3738)

BUG: fix an issue where missing files would be indexed without verification #3816 by @neutrinoceros (issue #2819 by @joostvan)

BUG: fix a typesetting bug with fractionnal units in colorbar labels for volume rendering #3755 by @neutrinoceros

BUG: fix buffer masking in cartesian geometry #3856 by @neutrinoceros (issue #3854)

BUG: fix a regression where calling annotate_timestamp on a plot from a dataset with code units would crash #3870 by @neutrinoceros (issue #3869, original report by @vdbma)

Frontend specific patches

Flash

BUG: Track blockless particles #3683 by @matthewturk

OWLS

Fix Electron number density field #3874 by @jzuhone (issue #3838 by @cphyc)

RAMSES

BUG: also validate unresolved symlinks (RAMSES) #3786 by @mtrebitsch (issue #3785)

BUG: fix error messages in RAMSESDataset validation #3801 by @neutrinoceros (issue #3800)

BUG: bring back support to group folder structure in RAMSES #3811 by @cphyc (issue #3785 by @mtrebitsch)

Add missing ionized species #3841 by @cphyc

Fix definition of particle age/times #3864 by @cphyc (issue #3861 by @lconaboy)

BUG: Use cm_per_mpc instead of 3.08e24 #3868 by @Iconaboy (issue #3867)

Requirements

Python 3.6 to 3.10 is supported. The next feature release, yt 4.1, will require Python 3.7 or newer.

Some small adjustments to yt's dependencies were done for this release, and listed here to the intention of downstream packagers

The unmaintained toml package isn't required anymore, yt now require tomli and tomli_w instead (issue #3830 by @henryiii)

pyyaml and IPython were previously declared as hard requirements when they are in fact not strictly necessary. pyaml is now used in tests instead of pyyaml, and IPython is now considered an optional dependency.

jinja2 is now pinned in docs builds (#3860 by @Xarthisius)

Source code(tar.gz)
Source code(zip)
yt-4.0.2(Feb 4, 2022)
Summary

This is the second bugfix release in the 4.0.x series. Most notably, this release fixes a number of compatibility issues with recent versions of Matplotlib (3.4 and 3.5). It is also the first release to officially support Python 3.10.

This release contains no new deprecation or removal and should be fully backward compatible, we recommend all users to upgrade.

How to upgrade

To upgrade via PyPI, run

python -m pip install --upgrade yt

or via conda-forge

conda update --channel conda-forge yt

Most impactful changes are categorised below. See the 4.0.2 milestone for a an comprehensive view of the release's content.

Matplotlib compatibility

typesetting issues

BUG: Set the display name for new fields #3282 by @chummels (issue #3269)

BUG: fix default math font in yt plots against matplotlib >= 3.4 #3520 by @neutrinoceros (issue #3514)

BUG: fix LaTeX typesetting for fractional ylabels in LinePlot #3637 by @neutrinoceros (issue #3636)

colorbar layout in plots using (sym)log norms

BUG: fix two bugs with symlog colorbar ticks for matplotlib 3.5 #3556 by @neutrinoceros (issues #3554 and #3560)

BUG: fix for matplotlib 3.5 + symlog + linthresh value outside data range #3565 by @neutrinoceros (issue #3564)

MNT: matplotlib 3.5 compat #3670 by @neutrinoceros (issue #3558)

2D Plotting

BUG: Checking if extrema are all None in PhasePlot #3432 by @chummels (issue #3431)

BUG: fix a bug where 2D plots in spherical geometries would always use code-units in plot axes #3618 by @neutrinoceros (issue #2913 by @n-claes)

BUG: fixes for Hammer-Aitoff projection (r-normal) 2D viz in spherical geometries #3628 by @neutrinoceros (issue #3610)

BUG: fix figure layout in case the long axis is vertical #3633 by @neutrinoceros

BUG: fix a bug where PlotContainer.save would wrongfully ignore specified field extension #3687 by @neutrinoceros (issue #3632)

Documentation

DOC: add an exhaustive list of deprecations to the yt4differences page #3665 by @neutrinoceros (issue #3451)

DOC: fix an error in AMRVAC specific documentation #3716 by @neutrinoceros

Frontend specific patches

Gadget

Make sure we add the Parameters group to hvals if it's present #3455 by @jzuhone

BUG: Take into account endianness when reading Gadget binary files #3688 by @cphyc (issue #3676 by @olebole)

Athena

These changes should actually not affect Athena users, they are only meant to open the design space for new frontends supporting vtk files (issue #3001)

BUG: implement AthenaDataset._is_valid #3424 by @neutrinoceros

ENH: loosen header matching for Athena vtk files to allow compatibility with single variable 'vtk outputs' #3490 by @neutrinoceros

GDF

dimensionality doesn't imply which dimension is reduced #3630 by @Xarthisius (issue #3629)

Misc

BUG: add support for setting up derived fields from lambdas #3440 by @neutrinoceros (issue #3434 by @forrestglines)

BUG: fix combining FITS images #3456 by @jzuhone

BUG: fix two bugs in yt.visualization.eps_writer #3496 by @neutrinoceros

Python 3.10 support

yt 4.0.2 is the first release for which we provide wheels for Python 3.10 We note that part of our test suite has yet to be ported from nosetest to pytest. Because nosetest isn't compatible with Python 3.10 (and has been unmaintained for years), this means that we currently run only a (large) subset of our tests with Python 3.10 In the unlikely event of a regression, please fill in a bug report

Requirements

For posterity, it may be worth noting that some of yt's minimal requirements were adjusted for this release:

IPython 2.0 or newer is now required (previously 1.0)

Matplotlib 2.1 or newer is now required (previously 2.0.2)

cmyt (ColorMaps from yt) is a new small dependency that was extracted from yt's main code base, this should have no impact on yt users.

Python 3.6 is still supported for this release. The next feature release, yt 4.1, will require Python 3.7 or newer.
Source code(tar.gz)
Source code(zip)