The versatile ocean simulator, in pure Python, powered by JAX.

TeamOcean

Last update: Dec 20, 2022

Related tags

Deep Learning python gpu climate parallel distributed geophysics oceanography multi-core jax

Overview

Veros is the versatile ocean simulator -- it aims to be a powerful tool that makes high-performance ocean modeling approachable and fun. Because Veros is a pure Python module, the days of struggling with complicated model setup workflows, ancient programming environments, and obscure legacy code are finally over.

Veros supports a NumPy backend for small-scale problems and a high-performance JAX backend with CPU and GPU support. It is fully parallelized via MPI and supports distributed execution.

The underlying numerics are based on pyOM2, a Fortran ocean model developed at Institut für Meereskunde, Hamburg University.

Veros is currently being developed at Niels Bohr Institute, Copenhagen University.

How about a demonstration?

(0.25×0.25° high-resolution model spin-up, click for better quality)

Features

Veros provides

a fully staggered 3-D grid geometry (C-grid)
support for both idealized and realistic configurations in Cartesian or pseudo-spherical coordinates
several friction and advection schemes to choose from
isoneutral mixing, eddy-kinetic energy, turbulent kinetic energy, and internal wave energy parameterizations
several pre-implemented diagnostics such as energy fluxes, variable time averages, and a vertical overturning stream function (written to netCDF output)
pre-configured idealized and realistic set-ups that are ready to run and easy to adapt
accessibility and extensibility - thanks to the power of Python!

Veros for the impatient

A minimal example to install and run Veros:

$ pip install veros
$ veros copy-setup acc --to /tmp/acc
$ veros run /tmp/acc/acc.py

For more detailed installation instructions, have a look at our documentation.

Basic usage

To run Veros, you need to set up a model - i.e., specify which settings and model domain you want to use. This is done by subclassing the VerosSetup base class in a setup script that is written in Python. You should use the veros copy-setup command to copy one into your current folder. A good place to start is the ACC model:

$ veros copy-setup acc

After setting up your model, all you need to do is call the setup and run methods on your setup class. The pre-implemented setups can all be executed via veros run:

$ veros run acc.py

For more information on using Veros, have a look at our documentation.

Contributing

Contributions to Veros are always welcome, no matter if you spotted an inaccuracy in the documentation, wrote a new setup, fixed a bug, or even extended Veros' core mechanics. There are 2 ways to contribute:

If you want to report a bug or request a missing feature, please open an issue. If you are reporting a bug, make sure to include all relevant information for reproducing it (ideally through a minimal code sample).
If you want to fix the issue yourself, or wrote an extension for Veros - great! You are welcome to submit your code for review by committing it to a repository and opening a pull request. However, before you do so, please check the contribution guide for some tips on testing and benchmarking, and to make sure that your modifications adhere with our style policies. Most importantly, please ensure that you follow the PEP8 guidelines, use meaningful variable names, and document your code using Google-style docstrings.

How to cite

If you use Veros in scientific work, please consider citing the following publication:

@article{hafner_veros_2018,
	title = {Veros v0.1 – a fast and versatile ocean simulator in pure {Python}},
	volume = {11},
	issn = {1991-959X},
	url = {https://gmd.copernicus.org/articles/11/3299/2018/},
	doi = {10.5194/gmd-11-3299-2018},
	number = {8},
	journal = {Geoscientific Model Development},
	author = {Häfner, Dion and Jacobsen, René Løwe and Eden, Carsten and Kristensen, Mads R. B. and Jochum, Markus and Nuterman, Roman and Vinter, Brian},
	month = aug,
	year = {2018},
	pages = {3299--3312},
}

Comments

Job resubmission with job scheduler doesn't work

I was not able to find out the reason behind resubmission issue with job scheduler, such as: veros-resubmit -i acc.lowres -n 50 -l 62208000 -c "python acc.py -b bohrium -v debug" --callback "/usr/bin/sbatch /groups/ocean/nutrik/veros_cases/paper/acc/veros_batch.sh" Although jobs with run length of up to 29 days are resubmitted fine, those with longer run length are not resubmitted and no errors or messages are reported.

In fact, jobs are successfully resubmitted without scheduler (--callback "./veros_batch.sh") for any run length.
bug

opened by nutrik 10
Changes in documentation

I put two tables with Veros runtime for different cases (not sure how they will be displayed in the readthedocs), also did updates of publication list and installation guide. Mixed current commit with previous commit with potential density (forgot to pull origin master before editing doc files). Fucked up and don't know how to change it :(

opened by nutrik 10
FIX: kappaM_0 and kappaH_0 are settings

I tured tke off, and got that vs.kappaH/M_0 do not exist - changing to settings seems to work.

This is only tested locally, but would benefit from being added to your test suite.

opened by jklymak 7
only acc and eady example setups work for me

Hi there

Firstly, thanks for sharing this code!

I can run the acc and eady models out of the box which is great but when I come to run any of global_1deg, global_4deg, north_atlantic or wave_propagation models, I get the following type of error. Is this something that you have seen before?

(my_root) clim01|Tue Jan 30|00:08:52|veros-run> cd global_1deg/ (my_root) clim01|Tue Jan 30|00:08:56|global_1deg> python global_one_degree.py /scale_akl_persistent/filesets/home/williamsjh/veros/veros/core/numerics.py:10: UserWarning: Special OpenCL implementations could not be imported warnings.warn("Special OpenCL implementations could not be imported") Traceback (most recent call last): File "global_one_degree.py", line 10, in DATA_FILES = veros.tools.get_assets("global_1deg", os.path.join(BASE_PATH, "assets.yml")) File "/scale_akl_persistent/filesets/home/williamsjh/veros/veros/tools/assets.py", line 43, in get_assets return {key: get_asset(val["url"], val.get("md5", None)) for key, val in assets.items()} File "/scale_akl_persistent/filesets/home/williamsjh/veros/veros/tools/assets.py", line 43, in return {key: get_asset(val["url"], val.get("md5", None)) for key, val in assets.items()} File "/scale_akl_persistent/filesets/home/williamsjh/veros/veros/tools/assets.py", line 36, in get_asset _download_file(url, target_path) File "/scale_akl_persistent/filesets/home/williamsjh/veros/veros/tools/assets.py", line 48, in _download_file with requests.get(url, stream=True, timeout=timeout) as response: AttributeError: exit (my_root) clim01|Tue Jan 30|00:09:02|global_1deg>

I have run wget on the source file for this example and it is there.

Thanks for any ideas, I'm a bit stuck!

Jonny

opened by jonnyhtw 7
Bump xarray from 0.21.0 to 0.21.1
Bumps xarray from 0.21.0 to 0.21.1.

Release notes

Sourced from xarray's releases.

v0.21.1

This is a bugfix release to specify packaging as a dependency.

Commits

fe491b1 Add release notes for 0.21.1

5973ef3 Fix missing dependecy definition of 'packaging' (#6207)

e939bfc GHA concurrency followup (#6223)

e8d4239 update release steps (#6214)

0a1e7e4 [pre-commit.ci] pre-commit autoupdate (#6221)

b09de81 GHA concurrency (#6210)

aab856b Xfail failing test (#6211)

5470d93 MNT: prepare h5netcdf backend for (coming) change in dimension handling (#6200)

9b8cba4 Whatsnew template for 0.21.1 (#6205)

See full diff in compare view

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

Dependabot commands and options

You can trigger Dependabot actions by commenting on this PR:

@dependabot rebase will rebase this PR

@dependabot recreate will recreate this PR, overwriting any edits that have been made to it

@dependabot merge will merge this PR after your CI passes on it

@dependabot squash and merge will squash and merge this PR after your CI passes on it

@dependabot cancel merge will cancel a previously requested merge and block automerging

@dependabot reopen will reopen this PR if it is closed

@dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually

@dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)

@dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)

@dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)

dependencies
opened by dependabot[bot] 6
Compute initial streamfunction from initial velocity

Not knowing the numerics, and possibly not reading the docs carefully enough, it is unclear to me how to initialize the velocities in the model.

For a channel run, re-entrant in x, with intial velocity 0.1 m/s everywhere, no forcing, I tried do in set_initial_conditions: s.u = update(vs.u, at[...], 0.1 * vs.maskU[..., None]).

The velocity signal only lasts for one time step, and then it is gone. It does create a small pressure perturbations that drive internal waves, but the mean flow of 0.1 m/s is immediately gone. Conversely, the initial conditions have psi=0 everywhere, and then immediately on the next time step there is a stream function, but if the units are really m^3/s it is far too small.

Was I to initialize psi at the beginning instead of u, or in addition to u?
enhancement

opened by jklymak 5
[doc] analysis tutorial

Hi Dion, I don't know why but Actions on my account don't currently work (trying to get in touch with GitHub support). Could you please review my commit for docs? I will probably do development in team-ocean next time (not sure it is going to help though 🤨) Sorry!

There are several hidden changes in attributes of coordinates. They are done to eliminate some problems related to textwrap module, which cannot handle the default attribute names from netCDF files 🤷‍♂️

opened by nutrik 5
Grid refinement

Hi Dion, Could you please take a look at wave propagation case with refined around equator latitude grid? The case has an issue (when grid refinement and coastal shelfs are included) with convergence of Poisson solver, but workers fine without coastal shelfs. I have tried to reduce time steps, but it didn't work :( Best, Roman

opened by nutrik 5
Bump numpy from 1.21.5 to 1.22.0
Bumps numpy from 1.21.5 to 1.22.0.

Release notes

Sourced from numpy's releases.

v1.22.0

NumPy 1.22.0 Release Notes

NumPy 1.22.0 is a big release featuring the work of 153 contributors spread over 609 pull requests. There have been many improvements, highlights are:

Annotations of the main namespace are essentially complete. Upstream is a moving target, so there will likely be further improvements, but the major work is done. This is probably the most user visible enhancement in this release.

A preliminary version of the proposed Array-API is provided. This is a step in creating a standard collection of functions that can be used across application such as CuPy and JAX.

NumPy now has a DLPack backend. DLPack provides a common interchange format for array (tensor) data.

New methods for quantile, percentile, and related functions. The new methods provide a complete set of the methods commonly found in the literature.

A new configurable allocator for use by downstream projects.

These are in addition to the ongoing work to provide SIMD support for commonly used functions, improvements to F2PY, and better documentation.

The Python versions supported in this release are 3.8-3.10, Python 3.7 has been dropped. Note that 32 bit wheels are only provided for Python 3.8 and 3.9 on Windows, all other wheels are 64 bits on account of Ubuntu, Fedora, and other Linux distributions dropping 32 bit support. All 64 bit wheels are also linked with 64 bit integer OpenBLAS, which should fix the occasional problems encountered by folks using truly huge arrays.

Expired deprecations

Deprecated numeric style dtype strings have been removed

Using the strings "Bytes0", "Datetime64", "Str0", "Uint32", and "Uint64" as a dtype will now raise a TypeError.

(gh-19539)

Expired deprecations for loads, ndfromtxt, and mafromtxt in npyio

numpy.loads was deprecated in v1.15, with the recommendation that users use pickle.loads instead. ndfromtxt and mafromtxt were both deprecated in v1.17 - users should use numpy.genfromtxt instead with the appropriate value for the usemask parameter.

(gh-19615)

... (truncated)

Commits

4adc87d Merge pull request #20685 from charris/prepare-for-1.22.0-release

fd66547 REL: Prepare for the NumPy 1.22.0 release.

125304b wip

c283859 Merge pull request #20682 from charris/backport-20416

5399c03 Merge pull request #20681 from charris/backport-20954

f9c45f8 Merge pull request #20680 from charris/backport-20663

794b36f Update armccompiler.py

d93b14e Update test_public_api.py

7662c07 Update init.py

311ab52 Update armccompiler.py

Additional commits viewable in compare view

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

Dependabot commands and options

You can trigger Dependabot actions by commenting on this PR:

@dependabot rebase will rebase this PR

@dependabot recreate will recreate this PR, overwriting any edits that have been made to it

@dependabot merge will merge this PR after your CI passes on it

@dependabot squash and merge will squash and merge this PR after your CI passes on it

@dependabot cancel merge will cancel a previously requested merge and block automerging

@dependabot reopen will reopen this PR if it is closed

@dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually

@dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)

@dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)

@dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)

dependencies
opened by dependabot[bot] 4

Lock files while downloading assets

I am trying to run global_flexible setup from Veros' setup gallery.

Run script:

#!/bin/bash -l
# 
#SBATCH -p aegir
#SBATCH -A ocean
#SBATCH --job-name=flexdeg
#SBATCH --time=23:59:59
#SBATCH --constraint=v1
#SBATCH --nodes=2
#SBATCH --ntasks=32
#SBATCH --cpus-per-task=1
#SBATCH --exclusive
##SBATCH --mail-type=ALL
##SBATCH --mail-user=<REDACTED>
##SBATCH --output=slurm.out

export OMP_NUM_THREADS=1
module load veros/23052019

srun -v --mpi=pmi2 --kill-on-bad-exit python -m mpi4py global_flexible.py -n 8 4 -b bohrium >& veros_run.log

and getting md5 sum mismatch of forcing & bathymetry files

Veros log file:

srun: defined options for program `srun'
srun: --------------- ---------------------
srun: user           : `nutrik'
srun: uid            : 16001
srun: gid            : 16000
srun: cwd            : /lustre/hpc/ocean/nutrik/veros_cases/global_flexible
srun: ntasks         : 32 (set)
srun: cpus_per_task  : 1
srun: nodes          : 2 (set)
srun: jobid          : 13466356 (default)
srun: partition      : default
srun: profile        : `NotSet'
srun: job name       : `2deg'
srun: reservation    : `(null)'
srun: burst_buffer   : `(null)'
srun: wckey          : `(null)'
srun: cpu_freq_min   : 4294967294
srun: cpu_freq_max   : 4294967294
srun: cpu_freq_gov   : 4294967294
srun: switches       : -1
srun: wait-for-switches : -1
srun: distribution   : unknown
srun: cpu_bind       : default (0)
srun: mem_bind       : default (0)
srun: verbose        : 1
srun: slurmd_debug   : 0
srun: immediate      : false
srun: label output   : false
srun: unbuffered IO  : false
srun: overcommit     : false
srun: threads        : 60
srun: checkpoint_dir : /var/slurm/checkpoint
srun: wait           : 0
srun: nice           : -2
srun: account        : (null)
srun: comment        : (null)
srun: dependency     : (null)
srun: exclusive      : false
srun: bcast          : false
srun: qos            : (null)
srun: constraints    : mincpus-per-node=1 mem-per-cpu=1024M
srun: geometry       : (null)
srun: reboot         : yes
srun: rotate         : no
srun: preserve_env   : false
srun: network        : (null)
srun: propagate      : NONE
srun: prolog         : (null)
srun: epilog         : (null)
srun: mail_type      : NONE
srun: mail_user      : (null)
srun: task_prolog    : (null)
srun: task_epilog    : (null)
srun: multi_prog     : no
srun: sockets-per-node  : -2
srun: cores-per-socket  : -2
srun: threads-per-core  : -2
srun: ntasks-per-node   : -2
srun: ntasks-per-socket : -2
srun: ntasks-per-core   : -2
srun: plane_size        : 4294967294
srun: core-spec         : NA
srun: power             :
srun: remote command    : `python -m mpi4py global_flexible.py -n 8 4 -b bohrium'
srun: launching 13466356.0 on host node172, 16 tasks: [0-15]
srun: launching 13466356.0 on host node173, 16 tasks: [16-31]
srun: route default plugin loaded
srun: Node node172, 16 tasks started
srun: Node node173, 16 tasks started
WARNING: Error in initializing MVAPICH2 ptmalloc library.Continuing without InfiniBand registration cache support.
2019-05-28 16:26:47.708 | INFO     | veros.tools.assets:get_asset:73 - Downloading asset ETOPO5_Ice_g_gmt4.nc ...
2019-05-28 16:26:47.708 | INFO     | veros.tools.assets:get_asset:73 - Downloading asset ETOPO5_Ice_g_gmt4.nc ...
2019-05-28 16:26:47.713 | INFO     | veros.tools.assets:get_asset:73 - Downloading asset ETOPO5_Ice_g_gmt4.nc ...
2019-05-28 16:26:47.708 | INFO     | veros.tools.assets:get_asset:73 - Downloading asset ETOPO5_Ice_g_gmt4.nc ...
2019-05-28 16:26:47.713 | INFO     | veros.tools.assets:get_asset:73 - Downloading asset ETOPO5_Ice_g_gmt4.nc ...
2019-05-28 16:26:47.708 | INFO     | veros.tools.assets:get_asset:73 - Downloading asset ETOPO5_Ice_g_gmt4.nc ...
2019-05-28 16:26:47.713 | INFO     | veros.tools.assets:get_asset:73 - Downloading asset ETOPO5_Ice_g_gmt4.nc ...
2019-05-28 16:26:47.708 | INFO     | veros.tools.assets:get_asset:73 - Downloading asset ETOPO5_Ice_g_gmt4.nc ...
2019-05-28 16:26:47.713 | INFO     | veros.tools.assets:get_asset:73 - Downloading asset ETOPO5_Ice_g_gmt4.nc ...
2019-05-28 16:26:47.708 | INFO     | veros.tools.assets:get_asset:73 - Downloading asset ETOPO5_Ice_g_gmt4.nc ...
2019-05-28 16:26:47.713 | INFO     | veros.tools.assets:get_asset:73 - Downloading asset ETOPO5_Ice_g_gmt4.nc ...
2019-05-28 16:26:47.708 | INFO     | veros.tools.assets:get_asset:73 - Downloading asset ETOPO5_Ice_g_gmt4.nc ...
2019-05-28 16:26:47.713 | INFO     | veros.tools.assets:get_asset:73 - Downloading asset ETOPO5_Ice_g_gmt4.nc ...
2019-05-28 16:26:47.708 | INFO     | veros.tools.assets:get_asset:73 - Downloading asset ETOPO5_Ice_g_gmt4.nc ...
2019-05-28 16:26:47.713 | INFO     | veros.tools.assets:get_asset:73 - Downloading asset ETOPO5_Ice_g_gmt4.nc ...
2019-05-28 16:26:47.708 | INFO     | veros.tools.assets:get_asset:73 - Downloading asset ETOPO5_Ice_g_gmt4.nc ...
2019-05-28 16:26:47.713 | INFO     | veros.tools.assets:get_asset:73 - Downloading asset ETOPO5_Ice_g_gmt4.nc ...
2019-05-28 16:26:47.708 | INFO     | veros.tools.assets:get_asset:73 - Downloading asset ETOPO5_Ice_g_gmt4.nc ...
2019-05-28 16:26:47.713 | INFO     | veros.tools.assets:get_asset:73 - Downloading asset ETOPO5_Ice_g_gmt4.nc ...
2019-05-28 16:26:47.708 | INFO     | veros.tools.assets:get_asset:73 - Downloading asset ETOPO5_Ice_g_gmt4.nc ...
2019-05-28 16:26:47.713 | INFO     | veros.tools.assets:get_asset:73 - Downloading asset ETOPO5_Ice_g_gmt4.nc ...
2019-05-28 16:26:47.708 | INFO     | veros.tools.assets:get_asset:73 - Downloading asset ETOPO5_Ice_g_gmt4.nc ...
2019-05-28 16:26:47.713 | INFO     | veros.tools.assets:get_asset:73 - Downloading asset ETOPO5_Ice_g_gmt4.nc ...
2019-05-28 16:26:47.708 | INFO     | veros.tools.assets:get_asset:73 - Downloading asset ETOPO5_Ice_g_gmt4.nc ...
2019-05-28 16:26:47.713 | INFO     | veros.tools.assets:get_asset:73 - Downloading asset ETOPO5_Ice_g_gmt4.nc ...
2019-05-28 16:26:47.708 | INFO     | veros.tools.assets:get_asset:73 - Downloading asset ETOPO5_Ice_g_gmt4.nc ...
2019-05-28 16:26:47.713 | INFO     | veros.tools.assets:get_asset:73 - Downloading asset ETOPO5_Ice_g_gmt4.nc ...
2019-05-28 16:26:47.708 | INFO     | veros.tools.assets:get_asset:73 - Downloading asset ETOPO5_Ice_g_gmt4.nc ...
2019-05-28 16:26:47.713 | INFO     | veros.tools.assets:get_asset:73 - Downloading asset ETOPO5_Ice_g_gmt4.nc ...
2019-05-28 16:26:47.708 | INFO     | veros.tools.assets:get_asset:73 - Downloading asset ETOPO5_Ice_g_gmt4.nc ...
2019-05-28 16:26:47.713 | INFO     | veros.tools.assets:get_asset:73 - Downloading asset ETOPO5_Ice_g_gmt4.nc ...
2019-05-28 16:26:47.713 | INFO     | veros.tools.assets:get_asset:73 - Downloading asset ETOPO5_Ice_g_gmt4.nc ...
2019-05-28 16:26:48.468 | INFO     | veros.tools.assets:get_asset:73 - Downloading asset forcing_1deg_global_interpolated.nc ...
2019-05-28 16:26:48.496 | INFO     | veros.tools.assets:get_asset:73 - Downloading asset forcing_1deg_global_interpolated.nc ...
2019-05-28 16:26:48.583 | INFO     | veros.tools.assets:get_asset:73 - Downloading asset forcing_1deg_global_interpolated.nc ...
2019-05-28 16:26:48.595 | INFO     | veros.tools.assets:get_asset:73 - Downloading asset forcing_1deg_global_interpolated.nc ...
2019-05-28 16:26:48.606 | INFO     | veros.tools.assets:get_asset:73 - Downloading asset forcing_1deg_global_interpolated.nc ...
2019-05-28 16:26:48.611 | INFO     | veros.tools.assets:get_asset:73 - Downloading asset forcing_1deg_global_interpolated.nc ...
Traceback (most recent call last):
  File "/groups/ocean/software/python/gcc/3.6.7/lib/python3.6/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "/groups/ocean/software/python/gcc/3.6.7/lib/python3.6/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/groups/ocean/software/mpi4py_mvapich231/gcc/3.0.1/lib/python3.6/site-packages/mpi4py/__main__.py", line 7, in <module>
    main()
  File "/groups/ocean/software/mpi4py_mvapich231/gcc/3.0.1/lib/python3.6/site-packages/mpi4py/run.py", line 196, in main
    run_command_line(args)
  File "/groups/ocean/software/mpi4py_mvapich231/gcc/3.0.1/lib/python3.6/site-packages/mpi4py/run.py", line 47, in run_command_line
    run_path(sys.argv[0], run_name='__main__')
  File "/groups/ocean/software/python/gcc/3.6.7/lib/python3.6/runpy.py", line 263, in run_path
    pkg_name=pkg_name, script_name=fname)
  File "/groups/ocean/software/python/gcc/3.6.7/lib/python3.6/runpy.py", line 96, in _run_module_code
    mod_name, mod_spec, pkg_name, script_name)
  File "/groups/ocean/software/python/gcc/3.6.7/lib/python3.6/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "global_flexible.py", line 16, in <module>
    DATA_FILES = veros.tools.get_assets('global_flexible', os.path.join(BASE_PATH, 'assets.yml'))
  File "/lustre/hpc/ocean/software/veros/repo23052019/veros/tools/assets.py", line 81, in get_assets
    return {key: get_asset(val['url'], val.get('md5', None)) for key, val in assets.items()}
  File "/lustre/hpc/ocean/software/veros/repo23052019/veros/tools/assets.py", line 81, in <dictcomp>
    return {key: get_asset(val['url'], val.get('md5', None)) for key, val in assets.items()}
  File "/lustre/hpc/ocean/software/veros/repo23052019/veros/tools/assets.py", line 77, in get_asset
    raise AssetError('Mismatching MD5 checksum on asset %s' % target_filename)
veros.tools.assets.AssetError: Mismatching MD5 checksum on asset forcing_1deg_global_interpolated.nc
srun: Complete job step 13466356.0 received
slurmstepd: error: *** STEP 13466356.0 ON node172 CANCELLED AT 2019-05-28T16:26:49 ***
srun: Job step aborted: Waiting up to 32 seconds for job step to finish.
srun: Complete job step 13466356.0 received
srun: Received task exit notification for 16 tasks (status=0x0009).
srun: error: node172: tasks 0-15: Killed
srun: Terminating job step 13466356.0
srun: Complete job step 13466356.0 received
srun: Received task exit notification for 16 tasks (status=0x0009).
srun: error: node173: tasks 16-31: Killed

opened by nutrik 4

RuntimeError: Two parallel Veros+Bohrium runs on two GPUs on the same node

Hi Dion,

It seems impossible to make 2 parallel/standalone Veros runs on 2 GPUs on the same node. The error message is below. Is it something to do with OpenCL keys? Do you think that should work if I would have two separate versions of veros and/or bohrium and use them for individual runs?

Time step took 4.14e+00s
Current iteration: 17
build program: binary cache hit (key: 219cf206d832af2614aabaa6095b2a6d)
build program: start
build program: completed, success
pyopencl-invoker-cache-v1: in mem cache hit [key=1fc007e39493fed6f5e0c45672144578c269a48fd12e4bc28dc2ce3b7c1dc753]
build program: binary cache hit (key: 219cf206d832af2614aabaa6095b2a6d)
build program: start
build program: completed, success
pyopencl-invoker-cache-v1: in mem cache hit [key=1fc007e39493fed6f5e0c45672144578c269a48fd12e4bc28dc2ce3b7c1dc753]
Error code: -4
terminate called after throwing an instance of 'cl::Error'
  what():  clEnqueueNDRangeKernel
Traceback (most recent call last):
  File "/groups/ocean/software/veros/inst06032018/bin/veros-resubmit", line 11, in <module>
    load_entry_point('veros', 'console_scripts', 'veros-resubmit')()
  File "/groups/ocean/software/python/gcc/2.7.14/lib/python2.7/site-packages/click/core.py", line 722, in __call__
    return self.main(*args, **kwargs)
  File "/groups/ocean/software/python/gcc/2.7.14/lib/python2.7/site-packages/click/core.py", line 697, in main
    rv = self.invoke(ctx)
  File "/groups/ocean/software/python/gcc/2.7.14/lib/python2.7/site-packages/click/core.py", line 895, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/groups/ocean/software/python/gcc/2.7.14/lib/python2.7/site-packages/click/core.py", line 535, in invoke
    return callback(*args, **kwargs)
  File "/lustre/hpc/ocean/software/veros/repo06032018/veros/cli/veros_resubmit.py", line 81, in cli
    resubmit(*args, **kwargs)
  File "/lustre/hpc/ocean/software/veros/repo06032018/veros/cli/veros_resubmit.py", line 61, in resubmit
    call_veros(veros_cmd, identifier, current_n, length_per_run)
  File "/lustre/hpc/ocean/software/veros/repo06032018/veros/cli/veros_resubmit.py", line 46, in call_veros
    raise RuntimeError("Run {} failed, exiting".format(n))
RuntimeError: Run 0 failed, exiting

opened by nutrik 4

Bump sphinx from 5.3.0 to 6.1.1
Bumps sphinx from 5.3.0 to 6.1.1.

Release notes

Sourced from sphinx's releases.

v6.1.1

Changelog: https://www.sphinx-doc.org/en/master/changes.html

v6.1.0

Changelog: https://www.sphinx-doc.org/en/master/changes.html

v6.0.1

Changelog: https://www.sphinx-doc.org/en/master/changes.html

v6.0.0

Changelog: https://www.sphinx-doc.org/en/master/changes.html

v6.0.0b2

Changelog: https://www.sphinx-doc.org/en/master/changes.html

v6.0.0b1

Changelog: https://www.sphinx-doc.org/en/master/changes.html

Changelog

Sourced from sphinx's changelog.

Release 6.1.1 (released Jan 05, 2023)

Bugs fixed

#11091: Fix util.nodes.apply_source_workaround for literal_block nodes with no source information in the node or the node's parents.

Release 6.1.0 (released Jan 05, 2023)

Dependencies

Adopted the Ruff_ code linter.

.. _Ruff: https://github.com/charliermarsh/ruff

Incompatible changes

#10979: gettext: Removed support for pluralisation in get_translation. This was unused and complicated other changes to sphinx.locale.

Deprecated

sphinx.util functions:

Renamed sphinx.util.typing.stringify() to sphinx.util.typing.stringify_annotation()

Moved sphinx.util.xmlname_checker() to sphinx.builders.epub3._XML_NAME_PATTERN

Moved to sphinx.util.display:

sphinx.util.status_iterator

sphinx.util.display_chunk

sphinx.util.SkipProgressMessage

sphinx.util.progress_message

Moved to sphinx.util.http_date:

sphinx.util.epoch_to_rfc1123

sphinx.util.rfc1123_to_epoch

Moved to sphinx.util.exceptions:

sphinx.util.save_traceback

... (truncated)

Commits

77aaa86 Bump to 6.1.1 final

476c115 Suppress ValueError in apply_source_workaround (#11092)

c80d656 Bump version

4e1004a Bump to 6.1.0 final

a2176d4 Fix deprecation warnings

2c104e9 Merge branch '6.0.x'

a27d262 Bump to 6.0.1 final

821569e Add note for Pygments

222d366 imgmath: Fix relative file path (#10965)

c499f66 Add SIM113 lint (#11057)

Additional commits viewable in compare view

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

Dependabot commands and options

You can trigger Dependabot actions by commenting on this PR:

@dependabot rebase will rebase this PR

@dependabot recreate will recreate this PR, overwriting any edits that have been made to it

@dependabot merge will merge this PR after your CI passes on it

@dependabot squash and merge will squash and merge this PR after your CI passes on it

@dependabot cancel merge will cancel a previously requested merge and block automerging

@dependabot reopen will reopen this PR if it is closed

@dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually

@dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)

@dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)

@dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)

dependencies
opened by dependabot[bot] 1
How to run veros with multi-GPU

Hi! I am trying to run veros with multi-gpu, it works when I run acc_benchmark.py. But when I try to run global_flexible.py with the instruction mpirun -np 2 veros run global_flexible/global_flexible.py -n 1 2 --force-overwrite -b jax --device gpu, it seems that only one GPU is working. Could you please tell me what should I do? Thanks in advance!

opened by HuangLianghong 1

How to run Veros via MPI on a cluster with slurm?

Hi, I am trying to run Veros on a cluster, but I cannot figure it out. Here is my batch script:

#!/bin/bash -l
#
#SBATCH -p work
#SBATCH --job-name=veros_hlh
#SBATCH --nodes=2
#SBATCH --ntasks=64
#SBATCH --cpus-per-task=1
#SBATCH --exclusive


# load module dependencies
# module load petsc4py mpi4py h5py ...

export OMP_NUM_THREADS=1

# adapt srun command to your available scheduler / MPI implementation
veros resubmit -i my_run -n 8 -l 7776000 \
    -c "srun --mpi=none -- veros run global_flexible/global_flexible.py -b numpy -n 4 4" \
    --callback "sbatch veros_batch.sh"

error informations：

 Fatal error in PMPI_Init_thread: Other MPI error, error stack:
MPIR_Init_thread(805).................: fail failed
MPID_Init(1859).......................: channel initialization failed
MPIDI_CH3_Init(147)...................: fail failed
dapl_rc_setup_all_connections_20(1394): generic failure with errno = 872598799
getConnInfoKVS(956)...................: PMI_KVS_Get failed
[unset]: readline failed
srun: error: cpn256: task 2: Exited with exit code 15
Fatal error in PMPI_Init_thread: Other MPI error, error stack:
MPIR_Init_thread(805).................: fail failed
MPID_Init(1859).......................: channel initialization failed
MPIDI_CH3_Init(147)...................: fail failed
dapl_rc_setup_all_connections_20(1394): generic failure with errno = 872598799
getConnInfoKVS(956)...................: PMI_KVS_Get failed
[unset]: readline failed
Fatal error in PMPI_Init_thread: Other MPI error, error stack:
MPIR_Init_thread(805).................: fail failed
MPID_Init(1859).......................: channel initialization failed
MPIDI_CH3_Init(147)...................: fail failed
dapl_rc_setup_all_connections_20(1394): generic failure with errno = 872598799
getConnInfoKVS(956)...................: PMI_KVS_Get failed
[unset]: readline failed
srun: First task exited 60s ago
srun: step:2344490.0 task 3: running
srun: step:2344490.0 tasks 0-2: exited abnormally
srun: Terminating job step 2344490.0
srun: Job step aborted: Waiting up to 62 seconds for job step to finish.
srun: got SIGCONT
slurmstepd: error: *** JOB 2344490 ON cpn34 CANCELLED AT 2022-12-17T18:31:56 ***
srun: error: cpn273: task 3: Killed

Do you think I didn't install mpi4py and h5py correctly or any orther advice? I used IMPI during my installation.

opened by HuangLianghong 1

Changing grid resolution and a closed ITF in global_4deg setup

How to change the grid resolution of global_flexible setup ?

Also while plotting some of the variables from the '4deg.snapshot.nc' output file from global_4deg setup I got a closed ITF ( Indonesian Throughflow) . How to open ITF channel so that I can include the throughflow in my output files.

Thanks!

opened by Sougata18 1
Document overturning variables

Hello everyone! I have been trying to run one of the basic setups available in Versos, i.e. , the gloabl_1deg setup on my PC. But each time I try to run, its showing a "killed" message in the terminal. My PC has 8 GB RAM ; Is it too low to run the setup on my PC?

Also I would like to know what the output variable names actually mean. There are some variables for example the "bolus_depth" and "bolus_iso" from the global_4deg setup which inspite of having the same coordinates and attributes (i.e., meridional transport), represent different things. I looked into the official documentation but it has description for only model variables. Thanks for any kind of help!
enhancement

opened by Sougata18 2
Non-hydrostatic solver

Hi, everyone, I just stumbled into this project after watching this JuliaCon session and let me start by saying that I really enjoy its vision and scope!

I've been reading through the docs and it seems like you don't have the capability of running nonhydrostatic LES, do I understand that correctly?

If so, do you envision them to be implemented soon? From the intro I get the feeling that the nonhydrostatic solver will be ported soon from pyOM2, but I see no mention of LES closures. I ask this because I think that if LES are possible with this package, it'll open up many more research avenues given the extensibility of the model and ability to run on multiple GPUs.

Thanks!
enhancement

opened by tomchor 5

Releases(v1.4.5)

v1.4.5(Aug 15, 2022)
What's Changed

Fix versioneer for PEP 660 editable installs by @dionhaefner in https://github.com/team-ocean/veros/pull/366

Fix veros.tools.setup.get_coastline_distance with recent SciPy versions

Full Changelog: https://github.com/team-ocean/veros/compare/v1.4.4...v1.4.5
Source code(tar.gz)
Source code(zip)
v1.4.4(Aug 11, 2022)
What's Changed

ensure that using the CLI does not import veros by @dionhaefner in https://github.com/team-ocean/veros/pull/323

[doc] erda install by @nutrik in https://github.com/team-ocean/veros/pull/344

:bug: missing net radiation forcing by @nutrik in https://github.com/team-ocean/veros/pull/348

Do not import plugins when finding setup files by @dionhaefner in https://github.com/team-ocean/veros/pull/356

Add version limit for jaxlib by @dionhaefner in https://github.com/team-ocean/veros/pull/362

Add dimensions parameter to plugin interfaces by @dionhaefner in https://github.com/team-ocean/veros/pull/364

Full Changelog: https://github.com/team-ocean/veros/compare/v1.4.3...v1.4.4
Source code(tar.gz)
Source code(zip)
v1.4.3(Mar 29, 2022)
What's Changed

[core] change to current JAX update syntax by @dionhaefner in https://github.com/team-ocean/veros/pull/243

[doc] analysis tutorial by @nutrik in https://github.com/team-ocean/veros/pull/249

Fix unhelpful error message for non-existing inputs to veros run by @dionhaefner in https://github.com/team-ocean/veros/pull/259

Add surface pressure solver by @dionhaefner in https://github.com/team-ocean/veros/pull/274

Automatically split barotropic mode from initial velocity by @dionhaefner in https://github.com/team-ocean/veros/pull/272

Re-structure tests by @dionhaefner in https://github.com/team-ocean/veros/pull/280

Fix CI error with Conda on Windows by @dionhaefner in https://github.com/team-ocean/veros/pull/286

Test Python 3.10 by @dionhaefner in https://github.com/team-ocean/veros/pull/287

Bump dependencies

Full Changelog: https://github.com/team-ocean/veros/compare/v1.4.2...v1.4.3
Source code(tar.gz)
Source code(zip)
v1.4.2(Sep 24, 2021)

Extension modules are now optional as intended.
Source code(tar.gz)
Source code(zip)
v1.4.1(Sep 22, 2021)

Bumped dependencies, fixes a path issue on some systems.
Source code(tar.gz)
Source code(zip)
v1.4.0(Aug 20, 2021)
The following setups are not part of the main Veros repository anymore:

acc_sector

fjord

wave_propagation

They are now found in the veros-extra-setups plugin.
Source code(tar.gz)
Source code(zip)
v1.3.4(Jul 8, 2021)
Benchmarks now use at most 120 vertical layers

Runtime settings are now correctly parsed from environment variables

Source code(tar.gz)
Source code(zip)
v1.3.3(Jul 2, 2021)
biharmonic_friction_cosPower now works as intended (a value of 3 has a total impact of cos(lat)^3, not cos(lat)^6).

Added multi-GPU support to benchmarks (via -c jax-gpu-mpi).

Source code(tar.gz)
Source code(zip)
v1.3.2(Jun 25, 2021)
Changes

New setting biharmonic_friction_cosPower to scale biharmonic friction with latitude

New runtime setting monitor_streamfunction_residual to switch off convergence checks in PETSc solver

Restart files are never written in the first iteration

Source code(tar.gz)
Source code(zip)
v1.3.1(Jun 17, 2021)

Source distributions now include files missing in v1.3.0.
Source code(tar.gz)
Source code(zip)
v1.3.0(Jun 17, 2021)
This is a huge release with many breaking changes.

Setups written for previous versions of Veros will not be compatible with this version.

New versioning scheme

We have decided to add +1 to the Veros version. Veros has been at a stage where it is mature enough for scientific applications for a while, and we feel that a version 1.x.x reflects that better than 0.x.x.

Major changes

A new command line interface to execute Veros setups: $ veros run mysetup.py

Bohrium is now replaced with JAX. JAX is easier to install and provides better performance on CPU and GPU.

Overhauled the internal data model. All model state is now encapsulated in a VerosState object that can be passed around and inspected.

Major performance increases on GPU.

The PETSc linear solver now works on GPUs.

Explicit support for custom variables and dimensions.

Many bug fixes and quality-of-life improvements.

Asset files are now in JSON format.

Internal changes

Enforcing of a code formatting standard via flake8 and black.

Hand-written TDMA kernels (CUDA kernel by @sefrin).

A more robust testing suite.

Better modularity of diagnostics and model restarts.

Source code(tar.gz)
Source code(zip)
v0.2.3(May 22, 2021)

This is the last release before replacing Bohrium with JAX, and before switching to a new versioning scheme.

Besides some bug fixes and tweaks this also introduces the setup acc_basic (a simplified version of ACC).
Source code(tar.gz)
Source code(zip)
v0.2.2(May 20, 2020)
Changes

Assets are now downloaded the first time they are accesses (instead of during import)

Dependencies are pinned by default

Streamfunction solver uses a new ILU preconditioner that shows very good performance. PyAMG support has been dropped.

Linear solver can now be chosen via the VEROS_LINEAR_SOLVER environment variable.

Wind stress (surface_taux, surface_tauy) now has the correct units in output files.

global_4deg setup now has an ITF (Indonesian Throughflow).

Fixed critical bugs related to wind stress in acc and global_flexible setups.

Source code(tar.gz)
Source code(zip)
v0.2.1(Nov 25, 2019)
Changes

A new plugin system that supports external development on Veros

Diagnostics now have their own log level to make sure output is displayed in most cases

Veros now supports system-wide setup folders that can be discovered by veros copy-setup

Fixed bugs in overturning diagnostic

Potential density is now computed as a model variable

Fixed missing attributes on netCDF outputs

Tweaked documentation

Source code(tar.gz)
Source code(zip)
v0.2.0(May 22, 2019)
This is a huge release with many exciting changes!

This release is not backwards-compatible. Setups created with v0.1.x will have to be rewritten to work with this version.

Changes

Veros now fully supports distributed architectures through MPI, for both NumPy and Bohrium backends. If you have MPI installed, just run something like

$ mpirun -n 4 python mysetup.py -n 2 2

to get started.

A new realistic flexible resolution setup (global_flexible) that automatically adapts to the chosen resolution. Just choose a resolution and time step and that's it! Also comes with automatic removal of marginal seas.

Better output and logging capabilities: terminal colors, a progress bar and throughput indicator, more verbosity options.

A re-implementation of the island routines, which are now faster and more stable.

A fully distributed Poisson solver via PETSc that is over 10x faster than PyOM's solver.

netCDF output via HDF5, so libnetcdf is not a dependency anymore.

The old Veros base class is now separated into VerosSetup (containing all setup-specific logic) and VerosState (holding the model settings and variables).

Global runtime settings:

from veros import runtime_settings as rs rs.backend = 'bohrium'

Polished documentation.

Various code quality improvements and polish.

Removed features

Dropped Python 2 support.

Removed Eady setup.

Removed logfile argument.

Source code(tar.gz)
Source code(zip)
v0.1.1(Jan 25, 2018)
Veros on PyPI

Veros is now installable via pip! This allows you to get started with Veros in lightning speed:

$ pip install veros $ veros copy-setup --to /tmp $ python /tmp/acc/acc.py

Other changes

Command line interfaces are now based on click instead of argparse.

Unless explicitly told, Veros does not consume any arguments from the command line any more. In setups meant to be run interactively, you can add the following to parse command line arguments:

@veros.tools.cli def run(*args, **kwargs): simulation = MySetup(*args, **kwargs) simulation.setup() simulation.run() if __name__ == "__main__": run()

Setup assets (i.e., external binary files required to run the standard setups) are now downloaded on demand to $HOME/.veros, or to wherever the environment variable VEROS_ASSET_DIR pointed when Veros was imported.

Git LFS is not required anymore.

Veros is now licensed under a MIT license.

Source code(tar.gz)
Source code(zip)
v0.1.0(Dec 9, 2017)

The first official release of Veros!

All core routines in this version are more or less direct (vectorized) translations of their respective pyOM 2.1.0 counterpart.

The code is optimized for use with NumPy or Bohrium, using either the OpenMP or OpenCL backend.

Supports both Python 2.7 and 3.x, but you might run into issues with Bohrium on Python 3.x.
Source code(tar.gz)
Source code(zip)