Processing and interpolating spatial data with a twist of machine learning

Overview

Verde

Documentation | Documentation (dev version) | Contact | Part of the Fatiando a Terra project

Latest version on PyPI TravisCI build status Azure Pipelines build status Test coverage status Code quality grade on codacy Compatible Python versions. Digital Object Identifier for the JOSS paper

About

Verde is a Python library for processing spatial data (bathymetry, geophysics surveys, etc) and interpolating it on regular grids (i.e., gridding).

Most gridding methods in Verde use a Green's functions approach. A linear model is estimated based on the input data and then used to predict data on a regular grid (or in a scatter, a profile, as derivatives). The models are Green's functions from (mostly) elastic deformation theory. This approach is very similar to machine learning so we implement gridder classes that are similar to scikit-learn regression classes. The API is not 100% compatible but it should look familiar to those with some scikit-learn experience.

Advantages of using Green's functions include:

  • Easily apply weights to data points. This is a linear least-squares problem.
  • Perform model selection using established machine learning techniques, like k-fold or holdout cross-validation.
  • The estimated model can be easily stored for later use, like spherical-harmonic coefficients are used in gravimetry.

The main disadvantage is the heavy memory and processing time requirement (it's a linear regression problem).

Project goals

  • Provide a machine-learning inspired interface for gridding spatial data
  • Integration with the Scipy stack: numpy, pandas, scikit-learn, and xarray
  • Include common processing and data preparation tasks, like blocked means and 2D trends
  • Support for gridding scalar and vector data (like wind speed or GPS velocities)
  • Support for both Cartesian and geographic coordinates

The first release of Verde was focused on meeting these initial goals and establishing the look and feel of the library. Later releases will focus on expanding the range of gridders available, optimizing the code, and improving algorithms so that larger-than-memory datasets can also be supported.

Contacting us

Citing Verde

This is research software made by scientists (see AUTHORS.md). Citations help us justify the effort that goes into building and maintaining this project. If you used Verde for your research, please consider citing us.

See our CITATION.rst file to find out more.

Contributing

Code of conduct

Please note that this project is released with a Contributor Code of Conduct. By participating in this project you agree to abide by its terms.

Contributing Guidelines

Please read our Contributing Guide to see how you can help and give feedback.

Imposter syndrome disclaimer

We want your help. No, really.

There may be a little voice inside your head that is telling you that you're not ready to be an open source contributor; that your skills aren't nearly good enough to contribute. What could you possibly offer?

We assure you that the little voice in your head is wrong.

Being a contributor doesn't just mean writing code. Equally important contributions include: writing or proof-reading documentation, suggesting or implementing tests, or even giving feedback about the project (including giving feedback about the contribution process). If you're coming to the project with fresh eyes, you might see the errors and assumptions that seasoned contributors have glossed over. If you can write any code at all, you can contribute code to open source. We are constantly trying out new skills, making mistakes, and learning from those mistakes. That's how we all improve and we are happy to help others learn.

This disclaimer was adapted from the MetPy project.

License

This is free software: you can redistribute it and/or modify it under the terms of the BSD 3-clause License. A copy of this license is provided in LICENSE.txt.

Documentation for other versions

Comments
  • Surfer reader feature

    Surfer reader feature

    ported the surfer reader from fatiando and changed the output to a xarray Data Array. Ran tests and formatted locally.

    I ported the surfer reader feature from fatiando and changed the output from a dict to an xarray data array. Will you look through it to make sure it's what you had in mind for this feature. Do you want me to add examples to the docstring as well? Right now it just has the docstring from fatiando modified as appropriate.

    Fixes #38

    Reminders

    • [x] Run make format and make check to make sure the code follows the style guide.
    • [x] Add tests for new features or tests that would have caught the bug that you're fixing.
    • [x] Add new public functions/methods/classes to doc/api/index.rst and verde/__init__.py.
    • [x] Write detailed docstrings for all functions/classes/methods.
    • [ ] If adding new functionality, add an example to the docstring, gallery, and/or tutorials.
    opened by jessepisel 22
  • Add enhanced support for geographic coordinates

    Add enhanced support for geographic coordinates

    Add latlon_continuity() function that moves a set of longitudinal coordinates either to the [0, 360) or [-180, 180) degrees intervals, depending which one is more suited for that particular set of coordinates. It also allow to modify an extra set of coordinates based on the decision made on the first set. Modify the inside() function to enhance how the inside points are selected, even if their longitudinal coordinates are not defined on the same degree interval.

    Fixes #171

    Reminders

    • [ ] Run make format and make check to make sure the code follows the style guide.
    • [ ] Add tests for new features or tests that would have caught the bug that you're fixing.
    • [ ] Add new public functions/methods/classes to doc/api/index.rst and verde/__init__.py.
    • [ ] Write detailed docstrings for all functions/classes/methods.
    • [ ] If adding new functionality, add an example to the docstring, gallery, and/or tutorials.
    opened by santisoler 14
  • Switch from 88 character lines back to 80

    Switch from 88 character lines back to 80

    As much as I like black, and as much as I understand its advantages, its default maximum line length drives me crazy. The maximum of 80 is there for a good reason, as terminals usually have this limit. So if edit in a traditional editor, or look at the docs or the code in a terminal, a longer line-length makes working very difficult. And this is not that uncommon, think, e.g., about working on a server without GUI. I guess this is a plea that not everyone is working in a Notebook or with Atom, Sublime, VSCode etc.

    I suggest to restrict the max-line-length to 80. As far as I could see, there are three places currently with a max-line-length:

    verde-master/> grep -rin 'line-length'
        .pylintrc:109: max-line-length=100
        .stickler.yml:6: max-line-length: 88
        setup.cfg:10: max-line-length = 88
    

    I attach here two screenshots, one from GVim and one from an IPython console.

    IPython: Desktop-verde-master_002 coordinates py (~-Desktop-verde-master-verde) - GVIM_001

    The examples are for documentation, but the same applies for the code.

    question 
    opened by prisae 14
  • Add a blocked K-Fold cross-validator

    Add a blocked K-Fold cross-validator

    The BlockKFold class splits the data into spatial blocks and does K-fold cross-validation splits on the blocks. Unlike BlockShuffleSplit it's difficult to balance the folds to have the same amount of data (since blocks can have different numbers of points) so each fold will have the same number of blocks but not necessarily data. Refactor code from BlockShuffleSplit into a base class that can be shared with the new class.

    Reminders:

    • [ ] Run make format and make check to make sure the code follows the style guide.
    • [ ] Add tests for new features or tests that would have caught the bug that you're fixing.
    • [ ] Add new public functions/methods/classes to doc/api/index.rst and the base __init__.py file for the package.
    • [ ] Write detailed docstrings for all functions/classes/methods. It often helps to design better code if you write the docstrings first.
    • [ ] If adding new functionality, add an example to the docstring, gallery, and/or tutorials.
    • [ ] Add your full name, affiliation, and ORCID (optional) to the AUTHORS.md file (if you haven't already) in case you'd like to be listed as an author on the Zenodo archive of the next release.
    opened by leouieda 13
  • Fix bug with profile distances being unprojected

    Fix bug with profile distances being unprojected

    BaseGridder.profile has the option to apply a projection to the coordinates before predicting so we can pass geographic coordinates to cartesian gridders. In these cases, the distance along the profile is calculated by profile_coordinates with the unprojected coordinates (in the geographic case it would be degrees). Returning unprojected distances is not very useful and instead we should have been returning projected distances. They need to be calculated from the projected coordinates (out of profile_coordinates) because the points are evenly spaced in unprojected coordinates.

    Reminders:

    • [ ] Run make format and make check to make sure the code follows the style guide.
    • [ ] Add tests for new features or tests that would have caught the bug that you're fixing.
    • [ ] Add new public functions/methods/classes to doc/api/index.rst and the base __init__.py file for the package.
    • [ ] Write detailed docstrings for all functions/classes/methods. It often helps to design better code if you write the docstrings first.
    • [ ] If adding new functionality, add an example to the docstring, gallery, and/or tutorials.
    • [ ] Add your full name, affiliation, and ORCID (optional) to the AUTHORS.md file (if you haven't already) in case you'd like to be listed as an author on the Zenodo archive of the next release.
    opened by leouieda 11
  • Tutorial about how grid coordinates are generated

    Tutorial about how grid coordinates are generated

    I created a tutorial for grid generation that covers region and spacing adjustment and pixel registration. Let me know if this is what you had in mind for this tutorial and I can change as needed. Addresses #161

    Reminders

    • [X] Run make format and make check to make sure the code follows the style guide.
    • [na ] Add tests for new features or tests that would have caught the bug that you're fixing.
    • [na] Add new public functions/methods/classes to doc/api/index.rst and verde/__init__.py.
    • [na] Write detailed docstrings for all functions/classes/methods.
    • [X] If adding new functionality, add an example to the docstring, gallery, and/or tutorials.
    opened by jessepisel 11
  • Use dask.delayed instead of the distributed client

    Use dask.delayed instead of the distributed client

    Started work on using dask.delayed instead of the futures interface (client.submit). It's easier and allows building the entire graph lazily before executing. Still need tests and to port the SplineCV code to this. Will deprecate the client argument and remove in 2.0.0.

    Reminders

    • [ ] Run make format and make check to make sure the code follows the style guide.
    • [ ] Add tests for new features or tests that would have caught the bug that you're fixing.
    • [ ] Add new public functions/methods/classes to doc/api/index.rst and verde/__init__.py.
    • [ ] Write detailed docstrings for all functions/classes/methods.
    • [ ] If adding new functionality, add an example to the docstring, gallery, and/or tutorials.
    opened by leouieda 8
  • Add function for rolling windows on irregular data

    Add function for rolling windows on irregular data

    The rolling_window function returns the indices of points falling on rolling/moving windows. It works for irregularly sampled data and is a nice complement to xarray.DataArray.rolling (which only works on grids). Uses a KDTree to select points but is not compatible with pykdtree (it's missing the method we need).

    Reminders:

    • [ ] Run make format and make check to make sure the code follows the style guide.
    • [ ] Add tests for new features or tests that would have caught the bug that you're fixing.
    • [ ] Add new public functions/methods/classes to doc/api/index.rst and the base __init__.py file for the package.
    • [ ] Write detailed docstrings for all functions/classes/methods. It often helps to design better code if you write the docstrings first.
    • [ ] If adding new functionality, add an example to the docstring, gallery, and/or tutorials.
    • [ ] Add your full name, affiliation, and ORCID (optional) to the AUTHORS.md file (if you haven't already) in case you'd like to be listed as an author on the Zenodo archive of the next release.
    opened by leouieda 7
  • Add implementation for geographic coordinates conversion

    Add implementation for geographic coordinates conversion

    Description of the desired feature

    The geographic coordinates conversion have been implemented on Harmonica. I don't remember exactly why we added the functions there and not on Verde. Would it be Verde a better place for them?

    enhancement 
    opened by santisoler 7
  • Improve support for geographic coordinates

    Improve support for geographic coordinates

    Description of the desired feature

    Some functions from verde are intended to work with both projected and geographic coordinates. Nevertheless, working with the latter imposes some requirements. For example, one must ensure continuity around the globe: setting a grid between longitudes 170° and -170° or between 350° and 10° is not possible at the time because numerically the east coordinate is lower than the west one. One way to fix this is to add a latlon=True argument to those functions in order to make clear that we are passing geographic coordinates.

    I'll try to make a complete list of the functions that needs to be enhanced:

    • inside()
    • grid_coordinates()
    • scatter_coordinates()
    • profile_coordinates()
    • get_region()
    • pad_region()

    Are you willing to help implement and maintain this feature? Yes

    enhancement 
    opened by santisoler 7
  • Change order of dims in example of make_xarray_grid

    Change order of dims in example of make_xarray_grid

    Change the order of the dimensions of DataArrays in the examples of make_xarray_grid, since xarray v0.19.0 prints them in a different order. Tell pytest to ignore the outputs of the print statements in the make_xarray_grid examples while running doctests.

    Reminders:

    • [ ] Run make format and make check to make sure the code follows the style guide.
    • [ ] Add tests for new features or tests that would have caught the bug that you're fixing.
    • [ ] Add new public functions/methods/classes to doc/api/index.rst and the base __init__.py file for the package.
    • [ ] Write detailed docstrings for all functions/classes/methods. It often helps to design better code if you write the docstrings first.
    • [ ] If adding new functionality, add an example to the docstring, gallery, and/or tutorials.
    • [ ] Add your full name, affiliation, and ORCID (optional) to the AUTHORS.md file (if you haven't already) in case you'd like to be listed as an author on the Zenodo archive of the next release.
    opened by santisoler 6
  • Use a better Spline Green's function for small distances

    Use a better Spline Green's function for small distances

    If distance is small change the Green's function to x(log(x**x)-1) which evaluates to the correct limit of 0. Deprecate the mindist parameters since it's no longer needed. Need to propagate changes to the SplineCV class now.

    Relevant issues/PRs:

    opened by leouieda 1
  • Allow block cross-validation classes to take extra coordinates

    Allow block cross-validation classes to take extra coordinates

    Description of the desired feature:

    Sometimes, we may want to pass extra variables to be split (like heights or other numerical/categorical values) to be split by BlockKFold and BlockShuffleSplit (see fatiando/community#86). Right now, there are two things blocking this:

    1. The BaseBlockCrossValidator checks for 2 dimensions on X: https://github.com/fatiando/verde/blob/main/verde/base/base_classes.py#L81
    2. The cross-validators assume easting=X[:,0] and northing=X[:,1]: https://github.com/fatiando/verde/blob/main/verde/model_selection.py#L184

    A solution would be to:

    1. Remove the check in the base class.
    2. Pass in an argument coordinate_columns=[0, 1] that specifies which columns of the feature matrix X correspond to the coordinates.

    I think the rest should work and make these classes more usable to general spatial machine learning.

    Are you willing to help implement and maintain this feature?

    Maybe but I have no bandwidth for this right now and don't really have a use for it just yet. Happy for anyone else to give it a shot.

    enhancement 
    opened by leouieda 0
  • Remove optional numpy-based backend and require numba

    Remove optional numpy-based backend and require numba

    We still support numpy-based functions for building Jacobians and forward modelling in Spline and elsewhere. These are a bit of a pain to maintain and we have to keep testing with and without numba. The numba version is much faster, parallel, and more memory efficient. With the state of numba these days, it's not worth keeping it as an optional dependency.

    We should remove the numpy-based functions and add numba as a required dependency. This can allow us to experience with numba functions for other performance critical parts of the project, like BlockReduce.

    maintenance 
    opened by leouieda 0
  • Remove the

    Remove the "client" parameter from dask enabled functions

    Passing client as a Dask distributed client was deprecated and the argument should be removed in Verde v2.0.0 #331. This needs to be done in all classes/functions that use Dask for parallel processing.

    maintenance 
    opened by leouieda 0
  • Remove the verde.datasets module

    Remove the verde.datasets module

    This module has been deprecated in #277. The CheckerBoard class will be the only one there and even it is getting moved to a different module (#349). In Verde 2.0.0 #331 we should remove the entire module.

    maintenance 
    opened by leouieda 1
Releases(v1.7.0)
  • v1.7.0(Mar 25, 2022)

    Released on: 2022/03/25

    DOI: https://doi.org/10.5281/zenodo.6384887

    Deprecation:

    • Move the CheckerBoard class to verde.synthetic (#353)
    • Deprecate the verde.test function which will be removed in v2.0.0 (#344)
    • Deprecate the datasets module, which will be replaced by Ensaio in the future (#277)
    • Warn that the default score will change from R² to negative RMSE in v2.0 (#352)

    New features:

    • Add option to pass coordinates to the grid method instead of just region and spacing (#326)
    • Add support for Python 3.9 (#323) and 3.10 (#346)

    Documentation:

    • Modernize the front page of the docs (#356)
    • Modernize the Installing page (#355)
    • Update the contact link in the docs (#347)
    • Switch the docs theme to the sphinx-book-theme (#343)
    • Update dims in example of make_xarray_grid (#329)
    • Explicitly pass default arguments with their corresponding keywords on tests and examples (#327)

    Maintenance:

    • Replace Google Analytics for Plausible one to make our docs more privacy-friendly (#358)
    • Move configuration from setup.py to setup.cfg (#348)
    • Link CoC, Authorship, Contributing, and Maintainers guides back to the Fatiando-wide pages (#338)
    • Replace pylint with more flake8 plugins (#337)
    • Rename the main branch from "master" to "main" (#335)
    • Remove normalize argument when creating scikit-learn solvers (#333)

    This release contains contributions from:

    • Santiago Soler
    • Leonardo Uieda
    Source code(tar.gz)
    Source code(zip)
  • v1.6.1(Mar 22, 2021)

    Released on: 2021/03/22

    image

    Minor changes:

    • Allow make_xarray_grid to receive data=None instead of raising an error. This is used to create an empty xarray.Dataset (#318)

    Maintenance:

    • Fix use of wrong version numbers for PyPI releases (#317)

    This release contains contributions from:

    • Santiago Soler
    • Leonardo Uieda
    Source code(tar.gz)
    Source code(zip)
  • v1.6.0(Mar 18, 2021)

    Released on: 2021/03/18

    image

    New features:

    • Allow specifing the scoring function in cross_val_score instead of always using the .score method of the gridder (#273)
    • New function verde.make_xarray_grid to simplify the creation of xarray.Dataset from individual numpy arrays that represent a 2D grid (#282 and #300)

    Enhancements:

    • Raise informative errors for invalid verde.rolling_window arguments, like missing spacing or shape and invalid window sizes (#280)
    • Replace DeprecationWarning with FutureWarning since these are intended for end-users, which allows us to avoid having to set warning.simplefilter (#305 and #293)

    Documentation:

    • Several typo fixes (#306 #303 #281)
    • Update link to the GMT website in the Baja bathymetry example (#298)
    • Fix issue with Cartopy 0.17 and require versions >= 0.18 for building the docs (#283)

    Maintenance:

    • Refactor internal function get_data_names and related check functions to simplify their logic and make them more useful (#295)
    • Require Black >=20.8b1 (#284)
    • Format the doc/conf.py sphinx configuration file with Black (#275)
    • Add a license and copyright notice to every source file (#308)
    • Replace versioneer for setuptools-scm (#307)
    • Replace Travis and Azure with GitHub Actions (#309)
    • Exclude Dask 2021.03.0 as a dependency. This release was causing the tests to fail under Python 3.8 on every OS. The problem seems to be originated in dask.distributed (#311)
    • Use the OSI version of item 3 in the license (#299)

    This release contains contributions from:

    • Santiago Soler
    • Leonardo Uieda
    • Federico Esteban
    • DC Slagel
    Source code(tar.gz)
    Source code(zip)
  • v1.5.0(Jun 4, 2020)

    Released on: 2020/06/04

    image

    Bug fixes:

    • Apply projections using only the first two coordinates instead all given coordinates. Projections only really involve the first two (horizontal) coordinates. Only affects users passing extra_coords to gridder methods. (#264)

    New features:

    • New blocked cross-validation classes BlockShuffleSplit and BlockKFold. These are scikit-learn compatible cross-validators that split the data into spatial blocks before assigning them to folds. Blocked cross-validation can help avoid overestimation of prediction accuracy for spatial data. The classes work with verde.cross_val_score and any other function/method/class that accepts a scikit-learn cross-validator. (#251 and #254)
    • Add the option for block-wise splitting in verde.train_test_split by passing in a spacing or shape parameters. (#253 and #257)

    Base classes:

    • Add optional argument to verde.base.least_squares to copy Jacobian matrix. (#255)
    • Add extra coordinates (specified by the extra_coords keyword argument to outputs of BaseGridder methods. (#265)

    Maintenance:

    • Update tests to repr changes in scikit-learn 0.23.0. (#267)

    Documentation:

    • Fix typo in README contributing section. (#258)

    This release contains contributions from:

    • Leonardo Uieda
    • Santiago Soler
    • Rowan Cockett
    Source code(tar.gz)
    Source code(zip)
  • v1.4.0(Apr 6, 2020)

    DOI: https://doi.org/10.5281/zenodo.3739449

    Bug fixes:

    • Profile distances are now returned in projected (Cartesian) coordinates by the profile method of gridders if a projection is given. The method has the option to apply a projection to the coordinates before predicting so we can pass geographic coordinates to Cartesian gridders. In these cases, the distance along the profile is calculated by the profile_coordinates function with the unprojected coordinates (in the geographic case it would be degrees). The profile point calculation is also done assuming that coordinates are Cartesian, which is clearly wrong if inputs are longitude and latitude. To fix this, we now project the input points prior to passing them to profile_coordinates. This means that the distances are Cartesian and generation of profile points is also Cartesian (as is assumed by the function). The generated coordinates are projected back so that the user gets longitude and latitude but distances are still projected Cartesian meters. (#231)
    • Function verde.grid_to_table now sets the correct order for coordinates. We were relying on the order of the coords attribute of the xarray.Dataset for the order of the coordinates. This is wrong because xarray takes the coordinate order from the dims attribute instead, which is what we should also have been doing. (#229)

    Documentation:

    • Generalize coordinate system specifications in verde.base.BaseGridder docstrings. Most methods don't really depend on the coordinate system so use a more generic language to allow derived classes to specify their coordinate systems without having to overload the base methods just to rewrite the docstrings. (#240)

    New features:

    • New function verde.convexhul_mask to mask points in a grid that fall outside the convex hull defined by data points. (#237)
    • New function verde.project_grid that transforms 2D gridded data using a given projection. It re-samples the data using ScipyGridder (by default) and runs a blocked mean (optional) to avoid aliasing when the points aren't evenly distributed in the projected coordinates (like in polar projections). Finally, it applies a convexhul_mask to the grid to avoid extrapolation to points that had no original data. (#246)
    • New function verde.expanding_window for selecting data that falls inside of an expanding window around a central point. (#238)
    • New function verde.rolling_window for rolling window selections of irregularly sampled data. (#236)

    Improvements:

    • Allow verde.grid_to_table to take xarray.DataArray as input. (#235)

    Maintenance:

    • Use newer MacOS images on Azure Pipelines. (#234)

    This release contains contributions from:

    • Leonardo Uieda
    • Santiago Soler
    • Jesse Pisel
    Source code(tar.gz)
    Source code(zip)
  • v1.3.0(Jan 22, 2020)

    DOI: https://doi.org/10.5281/zenodo.3620851

    DEPRECATIONS (the following features are deprecated and will be removed in Verde v2.0.0):

    • Functions and the associated sample dataset verde.datasets.fetch_rio_magnetic and verde.datasets.setup_rio_magnetic_map are deprecated. Please use another dataset instead. (#213)
    • Class verde.VectorSpline2D is deprecated. The class is specific for GPS/GNSS data and doesn't fit the general-purpose nature of Verde. The implementation will be moved to the Erizo package instead. (#214)
    • The client keyword argument for verde.cross_val_score and verde.SplineCV is deprecated in favor of the new delayed argument (see below). (#222)

    New features:

    • Use the dask.delayed interface for parallelism in cross-validation instead of the futures interface (dask.distributed.Client). It's easier and allows building the entire graph lazily before executing. To use the new feature, pass delayed=True to verde.cross_val_score and verde.SplineCV. The argument client in both of these is deprecated (see above). (#222)
    • Expose the optimal spline in verde.SplineCV.spline_. This is the fitted verde.Spline object using the optimal parameters. (#219)
    • New option drop_coords to allow verde.BlockReduce and verde.BlockMean to reduce extra elements in coordinates (basically, treat them as data). Default to True to maintain backwards compatibility. If False, will no longer drop coordinates after the second one but will apply the reduction in blocks to them as well. The reduced coordinates are returned in the same order in the coordinates. (#198)

    Improvements:

    • Use the default system cache location to store the sample data instead of ~/.verde/data. This is so users can more easily clean up unused files. Because this is system specific, function verde.datasets.locate was added to return the cache folder location. (#220)

    Bug fixes:

    • Correctly use parallel=True and numba.prange in the numba compiled functions. Using it on the Green's function was raising a warning because there is nothing to parallelize. (#221)

    Maintenance:

    • Add testing and support for Python 3.8. (#211)

    Documentation:

    • Fix a typo in the JOSS paper Bibtex entry. (#215)
    • Wrap docstrings to 79 characters for better integration with Jupyter and IPython. These systems display docstrings using 80 character windows, causing our larger lines to wrap around and become almost illegible. (#212)
    • Use napoleon instead of numpydoc to format docstrings. Results is slightly different layout in the website documentation. (#209)
    • Update contact information to point to the Slack chat instead of Gitter. (#204)

    This release contains contributions from:

    • Santiago Soler
    • Leonardo Uieda
    Source code(tar.gz)
    Source code(zip)
  • v1.2.0(Jul 23, 2019)

    DOI: https://doi.org/10.5281/zenodo.3347076

    Bug fixes:

    • Return the correct coordinates when passing pixel_register=True and shape to verde.grid_coordinates. The returned coordinates had 1 too few elements in each dimension (and the wrong values). This is because we generate grid-line registered points first and then shift them to the center of the pixels and drop the last point. This only works when specifying spacing because it will generate the right amount of points. When shape is given, we need to first convert it to “grid-line” shape (with 1 extra point per dimension) before generating coordinates. (#183)
    • Reset force coordinates when refitting splines. Previously, the splines set the force coordinates from the data coordinates only the first time fit was called. This means that when fitting on different data, the spline would still use the old coordinates leading to a poor prediction score. Now, the spline will use the coordinates of the current data passed to fit. This only affects cases where force_coords=None. It’s a slight change and only affects some of the scores for cross-validation. (#191)

    New functions/classes:

    • New class verde.SplineCV: a cross-validated version of Spline . that performs grid search cross-validation to automatically tune the parameters of a Spline. (#185)
    • New function verde.longitude_continuity to format longitudes to a continuous range so that they can be indexed with verde.inside (#181)
    • New function verde.load_surfer to load grid data from a Surfer ASCII file (a contouring, griding and surface mapping software from GoldenSoftware). (#169)
    • New function verde.median_distance that calculates the median near neighbor distance between each point in the given dataset. (#163)

    Improvements:

    • Allow verde.block_split and verde.BlockReduce to take a shape argument instead of spacing. Useful when the size of the block is less meaningful than the number of blocks. (#184)
    • Allow zero degree polynomials in verde.Trend, which represents a mean value. (#162)
    • Function verde.cross_val_score returns a numpy array instead of a list for easier computations on the results. (#160)
    • Function verde.maxabs now handles inputs with NaNs automatically. (#158)

    Documentation:

    • New tutorial to explain the intricacies of grid coordinates generation, adjusting spacing vs region, pixel registration, etc. (#192)

    Maintenance:

    • Drop support for Python 3.5. (#178)
    • Add support for Python 3.7. (#150)
    • More functions are now part of the base API: n_1d_arrays, check_fit_input and least_squares are now included in verde.base. (#156)

    This release contains contributions from:

    • Goto15
    • Lindsey Heagy
    • Jesse Pisel
    • Santiago Soler
    • Leonardo Uieda
    Source code(tar.gz)
    Source code(zip)
  • v1.1.0(Nov 6, 2018)

    https://doi.org/10.5281/zenodo.1478245

    New features:

    • New verde.grid_to_table function that converts grids to xyz tables with the coordinate and data values for each grid point (#148)
    • Add an extra_coords option to coordinate generators (grid_coordinates, scatter_points, and profile_coordinates) to specify a constant value to be used as an extra coordinate (#145)
    • Allow gridders to pass extra keyword arguments (**kwargs) for the coordinate generator functions (#144)

    Improvements:

    • Don’t use the Jacobian matrix for predictions to avoid memory overloads. Use dedicated and numba wrapped functions instead. As a consequence, predictions are also a bit faster when numba is installed (#149)
    • Set the default n_splits=5 when using KFold from scikit-learn (#143)

    Bug fixes:

    • Use the xarray grid’s pcolormesh method instead of matplotlib to plot grids in the examples. The xarray method takes care of shifting the pixels by half a spacing when grids are not pixel registered (#151)

    New contributors to the project:

    • Jesse Pisel
    Source code(tar.gz)
    Source code(zip)
  • v1.0.1(Oct 11, 2018)

    https://doi.org/10.5281/zenodo.1421979

    Changes:

    • Paper submission to JOSS (#134). This is the new default citation for Verde.
    • Remove default shape for the grid method (#140). There is no reason to have one and it wasn’t even implemented in grid_coordinates.
    • Fix typo in the weights tutorial (#136).
    Source code(tar.gz)
    Source code(zip)
  • v1.0.0(Sep 13, 2018)

    https://doi.org/10.5281/zenodo.1415281

    First release of Verde. Establishes the gridder API and includes blocked reductions, bi-harmonic splines, coupled 2D interpolation, chaining operations to form a pipeline, and more.

    Source code(tar.gz)
    Source code(zip)
  • v1.0.0rc2(Nov 6, 2018)

  • v1.0.0rc1(Sep 13, 2018)

Owner
Fatiando a Terra
Open-source Python tools for geophysics
Fatiando a Terra
OSMnx: Python for street networks. Retrieve, model, analyze, and visualize street networks and other spatial data from OpenStreetMap.

OSMnx OSMnx is a Python package that lets you download geospatial data from OpenStreetMap and model, project, visualize, and analyze real-world street

Geoff Boeing 4k Jan 8, 2023
A package built to support working with spatial data using open source python

EarthPy EarthPy makes it easier to plot and manipulate spatial data in Python. Why EarthPy? Python is a generic programming language designed to suppo

Earth Lab 414 Dec 23, 2022
Using Global fishing watch's data to build a machine learning model that can identify illegal fishing and poaching activities through satellite and geo-location data.

Using Global fishing watch's data to build a machine learning model that can identify illegal fishing and poaching activities through satellite and geo-location data.

Ayush Mishra 3 May 6, 2022
PySAL: Python Spatial Analysis Library Meta-Package

Python Spatial Analysis Library PySAL, the Python spatial analysis library, is an open source cross-platform library for geospatial data science with

Python Spatial Analysis Library 1.1k Dec 18, 2022
Using SQLAlchemy with spatial databases

GeoAlchemy GIS Support for SQLAlchemy. Introduction GeoAlchemy is an extension of SQLAlchemy. It provides support for Geospatial data types at the ORM

null 109 Dec 1, 2022
Software for Advanced Spatial Econometrics

GeoDaSpace Software for Advanced Spatial Econometrics GeoDaSpace current version 1.0 (32-bit) Development environment: Mac OSX 10.5.x (32-bit) wxPytho

GeoDa Center 38 Jan 3, 2023
Raster-based Spatial Analysis for Python

?? xarray-spatial: Raster-Based Spatial Analysis in Python ?? Fast, Accurate Python library for Raster Operations ⚡ Extensible with Numba ⏩ Scalable w

makepath 649 Jan 1, 2023
Python package for earth-observing satellite data processing

Satpy The Satpy package is a python library for reading and manipulating meteorological remote sensing data and writing it to various image and data f

PyTroll 882 Dec 27, 2022
framework for large-scale SAR satellite data processing

pyroSAR A Python Framework for Large-Scale SAR Satellite Data Processing The pyroSAR package aims at providing a complete solution for the scalable or

John Truckenbrodt 389 Dec 21, 2022
A toolbox for processing earth observation data with Python.

eo-box eobox is a Python package with a small collection of tools for working with Remote Sensing / Earth Observation data. Package Overview So far, t

null 13 Jan 6, 2022
Geospatial Image Processing for Python

GIPPY Gippy is a Python library for image processing of geospatial raster data. The core of the library is implemented as a C++ library, libgip, with

GIPIT 83 Aug 19, 2022
Open Data Cube analyses continental scale Earth Observation data through time

Open Data Cube Core Overview The Open Data Cube Core provides an integrated gridded data analysis environment for decades of analysis ready earth obse

Open Data Cube 410 Dec 13, 2022
A public data repository for datasets created from TransLink GTFS data.

TransLink Spatial Data What: TransLink is the statutory public transit authority for the Metro Vancouver region. This GitHub repository is a collectio

Henry Tang 3 Jan 14, 2022
Fiona reads and writes geographic data files

Fiona Fiona reads and writes geographic data files and thereby helps Python programmers integrate geographic information systems with other computer s

null 987 Jan 4, 2023
python toolbox for visualizing geographical data and making maps

geoplotlib is a python toolbox for visualizing geographical data and making maps data = read_csv('data/bus.csv') geoplotlib.dot(data) geoplotlib.show(

Andrea Cuttone 976 Dec 11, 2022
Tool to suck data from ArcGIS Server and spit it into PostgreSQL

chupaESRI About ChupaESRI is a Python module/command line tool to extract features from ArcGIS Server map services. Name? Think "chupacabra" or "Chupa

John Reiser 34 Dec 4, 2022
gpdvega is a bridge between GeoPandas and Altair that allows to seamlessly chart geospatial data

gpdvega gpdvega is a bridge between GeoPandas a geospatial extension of Pandas and the declarative statistical visualization library Altair, which all

Ilia Timofeev 49 Jul 25, 2022
Blender addons to make the bridge between Blender and geographic data

Blender GIS Blender minimal version : 2.8 Mac users warning : currently the addon does not work on Mac with Blender 2.80 to 2.82. Please do not report

null 5.9k Jan 2, 2023
EOReader is a multi-satellite reader allowing you to open optical and SAR data.

Remote-sensing opensource python library reading optical and SAR sensors, loading and stacking bands, clouds, DEM and index.

ICube-SERTIT 152 Dec 30, 2022