Python API for working with RESQML models

Overview

resqpy: Python API for working with RESQML models

License Documentation Status Python CI Python version PyPI Status codecov

Introduction

resqpy is a pure python package which provides a programming interface (API) for reading, writing, and modifying reservoir models in the RESQML format. It gives you the ability to work with reservoir models programmatically, without having to know the details of the RESQML standard.

The package is written and maintained by bp, and is made available under the MIT license as a contribution to the open-source community.

resqpy was created by Andy Beer. For enquires about resqpy, please contact Nathan Lane ([email protected])

Documentation

See the complete package documentation on readthedocs.

About RESQML

RESQML™ is an industry initiative to provide open, non-proprietary data exchange standards for reservoir characterization, earth and reservoir models. It is governed by the Energistics consortium.

Resqpy provides specialized classes for a subset of the RESQML high level object classes, as described in the docs. Furthermore, not all variations of these object types are supported; for example, radial IJK grids are not yet catered for, although the RESQML standard does allow for such grids.

It is envisaged that the code base will be expanded to include other classes of object and more fully cover the options permitted by the RESQML standard.

Modification functionality at the moment focuses on changes to grid geometry.

Installation

Resqpy can be installed with pip:

pip install resqpy

Alternatively, to install your working copy of the code in "editable" mode:

pip install -e /path/to/repo/

Contributing

Contributions of all forms are welcome and encouraged! See the Contributing Guide for guidance on how you can contribute, including bug reports, features requests and pull requests.

Repository structure

  • resqpy: high level modules providing classes for main RESQML object types and high level modification functions
  • resqpy/olio: low level modules, not often imported directly by calling code
  • tests: unit tests
  • example_data: small example datasets

Unit tests

Run the test suite locally with:

pytest tests/

Making a release

To make a release at a given commit, simply make a git tag:

# Make a tag
git tag -a v0.0.1 -m "Incremental release with some bugfixes"

# Push tag to github
git push origin v0.0.1

The tag must have the prefix v and have the form MAJOR.MINOR.PATCH.

Following semantic versioning, increment the:

  • MAJOR version when you make incompatible API changes,
  • MINOR version when you add functionality in a backwards compatible manner, and
  • PATCH version when you make backwards compatible bug fixes.
Comments
  • Add context manager for Models

    Add context manager for Models

    Adds a high-level way of opening and closing models:

    with ModelContext("my_model.epc") as model:
        print(model.uuids())
    

    This provides a convenient way to ensure all file handles are properly closed when the "with" clause exists

    Nb. an alternative implementation would be to make the Model class itself function as a context manager:

    with Model("my_model.epc") as model:
        do_stuff()
    

    However, I think it's probably simpler to keep the context manager as a separate class so we can have dedicated arguments (e.g. write=True if we want store_epc to be called on exit). These options would otherwise have to go in Model.__init__, which already has a large number of options.

    In future, it would be great if this method had the option of saving any modified objects (to XML and HDF5) upon exit, so users do not have to remember the "write_xml" and "write_hdf" method calls for each object.

    enhancement 
    opened by connortann 7
  • CopyAllParts fails with invalid UUID patterns

    CopyAllParts fails with invalid UUID patterns

    The little snippet fails due to invalid UUID patterns (in hdf5).

    import resqpy.model as rq
    model = rq.Model('d:/dev/resqpy/tests/test_data/UGRID_GRID.epc')
    new_model = rq.Model('d:/new.epc', new_epc=True)
    new_model.copy_all_parts_from_other_model(model)
    new_model.store_epc('d:/new.epc')
    
    

    It fails with:

    Traceback (most recent call last):
      File "D:\DEV\resqpy\test_local\rewrite_ugrid.py", line 14, in <module>
        new_model.copy_all_parts_from_other_model(model)
      File "D:\DEV\resqpy\resqpy\model.py", line 3348, in copy_all_parts_from_other_model
        self.copy_part_from_other_model(other_model,
      File "D:\DEV\resqpy\resqpy\model.py", line 3232, in copy_part_from_other_model
        hdf5_count = whdf5.copy_h5(other_h5_file_name, self_h5_file_name, uuid_inclusion_list = [uuid], mode = 'a')
      File "D:\DEV\resqpy\resqpy\olio\write_hdf5.py", line 167, in copy_h5
        uuid = bu.uuid_from_string(group)
      File "D:\DEV\resqpy\resqpy\olio\uuid.py", line 111, in uuid_from_string
    
    

    because it encounters

    'Fault_4346bb9c-68a5-4591-86df-9284316d0dd3' or 'UnstructuredGrid_492f069b-888a-4a17-8cc1-cdc845774f18_Geometry'

    I think the issue arises from the hdf paths:

    <eml20:PathInHdfFile xmlns:xsd="http://www.w3.org/2001/XMLSchema" xsi:type="xsd:string">RESQML/UnstructuredGrid_492f069b-888a-4a17-8cc1-cdc845774f18_Geometry/USER_CELL_INDEX</eml20:PathInHdfFile> or

    <eml20:PathInHdfFile xmlns:xsd="http://www.w3.org/2001/XMLSchema" xsi:type="xsd:string">RESQML/Fault_4346bb9c-68a5-4591-86df-9284316d0dd3/Fault_Cells</eml20:PathInHdfFile> It could well be that a very well-known vendor is ignoring standard or business rule with regard to naming the hdf path within the h5 file. I have always treated PathInHdfFile as is rather than assuming some semantics.

    ugrid.zip

    bug 
    opened by mgimhof 7
  • Support full set of RESQML units of measure

    Support full set of RESQML units of measure

    Presently, common units such as "v/v" are lost and stored as "EUC".

    A really exciting option could be to include pint, and include a short config file to handle the most common units used in reservoir modelling. This could be (hopefully) an elegant way to allow a wide range of common units, with simple configuration - and could be a "selling point" of resqpy more broadly.

    https://pint.readthedocs.io/en/stable/

    enhancement 
    opened by connortann 7
  • Integration - surface.py - Move classes to individual files in new surface directory

    Integration - surface.py - Move classes to individual files in new surface directory

    Initial changes to break out classes from surface.py into individual files, under the directory 'surface'. And additional unit/integration tests for PointSet class.

    refactor tests 
    opened by emmanesbit 6
  • initial code for stratigraphic classes

    initial code for stratigraphic classes

    This change introduces 5 more high level classes, in a new module named strata.py

    The aim is for resqpy to be able to work with the stratigraphic objects exported from RMS.

    resolves issue #175

    enhancement 
    opened by andy-beer 6
  • Merging timeseries

    Merging timeseries

    New timeseries functions to merge together several timeseries into one sorted timeseries and return the resulting timeseries object. Also created a convenience routine to return datetime objects which may entail slicing off the "+Z" timezone character from the timestamps.

    This works only based off of timeseries uuids.

    I created tests in tests/test_timeseries.py, but I'm not sure how to configure the CI workflow to run them automatically

    opened by jrt54 6
  • Pin sphinx_rtd_theme version

    Pin sphinx_rtd_theme version

    Closes #227

    From experimentation:

    • Pinning an older version of sphinx-rtd-theme seems to fix the issue
    • Removing the autoclasstoc extension does not fix the issue
    • Changing the theme to "classic" does not seem to fix the issue
    bug documentation 
    opened by connortann 5
  • Improve olio.RelPerm interoperability with equinor/pyscal

    Improve olio.RelPerm interoperability with equinor/pyscal

    The current olio.RelPerm.text_to_relperm_dict() function takes a filename as the argument. This issue suggests we make a version of this function which takes a string. This would allow easier integration (by calling code) with the equinor/pyscal library which includes methods for returning Nexus WOTABLE etc. keywords as a string.

    We could also look at more efficient ways of passing rel. perm. and cap. pressure data between the two libraries, in both directions, though it is desirable not to introduce a mandatory installation dependency.

    enhancement 
    opened by andy-beer 5
  • Implement a resqpy RelPerm class inheriting from DataFrame

    Implement a resqpy RelPerm class inheriting from DataFrame

    The existing olio.dataframe module includes a general purpose DataFrame class for storing and retrieving numerical pandas dataframe data in a RESQML object (creatively re-using the Grid2DRepresentation class and related classes).

    This issue proposes implementing a specialist RelPerm class which inherits from DataFrame.

    (Similarly capillary pressure data, rock compressibility etc. could have specialised resqpy classes.)

    enhancement good first issue 
    opened by andy-beer 5
  • Additional integration tests for property.py

    Additional integration tests for property.py

    This PR adds some additional tests for property.py. These new tests have been written and run against the master branch, to ensure no breaking changes due to refactoring.

    tests 
    opened by emmanesbit 4
  • Refactored subpackages not appearing in HTML docs

    Refactored subpackages not appearing in HTML docs

    I triggered a build of the docs for the integration branch, and noticed that classes and methods corresponding to refactored sub-packages are completely missing e.g. resqpy.lines :

    image

    bug documentation 
    opened by connortann 4
  • Add Type Hinting

    Add Type Hinting

    Currently importing RESQPY into a python file in another project and running the Mypy type checker generates a series of errors: error: Skipping analyzing "resqpy": module is installed, but missing library stubs or py.typed marker [import]. This is because Mypy cannot detect the types present in RESQPY due to a lack of type hints.

    enhancement 
    opened by cflynn3 1
  • Nexus vdb import needs to handle LAB units

    Nexus vdb import needs to handle LAB units

    At present, the resqpy code for importing from Nexus runs assumes that the length units are metres or feet. For LAB units, it should be cm. Units of measure for the various array properties need to be set appropriately for LAB units as well.

    opened by andy-beer 0
  • Bug affecting PolylineSet in the presence of mixed open and closed polylines

    Bug affecting PolylineSet in the presence of mixed open and closed polylines

    The RESQML schema includes an AbstractBooleanArray for the indicators of whether each polyline in a PolylineSetRepresentation is closed or not. The resqpy code implements the abstract array as a constant array if all the polylines in the set have the same closed or open status. When there is a mix, the abstract array is implemented as a BooleanArrayFromIndexArray. There are a bug and another possible bug in the code in this situation:

    • in the ...FromIndexArray case, the create_xml() code uses a tag of 'Value' where it should be 'IndexIsTrue'
    • the actual value of that node (the Text field) may have the opposite boolean value of that which is needed
    bug 
    opened by andy-beer 0
  • Add changelog information to docs

    Add changelog information to docs

    Many repos keep a changelog. Recommended practices / style are here: https://keepachangelog.com/en/1.0.0/

    Suggested actions:

    • [ ] Add CHANGELOG.md to repo, in format detailed above
    • [ ] Key changes of past few releases populated
    • [ ] Include changelog into build documentation (e.g. markdown-to-RST extension here)
    documentation enhancement 
    opened by connortann 2
  • Enable pre-commit hooks

    Enable pre-commit hooks

    Plenty of PRs have failed jobs due to code formatting and flake8 errors, which suggests that some time is being wasted by devs having to run these tools manually.

    We could use the pre-commit python package as a developer dependency, and add a .pre-commit-config.yaml file together with some instructions in the contributor guide to install.

    This would mean yapf is run locally automatically on relevant files before each commit is made, so devs can spend their time on more important things.

    opened by connortann 0
  • Add optional schema validation for methods that load in XML files

    Add optional schema validation for methods that load in XML files

    Functionality should be added to validate XML schemas when they are loaded in. This code should be run by the 'load' methods, which will have an optional parameter added to them called 'validate_schema' which will have a default value of false. This code should also be fully unit tested before being used.

    enhancement 
    opened by cflynn3 0
Releases(v3.10.0)
  • v3.10.0(Dec 31, 2022)

    This release includes:

    • support for the GenericInterpretation RESQML organizational object class
    • a property parts convenience function (in the property module)
    • option to generate 'shadow' properties when finding faces for a surface in a regular grid
    Source code(tar.gz)
    Source code(zip)
  • v3.9.0(Dec 18, 2022)

    This minor change reduces the use of the from...import form of import statements. This might be of help when running on Windows where mutual references between modules can cause load failures when the from form is used.

    Source code(tar.gz)
    Source code(zip)
  • v3.8.5(Dec 14, 2022)

  • v3.8.4(Dec 14, 2022)

  • v3.8.3(Dec 14, 2022)

  • v3.8.2(Dec 7, 2022)

  • v3.8.1(Dec 5, 2022)

    This patch adds a crs_uuid argument to the PolylineSet initialiser, to be used when importing from ascii formats. Some other small enhancements and fixes are also included.

    Source code(tar.gz)
    Source code(zip)
  • v3.8.0(Nov 30, 2022)

    This release includes improved support for coordinate reference systems (CRS) with differing units of measure for z values and xy values (projected axes). The improvements are in areas such as volume methods, working with normal vectors and other directional vectors.

    Many minor bug fixes and enhancements are also included, as is an update of versions of dependencies.

    Source code(tar.gz)
    Source code(zip)
  • v3.7.3(Oct 24, 2022)

    This patch reduces the execution time required to generate regular grid bisector properties when requested as a returned property from the find_faces... function.

    Source code(tar.gz)
    Source code(zip)
  • v3.7.2(Oct 20, 2022)

    This patch sets the local property kind for normal vector properties for grid connection sets, when deriving them from a surface property.

    See also notes for v3.7.0

    Source code(tar.gz)
    Source code(zip)
  • v3.7.1(Oct 19, 2022)

    This patch adds the generation of warnings.warn() messages when using one of the 3 main abstract property kinds – 'continuous', 'discrete' or 'categorical' – in either of two situations:

    • adding a property array to a PropertyCollection imported_list
    • selecting properties with a property_kind filter (including PropertyCollection.singleton() and PropertyCollection.single_array_ref())

    See also notes for v3.7.0

    Source code(tar.gz)
    Source code(zip)
  • v3.7.0(Oct 18, 2022)

    This minor release changes the behaviour when creating xml for a property with a specified property kind which is one of the three main abstract property kinds: 'continuous', 'discrete' or 'categorical'. In these cases, a local property kind will be used instead, with a title the same as the property title.

    This change is to improve inter-operability with Fesapi based applications, as that API now rejects datasets that use property kinds which are identified as abstract in an ancillary xml file in the RESQML v2.0.1 standard. Note that only the three abstract kinds listed above are replaced. Other abstract kinds such as volume per volume are not replaced and might still lead to inter-operability issues.

    This change might break some workflows. When upgrading to this version, check calling code for calls to the Model catalogue methods – parts(), uuids() etc. – which specify a property_kind filter set to one of the above 3 kinds. These will need to be changed to the local property kind.

    Although resqpy will generate local property kinds as needed, it is recommended that calling code explicitly generates kinds, for clarity.

    Source code(tar.gz)
    Source code(zip)
  • v3.6.1(Oct 17, 2022)

    This patch modifies the direction of normal vectors calculated by the Surface method. They now point upwards, which is in line with the functionality before a recent refactoring.

    Source code(tar.gz)
    Source code(zip)
  • v3.6.0(Oct 5, 2022)

    This release changes the (default) RESQML representation used for the geometry of regular grids. The new code uses a Geometry node in the xml with Points defined by a Point3dLatticeArray. This should be more interoperable with software making use of the FESAPI interfaces.

    (FESAPI is written and maintained by F2I Counsulting.)

    Source code(tar.gz)
    Source code(zip)
  • v3.5.2(Oct 3, 2022)

  • v3.5.1(Sep 28, 2022)

    This patch includes further optimisation of the find faces to represent surface functionality for regular grids, especially targetted at very large (fine) grids. The normal vector calculation is also relocated to higher level functions.

    Source code(tar.gz)
    Source code(zip)
  • v3.5.0(Sep 13, 2022)

    This minor release includes:

    • property collection selection options for None, or not None, const value
    • inhibition of min max xml node creation for categorical properties
    • blocked well method for sampling grid property
    • stripping of optional time element when loading Nexus wellspec datestamps
    • various minor bug fixes and enhancements

    Nexus is a trademark of Halliburton

    Source code(tar.gz)
    Source code(zip)
  • v3.4.0(Aug 30, 2022)

    This minor release includes multiprocessing wrappers for blocking well trajectories against a grid. Nexus fault data export is enhanced to support a grid connection set property as the source of transmissibility multiplier values. Miscellaneous other small bug fixes and enhancements are also included.

    Source code(tar.gz)
    Source code(zip)
  • v3.3.2(Aug 17, 2022)

    This patch removes the numba just in time compilation decorator from the grid_surface.bisector_from_faces() function. It was found to be causing worker processes to die unexpectedly for large cell counts. On-going work is looking into the underlying issue so this is a temporary fix.

    This release also includes support for USA date format (MM/DD/YYY) in Nexus wellspec data with dates, along with other minor enhancements.

    Nexus is a trademark of Halliburton.

    Source code(tar.gz)
    Source code(zip)
  • v3.3.1(Aug 5, 2022)

    This patch adds some logic for Lab units, when guessing uoms for properties being imported from Nexus.

    Nexus is a trademark of Halliburton.

    Source code(tar.gz)
    Source code(zip)
  • v3.3.0(Aug 5, 2022)

    The Nexus import from vdb functionality reads the Nexus summary file to establish the time series. Previously the import functionality assumed that the Nexus run had included dates. This minor release enhances the import functionality to handle dateless Nexus runs.

    Nexus is a trademark of Halliburton.

    Source code(tar.gz)
    Source code(zip)
  • v3.2.0(Jul 31, 2022)

    This release includes further speed improvements, and a reduction in memory requirement, for the aligned regular grid version of the find faces to represent a surface function.

    There are also a multi-processing wrapper and batch functions for use when generating Mesh objects from grid column properties.

    Note that the optimisation work has made a small change to the signature of the low level numba_intersect() function. In the unlikely event that calling code is making direct use of that function then changes will be required.

    Source code(tar.gz)
    Source code(zip)
  • v3.1.1(Jul 25, 2022)

    This patch adds an option to generate a grid cell boolean property whilst finding faces to represent a surface (optimised, multi-processing version). The boolean property contains True for cells on one side of the surface, and False for the other. If the surface is not a curtain (vertical) then True is used for the cells 'above' the surface.

    Source code(tar.gz)
    Source code(zip)
  • v3.1.0(Jul 20, 2022)

    This minor release includes the latest updates to the WELLSPEC import functionality, which now supports multi-timestamped entries and improved metadata for properties generated from columns and added options around the handling of null data. WELLSPEC is a Nexus keyword. Nexus is a trademark of Halliburton.

    The release also includes a fix for a bug affecting the Grid.coordinate_line_end_points() method when the grid contains K Gaps. This method is generally used when preparing grid geometry data for a popular old simulator keyword format.

    Source code(tar.gz)
    Source code(zip)
  • 3.1.0(Jul 20, 2022)

    Note: the v is missing from this tag. Please use v3.1.0 instead.

    This minor release includes the latest updates to the WELLSPEC import functionality, which now supports multi-timestamped entries and improved metadata for properties generated from columns and added options around the handling of null data. WELLSPEC is a Nexus keyword. Nexus is a trademark of Halliburton.

    The release also includes a fix for a bug affecting the Grid.coordinate_line_end_points() method when the grid contains K Gaps. This method is generally used when preparing grid geometry data for a popular old simulator keyword format.

    Source code(tar.gz)
    Source code(zip)
  • v3.0.8(Jul 18, 2022)

    This patch allows well data to be read from a WELLSPEC file at multiple timestamps, and combined into a single dataframe.

    WELLSPEC is a keyword used by the Nexus simulator. Nexus is a trademark of Halliburton.

    Source code(tar.gz)
    Source code(zip)
  • v3.0.7(Jul 13, 2022)

    This patch includes a few small changes. The main functional change is that the equivalence (.EQ.) method for the Crs class now requires that extra metadata is identical for a match. This test affects the consolidation of datasets when copying parts between models.

    Source code(tar.gz)
    Source code(zip)
  • v3.0.6(Jul 8, 2022)

    This patch includes fixes a bug in the FineCoarse class proportions_for_axis() method when equal proportions is True for the axis and constant ratios is None for the axis.

    Source code(tar.gz)
    Source code(zip)
  • v3.0.5(Jul 7, 2022)

  • v3.0.4(Jul 7, 2022)

    This patch strips out explicit hdf5 control from the add_surface() functionality, instead using the default resqpy behaviour. This is needed to fix a bug that was affecting import of multiple surfaces when creating a new dataset.

    Source code(tar.gz)
    Source code(zip)
Owner
BP
BP
Library for working with QIWI API.

Library for working with QIWI API.

qxtony 2 Apr 26, 2022
A.I and game for gomoku, working only on windows

Gomoku (A.I of gomoku) The goal of the project is to create an artificial intelligence of gomoku. Goals Beat the opponent. Requirements Python 3.7+ Wo

Luis Rosario 13 Jun 20, 2021
Working TikTok Username Auto-Claimer/Sniper/Swapper which will autoclaim username if it´s available

TikTok-AutoClaimer Working TikTok Username Auto-Claimer/Sniper/Swapper which will autoclaim username if it´s available Usage Python 3.6 or above is re

Kevin 18 Dec 8, 2022
A MassDM selfbot which is working in 2021

mass-dm-discord - Little preview of the Logger and the Spammer Features Logging User IDS Sending DMs to the logged IDs Blacklist IDs (add the ID of th

karma.meme 88 Dec 26, 2022
Easy & powerful bot to check if your all Telegram bots are working or not. This bot status bot updates every 45 minutes & runs for 24x7 hours.

PowerfulBotStatus-IDN-C-X Easy & powerful bot to check if your all Telegram bots are working or not. This bot status bot updates every 45 minutes & ru

IDNCoderX 5 Oct 6, 2022
A (probably) working Kik name checker

KikNameChecker !THIS ONLY CHECKS WS2.KIK.COM ENDPOINT! \ Will add user inputted endpoints thing \ A (probably) working Kik name checker Started as a s

insert edgy and cool name 1 Dec 17, 2022
A simple Discord Mass-Ban that's still working with Member Scraper.

Mass-Ban [!] This was made for education / you can use for revenge. Please don't skid it. [!] If you want to use it, please use member scraper before

WoahThatsHot 1 Nov 20, 2021
A Phyton script working for stream twits from twitter by tweepy to mongoDB

twitter-to-mongo A python script that uses the Tweepy library to pull Tweets with specific keywords from Twitter's Streaming API, and then stores the

null 3 Feb 3, 2022
A working selfbot for discord

React Selfbot Yes, for real ⚠ "Maintained" version: https://github.com/AquaSelfBot/AquaSelfbot ⚠ Why am I making this open source? Because can't stop

null 3 Jan 25, 2022
A working bypass for discord gc spamming

IllusionGcSpammer A working bypass for discord gc spamming Installation Run pip install pip install DiscordGcSpammer then your good to go. Usage You c

null 6 Sep 30, 2022
Automatically pulls specified repository whenever a specified file is pushed. Great for working collaboratively when you need to run something locally.

autopull Simple python tool that allows you to automatically pull from a github repository whenever a file with a specified name is uploaded installat

carreb 0 Sep 27, 2022
Instagram auto reporting tool 100% working

INSTA REPORTER Instagram auto reporting tool 100% working Description this tool is made by Guccifer Shubham (shubhushubhu99) and by using this tool yo

Guccifer Shubham 26 Dec 28, 2022
fair-test is a library to build and deploy FAIR metrics tests APIs supporting the specifications used by the FAIRMetrics working group.

☑️ FAIR test fair-test is a library to build and deploy FAIR metrics tests APIs supporting the specifications used by the FAIRMetrics working group. I

Maastricht University IDS 6 Oct 30, 2022
Official Python client for the MonkeyLearn API. Build and consume machine learning models for language processing from your Python apps.

MonkeyLearn API for Python Official Python client for the MonkeyLearn API. Build and run machine learning models for language processing from your Pyt

MonkeyLearn 157 Nov 22, 2022
PRAW, an acronym for "Python Reddit API Wrapper", is a python package that allows for simple access to Reddit's API.

PRAW: The Python Reddit API Wrapper PRAW, an acronym for "Python Reddit API Wrapper", is a Python package that allows for simple access to Reddit's AP

Python Reddit API Wrapper Development 3k Dec 29, 2022
PRAW, an acronym for "Python Reddit API Wrapper", is a python package that allows for simple access to Reddit's API.

PRAW: The Python Reddit API Wrapper PRAW, an acronym for "Python Reddit API Wrapper", is a Python package that allows for simple access to Reddit's AP

Python Reddit API Wrapper Development 3k Dec 29, 2022
alpaca-trade-api-python is a python library for the Alpaca Commission Free Trading API.

alpaca-trade-api-python is a python library for the Alpaca Commission Free Trading API. It allows rapid trading algo development easily, with support for both REST and streaming data interfaces

Alpaca 1.5k Jan 9, 2023
WhatsApp Api Python - This documentation aims to exemplify the use of Moorse Whatsapp API in Python

WhatsApp API Python ChatBot Este repositório contém uma aplicação que se utiliza

Moorse.io 3 Jan 8, 2022