Implements #23 by adding the following features:

• ee.Geometry constructors from full and short Plus Codes
• Methods to extract Plus Codes from ee.Geometry and ee.Feature objects

Hello, I was running through the tutorial notebooks (locally) and ran into the following errors:

004-Computing-Spectral-Indices-Landsat-8.ipynb, I received the following error in the final cell:

---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-16-e652de6f3203> in <module>
      2 Map.addLayer(L8.select('GNDVI'),visParamsGNDVI,'GNDVI')
      3 Map.addLayer(L8.select('NDWI'),visParamsNDWI,'NDWI')
----> 4 Map.add_colorbar(visParamsGNDVI['palette'], caption = 'GNDVI')
      5 Map.add_colorbar(visParamsNDWI['palette'], caption = 'NDWI')
      6 Map.addLayerControl()

~/anaconda3/envs/eemont/lib/python3.9/site-packages/geemap/geemap.py in add_colorbar(self, vis_params, cmap, discrete, label, orientation, position, transparent_bg, layer_name, **kwargs)
   2605 
   2606         if not isinstance(vis_params, dict):
-> 2607             raise TypeError("The vis_params must be a dictionary.")
   2608 
   2609         if orientation not in ["horizontal", "vertical"]:

TypeError: The vis_params must be a dictionary.

I had similar errors in tutorials 005 and 006 as well.

Then in 013-Time-Series-By-Region-Pandas.ipynb, I had the following code-error combo. Code block:

tsPandas = pd.melt(tsPandas,
                   id_vars = ['system:index','reducer','date'],
                   value_vars = ['GNDVI','EVI'],
                   var_name = 'Index',
                   value_name = 'Value')

Error:

---------------------------------------------------------------------------
KeyError                                  Traceback (most recent call last)
<ipython-input-11-bd846c0b8d50> in <module>
----> 1 tsPandas = pd.melt(tsPandas,
      2                    id_vars = ['system:index','reducer','date'],
      3                    value_vars = ['GNDVI','EVI'],
      4                    var_name = 'Index',
      5                    value_name = 'Value')

~/anaconda3/envs/eemont/lib/python3.9/site-packages/pandas/core/reshape/melt.py in melt(frame, id_vars, value_vars, var_name, value_name, col_level, ignore_index)
     62             missing = Index(com.flatten(id_vars)).difference(cols)
     63             if not missing.empty:
---> 64                 raise KeyError(
     65                     "The following 'id_vars' are not present "
     66                     f"in the DataFrame: {list(missing)}"

KeyError: "The following 'id_vars' are not present in the DataFrame: ['system:index']"

I ran into a similar error in tutorial 014 as well.

I had no issues running tutorials 001, 002, or 003. For tutorials 007 and 008, I wasn't able to get the maps to load, but this may be an error (or impatience) on my end, as the "Loading widget..." text did come up.

This issue relates to the ongoing JOSS Review of this package.

Hi @davemlz. First of all, eemont is a great tool. Thanks for creating it!

I'm not sure if you're familiar with open location code, but it's a geocoding tool that generates more readable coordinates (aka plus codes) from latitude/longitude (e.g. [39.777062, -105.006438] == 85FPQXGV+RC). Plus codes can also be shortened by including a nearby location (e.g. 85FPQXGV+RC == QXGV+RC Denver, CO).

It would be handy to have a tool for generating EE geometries from plus codes, and I was wondering if you think it would fit in eemont? Unfortunately there are no open web APIs that I know of to decode plus codes, so it would require adding openlocationcode as a dependency.

As far as implementation, I'm thinking it would extend ee.Geometry with a few static methods using your decorators. To handle shortened plus codes, an address could be geocoded using the same approach you use for ee.Geometry.PointFromQuery. Another feature might be the ability to retrieve a plus code from an ee.Geometry.Point. Usage might look something like below.

# Create a point from a full plus code
pt = ee.Geometry.PointFromPlusCode("85FPQXGV+RC")
# Create a point from a partial plus code and a nearby location reference
pt = ee.Geometry.PointFromPlusCode("QXGV+RC Denver, CO")
# Create a polygon from a list of plus codes
poly = ee.Geometry.PolygonFromPlusCodes(
    ["85GGXXJ2+R6"],
    ["85GVXVRV+5H"],
    ["859V2WJG+FQ"],
    ["859G2WQH+XF"]
)

# Create a point from coordinates
pt = ee.Geometry.Point([39.777062, -105.006438])
# Get the plus code of the point
pt.plus_code()

>> "85FPQXGV+RC"

If you think this would fit in eemont and it's worth adding another dependency for the feature, I'd be happy to implement and make a PR. If not, no worries! It could work as a little standalone package instead.

Repo:

• Suggest removing all .ipynb_checkpoints. You might want to add .ipynb_checkpoints to .gitignore so that these unnecessary folders won’t be pushed to GitHub. (see pull request https://github.com/davemlz/eemont/pull/21)
• Suggest adding the docs website (https://eemont.readthedocs.io) to the About section located at the upper-right corner of the repo page.
• Adding a Table of Content for README.rst, making it easier to navigate this long document.
• In README.rst, the Contribution Steps section probably belongs to a subsection of Contributing.
• You might want to consider publishing the package on conda-forge. Unless your package is available on conda-forge, other conda packages can’t list eermont as a dependency. Check out my tutorial on publishing a conda package if needed.
• The installation instructions can be improved by adding instructions such as how to update the package, install from GitHub, install from conda-forge (if it exists), etc.
• For all notebooks, !pip install earthengine-api is not needed because eemont/geemap already has earthengine-api as a dependency.
• Be cautious about using the GEE logo. Google is pretty picky about how their logos are used: https://www.google.com/permissions/logos-trademarks. I was warned before by a Google employee not to use the GEE logo. Unless you have received permission from Google to use/modify the GEE logo, you might want to design your own logo that is distinctive from the GEE logo.

Paper:

• The writing can be improved. I made some minor changes. See pull request https://github.com/davemlz/eemont/pull/21

Hey @davemlz, what do you think about adding an ee.Image.panSharpen() method to eemont? I've implemented some sharpening algorithms before in Javascript, but it would be cool to have them accessible in Python. I think Landsat 7/8 are the only platforms on GEE with panchromatic bands, so it would probably only work with those collections. If you think that would be a good feature, I'd be happy to work on it!

Thanks!

Describe the bug I am trying to use geemap,eemont and ee to derive some indices from sentinel-2 dataset from GEE python API in VSCode. the problem is that some methods and modules can not be recognized by python and I face error. I also copied the exact scripts from your tutorials, but the problem still exists.

To Reproduce import ee, eemont, geemap

   S2 = (ee.ImageCollection("COPERNICUS/S2_SR")
            .filterBounds(ROI)
            .filterDate("2020-01-01","2021-01-01")
            .preprocess()
            .maskClouds()
            .scaleAndOffset()
            )
    S2online = S2.spectralIndices(["NDVI","kNDVI","NDWI","SeLI"],online = True).median()

AttributeError: 'ImageCollection' object has no attribute 'preprocess'

Screenshots If applicable, add screenshots to help explain your problem.

Setup (please complete the following information):

• OS: Windows
• python version 3.10 & 3.8
• eemont version 0.3.5
• earthengine-api version [e.g. 0.1.32]

Additional context Add any other context about the problem here.

It is clear this is a substantial scholarly effort and represents an excellent contribution to the remote sensing/scientific computing community. I don't think the paper/repo needs any substantial changes. It installed easily in a fresh virtual envronment via pip and conda and documentation was relatively straightforward. All code examples worked for me, installation was as documented, and the QGIS functionality worked as documented. I think it will be easy for GEE practitioners to implement.

Comments:

• I do think the statement of need could be improved by stating the need first rather than stating what you've done first. I think this could be a single sentence or two clarifying that "The typical pre-processing and processing steps and long, and complex, making it challenging to move from data selection to analysis and these steps have been simplified with intelligent defaults and clearer more pythonic syntax." Just a template but I think that helps readers rapidly understand the main problem.
• The tutorials are excellent, ran correctly in colab, and I learned a few nice tricks from them. I would recommend linking them in a clear spot in the README document instead of just on the docs.

The eemont package is clearly a useful addition to the GEE API and improves the functionality and ease of use of Google Earth Engine, so congratulations to @davemlz for creating a useful package for the broader gee community.

That being said I think both the documentation and the JOSS paper could make a stronger case for the benefits of the eemont package. From the paper and documentation alone is not obvious to me why I would pick up eemont - potentially simpler code than the standard Earth Engine API? I would suggest showing the necessary code without eemont to perform the same analysis for one or two of the examples in the Features section of the documentation alongside the eemont implementation. Having a direct comparison of the (presumably longer) standard Earth Engine code required to accomplish the same task you can perform with eemont in just a few lines provides a much more compelling reason for someone who sees the package to use it.

In a similar vein, in the context of the paper, I think it would at least be beneficial to state the number of lines required to perform the process in the example given with and without eemont. This way a reader of the JOSS paper will be able to more intuitively grasp the value of using eemont. Even better, if there is some estimated "average" simplification or line-shortening accomplished by using eemont that'd be great to include, although I can understand that such a quantification may be impossible due to the diversity of remote sensing applications and processes eemont can be used for.

Regarding the "State of the field", I believe there are a handful of packages out there that provide additional functionality on-top of the standard Earth Engine API (e.g., geemap, geetools, some packages by GitHub users fitoprincipe and samapriya, just to highlight a few). I think the paper should mention more of these other tools and packages, as eemont can likely improve users' experiences with them as well (at least where calls to Earth Engine are made). Furthermore it would provide a good place to highlight the unique niche eemont fills amongst other tools and packages that build off of the standard Earth Engine API.

This comment relates to the ongoing JOSS review of this package, comment pertains to the "Documentation" and "Software paper" sections of the JOSS reviewer guidelines:

A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?

A statement of need: Does the paper have a section titled 'Statement of Need' that clearly states what problems the software is designed to solve and who the target audience is?

State of the field: Do the authors describe how this software compares to other commonly-used packages?

MOD09A1.006 Terra Surface Reflectance 8-Day Global 500m has been superseded by MOD09A1.061 Terra Surface Reflectance 8-Day Global 500m. Please add support for MODIS/061/MOD09A1.

Source: https://developers.google.com/earth-engine/datasets/catalog/MODIS_006_MOD09A1

Hi @davemlz ! I recently want to combine part of the extended methods for the ee.Image class into my personal image processing workflow. However, I found that those methods cannot be thrown into map. As a simple illustration, the following code excerpt will prompts the error of A mapped function's arguments cannot be used in client-side operations.

import ee, eemont
ee.Initialize()

def user_process(image):
    return image.scaleAndOffset()

image_coll = ee.ImageCollection("COPERNICUS/S2_SR").limit(3).map(user_process)

I definitely understand that the above stupid example can be simply resolved by:

image_coll = ee.ImageCollection("COPERNICUS/S2_SR").limit(3).scaleAndOffset()

However, won't you think it would be great if extended methods for the ee.Image class can be used as identically as standard methods for the ee.Image class? For instance, the following another stupid example would work perfectly:

import ee, eemont
ee.Initialize()

def user_process(image):
    return image.add(1).subtract(1)

image_coll = ee.ImageCollection("COPERNICUS/S2_SR").limit(3).map(user_process)

I also checked that eemont is powered by eeExtra and throwing to ee_extra.STAC.core.scaleAndOffset(image) to map would cause the same issue. As such, I am not sure if it would be too complicated to have this feature which although may be helpful at some cases.

All the best, Fei

Hello @davemlz! Thank you for your job with eemont! I was thinking if there is possible to export a time series of an area of interest to pandas. After checking the documentation I think, that is only avaiable within a Geometry.Point. by getTimeSeriesByRegion function. Am I wrong or that is not possible with eemont? I want to have a pandas dataframe like this.

I need it to do the unsupervised classification with kmeans. Do you know how can I create such a dataframe with gee? Thank you in advance! I'd be happy to work on it!

Describe the bug I am using eemont to preprocess a series of Sentinel-2 L2A images. Unfortunately, it cannot preprocess certain images as follows.

To Reproduce

import ee, eemont
ee.Initialize()
image = ee.Image("COPERNICUS/S2_SR/20220429T070619_20220429T071819_T39SYC")
image = image.preprocess()

prompts the following errors:

---------------------------------------------------------------------------
HttpError                                 Traceback (most recent call last)
File /exports/csce/datastore/geos/users/s1855106/miniconda/base/envs/gee/lib/python3.8/site-packages/ee/data.py:328, in _execute_cloud_call(call, num_retries)
    327 try:
--> 328   return call.execute(num_retries=num_retries)
    329 except googleapiclient.errors.HttpError as e:

File /exports/csce/datastore/geos/users/s1855106/miniconda/base/envs/gee/lib/python3.8/site-packages/googleapiclient/_helpers.py:134, in positional.<locals>.positional_decorator.<locals>.positional_wrapper(*args, **kwargs)
    133         logger.warning(message)
--> 134 return wrapped(*args, **kwargs)

File /exports/csce/datastore/geos/users/s1855106/miniconda/base/envs/gee/lib/python3.8/site-packages/googleapiclient/http.py:915, in HttpRequest.execute(self, http, num_retries)
    914 if resp.status >= 300:
--> 915     raise HttpError(resp, content, uri=self.uri)
    916 return self.postproc(resp, content)

HttpError: <HttpError 400 when requesting https://earthengine.googleapis.com/v1alpha/projects/fluent-imprint-343810/value:compute?prettyPrint=false&alt=json returned "Image.select: Parameter 'input' is required.". Details: "Image.select: Parameter 'input' is required.">

During handling of the above exception, another exception occurred:

EEException                               Traceback (most recent call last)
Input In [15], in <cell line: 2>()
      1 image = ee.Image("COPERNICUS/S2_SR/20220429T070619_20220429T071819_T39SYC")
----> 2 image = image.preprocess()

File /exports/csce/datastore/geos/users/s1855106/miniconda/base/envs/gee/lib/python3.8/site-packages/eemont/image.py:1235, in preprocess(self, **kwargs)
   1200 @extend(ee.image.Image)
   1201 def preprocess(self, **kwargs):
   1202     """Pre-processes the image: masks clouds and shadows, and scales and offsets the image.
   1203 
   1204     Tip
   (...)
   1233     >>> S2 = ee.ImageCollection('COPERNICUS/S2_SR').first().preprocess()
   1234     """
-> 1235     return ee_extra.QA.pipelines.preprocess(self, **kwargs)

File /exports/csce/datastore/geos/users/s1855106/miniconda/base/envs/gee/lib/python3.8/site-packages/ee_extra/QA/pipelines.py:38, in preprocess(x, **kwargs)
     35         kwargs[key] = value
     37 x = maskClouds(x, **kwargs)
---> 38 x = scaleAndOffset(x)
     40 return x

File /exports/csce/datastore/geos/users/s1855106/miniconda/base/envs/gee/lib/python3.8/site-packages/ee_extra/STAC/core.py:116, in scaleAndOffset(x)
     98 def scaleAndOffset(
     99     x: Union[ee.Image, ee.ImageCollection]
    100 ) -> Union[ee.Image, ee.ImageCollection]:
    101     """Scales and offsets bands on an Image or Image Collection.
    102 
    103     Args:
   (...)
    114         >>> scaleAndOffset(S2)
    115     """
--> 116     scaleParams = getScaleParams(x)
    117     offsetParams = getOffsetParams(x)
    119     if scaleParams is None or offsetParams is None:

File /exports/csce/datastore/geos/users/s1855106/miniconda/base/envs/gee/lib/python3.8/site-packages/ee_extra/STAC/core.py:60, in getScaleParams(x)
     44 def getScaleParams(x: Union[ee.Image, ee.ImageCollection]) -> dict:
     45     """Gets the scale parameters for each band of the image or image collection.
     46 
     47     Args:
   (...)
     58         >>> getScaleParams(S2)
     59     """
---> 60     platformDict = _get_platform_STAC(x)
     61     eeDict = _load_JSON("ee-catalog-scale.json")
     62     platforms = list(eeDict.keys())

File /exports/csce/datastore/geos/users/s1855106/miniconda/base/envs/gee/lib/python3.8/site-packages/ee_extra/STAC/utils.py:25, in _get_platform_STAC(args)
     22 eeDict = _load_JSON()
     23 platforms = list(eeDict.keys())
---> 25 ID = args.get("system:id").getInfo()
     27 plt = None
     29 for platform in platforms:

File /exports/csce/datastore/geos/users/s1855106/miniconda/base/envs/gee/lib/python3.8/site-packages/ee/computedobject.py:98, in ComputedObject.getInfo(self)
     92 def getInfo(self):
     93   """Fetch and return information about this object.
     94 
     95   Returns:
     96     The object can evaluate to anything.
     97   """
---> 98   return data.computeValue(self)

File /exports/csce/datastore/geos/users/s1855106/miniconda/base/envs/gee/lib/python3.8/site-packages/ee/data.py:738, in computeValue(obj)
    735 if workload_tag:
    736   body['workloadTag'] = workload_tag
--> 738 return _execute_cloud_call(
    739     _get_cloud_api_resource().projects().value().compute(
    740         body=body,
    741         project=_get_projects_path(),
    742         prettyPrint=False))['result']

File /exports/csce/datastore/geos/users/s1855106/miniconda/base/envs/gee/lib/python3.8/site-packages/ee/data.py:330, in _execute_cloud_call(call, num_retries)
    328   return call.execute(num_retries=num_retries)
    329 except googleapiclient.errors.HttpError as e:
--> 330   raise _translate_cloud_exception(e)

EEException: Image.select: Parameter 'input' is required.

Setup (please complete the following information):

• OS: Ubuntu 20.04.4 LTS
• python version: 3.8.13
• eemont version: 0.3.5
• earthengine-api version: 0.1.318

Additional context I did some check and found that "system:id" disappears following maskClouds() that subsequently leads to the error of scaleAndOffset(). But I have no idea on how to fix the problem. I'd be grateful if you could help!

Is this a Feature Request for eemont or for eeExtra? eemont

Is your feature request related to a problem? Please describe. I have been trying to compute spectral indices listed in espectro for S1 but have had no success- I am getting the impression that it is not yet supported?

I keep getting an error stating the platform is not supported.

Describe the solution you'd like I preprocess Sentinel-1 data using this package and then need to compute radar indices.

Hi @davemlz !

I just found a VERY weird behavior in eemont 0.2.5. Before importing the eemont, the following code excerpt works perfectly:

import ee
ee.Initialize()

image = ee.ImageCollection("COPERNICUS/S2_SR").first().set("lon",5.9053, "lat",31.6585)

poi = ee.Geometry.Point([image.getNumber("lon"), image.getNumber("lat")]) # A list of two [x,y] coordinates.
print(poi.getInfo())

poi = ee.Geometry.Point(image.getNumber("lon"), image.getNumber("lat")) # varargs.
print(poi.getInfo())

>>>
{'type': 'Point', 'coordinates': [5.9053, 31.6585]}
{'type': 'Point', 'coordinates': [5.9053, 31.6585]}

However, after importing eemont, the last two sentences will prompts the error of "EEException: Invalid argument specified for ee.Number(): <object object at 0x2b9f90c8ed80>":

import eemont

poi = ee.Geometry.Point([image.getNumber("lon"), image.getNumber("lat")]) # A list of two [x,y] coordinates.
print(poi.getInfo())

poi = ee.Geometry.Point(image.getNumber("lon"), image.getNumber("lat")) # varargs.
print(poi.getInfo())

I spent a couple of hours to identify the root of this weird, hard-to-be-spotted issue, so would like to let you and others know about it, although I am not sure if this still exists in the latest version of the eemont. Also not sure if this issue goes to other functions that accept varargs.

All the best, Fei

Hi @davemlz ! Thanks for creating the amazing eemont and making it public. I love its maskClouds method but see that it currently only supports Surface Reflectance products. However, for the s2cloudless algorithm, it seems that it can also be applied to Level 1C data, as described here. As such, would it be nice to expose the s2cloudless algorithm to both Sentinel-2 Level 1C and 2A data?

All the best, Fei

** Feature Request for eemont or ee extra** Hello Dr David Congratulations for nice contribution to making easy use of GEE. I request to add in your eemont package few mode drought indices. I did not find way to fit distribution in GEE or in GGE related packages in python.

I request to add some more indices like standardized precipitation index(SPI). The SPI calculation for any location is based on the long-term precipitation record for a desired period. This long-term record is fitted to a probability distribution, which is then transformed into a normal distribution so that the mean SPI for the location and desired period is zero (Edwards and McKee, 1997). Please find link of different drought indices https://www.droughtmanagement.info/indices/. Thanks P.K.Pandey