A python package that extends Google Earth Engine.

Overview

header

A python package that extends Google Earth Engine

PyPI conda-forge License Documentation Status Tests GitHub Sponsors Buy me a coffee Ko-fi GEE Community Twitter JOSS Black isort


GitHub: https://github.com/davemlz/eemont

Documentation: https://eemont.readthedocs.io/

PyPI: https://pypi.org/project/eemont/

Conda-forge: https://anaconda.org/conda-forge/eemont

Tutorials: https://github.com/davemlz/eemont/tree/master/tutorials

Paper: https://joss.theoj.org/papers/10.21105/joss.03168


Overview

Google Earth Engine is a cloud-based service for geospatial processing of vector and raster data. The Earth Engine platform has a JavaScript and a Python API with different methods to process geospatial objects. Google Earth Engine also provides a HUGE PETABYTE-SCALE CATALOG of raster and vector data that users can process online (e.g. Landsat Missions Image Collections, Sentinel Missions Image Collections, MODIS Products Image Collections, World Database of Protected Areas, etc.). The eemont package extends the Google Earth Engine Python API with pre-processing and processing tools for the most used satellite platforms by adding utility methods for different Earth Engine Objects that are friendly with the Python method chaining.

Google Earth Engine Community: Developer Resources

The eemont Python package can be found in the Earth Engine Community: Developer Resources together with other awesome resources such as geemap and rgee.

How does it work?

The eemont python package extends the following Earth Engine classes:

New utility methods and constructors are added to above-mentioned classes in order to create a more fluid code by being friendly with the Python method chaining. These methods are mandatory for some pre-processing and processing tasks (e.g. clouds masking, shadows masking, image scaling, spectral indices computation, etc.), and they are presented as simple functions that give researchers, students and analysts the chance to analyze data with far fewer lines of code.

Look at this simple example where a Sentinel-2 Surface Reflectance Image Collection is pre-processed and processed in just one step:

import ee, eemont
   
ee.Authenticate()
ee.Initialize()

point = ee.Geometry.PointFromQuery(
    'Cali, Colombia',
    user_agent = 'eemont-example'
) # Extended constructor

S2 = (ee.ImageCollection('COPERNICUS/S2_SR')
    .filterBounds(point)
    .closest('2020-10-15') # Extended (pre-processing)
    .maskClouds(prob = 70) # Extended (pre-processing)
    .scaleAndOffset() # Extended (pre-processing)
    .spectralIndices(['NDVI','NDWI','BAIS2'])) # Extended (processing)

And just like that, the collection was pre-processed, processed and ready to be analyzed!

Installation

Install the latest version from PyPI:

pip install eemont

Upgrade eemont by running:

pip install -U eemont

Install the latest version from conda-forge:

conda install -c conda-forge eemont

Install the latest dev version from GitHub by running:

pip install git+https://github.com/davemlz/eemont

Features

Let's see some of the main features of eemont and how simple they are compared to the GEE Python API original methods:

Overloaded Operators

The following operators are overloaded: +, -, *, /, //, %, **\ , <<, >>, &, |, <, <=, ==, !=, >, >=, -, ~. (and you can avoid the ee.Image.expression() method!)

GEE Python API eemont-style
ds = 'COPERNICUS/S2_SR'
          
S2 = (ee.ImageCollection(ds)
.first())

def scaleImage(img):
    scaling = img.select('B.*')
    x = scaling.multiply(0.0001)
    scaling = img.select(['AOT','WVP'])
    scaling = scaling.multiply(0.001)
    x = x.addBands(scaling)
    notScaling = img.select([
        'SCL',
        'TCI.*',
        'MSK.*',
        'QA.*'
    ]))
    return x.addBands(notScaling)
    
S2 = scaleImage(S2)

exp = '2.5*(N-R)/(N+(6*R)-(7.5*B)+1)'

imgDict = {
'N': S2.select('B8'),
'R': S2.select('B4'),
'B': S2.select('B2')
}

EVI = S2.expression(exp,imgDict)
ds = 'COPERNICUS/S2_SR'
          
S2 = (ee.ImageCollection(ds)
.first()
.scale())

N = S2.select('B8')
R = S2.select('B4')
B = S2.select('B2')

EVI = 2.5*(N-R)/(N+(6*R)-(7.5*B)+1)

Clouds and Shadows Masking

Masking clouds and shadows can be done using eemont with just one method: maskClouds()!

GEE Python API eemont-style
ds = 'LANDSAT/LC08/C01/T1_SR'
          
def maskCloudsShadows(img):
    c = (1 << 3)
    s = (1 << 5)
    qa = 'pixel_qa'
    qa = img.select(qa)
    cm = qa.bitwiseAnd(c).eq(0)
    sm = qa.bitwiseAnd(s).eq(0)
    mask = cm.And(sm)
    return img.updateMask(mask)
    
(ee.ImageCollection(ds)
    .map(maskCloudsShadows))
ds = 'LANDSAT/LC08/C01/T1_SR'
          
(ee.ImageCollection(ds)
    .maskClouds())

Image Scaling and Offsetting

Scaling and offsetting can also be done using eemont with just one method: scale()!

GEE Python API eemont-style
def scaleBands(img):
    scaling = img.select([
    'NDVI',
    'EVI',
    'sur.*'
    ])
    x = scaling.multiply(0.0001)
    scaling = img.select('.*th')
    scaling = scaling.multiply(0.01)
    x = x.addBands(scaling)
    notScaling = img.select([
    'DetailedQA',
    'DayOfYear',
    'SummaryQA'
    ])
    return x.addBands(notScaling)              

ds = 'MODIS/006/MOD13Q1'

(ee.ImageCollection(ds)
    .map(scaleBands))
ds = 'MODIS/006/MOD13Q1'
          
(ee.ImageCollection(ds)
    .scaleAndOffset())

Complete Preprocessing

The complete preprocessing workflow (Masking clouds and shadows, and image scaling and offsetting) can be done using eemont with just one method: preprocess()!

GEE Python API eemont-style
ds = 'LANDSAT/LC08/C01/T1_SR'
          
def maskCloudsShadows(img):
    c = (1 << 3)
    s = (1 << 5)
    qa = 'pixel_qa'
    qa = img.select(qa)
    cm = qa.bitwiseAnd(c).eq(0)
    sm = qa.bitwiseAnd(s).eq(0)
    mask = cm.And(sm)
    return img.updateMask(mask)
    
def scaleBands(img):
    scaling = img.select('B[1-7]')
    x = scaling.multiply(0.0001)
    scaling = img.select([
    'B10','B11'
    ])
    scaling = scaling.multiply(0.1)
    x = x.addBands(scaling)
    notScaling = img.select([
    'sr_aerosol',
    'pixel_qa',
    'radsat_qa'
    ])
    return x.addBands(notScaling)
    
(ee.ImageCollection(ds)
    .map(maskCloudsShadows)
    .map(scaleBands))
ds = 'LANDSAT/LC08/C01/T1_SR'
          
(ee.ImageCollection(ds)
    .preprocess())

Spectral Indices

Do you need to compute several spectral indices? Use the spectralIndices() method! The indices are taken from Awesome Spectral Indices.

GEE Python API eemont-style
ds = 'LANDSAT/LC08/C01/T1_SR'
          
def scaleImage(img):
    scaling = img.select('B[1-7]')
    x = scaling.multiply(0.0001)
    scaling = img.select(['B10','B11'])
    scaling = scaling.multiply(0.1)
    x = x.addBands(scaling)
    notScaling = img.select([
        'sr_aerosol',
        'pixel_qa',
        'radsat_qa'
    ]))
    return x.addBands(notScaling)

def addIndices(img):
    x = ['B5','B4']
    a = img.normalizedDifference(x)
    a = a.rename('NDVI')
    x = ['B5','B3']
    b = img.normalizedDifference(x)
    b = b.rename('GNDVI')
    x = ['B3','B6']
    c = img.normalizedDifference(x)
    c = b.rename('NDSI')
    return img.addBands([a,b,c])                    

(ee.ImageCollection(ds)
    .map(scaleImage)
    .map(addIndices))
ds = 'LANDSAT/LC08/C01/T1_SR'
          
(ee.ImageCollection(ds)
    .scaleAndOffset()
    .spectralIndices([
        'NDVI',
        'GNDVI',
        'NDSI'])
)

The list of available indices can be retrieved by running:

eemont.listIndices()

Information about the indices can also be checked:

indices = eemont.indices() 
indices.BAIS2.formula
indices.BAIS2.reference

Closest Image to a Specific Date

Struggling to get the closest image to a specific date? Here is the solution: the closest() method!

GEE Python API eemont-style
ds = 'COPERNICUS/S5P/OFFL/L3_NO2'
          
xy = [-76.21, 3.45]
poi = ee.Geometry.Point(xy)

date = ee.Date('2020-10-15')
date = date.millis()

def setTimeDelta(img):              
    prop = 'system:time_start'
    prop = img.get(prop)
    prop = ee.Number(prop)              
    delta = prop.subtract(date)
    delta = delta.abs()              
    return img.set(
    'dateDist',
    delta)                     

(ee.ImageCollection(ds)
    .filterBounds(poi)
    .map(setTimeDelta)
    .sort('dateDist')
    .first())
ds = 'COPERNICUS/S5P/OFFL/L3_NO2'
          
xy = [-76.21, 3.45]
poi = ee.Geometry.Point(xy)

(ee.ImageCollection(ds)
    .filterBounds(poi)
    .closest('2020-10-15'))

Time Series By Regions

The JavaScript API has a method for time series extraction (included in the ui.Chart module), but this method is missing in the Python API... so, here it is!

PD: Actually, there are two methods that you can use: getTimeSeriesByRegion() and getTimeSeriesByRegions()!

f1 = ee.Feature(ee.Geometry.Point([3.984770,48.767221]).buffer(50),{'ID':'A'})
f2 = ee.Feature(ee.Geometry.Point([4.101367,48.748076]).buffer(50),{'ID':'B'})
fc = ee.FeatureCollection([f1,f2])

S2 = (ee.ImageCollection('COPERNICUS/S2_SR')
    .filterBounds(fc)
    .filterDate('2020-01-01','2021-01-01')
    .maskClouds()
    .scaleAndOffset()
    .spectralIndices(['EVI','NDVI']))

# By Region
ts = S2.getTimeSeriesByRegion(reducer = [ee.Reducer.mean(),ee.Reducer.median()],
                                geometry = fc,
                                bands = ['EVI','NDVI'],
                                scale = 10)

# By Regions
ts = S2.getTimeSeriesByRegions(reducer = [ee.Reducer.mean(),ee.Reducer.median()],
                                collection = fc,
                                bands = ['EVI','NDVI'],
                                scale = 10)

Constructors by Queries

Don't you have the coordinates of a place? You can construct them by using queries!

usr = 'my-eemont-query-example'
   
seattle_bbox = ee.Geometry.BBoxFromQuery('Seattle',user_agent = usr)
cali_coords = ee.Feature.PointFromQuery('Cali, Colombia',user_agent = usr)
amazonas_river = ee.FeatureCollection.MultiPointFromQuery('Río Amazonas',user_agent = usr)

JavaScript Modules

This is perhaps the most important feature in eeExtra! What if you could use a JavaScript module (originally just useful for the Code Editor) in python or R? Well, wait no more for it!

JS (Code Editor) Python (eemont) R (rgee+)
var usr = 'users/sofiaermida/'
var rep = 'landsat_smw_lst:'
var fld = 'modules/'
var fle = 'Landsat_LST.js'
var pth = usr+rep+fld+fle
var mod = require(pth)
var LST = mod.collection(
    ee.Geometry.Rectangle([
        -8.91,
        40.0,
        -8.3,
        40.4
    ]),
    'L8',
    '2018-05-15',
    '2018-05-31',
    true
)
import ee, eemont
ee.Initialize()
usr = 'users/sofiaermida/'
rep = 'landsat_smw_lst:'
fld = 'modules/'
fle = 'Landsat_LST.js'
pth = usr+rep+fld+fle
ee.install(pth)
mod = ee.require(pth)
LST = mod.collection(
    ee.Geometry.Rectangle([
        -8.91,
        40.0,
        -8.3,
        40.4
    ]),
    'L8',
    '2018-05-15',
    '2018-05-31',
    True
)
library(rgee)
library(rgeeExtra)
ee_Initialize()
usr <- 'users/sofiaermida/'
rep <- 'landsat_smw_lst:'
fld <- 'modules/'
fle <- 'Landsat_LST.js'
pth <- paste0(usr,rep,fld,fle)
mod <- ee$require(pth)
LST = mod$collection(
    ee$Geometry$Rectangle(c(
        -8.91,
        40.0,
        -8.3,
        40.4
    )),
    'L8',
    '2018-05-15',
    '2018-05-31',
    TRUE
)

License

The project is licensed under the MIT license.

How to cite

Do you like using eemont and think it is useful? Share the love by citing it!:

Montero, D., (2021). eemont: A Python package that extends Google Earth Engine. 
Journal of Open Source Software, 6(62), 3168, https://doi.org/10.21105/joss.03168

If required, here is the BibTex!:

@article{Montero2021,
    doi = {10.21105/joss.03168},
    url = {https://doi.org/10.21105/joss.03168},
    year = {2021},
    publisher = {The Open Journal},
    volume = {6},
    number = {62},
    pages = {3168},
    author = {David Montero},
    title = {eemont: A Python package that extends Google Earth Engine},
    journal = {Journal of Open Source Software}
}

Artists

  • David Montero Loaiza: Lead Developer of eemont and eeExtra.
  • César Aybar: Lead Developer of rgee and eeExtra.
  • Aaron Zuspan: Plus Codes Constructors and Methods, Panchromatic Sharpening and Histogram Matching Developer.

Credits

Special thanks to Justin Braaten for featuring eemont in tutorials and the GEE Community: Developer Resources Page, to César Aybar for the formidable help with Awesome Spectral Indices and to the JOSS Review Team (Katy Barnhart, Jayaram Hariharan, Qiusheng Wu and Patrick Gray) for the comments, suggestions and contributions!

Comments
  • Implement Plus Codes

    Implement Plus Codes

    Implements #23 by adding the following features:

    • ee.Geometry constructors from full and short Plus Codes
    • Methods to extract Plus Codes from ee.Geometry and ee.Feature objects
    opened by aazuspan 10
  • [JOSS Review] Errors in tutorial notebooks

    [JOSS Review] Errors in tutorial notebooks

    Hello, I was running through the tutorial notebooks (locally) and ran into the following errors:

    004-Computing-Spectral-Indices-Landsat-8.ipynb, I received the following error in the final cell:

    ---------------------------------------------------------------------------
    TypeError                                 Traceback (most recent call last)
    <ipython-input-16-e652de6f3203> in <module>
          2 Map.addLayer(L8.select('GNDVI'),visParamsGNDVI,'GNDVI')
          3 Map.addLayer(L8.select('NDWI'),visParamsNDWI,'NDWI')
    ----> 4 Map.add_colorbar(visParamsGNDVI['palette'], caption = 'GNDVI')
          5 Map.add_colorbar(visParamsNDWI['palette'], caption = 'NDWI')
          6 Map.addLayerControl()
    
    ~/anaconda3/envs/eemont/lib/python3.9/site-packages/geemap/geemap.py in add_colorbar(self, vis_params, cmap, discrete, label, orientation, position, transparent_bg, layer_name, **kwargs)
       2605 
       2606         if not isinstance(vis_params, dict):
    -> 2607             raise TypeError("The vis_params must be a dictionary.")
       2608 
       2609         if orientation not in ["horizontal", "vertical"]:
    
    TypeError: The vis_params must be a dictionary.
    

    I had similar errors in tutorials 005 and 006 as well.


    Then in 013-Time-Series-By-Region-Pandas.ipynb, I had the following code-error combo. Code block:

    tsPandas = pd.melt(tsPandas,
                       id_vars = ['system:index','reducer','date'],
                       value_vars = ['GNDVI','EVI'],
                       var_name = 'Index',
                       value_name = 'Value')
    

    Error:

    ---------------------------------------------------------------------------
    KeyError                                  Traceback (most recent call last)
    <ipython-input-11-bd846c0b8d50> in <module>
    ----> 1 tsPandas = pd.melt(tsPandas,
          2                    id_vars = ['system:index','reducer','date'],
          3                    value_vars = ['GNDVI','EVI'],
          4                    var_name = 'Index',
          5                    value_name = 'Value')
    
    ~/anaconda3/envs/eemont/lib/python3.9/site-packages/pandas/core/reshape/melt.py in melt(frame, id_vars, value_vars, var_name, value_name, col_level, ignore_index)
         62             missing = Index(com.flatten(id_vars)).difference(cols)
         63             if not missing.empty:
    ---> 64                 raise KeyError(
         65                     "The following 'id_vars' are not present "
         66                     f"in the DataFrame: {list(missing)}"
    
    KeyError: "The following 'id_vars' are not present in the DataFrame: ['system:index']"
    

    I ran into a similar error in tutorial 014 as well.


    I had no issues running tutorials 001, 002, or 003. For tutorials 007 and 008, I wasn't able to get the maps to load, but this may be an error (or impatience) on my end, as the "Loading widget..." text did come up.

    This issue relates to the ongoing JOSS Review of this package.

    opened by elbeejay 9
  • [Possible feature?] Points and polygons from plus codes

    [Possible feature?] Points and polygons from plus codes

    Hi @davemlz. First of all, eemont is a great tool. Thanks for creating it!

    I'm not sure if you're familiar with open location code, but it's a geocoding tool that generates more readable coordinates (aka plus codes) from latitude/longitude (e.g. [39.777062, -105.006438] == 85FPQXGV+RC). Plus codes can also be shortened by including a nearby location (e.g. 85FPQXGV+RC == QXGV+RC Denver, CO).

    It would be handy to have a tool for generating EE geometries from plus codes, and I was wondering if you think it would fit in eemont? Unfortunately there are no open web APIs that I know of to decode plus codes, so it would require adding openlocationcode as a dependency.

    As far as implementation, I'm thinking it would extend ee.Geometry with a few static methods using your decorators. To handle shortened plus codes, an address could be geocoded using the same approach you use for ee.Geometry.PointFromQuery. Another feature might be the ability to retrieve a plus code from an ee.Geometry.Point. Usage might look something like below.

    # Create a point from a full plus code
    pt = ee.Geometry.PointFromPlusCode("85FPQXGV+RC")
    # Create a point from a partial plus code and a nearby location reference
    pt = ee.Geometry.PointFromPlusCode("QXGV+RC Denver, CO")
    # Create a polygon from a list of plus codes
    poly = ee.Geometry.PolygonFromPlusCodes(
        ["85GGXXJ2+R6"],
        ["85GVXVRV+5H"],
        ["859V2WJG+FQ"],
        ["859G2WQH+XF"]
    )
    
    # Create a point from coordinates
    pt = ee.Geometry.Point([39.777062, -105.006438])
    # Get the plus code of the point
    pt.plus_code()
    
    >> "85FPQXGV+RC"
    

    If you think this would fit in eemont and it's worth adding another dependency for the feature, I'd be happy to implement and make a PR. If not, no worries! It could work as a little standalone package instead.

    enhancement 
    opened by aazuspan 8
  • [JOSS Review] Comments and Suggestions

    [JOSS Review] Comments and Suggestions

    Repo:

    • Suggest removing all .ipynb_checkpoints. You might want to add .ipynb_checkpoints to .gitignore so that these unnecessary folders won’t be pushed to GitHub. (see pull request https://github.com/davemlz/eemont/pull/21)
    • Suggest adding the docs website (https://eemont.readthedocs.io) to the About section located at the upper-right corner of the repo page.
    • Adding a Table of Content for README.rst, making it easier to navigate this long document.
    • In README.rst, the Contribution Steps section probably belongs to a subsection of Contributing.
    • You might want to consider publishing the package on conda-forge. Unless your package is available on conda-forge, other conda packages can’t list eermont as a dependency. Check out my tutorial on publishing a conda package if needed.
    • The installation instructions can be improved by adding instructions such as how to update the package, install from GitHub, install from conda-forge (if it exists), etc.
    • For all notebooks, !pip install earthengine-api is not needed because eemont/geemap already has earthengine-api as a dependency.
    • Be cautious about using the GEE logo. Google is pretty picky about how their logos are used: https://www.google.com/permissions/logos-trademarks. I was warned before by a Google employee not to use the GEE logo. Unless you have received permission from Google to use/modify the GEE logo, you might want to design your own logo that is distinctive from the GEE logo.

    Paper:

    • The writing can be improved. I made some minor changes. See pull request https://github.com/davemlz/eemont/pull/21
    opened by giswqs 8
  • Panchromatic image sharpening method

    Panchromatic image sharpening method

    Hey @davemlz, what do you think about adding an ee.Image.panSharpen() method to eemont? I've implemented some sharpening algorithms before in Javascript, but it would be cool to have them accessible in Python. I think Landsat 7/8 are the only platforms on GEE with panchromatic bands, so it would probably only work with those collections. If you think that would be a good feature, I'd be happy to work on it!

    Thanks!

    enhancement 
    opened by aazuspan 6
  • some methods and modules do not work

    some methods and modules do not work

    Describe the bug I am trying to use geemap,eemont and ee to derive some indices from sentinel-2 dataset from GEE python API in VSCode. the problem is that some methods and modules can not be recognized by python and I face error. I also copied the exact scripts from your tutorials, but the problem still exists.

    To Reproduce import ee, eemont, geemap

       S2 = (ee.ImageCollection("COPERNICUS/S2_SR")
                .filterBounds(ROI)
                .filterDate("2020-01-01","2021-01-01")
                .preprocess()
                .maskClouds()
                .scaleAndOffset()
                )
        S2online = S2.spectralIndices(["NDVI","kNDVI","NDWI","SeLI"],online = True).median()
    

    AttributeError: 'ImageCollection' object has no attribute 'preprocess'

    Screenshots If applicable, add screenshots to help explain your problem.

    Setup (please complete the following information):

    • OS: Windows
    • python version 3.10 & 3.8
    • eemont version 0.3.5
    • earthengine-api version [e.g. 0.1.32]

    Additional context Add any other context about the problem here.

    bug 
    opened by navidboy70 5
  • [JOSS Review] Comments

    [JOSS Review] Comments

    It is clear this is a substantial scholarly effort and represents an excellent contribution to the remote sensing/scientific computing community. I don't think the paper/repo needs any substantial changes. It installed easily in a fresh virtual envronment via pip and conda and documentation was relatively straightforward. All code examples worked for me, installation was as documented, and the QGIS functionality worked as documented. I think it will be easy for GEE practitioners to implement.

    Comments:

    • I do think the statement of need could be improved by stating the need first rather than stating what you've done first. I think this could be a single sentence or two clarifying that "The typical pre-processing and processing steps and long, and complex, making it challenging to move from data selection to analysis and these steps have been simplified with intelligent defaults and clearer more pythonic syntax." Just a template but I think that helps readers rapidly understand the main problem.
    • The tutorials are excellent, ran correctly in colab, and I learned a few nice tricks from them. I would recommend linking them in a clear spot in the README document instead of just on the docs.
    opened by patrickcgray 5
  • [JOSS Review] Paper/documentation comments and suggestions

    [JOSS Review] Paper/documentation comments and suggestions

    The eemont package is clearly a useful addition to the GEE API and improves the functionality and ease of use of Google Earth Engine, so congratulations to @davemlz for creating a useful package for the broader gee community.

    That being said I think both the documentation and the JOSS paper could make a stronger case for the benefits of the eemont package. From the paper and documentation alone is not obvious to me why I would pick up eemont - potentially simpler code than the standard Earth Engine API? I would suggest showing the necessary code without eemont to perform the same analysis for one or two of the examples in the Features section of the documentation alongside the eemont implementation. Having a direct comparison of the (presumably longer) standard Earth Engine code required to accomplish the same task you can perform with eemont in just a few lines provides a much more compelling reason for someone who sees the package to use it.

    In a similar vein, in the context of the paper, I think it would at least be beneficial to state the number of lines required to perform the process in the example given with and without eemont. This way a reader of the JOSS paper will be able to more intuitively grasp the value of using eemont. Even better, if there is some estimated "average" simplification or line-shortening accomplished by using eemont that'd be great to include, although I can understand that such a quantification may be impossible due to the diversity of remote sensing applications and processes eemont can be used for.

    Regarding the "State of the field", I believe there are a handful of packages out there that provide additional functionality on-top of the standard Earth Engine API (e.g., geemap, geetools, some packages by GitHub users fitoprincipe and samapriya, just to highlight a few). I think the paper should mention more of these other tools and packages, as eemont can likely improve users' experiences with them as well (at least where calls to Earth Engine are made). Furthermore it would provide a good place to highlight the unique niche eemont fills amongst other tools and packages that build off of the standard Earth Engine API.


    This comment relates to the ongoing JOSS review of this package, comment pertains to the "Documentation" and "Software paper" sections of the JOSS reviewer guidelines:

    A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?

    A statement of need: Does the paper have a section titled 'Statement of Need' that clearly states what problems the software is designed to solve and who the target audience is?

    State of the field: Do the authors describe how this software compares to other commonly-used packages?

    opened by elbeejay 5
  • MODIS/061/MOD09A1

    MODIS/061/MOD09A1

    MOD09A1.006 Terra Surface Reflectance 8-Day Global 500m has been superseded by MOD09A1.061 Terra Surface Reflectance 8-Day Global 500m. Please add support for MODIS/061/MOD09A1.

    Source: https://developers.google.com/earth-engine/datasets/catalog/MODIS_006_MOD09A1

    enhancement 
    opened by masands 4
  • Extended methods for the ee.Image class cannot be thrown to map

    Extended methods for the ee.Image class cannot be thrown to map

    Hi @davemlz ! I recently want to combine part of the extended methods for the ee.Image class into my personal image processing workflow. However, I found that those methods cannot be thrown into map. As a simple illustration, the following code excerpt will prompts the error of A mapped function's arguments cannot be used in client-side operations.

    import ee, eemont
    ee.Initialize()
    
    def user_process(image):
        return image.scaleAndOffset()
    
    image_coll = ee.ImageCollection("COPERNICUS/S2_SR").limit(3).map(user_process)
    

    I definitely understand that the above stupid example can be simply resolved by:

    image_coll = ee.ImageCollection("COPERNICUS/S2_SR").limit(3).scaleAndOffset()
    

    However, won't you think it would be great if extended methods for the ee.Image class can be used as identically as standard methods for the ee.Image class? For instance, the following another stupid example would work perfectly:

    import ee, eemont
    ee.Initialize()
    
    def user_process(image):
        return image.add(1).subtract(1)
    
    image_coll = ee.ImageCollection("COPERNICUS/S2_SR").limit(3).map(user_process)
    

    I also checked that eemont is powered by eeExtra and throwing to ee_extra.STAC.core.scaleAndOffset(image) to map would cause the same issue. As such, I am not sure if it would be too complicated to have this feature which although may be helpful at some cases.

    All the best, Fei

    bug 
    opened by FeiYao-Edinburgh 4
  • toPandas

    toPandas

    Hello @davemlz! Thank you for your job with eemont! I was thinking if there is possible to export a time series of an area of interest to pandas. After checking the documentation I think, that is only avaiable within a Geometry.Point. by getTimeSeriesByRegion function. Am I wrong or that is not possible with eemont? I want to have a pandas dataframe like this. grafik

    I need it to do the unsupervised classification with kmeans. Do you know how can I create such a dataframe with gee? Thank you in advance! I'd be happy to work on it!

    opened by inter8888 4
  • "system:id" disappears following maskClouds() that subsequently leads to the error of scaleAndOffset()

    Describe the bug I am using eemont to preprocess a series of Sentinel-2 L2A images. Unfortunately, it cannot preprocess certain images as follows.

    To Reproduce

    import ee, eemont
    ee.Initialize()
    image = ee.Image("COPERNICUS/S2_SR/20220429T070619_20220429T071819_T39SYC")
    image = image.preprocess()
    

    prompts the following errors:

    ---------------------------------------------------------------------------
    HttpError                                 Traceback (most recent call last)
    File /exports/csce/datastore/geos/users/s1855106/miniconda/base/envs/gee/lib/python3.8/site-packages/ee/data.py:328, in _execute_cloud_call(call, num_retries)
        327 try:
    --> 328   return call.execute(num_retries=num_retries)
        329 except googleapiclient.errors.HttpError as e:
    
    File /exports/csce/datastore/geos/users/s1855106/miniconda/base/envs/gee/lib/python3.8/site-packages/googleapiclient/_helpers.py:134, in positional.<locals>.positional_decorator.<locals>.positional_wrapper(*args, **kwargs)
        133         logger.warning(message)
    --> 134 return wrapped(*args, **kwargs)
    
    File /exports/csce/datastore/geos/users/s1855106/miniconda/base/envs/gee/lib/python3.8/site-packages/googleapiclient/http.py:915, in HttpRequest.execute(self, http, num_retries)
        914 if resp.status >= 300:
    --> 915     raise HttpError(resp, content, uri=self.uri)
        916 return self.postproc(resp, content)
    
    HttpError: <HttpError 400 when requesting https://earthengine.googleapis.com/v1alpha/projects/fluent-imprint-343810/value:compute?prettyPrint=false&alt=json returned "Image.select: Parameter 'input' is required.". Details: "Image.select: Parameter 'input' is required.">
    
    During handling of the above exception, another exception occurred:
    
    EEException                               Traceback (most recent call last)
    Input In [15], in <cell line: 2>()
          1 image = ee.Image("COPERNICUS/S2_SR/20220429T070619_20220429T071819_T39SYC")
    ----> 2 image = image.preprocess()
    
    File /exports/csce/datastore/geos/users/s1855106/miniconda/base/envs/gee/lib/python3.8/site-packages/eemont/image.py:1235, in preprocess(self, **kwargs)
       1200 @extend(ee.image.Image)
       1201 def preprocess(self, **kwargs):
       1202     """Pre-processes the image: masks clouds and shadows, and scales and offsets the image.
       1203 
       1204     Tip
       (...)
       1233     >>> S2 = ee.ImageCollection('COPERNICUS/S2_SR').first().preprocess()
       1234     """
    -> 1235     return ee_extra.QA.pipelines.preprocess(self, **kwargs)
    
    File /exports/csce/datastore/geos/users/s1855106/miniconda/base/envs/gee/lib/python3.8/site-packages/ee_extra/QA/pipelines.py:38, in preprocess(x, **kwargs)
         35         kwargs[key] = value
         37 x = maskClouds(x, **kwargs)
    ---> 38 x = scaleAndOffset(x)
         40 return x
    
    File /exports/csce/datastore/geos/users/s1855106/miniconda/base/envs/gee/lib/python3.8/site-packages/ee_extra/STAC/core.py:116, in scaleAndOffset(x)
         98 def scaleAndOffset(
         99     x: Union[ee.Image, ee.ImageCollection]
        100 ) -> Union[ee.Image, ee.ImageCollection]:
        101     """Scales and offsets bands on an Image or Image Collection.
        102 
        103     Args:
       (...)
        114         >>> scaleAndOffset(S2)
        115     """
    --> 116     scaleParams = getScaleParams(x)
        117     offsetParams = getOffsetParams(x)
        119     if scaleParams is None or offsetParams is None:
    
    File /exports/csce/datastore/geos/users/s1855106/miniconda/base/envs/gee/lib/python3.8/site-packages/ee_extra/STAC/core.py:60, in getScaleParams(x)
         44 def getScaleParams(x: Union[ee.Image, ee.ImageCollection]) -> dict:
         45     """Gets the scale parameters for each band of the image or image collection.
         46 
         47     Args:
       (...)
         58         >>> getScaleParams(S2)
         59     """
    ---> 60     platformDict = _get_platform_STAC(x)
         61     eeDict = _load_JSON("ee-catalog-scale.json")
         62     platforms = list(eeDict.keys())
    
    File /exports/csce/datastore/geos/users/s1855106/miniconda/base/envs/gee/lib/python3.8/site-packages/ee_extra/STAC/utils.py:25, in _get_platform_STAC(args)
         22 eeDict = _load_JSON()
         23 platforms = list(eeDict.keys())
    ---> 25 ID = args.get("system:id").getInfo()
         27 plt = None
         29 for platform in platforms:
    
    File /exports/csce/datastore/geos/users/s1855106/miniconda/base/envs/gee/lib/python3.8/site-packages/ee/computedobject.py:98, in ComputedObject.getInfo(self)
         92 def getInfo(self):
         93   """Fetch and return information about this object.
         94 
         95   Returns:
         96     The object can evaluate to anything.
         97   """
    ---> 98   return data.computeValue(self)
    
    File /exports/csce/datastore/geos/users/s1855106/miniconda/base/envs/gee/lib/python3.8/site-packages/ee/data.py:738, in computeValue(obj)
        735 if workload_tag:
        736   body['workloadTag'] = workload_tag
    --> 738 return _execute_cloud_call(
        739     _get_cloud_api_resource().projects().value().compute(
        740         body=body,
        741         project=_get_projects_path(),
        742         prettyPrint=False))['result']
    
    File /exports/csce/datastore/geos/users/s1855106/miniconda/base/envs/gee/lib/python3.8/site-packages/ee/data.py:330, in _execute_cloud_call(call, num_retries)
        328   return call.execute(num_retries=num_retries)
        329 except googleapiclient.errors.HttpError as e:
    --> 330   raise _translate_cloud_exception(e)
    
    EEException: Image.select: Parameter 'input' is required.
    

    Setup (please complete the following information):

    • OS: Ubuntu 20.04.4 LTS
    • python version: 3.8.13
    • eemont version: 0.3.5
    • earthengine-api version: 0.1.318

    Additional context I did some check and found that "system:id" disappears following maskClouds() that subsequently leads to the error of scaleAndOffset(). But I have no idea on how to fix the problem. I'd be grateful if you could help!

    bug 
    opened by FeiYao-Edinburgh 1
  • Support for computing indices for Sentinel-1

    Support for computing indices for Sentinel-1

    Is this a Feature Request for eemont or for eeExtra? eemont

    Is your feature request related to a problem? Please describe. I have been trying to compute spectral indices listed in espectro for S1 but have had no success- I am getting the impression that it is not yet supported?

    I keep getting an error stating the platform is not supported.

    Describe the solution you'd like I preprocess Sentinel-1 data using this package and then need to compute radar indices.

    enhancement 
    opened by Geethen 1
  • ee.Geometry.Point does not accept varargs after importing eemont

    ee.Geometry.Point does not accept varargs after importing eemont

    Hi @davemlz !

    I just found a VERY weird behavior in eemont 0.2.5. Before importing the eemont, the following code excerpt works perfectly:

    import ee
    ee.Initialize()
    
    image = ee.ImageCollection("COPERNICUS/S2_SR").first().set("lon",5.9053, "lat",31.6585)
    
    poi = ee.Geometry.Point([image.getNumber("lon"), image.getNumber("lat")]) # A list of two [x,y] coordinates.
    print(poi.getInfo())
    
    poi = ee.Geometry.Point(image.getNumber("lon"), image.getNumber("lat")) # varargs.
    print(poi.getInfo())
    
    >>>
    {'type': 'Point', 'coordinates': [5.9053, 31.6585]}
    {'type': 'Point', 'coordinates': [5.9053, 31.6585]}
    

    However, after importing eemont, the last two sentences will prompts the error of "EEException: Invalid argument specified for ee.Number(): <object object at 0x2b9f90c8ed80>":

    import eemont
    
    poi = ee.Geometry.Point([image.getNumber("lon"), image.getNumber("lat")]) # A list of two [x,y] coordinates.
    print(poi.getInfo())
    
    poi = ee.Geometry.Point(image.getNumber("lon"), image.getNumber("lat")) # varargs.
    print(poi.getInfo())
    

    I spent a couple of hours to identify the root of this weird, hard-to-be-spotted issue, so would like to let you and others know about it, although I am not sure if this still exists in the latest version of the eemont. Also not sure if this issue goes to other functions that accept varargs.

    All the best, Fei

    bug 
    opened by FeiYao-Edinburgh 4
  • Would it be nice to expose the s2cloudless algorithm to Sentinel-2 Level 1C data?

    Would it be nice to expose the s2cloudless algorithm to Sentinel-2 Level 1C data?

    Hi @davemlz ! Thanks for creating the amazing eemont and making it public. I love its maskClouds method but see that it currently only supports Surface Reflectance products. However, for the s2cloudless algorithm, it seems that it can also be applied to Level 1C data, as described here. As such, would it be nice to expose the s2cloudless algorithm to both Sentinel-2 Level 1C and 2A data?

    All the best, Fei

    enhancement 
    opened by FeiYao-Edinburgh 1
  • Request to add drought indices in  eemont

    Request to add drought indices in eemont

    ** Feature Request for eemont or ee extra** Hello Dr David Congratulations for nice contribution to making easy use of GEE. I request to add in your eemont package few mode drought indices. I did not find way to fit distribution in GEE or in GGE related packages in python.

    I request to add some more indices like standardized precipitation index(SPI). The SPI calculation for any location is based on the long-term precipitation record for a desired period. This long-term record is fitted to a probability distribution, which is then transformed into a normal distribution so that the mean SPI for the location and desired period is zero (Edwards and McKee, 1997). Please find link of different drought indices https://www.droughtmanagement.info/indices/. Thanks P.K.Pandey

    enhancement 
    opened by pkpnerist 0
Releases(0.3.5)
Owner
David Montero Loaiza
PhD Student at UniLeipzig | Research Assistant at RSC4Earth | Creator of #eemont #awesome-spectral-indices #spectral and #spyndex
David Montero Loaiza
A ninja python package that unifies the Google Earth Engine ecosystem.

A Python package that unifies the Google Earth Engine ecosystem. EarthEngine.jl | rgee | rgee+ | eemont GitHub: https://github.com/r-earthengine/ee_ex

null 47 Dec 27, 2022
Simple CLI for Google Earth Engine Uploads

geeup: Simple CLI for Earth Engine Uploads with Selenium Support This tool came of the simple need to handle batch uploads of both image assets to col

Samapriya Roy 79 Nov 26, 2022
Get Landsat surface reflectance time-series from google earth engine

geextract Google Earth Engine data extraction tool. Quickly obtain Landsat multispectral time-series for exploratory analysis and algorithm testing On

Loïc Dutrieux 50 Dec 15, 2022
Enable geospatial data mining through Google Earth Engine in Grasshopper 3D, via its most recent Hops component.

AALU_Geo Mining This repository is produced for a masterclass at the Architectural Association Landscape Urbanism programme. Requirements Rhinoceros (

null 4 Nov 16, 2022
A Python interface between Earth Engine and xarray

eexarray A Python interface between Earth Engine and xarray Description eexarray was built to make processing gridded, mesoscale time series data quic

Aaron Zuspan 159 Dec 23, 2022
Python package for earth-observing satellite data processing

Satpy The Satpy package is a python library for reading and manipulating meteorological remote sensing data and writing it to various image and data f

PyTroll 882 Dec 27, 2022
ESMAC diags - Earth System Model Aerosol-Cloud Diagnostics Package

Earth System Model Aerosol-Cloud Diagnostics Package This Earth System Model (ES

Pacific Northwest National Laboratory 1 Jan 4, 2022
A toolbox for processing earth observation data with Python.

eo-box eobox is a Python package with a small collection of tools for working with Remote Sensing / Earth Observation data. Package Overview So far, t

null 13 Jan 6, 2022
A simple python script that, given a location and a date, uses the Nasa Earth API to show a photo taken by the Landsat 8 satellite. The script must be executed on the command-line.

What does it do? Given a location and a date, it uses the Nasa Earth API to show a photo taken by the Landsat 8 satellite. The script must be executed

Caio 42 Nov 26, 2022
Open Data Cube analyses continental scale Earth Observation data through time

Open Data Cube Core Overview The Open Data Cube Core provides an integrated gridded data analysis environment for decades of analysis ready earth obse

Open Data Cube 410 Dec 13, 2022
Digital Earth Australia notebooks and tools repository

Repository for Digital Earth Australia Jupyter Notebooks: tools and workflows for geospatial analysis with Open Data Cube and xarray

Geoscience Australia 335 Dec 24, 2022
Calculate & view the trajectory and live position of any earth-orbiting satellite

satellite-visualization A cross-platform application to calculate & view the trajectory and live position of any earth-orbiting satellite in 3D. This

Space Technology and Astronomy Cell - Open Source Society 3 Jan 8, 2022
A package to fetch sentinel 2 Satellite data from Google.

Sentinel 2 Data Fetcher Installation Create a Virtual Environment and activate it. python3 -m venv venv . venv/bin/activate Install the Package via pi

null 1 Nov 18, 2021
Google maps for Jupyter notebooks

gmaps gmaps is a plugin for including interactive Google maps in the IPython Notebook. Let's plot a heatmap of taxi pickups in San Francisco: import g

Pascal Bugnion 747 Dec 19, 2022
Replace MSFS2020's bing map to google map

English verison here 中文 免责声明 本教程提到的方法仅用于研究和学习用途。我不对使用、拓展该教程及方法所造成的任何法律责任和损失负责。 背景 微软模拟飞行2020的地景使用了Bing的卫星地图,然而卫星地图比较老旧,很多地区都是几年前的图设置直接是没有的。这种现象在全球不同地区

hesicong 272 Dec 24, 2022
Google Maps keeps old satellite imagery around for a while – this tool collects what's available for a user-specified region in the form of a GIF.

google-maps-at-88-mph The folks maintaining Google Maps regularly update the satellite imagery it serves its users, but outdated versions of the image

Noah Doersing 111 Sep 27, 2022
Location field and widget for Django. It supports Google Maps, OpenStreetMap and Mapbox

django-location-field Let users pick locations using a map widget and store its latitude and longitude. Stable version: django-location-field==2.1.0 D

Caio Ariede 481 Dec 29, 2022
Implemented a Google Maps prototype that provides the shortest route in terms of distance

Implemented a Google Maps prototype that provides the shortest route in terms of distance, the fastest route, the route with the fewest turns, and a scenic route that avoids roads when provided a source and destination. The algorithms used were DFS, BFS, A*, and Iterative Depth First Search.

null 1 Dec 26, 2021
PySAL: Python Spatial Analysis Library Meta-Package

Python Spatial Analysis Library PySAL, the Python spatial analysis library, is an open source cross-platform library for geospatial data science with

Python Spatial Analysis Library 1.1k Dec 18, 2022