Color correction plugin for rasterio

Mapbox

Last update: Nov 15, 2022

Overview

rio-color

A rasterio plugin for applying basic color-oriented image operations to geospatial rasters.

Goals

No heavy dependencies: rio-color is purposefully limited in scope to remain lightweight
Use the image structure: By iterating over the internal blocks of the input image, we keep memory usage low and predictable while gaining the ability to
Use multiple cores: thanks to rio-mucho
Retain all the GeoTIFF info and TIFF structure: nothing is lost. A GeoTIFF input → GeoTIFF output with the same georeferencing, internal tiling, compression, nodata values, etc.
Efficient colorspace conversions: the intensive math is written in highly optimized C functions and for use with scalars and numpy arrays.
CLI and Python module: accessing the functionality as a python module that can act on in-memory numpy arrays opens up new opportunities for composing this with other array operations without using intermediate files.

Operations

Gamma adjustment adjusts RGB values according to a power law, effectively brightening or darkening the midtones. It can be very effective in satellite imagery for reducing atmospheric haze in the blue and green bands.

Sigmoidal contrast adjustment can alter the contrast and brightness of an image in a way that matches human's non-linear visual perception. It works well to increase contrast without blowing out the very dark shadows or already-bright parts of the image.

Saturation can be thought of as the "colorfulness" of a pixel. Highly saturated colors are intense and almost cartoon-like, low saturation is more muted, closer to black and white. You can adjust saturation independently of brightness and hue but the data must be transformed into a different color space.

Examples

Sigmoidal

Contrast

Bias

Gamma

Red

Green

Blue

Saturation

Combinations of operations

Install

We highly recommend installing in a virtualenv. Once activated,

pip install -U pip
pip install rio-color

Or if you want to install from source

git checkout https://github.com/mapbox/rio-color.git
cd rio-color
pip install -U pip
pip install -r requirements-dev.txt
pip install -e .

Python API

`rio_color.operations`

The following functions accept and return numpy ndarrays. The arrays are assumed to be scaled 0 to 1. In some cases, the input array is assumed to be in the RGB colorspace.

All arrays use rasterio ordering with the shape as (bands, columns, rows). Be aware that other image processing software may use the (columns, rows, bands) axis order.

sigmoidal(arr, contrast, bias)
gamma(arr, g)
saturation(rgb, proportion)
simple_atmo(rgb, haze, contrast, bias)

The rio_color.operations.parse_operations function takes an operations string and returns a list of python functions which can be applied to an array.

ops = "gamma b 1.85, gamma rg 1.95, sigmoidal rgb 35 0.13, saturation 1.15"

assert arr.shape[0] == 3
assert arr.min() >= 0
assert arr.max() <= 1

for func in parse_operations(ops):
    arr = func(arr)

This provides a tiny domain specific language (DSL) to allow you to compose ordered chains of image manipulations using the above operations. For more information on operation strings, see the rio color command line help.

`rio_color.colorspace`

The colorspace module provides functions for converting scalars and numpy arrays between different colorspaces.

>>> from rio_color.colorspace import ColorSpace as cs  # enum defining available color spaces
>>> from rio_color.colorspace import convert, convert_arr
>>> convert_arr(array, src=cs.rgb, dst=cs.lch) # for arrays
...
>>> convert(r, g, b, src=cs.rgb, dst=cs.lch)  # for scalars
...
>>> dict(cs.__members__)  # can convert to/from any of these color spaces
{
 'rgb': <ColorSpace.rgb: 0>,
 'xyz': <ColorSpace.xyz: 1>,
 'lab': <ColorSpace.lab: 2>,
 'lch': <ColorSpace.lch: 3>,
 'luv': <ColorSpace.luv: 4>
 }

Command Line Interface

Rio color provides two command line interfaces:

`rio color`

A general-purpose color correction tool to perform gamma, contrast and saturation adjustments.

The advantages over Imagemagick convert: rio color is geo-aware, retains the profile of the source image, iterates efficiently over interal tiles and can use multiple cores.

Usage: rio color [OPTIONS] SRC_PATH DST_PATH OPERATIONS...

  Color correction

  Operations will be applied to the src image in the specified order.

  Available OPERATIONS include:

      "gamma BANDS VALUE"
          Applies a gamma curve, brightening or darkening midtones.
          VALUE > 1 brightens the image.

      "sigmoidal BANDS CONTRAST BIAS"
          Adjusts the contrast and brightness of midtones.
          BIAS > 0.5 darkens the image.

      "saturation PROPORTION"
          Controls the saturation in LCH color space.
          PROPORTION = 0 results in a grayscale image
          PROPORTION = 1 results in an identical image
          PROPORTION = 2 is likely way too saturated

  BANDS are specified as a single arg, no delimiters

      `123` or `RGB` or `rgb` are all equivalent

  Example:

      rio color -d uint8 -j 4 input.tif output.tif \
          gamma 3 0.95, sigmoidal rgb 35 0.13


Options:
  -j, --jobs INTEGER              Number of jobs to run simultaneously, Use -1
                                  for all cores, default: 1
  -d, --out-dtype [uint8|uint16]  Integer data type for output data, default:
                                  same as input
  --co NAME=VALUE                 Driver specific creation options.See the
                                  documentation for the selected output driver
                                  for more information.
  --help                          Show this message and exit.

Example:

$ rio color -d uint8 -j 4 rgb.tif test.tif \
    gamma G 1.85 gamma B 1.95 sigmoidal RGB 35 0.13 saturation 1.15

`rio atmos`

Provides a higher-level tool for general atmospheric correction of satellite imagery using a proven set of operations to adjust for haze.

Usage: rio atmos [OPTIONS] SRC_PATH DST_PATH

  Atmospheric correction

Options:
  -a, --atmo FLOAT                How much to dampen cool colors, thus cutting
                                  through haze. 0..1 (0 is none), default:
                                  0.03.
  -c, --contrast FLOAT            Contrast factor to apply to the scene.
                                  -infinity..infinity(0 is none), default: 10.
  -b, --bias FLOAT                Skew (brighten/darken) the output. Lower
                                  values make it brighter. 0..1 (0.5 is none),
                                  default: 0.15
  -d, --out-dtype [uint8|uint16]  Integer data type for output data, default:
                                  same as input
  --as-color                      Prints the equivalent rio color command to
                                  stdout.Does NOT run either command, SRC_PATH
                                  will not be created
  -j, --jobs INTEGER              Number of jobs to run simultaneously, Use -1
                                  for all cores, default: 1
  --co NAME=VALUE                 Driver specific creation options.See the
                                  documentation for the selected output driver
                                  for more information.
  --help                          Show this message and exit.

Comments

Compatibility with existing color forumulas
The color formulas for each source are defined in a color file containing the command line arguments to imagemagick convert

We need to translate these to rio color arguments. Two main options

~~Have a rio color --compatability option that will read convert args and interpret them on the fly~~

Script/manually adjust all the color formulas in pxm-sources. Create an alternate color.rio file that mirrors the intent of color but with rio color syntax

Questions:

what do we do with convert options that don't translate well to rio color?

how are we going to visually test to make sure rio color output matches convert's?
opened by perrygeo 16

Performance of saturation

Saturation operator needs to be faster. It's really slow compared to imagemagick's modulate.

$ rio info $orig | jq .shape
[
  8192,
  8192
]
$ time rio color $orig f416_rio_lch.tif "saturation 130"

real    1m10.490s
$ time convert $orig -modulate 100,130 f416_magick.tif

real    0m6.954s

Investigating...

opened by perrygeo 11

Add numpy as setup requirement
We got a downstream bug report of rio-color's install failing because the user didn't have numpy installed when he tried to install rio-color, so the headers couldn't be found.

This is a bit of a chicken-and-egg issue, because the numpy headers are needed to build, but you can't find them before installing numpy. This PR uses a technique I learned from pybind11 (though I can't find now exactly where in the documentation it was referenced), which uses a class to postpone finding numpy's headers until numpy is installed.

Tested locally (using conda just to create a virtual env):

Setup:

> conda create -n minimal-test -c conda-forge python -y > source activate minimal-test

Before:

> python setup.py bdist_wheel Numpy and its headers are required to run setup(). Exiting.

After, creating the wheel works.
opened by kylebarron 10
rio color and rio atmos
As discussed in #5, we need a low level tool to apply color corrections (rio color) in addition to a high-level atmospheric correction command (rio atmos).

Related to #4 Resolves #5 Also resolves #6 - adding a saturation operator in LCH color space

See the README in this branch for usage.

This is ready but for the travis thing

TODO

[x] compare results to imagemagick convert

[x] benchmarking

[x] document examples

[x] unit tests for CLI and Python API (100% coverage!)

[x] clean up python api into modules, document in readme

[x] travis
opened by perrygeo 9
Channel selectors
@virginiayung and I want a syntax and mechanism for rio color to apply given operations only to selected channels. For comparison, ImageMagick’s convert works like this:

convert input.tif -channel G -gamma 1.5 -channel B -gamma 0.5 -sigmoidal-contrast 3,50% -channel RGB output.tif

Which is something like this in pseudocode:

G = gamma(G, 1.5); B = gamma(B, 0.5); B = sigmoid(B, 3, 50); write([R, G, B], output.tif);

Implicitly, R is passed through. It amounts to a simple system for creating blocks of operations.

I don’t love the convert syntax (for example, it’s easy to forget to switch back to RGB at the end) but it seems to mostly work. I wonder about something maybe more like this:

rio color 'G(gamma 1.6) B(gamma 0.5, sigmoid 3 50)' input.tif output.tif

Or maybe full s-expressions, or…?
resolved-by-pr
opened by celoyd 8
Optimized saturation operator
Resolves #21

The previous saturation operation went something like:

swap axes to image order

convert colorspace using skimage

swap axes to rasterio order

apply saturation mutliplier

swap axes to image order

convert colorspace using skimage

swap axes to rasterio order

This PR provides a cython rio_color.colorspace module which provides c functions to do the math directly, then some python wrappers to work efficiently against ndarrays.

See #21 for details of testing and benchmarks.
opened by perrygeo 7
Example wheel builds using cibuildwheel

:wave:

In rio-tiler we're considering adding a dependency on rio-color and I saw #65. I've had a lot of success using the https://github.com/joerick/cibuildwheel project, which uses a CI platform of your choice to simplify the wheel-building process (especially for simple wheels that don't have any complex dependencies). I thought it might help to move the conversation forward to share a working example. This example uses Github Actions but it should be possible to use Travis CI instead if you wish.

This ran on my fork and created the following set of wheels: artifact.zip. (I'm not totally sure why it didn't build wheels for 3.9; a similar script did build 3.9 wheels for me).

opened by kylebarron 6

How to rescale the image after rio-color operations?

According to documentation, rio-color operations only take rasters of dimensions (3, x, y) for processing, in 0 to 1 range for pixel values.

I'm facing some issues getting the image into the same state as doing rio color --co photometric=rgb stack.tiff landsat8_color.tiff sigmoidal RGB 20 0.2 -j 1

The code I've used is as follows.

# initializing and reading the bands

import rasterio as rio
import numpy as np
from rio_color import operations, utils

R10 = '/Users/shivashis.ext/felicette-data/LC81390462020136'
b4 = rio.open(R10+'/LC81390462020136-b4.tiff')
b3 = rio.open(R10+'/LC81390462020136-b3.tiff')
b2 = rio.open(R10+'/LC81390462020136-b2.tiff')
# getting the bands in the range of [0..1]
r = b4.read(1)
g = b3.read(1)
b = b2.read(1)
norm_r = np.linalg.norm(r)
norm_g = np.linalg.norm(g)
norm_b = np.linalg.norm(b)
r = r / norm_r
g = g / norm_g
b = b / norm_b

# making and processing image
img = np.array([r,g,b])

img = operations.sigmoidal(img, 20, 0.2)

# from matplotlib import pyplot as plt
norm_r = img[0]
norm_r = utils.scale_dtype(img[0], np.uint16)# np.interp(norm_r, (norm_r.min(), norm_r.max()), (0, 65535))
norm_g = img[1]
norm_g = utils.scale_dtype(img[1], np.uint16)#np.interp(norm_g, (norm_g.min(), norm_g.max()), (0, 65535))
norm_b = img[2]
norm_b = utils.scale_dtype(img[2], np.uint16)#np.interp(norm_b, (norm_b.min(), norm_b.max()), (0, 65535))

# writing back to file

out_tiff = R10 + '/stack_prog_color_2.tiff'
with rio.open(out_tiff,'w',driver='Gtiff', width=b4.width, height=b4.height, 
              count=3,crs=b4.crs,transform=b4.transform, dtype=np.uint16, photometric="RGB") as rgb:
    rgb.write(norm_r.astype(np.uint16),1) 
    rgb.write(norm_g.astype(np.uint16),2) 
    rgb.write(norm_b.astype(np.uint16),3) 
    rgb.close()

As one can see, I've used both in-house scale_dtype and Numpy's linear interpolation to scale the array back, without success.

Also, I planned to save the numpy response of sigmoid function as a pickle file, and debug keeping it as reference, but since the job is parallel by rio-mucho, it got too complex in very short time.

I am almost sure that I'm scaling the image back wrong, because size of the the output tiffs with a) command line and b) Python API are same. (Both use np.uint16 to store data)

Please let me also know, if any other detail is required to understand/debug/help this issue.

Thank you for your time in advance!

opened by plant99 6

Saturation operator
One of the convert flags we want to replace is the second argument to -modulate, which adjusts saturation. (Pulled out of #5.) Braindumping some options, in order of increasing hackiness:

The conceptually straightest path here is to convert RGB to a radial colorspace that has a dedicated saturation channel, multiply that channel, and convert it back. Our default radial colorspace is LCH, because it combines a nicely grounded definition with a good approximation of perceptual uniformity. Unfortunately, the conversion is relatively complex and resource-hungry: skimage, for example, goes RGB → Lab → LCH and back.

We could use HSV, which has a simpler definition.* The :-1: I see is setting a precedent of using HSV, which isn’t our best practice. I think it’s wiser to keep rio color in a “gold standard” role instead of a “workable approximation” role – for that, you might as well write your own functions. *Though possibly not faster, given that it has a 6-part case, as opposed to the trig functions of the LCH route. I don’t have good intuition about numpy optimization.

We could work out an equivalent (ish) saturation operator that works directly on RGB pixels. I see this as even more of a hack: by the time it was well defined, it’d be doing 80% of the work of converting to and from a radial colorspace anyway. On the :+1:, it would probably be very fast and do a fairly good job of matching any other saturation operator.

How do you see this, @perrygeo?
resolved-by-pr
opened by celoyd 5
Update numpy dependency to reflect ABI change

When I ran pip3 install rio-color on a Debian system that had numpy 1.19.5 and Python 3.9 installed, it seems to have downloaded a rio-color wheel that was built against a newer version of numpy that broke backwards ABI compatibility. That resulted in a "ValueError: numpy.ndarray size changed, may indicate binary incompatibility. Expected 88 from C header, got 80 from PyObject", which led me to this page: https://stackoverflow.com/questions/66060487/valueerror-numpy-ndarray-size-changed-may-indicate-binary-incompatibility-exp

I know this isn't the most ideal change to make, but I figure it might help people in a similar situation in the future?

opened by aripollak 4
Manylinux wheels

As part of the release process, we might consider building manylinux wheels to allow easy installation on linux without a build toolchain. I think that, because colorspace.so does not link to any external shared objects, that this should be fairly lightweight and rely on the libgdal that is linked to Rasterio.

@sgillies do we want to include this in frs-wheel-builds?

cc @vincentsarago
enhancement

opened by perrygeo 4