Squidpy is a tool for the analysis and visualization of spatial molecular data.

TODOs:

• [x] Implement the interface to DB
• [ ] ~~Verify that the impl. is correct (by comparing with CellPhoneDB)~~
• [x] Find remaining bottlenecks in the code
• [ ] ~~Add unit tests!~~

closes #123

closes https://github.com/theislab/squidpy/issues/89 closes #161 expand anndata interaction with napari by enabling visualization of any adata.obs or adata.var_names with napari.layers

In this PR:

• add basic docs (image API still missing because init.py is empty)
• improve tox and and switch to GitHub actions
• fix some warning with docrep
• clean up docstring + some types

Types of changes

• [ ] Bug fix (non-breaking change which fixes an issue)
• [ ] Breaking change (change that would cause existing functionality to not work as before)
• [x] New feature (non-breaking change which adds functionality)

Description

Enables ImageContainer to deal with z-stacks by extending representation to y,x,z,channels

• feature extraction:
  • summary+histogram: work with 3D
  • segmentation: can only be extracted from ImageContainer with len(img.data.z) == 1
• processing:
  • smoothing: works per z-dim, can apply to several z-dims

merge after #324 (Das image integration)

TODO

• [x] add concatenation function for creating img with z-stacks
• [x] add library_id argument to processing functions
• [x] check processing functions
• [ ] texture feature extraction
• [x] how to deal with ImageContainer attributes? -> check disc below
• [ ] tests:
  • [x] fix existing tests (TODO: segmentation, features, processing, DONE: container, io)
  • [x] test for new infer_dimensions option (is z dim loaded correctly)?
  • [x] test for show on zstack image
  • z-stack tests:
  • [x] test for concat
  • [x] test for cropping different library_ids
  • [x] test for crop_corner with scale and mask_circle
  • [x] test for generate_spot_crops with different library ids
  • [x] test for apply with different library_ids
  • [x] test for apply with overwriting existing layers
  • [x] test for assigning library_id on loading image
  • [x] test process / segment with more than one library_id - make sure that non-processed library_ids are either identity or 0s.
  • [x] test img._library_id_list

Closes

closes #321

TODOs:

• [x] delete sq.im.segment_crops (dead code)
• [x] generate_crops improvements
• [x] not returning (y, x) for crops (no longer necessary)
• [x] yr + xr => ryrx - still needs renaming
• [x] ~~unify channel_dim name.~~ not possible
• [ ] ~~jpeg lazy loading~~ does not seem to be possible (maybe through dask, but still a low-prio)
• [x] add_img counter
• [x] fix running operations on empty container
• [ ] ~~too long docs for get_segmentation_features~~ keep it (but some validation/more explanation is desired)
• [x] rename to SegmentationTensorflow
• [x] segmentation directy on ImageContainer (additionally to arrays)
• [x] custom Callable SegmentationModel
• [x] sq.im.plot_segmentation (dead code)
• [x] singledispatch _load_img
• [x] laziness fix
• [x] as_array option for crop_spot_iterator

closes #220 #240 #252 #253 #254

Hi, when I plot leiden clusters with sc.pl.spatial(adata[adata.obs['sample']=="C8"],library_id="C8",color=['n_genes_by_counts','leiden']), the color of each leiden cluster became disordered. There are 3 samples (C6, C7, C8) in my anndata object and was clustered into 13 clusters. If each sample has all the 13 clusters, then the color will be right, but when the cluster number is different (such as C7 has 12 clusters, while C8 and C6 has 13 clusters, the color will be disordered. It seems that squidpy assign leiden colors by the sequence of the color, not the cluster names. I think It is the case in scanpy and squidpy.

Currently for adatas (wraps scanpy.read). @giovp I've only added support for a header in the docstring, see https://github.com/theislab/squidpy/pull/228/files#diff-88bef3a9698e1fdc3c95ecf443e3cab48c14d9360187ffdadc7c00a4b3baf1edR68

If you want to write full (e.g. more explanation), just write it in the 1 line docstring and I will implement the multi-line.

I had another implementation which did:

@_dataset
def seqfish(path: PathLike = None, **kwargs: Any) -> AnnData:
    """
    TODO.

    foo.
    """
    <you do not write the function body, it's based on the decorator>

but I think it was too complicated (i.e. the Metadata variables required to have similar name as the functions). All the images are tiffs, right? Is there a scanpy fn to download them as well?

In this PR, we have:

• more docrep (some still missing/not ideal, see https://github.com/Chilipp/docrep/issues/21)
• more constants/Enums
• added a bunch of TODOs
• deals with #166
• deals with #168

I think the ImageContainer/SegmentationModel and the image utils need to be really refactored (i.e. I think the crop functions should be working on the container itself and for high-lvl api, it can accept some array and we will create the container itself), as well as some things in the graph module (key exposure) + plotting (common arguments, see #155 )

TODOs:

• verify that there isn't mistake with docrep (look for % in the built documentation)
• deal with as much TODOs in the source as possible/feasible

closes #153 #155 #166 #168

IMPORTANT: Please search among the Pull requests before creating one.

Description

Add :func:squidpy.pl.spatial_scatter and :func:squidpy.pl.spatial_segment to statically plot spatial omics data.

TODO

• [x] add segmentation plotting capabilities #427
• [x] add some type of behaviour for alpha in continuous values. The reason for this is that if with umap you don't care to see points where the value is e.g. 0 (and you can just subset adata), in spatial you do cause you still want to see the full tissue (and this doesn't happen by proxy if you subset adata, cause it also crop the tissue). Not super sure how to handle it yet.
• [x] saving
• [x] docs
• [x] tests

closes #242 #247 #444

Types of changes

• [ ] Bug fix (non-breaking change which fixes an issue)
• [ ] Breaking change (change that would cause existing functionality to not work as before)
• [x] New feature (non-breaking change which adds functionality)

Description

Add dask support.

How has this been tested?

Many times.

Closes

closes #296 closes #327 closes #328 closes #333 closes #336 closes #337

Description

From a few discussions with @AnnaChristina, @hspitzer, and @giovp it sounds like being able to color cell segments with values of the cells is a commonly desired feature.

I've made a very quick proof on concept here which mostly deals with mapping the values correctly. Definitely still needs to be wrapped up with some nicer tooling and arguments, but it gets the data structures and a plotted image. Examples:

I am trying to use the interactive mode to annotate cell

viewer = img.interactive(adata)
shapes_layer = viewer.add_shapes(polygons, shape_type='polygon', edge_width=5,
                          edge_color='coral', face_color='royalblue')

however I encountered

Traceback (most recent call last):
  File "/Users/xiangwang/opt/anaconda3/lib/python3.9/site-packages/squidpy/pl/_interactive/_widgets.py", line 41, in <lambda>
    self.itemDoubleClicked.connect(lambda item: self._onAction((item.text(),)))
  File "/Users/xiangwang/opt/anaconda3/lib/python3.9/site-packages/squidpy/pl/_interactive/_widgets.py", line 146, in _onAction
    self._controller.add_points(vec, key=item, layer_name=name)
  File "/Users/xiangwang/opt/anaconda3/lib/python3.9/site-packages/squidpy/pl/_interactive/_controller.py", line 175, in add_points
    layer: Points = self.view.viewer.add_points(
  File "/Users/xiangwang/opt/anaconda3/lib/python3.9/site-packages/napari/components/viewer_model.py", line 5, in add_points
    import os
  File "/Users/xiangwang/opt/anaconda3/lib/python3.9/_collections_abc.py", line 1073, in append
    self.insert(len(self), value)
  File "/Users/xiangwang/opt/anaconda3/lib/python3.9/site-packages/napari/components/layerlist.py", line 135, in insert
    super().insert(index, new_layer)
  File "/Users/xiangwang/opt/anaconda3/lib/python3.9/site-packages/napari/utils/events/containers/_selectable_list.py", line 65, in insert
    super().insert(index, value)
  File "/Users/xiangwang/opt/anaconda3/lib/python3.9/site-packages/napari/utils/events/containers/_evented_list.py", line 183, in insert
    self.events.inserted(index=index, value=value)
  File "/Users/xiangwang/opt/anaconda3/lib/python3.9/site-packages/napari/utils/events/event.py", line 715, in __call__
    self._invoke_callback(cb, event if pass_event else None)
  File "/Users/xiangwang/opt/anaconda3/lib/python3.9/site-packages/napari/utils/events/event.py", line 752, in _invoke_callback
    _handle_exception(
  File "/Users/xiangwang/opt/anaconda3/lib/python3.9/site-packages/napari/utils/events/event.py", line 739, in _invoke_callback
    cb(event)
  File "/Users/xiangwang/opt/anaconda3/lib/python3.9/site-packages/napari/_qt/qt_viewer.py", line 488, in _on_add_layer_change
    self._add_layer(layer)
  File "/Users/xiangwang/opt/anaconda3/lib/python3.9/site-packages/napari/_qt/qt_viewer.py", line 498, in _add_layer
    vispy_layer = create_vispy_visual(layer)
  File "/Users/xiangwang/opt/anaconda3/lib/python3.9/site-packages/napari/_vispy/utils/visual.py", line 63, in create_vispy_visual
    return visual_class(layer)
  File "/Users/xiangwang/opt/anaconda3/lib/python3.9/site-packages/napari/_vispy/layers/points.py", line 39, in __init__
    self._on_data_change()
  File "/Users/xiangwang/opt/anaconda3/lib/python3.9/site-packages/napari/_vispy/layers/points.py", line 82, in _on_data_change
    self.reset()
  File "/Users/xiangwang/opt/anaconda3/lib/python3.9/site-packages/napari/_vispy/layers/points.py", line 181, in reset
    self._on_symbol_change()
  File "/Users/xiangwang/opt/anaconda3/lib/python3.9/site-packages/napari/_vispy/layers/points.py", line 85, in _on_symbol_change
    self.node.symbol = self.layer.symbol
  File "/Users/xiangwang/opt/anaconda3/lib/python3.9/site-packages/vispy/util/frozen.py", line 17, in __setattr__
    object.__setattr__(self, key, value)
  File "/Users/xiangwang/opt/anaconda3/lib/python3.9/site-packages/napari/_vispy/visuals/points.py", line 32, in symbol
    marker.symbol = value
  File "/Users/xiangwang/opt/anaconda3/lib/python3.9/site-packages/vispy/util/frozen.py", line 13, in __setattr__
    if self.__isfrozen and not hasattr(self, key):
  File "/Users/xiangwang/opt/anaconda3/lib/python3.9/site-packages/vispy/visuals/markers.py", line 650, in symbol
    return np.vectorize(value_to_symbol.get)(self._data['a_symbol'])
TypeError: 'NoneType' object is not subscriptable

With the plot not showing what layer I am trying to see and add shape to it

... To begin with, how we should introduce the data to start the analysis? In the tutorial, it's mentioned that loading data is like this:

img = sq.datasets.visium_hne_image()
adata = sq.datasets.visium_hne_adata()

Now, for the image, I think it is clear what is the input but for the adata, should we use "filtered_feature_bc_matrix.h5"? Because I used it and got an error.

I used this code:

adata = sq.read.visium(path, counts_file='filtered_feature_bc_matrix.h5',
               library_id=None, load_images=True, source_image_path= "../Data_Analysis/")

And then followed by:

sc.pp.normalize_total(adata)
sc.pp.log1p(adata)
sc.tl.umap(adata)
sc.tl.leiden(adata)

And finally: adata This is what I get:

AnnData object with n_obs × n_vars = 3870 × 17943
    obs: 'in_tissue', 'array_row', 'array_col', 'leiden'
    var: 'gene_ids', 'feature_types', 'genome'
    uns: 'spatial', 'log1p', 'pca', 'neighbors', 'umap', 'leiden'
    obsm: 'spatial', 'X_pca', 'X_umap'
    varm: 'PCs'
    obsp: 'distances', 'connectivities'

But when I want to it, there is an error:

sc.pl.spatial(adata,'leiden',spot_size=1)

TypeError                                 Traceback (most recent call last)
/var/folders/1m/217nxw_53mg35gpt1pm70hdh0000gn/T/ipykernel_4901/1744331117.py in <module>
----> 1 sc.pl.spatial(adata,'leiden',spot_size=1)

TypeError: spatial() takes 1 positional argument but 2 positional arguments (and 1 keyword-only argument) were given

@giovp

Description

I followed the tutorial https://scanpy-tutorials.readthedocs.io/en/latest/spatial/integration-scanorama.html and merged 4 datasets using

spatial = sc.concat(
    adata,
    label="library_id",
    uns_merge="unique",
    keys=[
        k
        for d in [
            adata[0].uns["spatial"],
            adata[1].uns["spatial"],
            adata[2].uns["spatial"],
            adata[3].uns["spatial"],
        ]
        for k, v in d.items()
    ],
    index_unique="-",
)

with adata obviousely being a list of 4 squidpy objects. The image data was also incorrect. Instead of maintaining the images of the respective plot the images were replaced by the first image. But this might also be incorrectly observed by me. Can not check any more as I have corrected that. ...

Minimal reproducible example

So I simply hope this is true, too: Just follow the tutorial and see that the plots look different for

sc.pl.spatial(
   adata_spatial [adata_spatial.obs['library_id'] == adata_spatial.obs['library_id'].unique()[0] ],
    img_key="hires",
    color=['SPINK1'],
    size=1.5,library_id=adata_spatial.obs['library_id'].unique()[0] 
)

sc.pl.spatial( adata_spatial, img_key="hires", color=['SPINK1'], size=1.5,library_id=adata_spatial.obs['library_id'].unique()[0] )

The second plots all data even though I only ask for library_id = adata_spatial.obs['library_id'].unique()[0]

Traceback

There is no traceback as it is a logical, no programming error. Of casue only if you agree that only one slide of a merged obejct should be plotted at a time.

Version

squidpy==1.2.2 scanorama==1.7.2

...

Description

Dear Squidpy Team,

Previously I have used Squidpy's 'gr.spatial_autocorr` function to calculate the Moran's I values for genes to great effect. However, recently, I find that using the function, for however many genes, causes my Python to crash. I find that it happens even when I try to reproduce the Visium H&E tutorial, and it also happens when I try and run the code using either Python 3.8 or Python 3.10.

Below is the code I ran:

import scanpy as sc
import anndata as ad
import squidpy as sq

import numpy as np
import pandas as pd

sc.logging.print_header()
print(f"squidpy=={sq.__version__}")

# load the pre-processed dataset
img = sq.datasets.visium_hne_image()
adata = sq.datasets.visium_hne_adata()

sq.gr.spatial_neighbors(adata)

genes = adata[:, adata.var.highly_variable].var_names.values[:1000]
sq.gr.spatial_autocorr(
    adata,
    mode="moran",
    genes=genes,
    n_perms=100,
    n_jobs=1,
)

The versions of all the Python modules I am using are:

scanpy==1.9.1 anndata==0.8.0 umap==0.5.3 numpy==1.23.3 scipy==1.9.1 pandas==1.5.0 scikit-learn==1.1.2 statsmodels==0.13.2 python-igraph==0.10.1 pynndescent==0.5.7
squidpy==1.2.2

And this is the following error I get:

OMP: Info #276: omp_set_nested routine deprecated, please use omp_set_max_active_levels instead.
Segmentation fault: 11

The above warning about omp_set_nested has happened in the past but has not been an issue. The segmentation fault is the new occurrence that I do not get. If you have any advice on how to fix this, I would greatly appreciate it. Thank you!

Best wishes, Axel.

Description

When following the squidpy tutorials plotting of tissue slides is using the scanpy package leading to figures that are cropped to the areas actually containing cells. If I use the squidpy plotting functions instead I see the total area, with the cells only occupying a small area. Now I would need to identify the area of interest myself. But this is inherently complicated without even having a possibility to show the coordinates in these plots. So as scanpy already is able to identify the areas of interest, could squipdy please also use this functionality? I would not mind to just find the automatically estimated cropping area coordinates in either the scanpy object or the squidpy object and having an option to using them in the plot or other functions.

The problem occurred for me when needing these coordinates for the deconvoluting the spots into single cell expression profiles.

I hope this is possible without too much actual coding.