Dear gos, I am going to visualize my own data (BED, bedgraph or bigwig), my data format is very simple. But I can not find from your document how to visualized a user's data. Thank you!

Data format of bedgraph: chr1 10468 10469 1.000 chr1 10469 10470 0.667 chr1 10470 10471 1.000

Thank you! Yang

The base branch of this PR is the experimental-data-server branch. This PR is not ready to merge.

I wanted to fix the issue related to bam with the updated Gosling.js schema that requires indexUrl as a sibling property of url for the bam file. I basically changed the following code blocks:

def _create_loader(type_: str, create_ts: Optional[CreateTileset] = None):
-    def load(url: Union[pathlib.Path, str], **kwargs):
+    def load(url: Union[pathlib.Path, str], indexUrl: str = None, **kwargs):
        """Adds resource to data_server if local file is detected."""
        fp = pathlib.Path(url)
        if fp.is_file():
            data = create_ts(fp) if create_ts else fp
            url = data_server(data)

+        # bam's index file URL
+        if indexUrl != None and pathlib.Path(indexUrl).is_file():
+            indexUrl = data_server(indexUrl)

-        return dict(type=type_, url=url, **kwargs)
+        return dict(type=type_, url=url, indexUrl=indexUrl, **kwargs)

    return load

- def bam(url: str, **kwargs):
-    return dict(type="bam", url=url, **kwargs)
+ def bam(url: str, indexUrl: str, **kwargs):
+    return dict(type="bam", url=url, indexUrl=indexUrl, **kwargs)

While the spec seems to be generated properly:

!ls ../../gos-data

import gosling as gos
from gosling.experimental.data import bam

bd = bam(
    "../../gos-data/example_higlass.bam",
    "../../gos-data/example_higlass.bai",
)

gos.Track(bd).transform_coverage(
    startField="from", endField="to"
).mark_point(outlineWidth=0).encode(
    x="from:G",
    xe="to:G",
    y="coverage:Q",
    color=gos.value("grey")
).view(xDomain={"chromosome": "1", "interval": [136750, 139450]})

{
  "tracks": [
    {
      "color": {
        "value": "grey"
      },
      "data": {
        "indexUrl": "http://localhost:21227/611af51a-c41b-4255-a76f-1b8ab402f698",
        "type": "bam",
        "url": "http://localhost:21227/c5031c79-1e2c-4269-8cce-9e668e69c3bf"
      },
      "dataTransform": [
        {
          "endField": "to",
          "startField": "from",
          "type": "coverage"
        }
      ],
      "height": 180,
      "mark": "point",
      "style": {
        "outlineWidth": 0
      },
      "width": 800,
      "x": {
        "field": "from",
        "type": "genomic"
      },
      "xe": {
        "field": "to",
        "type": "genomic"
      },
      "y": {
        "field": "coverage",
        "type": "quantitative"
      }
    }
  ],
  "xDomain": {
    "chromosome": "1",
    "interval": [
      136750,
      139450
    ]
  }
}

I get the following CORS errors.

From the jupyter notebook:

From the browser:

Do you have any thoughts on this issue?

I used the following two files:

• https://aveit.s3.amazonaws.com/higlass/bam/example_higlass.bam.bai
• https://aveit.s3.amazonaws.com/higlass/bam/example_higlass.bam

Since both Track and View are important concepts in Gosling, I think we can support explicit functions for both in the python package as well. In this regard, I was wondering whether it makes sense to rename Chart to View since Chart() looks to construct Gosling's views.

// from
gos.Chart(
    title='Overview and Detail Views',
    arrangement='horizontal',
    views=[overview, detail_view]
)

// to
gos.View(
    title='Overview and Detail Views',
    arrangement='horizontal',
    views=[overview, detail_view] // two children views that are nested in this parent view
)

So, at the root level, users define a View, and it can contain either multiple Views or Tracks.

Is there a way to provide a constant value to y-axis in the bar chart (and others as well)? For example, in the gallery example of bar chart, is it possible to a set constant value 0.0005 to y instead of reading specifying it from the data object?

Potentially related to #80.

I am unable to reproduce (as the error has now gone away), but I ran into a CORs issue with Chrome 95 the first time I tried to load a local CSV. I switched to Firefox and the visualization worked fine. When I navigated back to Chrome, the CORs issue went away and hasn't come back.

I will add more information here as I learn more, but I wanted to file something to note the issue is known.

Updates gos for changes in https://github.com/gosling-lang/gosling.js/pull/533

Once the changes in gosling.js are merged and there is a new release, we will replace the schema URL in tool/generate_schema_wrappers.py. Currently a URL is used tracking the PR in gosling.js

Now that gosling works without jupyter extension (no install), I would like to remove the Jupyter extension-based renderer for the moment. Currently this just adds a lot of code complexity and limited "features" that we aren't using to the repo.

We can always add these back in the future (either in this repo or another) if there is a motivating use case. Removed features:

• Offline rendering (HTMLRenderer loads assets from https://unpkg.com)
• Embeded PNG in jupyter notebooks. The default HTMLRender cannot save a static image to the jupyter. I think the benefit of a no-install renderer out-weighs the feature of embedding a JPEG in notebooks.
• GoslingWidget - the widget has no immediate utility. If we want to explore a deeper integration with gosling.js API we can in the future when the use case is better understood.

It might be nice to add some convenience exports for reusable example datasets for gosling. This could remove some of the boilerplate in the examples for:

import gosling as gos
- from gosling.data import multivec
+ from gosling.datasets import cistrome_multivec


- data = multivec(
-    url="https://server.gosling-lang.org/api/v1/tileset_info/?d=cistrome-multivec",
-    row="sample",
-    column="position",
-    value="peak",
-    categories=["sample 1", "sample 2", "sample 3", "sample 4"],
-    binSize=5,
- )
- base_track = gos.Track(data, width=800, height=100)

+ base_track = gos.Track(cistrome_multivec, width=800, height=100)

Do you by any chance know if a bigwig file can be used directly without any pre-aggregation using vector()? It looked so according to the HiGlass docs, but when I tested with a local bigwig file, it did not show any visual elements in the rendered track, having some error message.

v = vector(
    "../data/Astrocytes-insertions_bin100_RIPnorm.bw",
    value="value",
    column="position",
    binSize=4
)

gos.Track(v).mark_bar(outlineWidth=0).encode(
    x="position:G",
    y=gos.Channel("value:Q"),
    color=gos.Channel("value:Q", legend=True),
    stroke=gos.value("black"),
    strokeWidth=gos.value(0.3),
).view() # Uncaught (in promise) TypeError: Cannot read property 'chromsizes' of null

I wonder if chromsizes should be somehow provided before storing a bigwig file to the server.

If a coordSystem is specified for the bigWig, but no chromsizes are found on the server, the import will fail.

Originally posted by @sehilyi in https://github.com/manzt/gos/pull/39#discussion_r695857459

Changed chromosome names in the examples to use chr prefix (e.g., 1 --> chr1) since the next version of Gosling will let ones use exact chromosome names (https://github.com/gosling-lang/gosling.js/pull/796).

We have the .save() function explained on the doc under API Reference, but it is quite hidden to users and does not provide detailed explanations and examples. I think we need to add a dedicated page or section that describes ways to export Gos plots (e.g., html, json). Maybe we can add a section near the Local Data?:

Related tweet: https://twitter.com/sminot/status/1559746056669126657?s=21&t=yCLH8vli7kbjDf8foZhy5g

Hi,

First, thanks for the great work! I am really liking the layout of Gosling so far!

I am having an issue plotting some data. I have a microbial assembly with two chromosomes and a tsv file with three columns: chromosome, position, value (the file does not have a header). I want to plot the values along each chromosome, so have code like this:

data = gos.csv(
    "../data/06_coverage/23b_coverage.bed",
    separator="\t",
    headerNames=["chrom", "position", "value"],
    chromosomeField="chrom",
    genomicFields=["position"]
)

gos.Track(data).mark_bar().encode(
    x=gos.X("position:G"),
    y=gos.Y("value:Q", axis="left"),
).view(
    assembly=[
        ("NC_012345.1", 1_234_567),
        ("NC_012346.1", 45000)
    ]
)

When I run the plot, I just get a blank canvas with the chromosome labels/positions along the top, but no data is showing up. I am not sure what I am missing here. Does the file have to have two coordinates for each feature?

Thanks in advance for the help! I hope this is not something obvious that I am missing.

Keep up the great work! Excited to see this tool continue to develop!

In the gosling website, there is an example of setting up visibility based on zoomLevel instead of width|height. If I try to set the visibility by passing zoomLevel to the measure parameter, I get schema validation error.

seq_info = seq_track.mark_text(
).encode(
    text=gos.Text("seq:N"),
    color=gos.value("black"),
    stroke=gos.value("white"),
    strokeWidth=gos.value(3),
    x=gos.X("start:G", linkingId='detail-1'),
    xe="end:G",
    row=gos.value(80)
).visibility_le(
    target="mark",
    measure="zoomLevel",
    threshold=10000,
    transitionPadding=5,
)

The code above gives:

SchemaValidationError: Invalid specification

        gosling.schema.core.VisibilityCondition->0->measure, validating 'enum'

        'zoomLevel' is not one of ['width', 'height']

However, I can pass width as a measure which passes the schema validation, then manually edit the json to zoomLevel and it will give me the exact behavior I am expecting i.e. mark appears when the zoomLevel is at < 10,000 bp. It would be nice to be able to use the function to set measure as zoomLevel instead of setting it up manually.

Just learned of gosling today via twitter. I used to work at heavy.ai on large-scale dataviz and have been pretty close to the Vega/VegaLite ecosystem for a few years, including collaborations with that team.

First of all, wanted to say that this is a FANTASTIC project - kudos to everyone involved for a truly next-gen way of taking genomics data and turning it into a comprehensive set of visualizations. I've always felt that this area has been underserved and you've all done a tremendous job!

I work at sneller.io now - which may be of some interest. We allow for SQL on very large-scale JSON collections - see here. It may be of interest if you have data collections of this sort.

Next - One of the projects I worked on in the past is IbisData - This allows you to use SQL databases/DWs as backends in a pythonic data workflow without needing SQL directly. It might be interesting to explore how to back gosling/gos with Ibis so you can put large genomic data sets in central storage and use Ibis to only pull in the data needed on demand.

Hope this helps - again, fantastic work and will stay close to this!

I think we can at least make the width and height optional so that users do not have to specify them when users use responsiveSize. But, yes, it is somewhat challenging to provide non-constant optimal width and height to individual tracks when the track/view structure becomes complicated.

Also, I was thinking that the appearance of some visual elements can be adaptive by default depending on the assigned size (e.g., hiding a color legend when the track height is too narrow). Another thing that we can figure out to make the responsiveSize more useful.

Originally posted by @sehilyi in https://github.com/gosling-lang/gos/issues/85#issuecomment-1021380417

There appear to be temperamental issues with loading the JS to power the gos visualization in Jupyter Notebooks. It is difficult to debug due to the browser cache and (hidden) state of saved jupyter notebooks as well.

Steps to reproduce:

• open a new blank notebook (jupyter notebook) (Python 3.9, Chrome 95)
• load gos and execute a cell which renders a gos.View
• re-run the same cell

https://user-images.githubusercontent.com/24403730/140839455-c8a2337e-5568-4c50-86cd-1a43e9258a87.mov

However, if a visualization is executed in a different cell, the expected rendering behavior is restored.

https://user-images.githubusercontent.com/24403730/140839595-4a54d895-8ebf-4b62-b383-7f9cac1fa7f3.mov

My best guess is that this is related to the custom JS loading code in gosling/display.py, but this is a requirement currently since some of our JS is incompatible with the global requirejs in Jupyter Notebooks.

TL;DR - If you are having trouble with rendering, try executing a different cell and re-running the previous cell. This step only needs to be done once, and will likely occur naturally during a typical workflow. Hopefully we will have a better solution soon.