An Official Repo of CVPR '20 "MSeg: A Composite Dataset for Multi-Domain Segmentation"

When I try to run the verification script

python -u ../tests/verify_all_dataset_paths_exist.py

I get the following error:

Finding matches for cityscapes-19-relabeled...
Finding matches for ade20k-150-relabeled...
Finding matches for bdd-relabeled...
Finding matches for coco-panoptic-133-relabeled...
Finding matches for idd-39-relabeled...
Finding matches for sunrgbd-37-relabeled...
Finding matches for mapillary-public65-relabeled...
Writing visual sanity checks for ade20k-151-inst...
Writing visual sanity checks for ade20k-151...
On 0 of ade20k-151
Traceback (most recent call last):
  File "../tests/verify_all_dataset_paths_exist.py", line 250, in <module>
    visual_sanitychecks()
  File "../tests/verify_all_dataset_paths_exist.py", line 94, in visual_sanitychecks
    id_to_class_name_map=id_to_classname_map
  File "/<path>/mseg-api/mseg/utils/mask_utils_detectron2.py", line 477, in overlay_instances
    class_mode_idx = get_most_populous_class(segment_mask, label_map)
  File "/<path>/mseg-api/mseg/utils/mask_utils.py", line 992, in get_most_populous_class
    class_mode_idx = get_np_mode(class_indices)
  File "/<path>/mseg-api/mseg/utils/mask_utils.py", line 931, in get_np_mode
    return np.argmax(counts)
  File "<__array_function__ internals>", line 6, in argmax
  File "/<path>/anaconda3/lib/python3.7/site-packages/numpy/core/fromnumeric.py", line 1186, in argmax
    return _wrapfunc(a, 'argmax', axis=axis, out=out)
  File "/<path>/anaconda3/lib/python3.7/site-packages/numpy/core/fromnumeric.py", line 61, in _wrapfunc
    return bound(*args, **kwds)
ValueError: attempt to get argmax of an empty sequence

Additonally adding 'ade20k-151', 'ade20k-150', and 'ade20k-150-relabeled' in line 58 to skip them, leads to the same error thrown for the BDD dataset. Thus, the error doesn't seem to be dataset specific.

I'm running python 3.7.6 and numpy 1.18.2.

Hello I have some problem with remap and relabelling of COCOPanoptic dataset. Firstly I needed to download it separately due to some script problem.

First here is error from remap log, but the script completed 100% train and validation 0%.

Second error in relabeling log. it also says it has completed 100% train and validation 0%.

After this I tried to run the unit test, and it crashes at cocopanoptic dataset.

Hello, I want to know the datasets that were not used in MSeg and the reasons for not including them. However, I can not find supplement through google. Can you give me the link of supplement?

Hi, I'm having a hard time finding the supplement material described in the paper. In a previous issue you mentioned that it will get updated to arxiv but I can't find it there. Can you let me know where should I search for this document?

Thanks!

Hello, I want to re-add the crosswalk class to the MSeg dataset. I have seen that mapillary vistas dataset contains the crosswalk class, and that it is relabeled to the road class. Also, I know that other datasets (such as cityscapes) contains crosswalks but they are not labeled and are considered plain road. My question is: How would you add the crosswalk class back into the dataset? . I'm thinking of 3 options, and I would appreciate your insight:

1. Use mseg-turk to label each dataset which does not contain crosswalks. This might be the best and most straight-forward option, but this can prove labor-intensive and costly
2. For the datasets that do not contain crosswalks, ignore the road class. For example, ignore all possible mislabels of crosswalks in the cityscapes dataset. I expect this to lower the confidence for the road class, as there will be considerably less data.
3. Simply add the crosswalk class and revert the relabel action applied on mapillary. I woudn't try this, because it might be problematic for the model to learn crosswalks only sometimes (in case of mapillary frames) while ignoring them alltogether in other cases(e.g. cityscapes frames) - which may also create confusion between the road class and crosswalk class.

Thanks!

I noticed that for camvid the test set seems to be listed (ie the TSV file camvid-11/list/test.txt exists).

However after downloading and remapping, the semseg11 segmentation mask is missing (eg semseg11/0001TP_008550_L.png is missing), while the original 32 class segmentation mask does exists (Labels32-RGB/0001TP_008550_L.png).

How can I remap the test set as well?

In the WildDash train file list, each entry is repeated 16 times. Example repetition: https://github.com/mseg-dataset/mseg-api/blob/master/mseg/dataset_lists/wilddash-19/list/train.txt#L1 https://github.com/mseg-dataset/mseg-api/blob/master/mseg/dataset_lists/wilddash-19/list/train.txt#L71

Is there any particular reason for that or is that an error?

Best, Michael

Check if polygon is degenerate, i.e. all vertices are on a horizontal line. In that case, use ImageDraw.Draw.line instead.

Generalize type assertions by checking for subtype of np.integer instead of using a list of some integer types.

Hello,

thank you for interesting work.

I have a question regarding the final class name -> id mapping. How can one obtain such mapping using the mseg-api? Are the label IDs sorted alphabetiaclly according to the universal column?

Really nice work and I hope you could release the remained 3 repos so that the community could conduct research along this direction.

This repo is the first of 4 repos that introduce our work. It provides utilities to download the MSeg dataset (which is nontrivial), and prepare the data on disk in a unified taxonomy.

In a few weeks, we will add the TaxonomyConverter class to this repo that supports on-the-fly mapping to a unified taxonomy during training.

• Logits space of "naive-merge" model should have had 316 classes, not 320 (cityscapes-34 was erroneously included in list of train datasets)
• Style fixes (return early instead of nesting)

Hey, here!

I am the member of OpenMMLab and we think this dataset is very valuable for many tasks such as segmentation, we want to make more researchers and people in industry use this dataset so would you like to make a new pr on MMSegmentation with us? That could be more excellent if you would like to do it.

Best,

Hi, @johnwlambert I want to make something similar with the dataset (as mentioned by someone in the issues, who wanted to add crosswalk), but also add curb and curb cut. So for this until now i figured out, that I should modify the tsv files of dataset you mentioned above. But I dont know if this is complete, so I have two questions:

• When you said add 'crosswalk' to universal taxonomy you refer to add a new line in MSeg_master.tsv for this class? I should do this also for, curb (curb, curb cut)? are there other files, i should change?

• I want to keep only 64 classes, so this means I only remove them (their lines) from MSeg_master.tsv and add to final line to unlabeled, and also modify the tsv file for every dataset (ex. ade20k-151_to_ade20k-150.tsv and ade20k-151_to_ade20k-150-relabeled.tsv).

Until now I changed MSeg_master.tsv to have my selected classes, and move the other to unlabeled, and then also changed the state in every 'name_of_dataset.tsv' to unlabeled for the deleted one. I run, remap functions and everything works ok. But when I try to relabel, on ade_20k it worked, but stopped on BDD. When I run with old bdd.tsv works, with the new tsv it catch an assert:

File "/home/ubuntu/data/datasets/mseg/mseg_files/mseg-api/mseg/utils/mask_utils.py", line 920, in swap_px_inside_mask assert np.allclose(np.unique(label_img[y,x]), np.array([old_val], dtype=np.uint8))

This is the new bdd.tsv: bdd bdd-relabeled building building road road sidewalk sidewalk_pavement terrain terrain person unlabeled rider rider_other traffic sign traffic_sign traffic light traffic_light sky sky pole unlabeled fence unlabeled vegetation vegetation bicycle bicycle car car motorcycle motorcycle bus bus train unlabeled truck unlabeled wall wall unlabeled unlabeled

Waiting for you response.

Downloading instruction is stating: "This dataset is not available via wget, so we ask that you download it in a browser, and upload it (1.3 GB) to the desired location on your server.

Log in, click the "Download" tab on the left, accept terms, then click the link for "Segmentation" (file should appear under name "bdd100k_seg.zip")"

When I try to download the file size is ~300Mb and also its name is bdd100k_sem_seg_labels_trainval. There is no file with 1.3G size and the name bdd100k_seg.zip on BDD official page (https://bdd-data.berkeley.edu/login.html)

Hi, thanks very much for your nice work. currently, I just follow your instructions and it reports that : Archive: ADE20K_2016_07_26.zip End-of-central-directory signature not found. Either this file is not a zipfile, or it constitutes one disk of a multi-part archive. In the latter case the central directory and zipfile comment will be found on the last disk(s) of this archive.

This branch provides support for our submissions to the the 2020 Robust Vision Challenge (RVC).

It includes:

• mappings from our universal taxonomy backwards to ADE20K, Cityscapes, KITTI, Mapillary Vistas, ScanNet, VIPER, WildDashV2. Please keep in mind that some of these backwards mappings are lossy, for example if a merge previously occurred.
• paired test set datalists to use the mseg/label_preparation/remap_dataset.py script to prepare submissions on disk.
• A script to analyze the per-class frequencies of any dataset, using tests/class_density_analysis.py
• A script to visualize individual masks of all classes from any dataset, using tests/visualize_specific_class_examples.py