I have trained SOLO with cityscape datasets, without modifying any settings except for datasets part. Here is my result. Obviously, solo performs very badly on the cityscape. Maybe I need to tune some hyper-parameters. Do you have any cues?

 Average Precision  (AP) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.075
 Average Precision  (AP) @[ IoU=0.50      | area=   all | maxDets=100 ] = 0.157
 Average Precision  (AP) @[ IoU=0.75      | area=   all | maxDets=100 ] = 0.068
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.004
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.063
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.166
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=  1 ] = 0.096
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets= 10 ] = 0.158
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.168
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.008
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.142
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.335

Dear @WXinlong ,

After training on a custom dataset (inherited from the CocoDataset, just like the way in the cityscapes.py), the visualization looks great, but mAP = 0. I see in the test_insts.py coco_eval(result_file, eval_types, dataset.coco)

Is it the right way?

I read your papers on SOLO and SOLOv2. One part that I didn't understand was that you said the dynamic version is more efficient than the vanilla version. How is efficiency measured here? If it's measured by computation (i.e. MACC/FLOP), then since you eventually generate the HxWxS^2 feature map, isn't the efficiency similar? If it's measure by parameters, than the dynamic version is indeed more efficient.

command

python tools/test_ins.py configs/solov2/solov2_light_448_r34_fpn_8gpu_3x.py  work_dirs/solov2_light_release_r34_fpn_8gpu_3x/epoch_36.pth --show --out  results_solo.pkl
 --eval segm

bug [>>>>>>>>>>>>> ] 20/76, 0.3 task/s, elapsed: 59s, ETA: 165s Traceback (most recent call last): ... RuntimeError: CUDA out of memory. Tried to allocate 3.30 GiB (GPU 0; 8.00 GiB total capacity; 973.14 MiB already allocated; 2.13 GiB free; 3.74 GiB reserved in total by PyTorch) Then I shrinked my test set to 14 images, same error occurred when [>> ] 2/14.

Environment python 3.7 CUDA 11.1 PyTorch 1.7.0+cu110

Supplement The epoch_36.pth file is generated from the training on my own dataset. And performed pretty good when single-tested by inference_demo.py but fail with this batch-test command.

from . import nms_cpu, nms_cuda from .soft_nms_cpu import soft_nms_cpu

help

git clone https://github.com/WXinlong/SOLO.git cd SOLO pip install -r requirements/build.txt pip install "git+https://github.com/cocodataset/cocoapi.git#subdirectory=PythonAPI" pip install -v -e .

I have successfully installed these command，however, there are no nms_cpu, nms_cuda or soft_nms_cpu.

Backend: pytorch1.1.0

1. How does matrix-NMS eliminate the sequential processing nature of NMS ?

2. What is the difference in terms of physical meaning or interpretation between sum_masks and seg_masks ?

3. Why is intersection defined as inter_matrix = torch.mm(seg_masks, seg_masks.transpose(1, 0)) ?

4. Why iou_matrix = (inter_matrix / (sum_masks_x + sum_masks_x.transpose(1, 0) - inter_matrix)).triu(diagonal=1) uses triu(diagonal=1) and transposed of sum_masks_x ?

Hi, I just read the SOLOv2 paper and found it very inspiring. But I got a question when I read the code

In mmdet/models/anchor_heads/solov2_head.py:

    def forward(self, feats, eval=False):
        new_feats = self.split_feats(feats)
        featmap_sizes = [featmap.size()[-2:] for featmap in new_feats]
        upsampled_size = (featmap_sizes[0][0] * 2, featmap_sizes[0][1] * 2)
        cate_pred, kernel_pred = multi_apply(self.forward_single, new_feats,
                                                       list(range(len(self.seg_num_grids))),
                                                       eval=eval, upsampled_size=upsampled_size)
        return cate_pred, kernel_pred

It seems that the code still uses different mask encoders and grid numbers for feature of each FPN level. Is the Unified mask feature branch function mentioned in the SOLOv2 paper included in the code? Or are there any settings related?

Thank you!

I tried to run inference of SOLOv2 using the command python tools/test.py configs/solov2/solov2_r101_dcn_fpn_8gpu_3x.py model_zoo/solov2_r101_dcn_fpn_8gpu_3x.pth --eval segm --json_out coco_json/solov2_r101_dcn_fpn_8gpu_3x_coco_val_results but I got CUDA out of memory error after evaluating around 1000 images.

Has anyone encountered this problem as well?

Step 12/12 : RUN pip install --no-cache-dir -e .
 ---> Running in 10680a3a2218
Obtaining file:///SOLO
    ERROR: Command errored out with exit status 1:
     command: /opt/conda/bin/python -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/SOLO/setup.py'"'"'; __file__='"'"'/SOLO/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' egg_info
         cwd: /SOLO/
    Complete output (8 lines):
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/SOLO/setup.py", line 251, in <module>
        sources=['src/compiling_info.cpp']),
      File "/SOLO/setup.py", line 103, in make_cuda_ext
        raise EnvironmentError('CUDA is required to compile MMDetection!')
    OSError: CUDA is required to compile MMDetection!
    No CUDA runtime is found, using CUDA_HOME='/usr/local/cuda'
    ----------------------------------------
ERROR: Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.

Hello,

If my understanding is correct, I see the SOLOv2 only uses a single combined feature map (Fig3 in the paper), which is different from SOLOv1 that uses multi-scale FPN features.

Is the proposed dynamic head applied to this single combined feature map?

Thanks

When I try to train this model, this error happens.

../mmdet/models/anchor_heads/solo_head.py", line 203, in loss num_ins = flatten_ins_ind_labels.sum() RuntimeError: "sum_cuda" not implemented for 'Bool'

i have run 'python setup.py build_ext --inplace'

Hi,

Solov2 is a very powerful tool for instance segmentation. I am trying to experiment with its output but this implementation uses mmdetection APIs which output a bunch of images showing instances on them. Is there a way to get the actual output from the model, i.e. the class and segmentation masks? I want to pass an image to the model at inference and get the raw output from the model. Any help is greatly appreciated.

@WXinlong ，Hello, thank you very much for your excellent open source project. When I read your code, I still can't find where your code for dynamically performing convolution is. Forgive my stupidity, can you enlighten me.

I want to try a simple training on my own datasets, but got this error, how to solve it, please?

___Traceback (most recent call last): File "c:\users\echo\mmcv\mmcv\utils\registry.py", line 69, in build_from_cfg return obj_cls(**args) TypeError: init() got an unexpected keyword argument 'mask_feat_head'

During handling of the above exception, another exception occurred:_

Traceback (most recent call last): File "E:\DeepLearning\Pytorch\Projects\SOLO-master\SOLO-master\tools\train.py", line 127, in main() File "E:\DeepLearning\Pytorch\Projects\SOLO-master\SOLO-master\tools\train.py", line 100, in main model = build_detector( File "D:\Anaconda\Anaconda3\envs\mmlab\lib\site-packages\mmdet\models\builder.py", line 58, in build_detector return DETECTORS.build( File "c:\users\echo\mmcv\mmcv\utils\registry.py", line 237, in build return self.build_func(*args, **kwargs, registry=self) File "c:\users\echo\mmcv\mmcv\cnn\builder.py", line 27, in build_model_from_cfg return build_from_cfg(cfg, registry, default_args) File "c:\users\echo\mmcv\mmcv\utils\registry.py", line 72, in build_from_cfg raise type(e)(f'{obj_cls.name}: {e}') TypeError: SOLOv2: init() got an unexpected keyword argument 'mask_feat_head'__

Hi Xinlong, could you provide your solov2 training config file on LVIS dataset? I met some problem when reimplementing the results of solov2 on LVIS v0.5 / v1 dataset. So I want to use the official config as reference to retry my experiment.

This is a request to specify that the maximum allowed version for Pytorch should be 1.10.1 or 1.10.2.

Motivation After Pytorch 1.11, there were some changes to the CPP code for the THC.h headers to be now part of ATen.h. Knowing this, Pytorch 1.10.1 (no matter if CUDA 10..2 or 11.3), should be the highest allowed Pytorch version unless the CUDA source files for the following source files are updated:

mmdet
├── ops
│     ├── nms/src/
│     │     ├── nms_kernel.cu
│     ├── roi_align/src/
│     │     ├── roi_align_kernel.cu
│     ├── roi_pool/src/
│     │     ├── roi_pool_kernel.cu
│     ├── dcn/src/
│     │     ├── deform_conv_cuda_kernel.cu
│     │     ├── deform_pool_cuda_kernel.cu
│     ├── sigmoid_focal_loss/src/
│     │     ├── sigmoid_focal_loss_cuda.cu
│     ├── masked_conv/src/
│     │     ├── masked_conv2d_kernel.cu