I think I did follow all the steps noted in the deployment docs. Everything goes well apart from parsing the deformconvfunction, any help is appreciated!

The code runs well whilst training, but failed on converting. And I have checked the log whilst pip install -v -e . which indicated that the DCN has been properly compiled and installed.

Traceback (most recent call last):
  File "onnx_exporter_solov2.py", line 223, in <module>
    convert2onnx(args, dummy_input)
  File "onnx_exporter_solov2.py", line 99, in convert2onnx
    torch.onnx.export(model, dummy_input, args.out, input_names=input_names, output_names=output_names, verbose=True, opset_version=11)
  File "/home/10t/jian/anaconda3/envs/lane/lib/python3.7/site-packages/torch/onnx/__init__.py", line 280, in export
    custom_opsets, enable_onnx_checker, use_external_data_format)
  File "/home/10t/jian/anaconda3/envs/lane/lib/python3.7/site-packages/torch/onnx/utils.py", line 94, in export
    use_external_data_format=use_external_data_format)
  File "/home/10t/jian/anaconda3/envs/lane/lib/python3.7/site-packages/torch/onnx/utils.py", line 706, in _export
    val_add_node_names, val_use_external_data_format, model_file_location)
RuntimeError: ONNX export failed: Couldn't export Python operator DeformConvFunction

[TensorRT] VERBOSE: builtin_op_importers.cpp:534: Convolution output dimensions: (-1, 256, -1, -1) [TensorRT] VERBOSE: ImporterContext.hpp:116: Registering tensor: 510 for ONNX tensor: 510 [TensorRT] VERBOSE: ModelImporter.cpp:179: [Conv] outputs: [510 -> (-1, 256, -1, -1)], [TensorRT] VERBOSE: ModelImporter.cpp:103: Parsing node: [Constant] [TensorRT] VERBOSE: ModelImporter.cpp:125: [Constant] inputs: [TensorRT] VERBOSE: ModelImporter.cpp:179: [Constant] outputs: [511 -> (3)], [TensorRT] VERBOSE: ModelImporter.cpp:103: Parsing node: [Reshape] [TensorRT] VERBOSE: ModelImporter.cpp:119: Searching for input: 510 [TensorRT] VERBOSE: ModelImporter.cpp:119: Searching for input: 511 [TensorRT] VERBOSE: ModelImporter.cpp:125: [Reshape] inputs: [510 -> (-1, 256, -1, -1)], [511 -> (3)], [TensorRT] VERBOSE: ImporterContext.hpp:141: Registering layer: (Unnamed Layer* 234) [Shuffle] for ONNX node: [TensorRT] VERBOSE: ImporterContext.hpp:116: Registering tensor: 512 for ONNX tensor: 512 [TensorRT] VERBOSE: ModelImporter.cpp:179: [Reshape] outputs: [512 -> (-1, 32, -1)], [TensorRT] VERBOSE: ModelImporter.cpp:103: Parsing node: [Constant] [TensorRT] VERBOSE: ModelImporter.cpp:125: [Constant] inputs: [TensorRT] VERBOSE: ModelImporter.cpp:179: [Constant] outputs: [513 -> (32)], [TensorRT] VERBOSE: ModelImporter.cpp:103: Parsing node: [Constant] [TensorRT] VERBOSE: ModelImporter.cpp:125: [Constant] inputs: [TensorRT] VERBOSE: ModelImporter.cpp:179: [Constant] outputs: [514 -> (32)], [TensorRT] VERBOSE: ModelImporter.cpp:103: Parsing node: [InstanceNormalization] [TensorRT] VERBOSE: ModelImporter.cpp:119: Searching for input: 512 [TensorRT] VERBOSE: ModelImporter.cpp:119: Searching for input: 513 [TensorRT] VERBOSE: ModelImporter.cpp:119: Searching for input: 514 [TensorRT] VERBOSE: ModelImporter.cpp:125: [InstanceNormalization] inputs: [512 -> (-1, 32, -1)], [513 -> (32)], [514 -> (32)], ERROR: Failed to parse the ONNX file. In node -1 (importInstanceNormalization): UNSUPPORTED_NODE: Assertion failed: !isDynamic(tensorPtr->getDimensions()) && "InstanceNormalization does not support dynamic inputs!

Hi, thanks for sharing the code. I follow your instruction and convert the pytorch model to onnx and tensorrt model. Tested by deploy/images/demo.jpg, the onnx output is almost the same as pytorch output, however, the tensorrt output is wrong: pytorch output:

onnx output:

tensorrt output:

also, I print the result of onnx model and tensorrt model, which are mask, category and score respectively:

onnx result:

tensorrt result:

Obviously, the score is mismatch. My question is what cause this mismatch? Can you tell me how to solve it ?

the environment setting is the same with yours: CUDA=10.2 cudnn=8.0.5 tensorrt==7.2.1.6 pytorch==1.4.0 torchvision==0.5.0 onnx==1.8.0 onnxruntime==1.6.0 onnx-tensorrt==7.2.1

As my question shows, I cannot get the correct prediction result when using float16. Obtained predictions often do not have any segmentation results. When I converted tensorRt to float32, all the problems were solved.

I have converted the trained SOLOv2_R50_3x.pth model into onnx model using this repository. I used the solov2_r50_fpn_8gpu_3x.py config file and I'm trying to get masks using opencv.

Here is how I convert trained pytorch model into onnx model.

python onnx_exporter_solov2.py   solov2_r50_fpn_8gpu_3x.py    SOLOv2_R50.onnx   --checkpoint SOLOv2_R50_3x.pth   --shape 256 256

Here is my onnx model. Link

Here is my code to read the network and get masks .

import cv2
import numpy as np 

net = cv2.dnn.readNetFromONNX("SOLOv2_R50.onnx")

img = cv2.imread("elephant-symbolism6.jpg")

img = cv2.resize(img, (256,256))

blob = cv2.dnn.blobFromImage(img, swapRB=True)

net.setInput(blob)

cate_pred, kernel_pred, seg_pred = net.forward(['cate_pred', 'kernel_pred', 'seg_pred'])

Here is the size of this arrays.

cate_pred:    (3872, 80)
kernel_pred:  (3872, 256)
seg_pred:     (1,256,64,64)

This code runs without any errors but I'm not sure what are cate_pred, kernel_pred, seg_pred and what they represent.

What are cate_pred, kernel_pred, seg_pred used for and how can I get masks with them?

Hi developer, thanks for your great repo, how to convert onnx to tensorrt model, do you use onnx2trt tool, or this repo only supports conversion to onnx model？

much appreciation

运行命令： python onnx_exporter_solov2.py ../configs/solov2/solov2_r101_fpn_8gpu_3x-kxm.py ../work_dirs/SOLOv2.onnx --checkpoint ../work_dirs/solov2_release_r101_fpn_8gpu_3x/epoch_3042.pth --shape 1333 800

运行结果：
File "onnx_exporter_solov2.py", line 222, in convert2onnx(args, dummy_input) File "onnx_exporter_solov2.py", line 97, in convert2onnx torch.onnx.export(model, dummy_input, args.out, input_names=input_names, output_names=output_names, verbose=True, opset_version=11) File "/home/kxm/.conda/envs/solov2onnx/lib/python3.7/site-packages/torch/onnx/init.py", line 148, in export strip_doc_string, dynamic_axes, keep_initializers_as_inputs) File "/home/kxm/.conda/envs/solov2onnx/lib/python3.7/site-packages/torch/onnx/utils.py", line 66, in export dynamic_axes=dynamic_axes, keep_initializers_as_inputs=keep_initializers_as_inputs) File "/home/kxm/.conda/envs/solov2onnx/lib/python3.7/site-packages/torch/onnx/utils.py", line 416, in _export fixed_batch_size=fixed_batch_size) File "/home/kxm/.conda/envs/solov2onnx/lib/python3.7/site-packages/torch/onnx/utils.py", line 296, in _model_to_graph fixed_batch_size=fixed_batch_size, params_dict=params_dict) File "/home/kxm/.conda/envs/solov2onnx/lib/python3.7/site-packages/torch/onnx/utils.py", line 135, in _optimize_graph graph = torch._C._jit_pass_onnx(graph, operator_export_type) File "/home/kxm/.conda/envs/solov2onnx/lib/python3.7/site-packages/torch/onnx/init.py", line 179, in _run_symbolic_function return utils._run_symbolic_function(*args, **kwargs) File "/home/kxm/.conda/envs/solov2onnx/lib/python3.7/site-packages/torch/onnx/utils.py", line 657, in run_symbolic_function return op_fn(g, *inputs, **attrs) File "/home/kxm/.conda/envs/solov2onnx/lib/python3.7/site-packages/torch/onnx/symbolic_opset11.py", line 424, in linspace steps = sym_help._get_const(steps, 'i', 'steps') File "/home/kxm/.conda/envs/solov2onnx/lib/python3.7/site-packages/torch/onnx/symbolic_helper.py", line 105, in _get_const raise RuntimeError("ONNX symbolic expected a constant value of the {} argument, got {}".format(arg_name, value)) RuntimeError: ONNX symbolic expected a constant value of the steps argument, got 2719 defined in (%2719 : Long() = onnx::Gather[axis=0](%2716, %2718) # /home/kxm/SOLOv2.tensorRT/mmdet/models/anchor_heads/solov2_head.py:195:0 )

Traceback (most recent call last):
  File "onnx_exporter_solov2.py", line 222, in <module>
    convert2onnx(args, dummy_input)
  File "onnx_exporter_solov2.py", line 87, in convert2onnx
    model = init_detector(args.config, args.checkpoint, device='cuda:0')
  File "/usr/local/lib/python3.6/dist-packages/mmdet/apis/inference.py", line 39, in init_detector
    model = build_detector(config.model, test_cfg=config.get('test_cfg'))
  File "/usr/local/lib/python3.6/dist-packages/mmdet/models/builder.py", line 58, in build_detector
    cfg, default_args=dict(train_cfg=train_cfg, test_cfg=test_cfg))
  File "/usr/local/lib/python3.6/dist-packages/mmcv/utils/registry.py", line 210, in build
    return self.build_func(*args, **kwargs, registry=self)
  File "/usr/local/lib/python3.6/dist-packages/mmcv/cnn/builder.py", line 26, in build_model_from_cfg
    return build_from_cfg(cfg, registry, default_args)
  File "/usr/local/lib/python3.6/dist-packages/mmcv/utils/registry.py", line 44, in build_from_cfg
    f'{obj_type} is not in the {registry.name} registry')
KeyError: 'SOLOv2 is not in the models registry'

I'm running on docker, but I keep getting this error. Any help would be appreciated.

I have been training a SOLOv2 and a Solov2 light model on the Resnet 50 backbone. I was wondering if this codebase can be used to convert the model to tensorrt

I run

python deploy/onnx_exporter_solov2.py configs/solov2/solov2_light_448_r18_fpn_8gpu_3x.py weights/SOLOv2_LIGHT_448_R18_3x.onnx --checkpoint ./checkpoints/SOLOv2_LIGHT_448_R18_3x.pth --shape 448 672

and got,

Traceback (most recent call last): File "deploy/onnx_exporter_solov2.py", line 225, in <module> convert2onnx(args, dummy_input) File "deploy/onnx_exporter_solov2.py", line 100, in convert2onnx torch.onnx.export(model, dummy_input, args.out, input_names=input_names, output_names=output_names, verbose=True, opset_version=11) File "/opt/conda/lib/python3.7/site-packages/torch/onnx/__init__.py", line 276, in export custom_opsets, enable_onnx_checker, use_external_data_format) File "/opt/conda/lib/python3.7/site-packages/torch/onnx/utils.py", line 94, in export use_external_data_format=use_external_data_format) File "/opt/conda/lib/python3.7/site-packages/torch/onnx/utils.py", line 701, in _export dynamic_axes=dynamic_axes) File "/opt/conda/lib/python3.7/site-packages/torch/onnx/utils.py", line 468, in _model_to_graph module=module) File "/opt/conda/lib/python3.7/site-packages/torch/onnx/utils.py", line 206, in _optimize_graph graph = torch._C._jit_pass_onnx(graph, operator_export_type) File "/opt/conda/lib/python3.7/site-packages/torch/onnx/__init__.py", line 309, in _run_symbolic_function return utils._run_symbolic_function(*args, **kwargs) File "/opt/conda/lib/python3.7/site-packages/torch/onnx/utils.py", line 993, in _run_symbolic_function symbolic_fn = _find_symbolic_in_registry(domain, op_name, opset_version, operator_export_type) File "/opt/conda/lib/python3.7/site-packages/torch/onnx/utils.py", line 950, in _find_symbolic_in_registry return sym_registry.get_registered_op(op_name, domain, opset_version) File "/opt/conda/lib/python3.7/site-packages/torch/onnx/symbolic_registry.py", line 116, in get_registered_op raise RuntimeError(msg) RuntimeError: Exporting the operator linspace to ONNX opset version 11 is not supported. Please feel free to request support or submit a pull request on PyTorch GitHub.

Traceback (most recent call last): File "onnx_exporter.py", line 232, in check(args, dummy_input, check_onnx=True, check_trt=False) File "onnx_exporter.py", line 127, in check sess = rt.InferenceSession(args.out) File "/home/tao/anaconda3/lib/python3.8/site-packages/onnxruntime/capi/session.py", line 195, in init self._create_inference_session(providers, provider_options) File "/home/tao/anaconda3/lib/python3.8/site-packages/onnxruntime/capi/session.py", line 205, in _create_inference_session sess.initialize_session(providers or [], provider_options or []) onnxruntime.capi.onnxruntime_pybind11_state.Fail: [ONNXRuntimeError] : 1 : FAIL : Node (Concat_601) Op (Concat) [ShapeInferenceError] Can't merge shape info. Both source and target dimension have values but they differ. Source=49 Target=48 Dimension=2

I want to use " multiple images per gpu".in "# TODO: support multiple images per gpu (only minor changes are needed) dataset = build_dataset(cfg.data.test) data_loader = build_dataloader( dataset, imgs_per_gpu=2, workers_per_gpu=1, dist=distributed, shuffle=False)" I change the imgs_per_gpu=1 to imgs_per_gpu=2,but I got the error 'RuntimeError: stack expects each tensor to be equal size, but got [3, 128, 128] at entry 0 and [3, 128, 96] at entry 1' ,but I search the code ,i did not find the size of (128,96),so ,how can I use 'multiple images per gpu' to test.