Hi,

I'm trying to reproduce the exact same steps from your Object Detection tutorial on Jetson Nano.

I've started from a fresh Jetpack 4.5.1 installation and executed the commands:

sudo apt remove cmake sudo snap install cmake --classic sudo reboot

cd ~ git clone https://github.com/NobuoTsukamoto/tensorrt-examples cd ./tensorrt-examples git submodule update --init --recursive

export TRT_LIBPATH=pwd/TensorRT export PATH=${PATH}:/usr/local/cuda/bin cd $TRT_LIBPATH mkdir -p build && cd build

Until now, everything works fine. But when I execute the cmake command:

cmake .. -DTRT_LIB_DIR=$TRT_LIBPATH -DTRT_OUT_DIR=pwd/out -DTRT_PLATFORM_ID=aarch64 -DCUDA_VERSION=10.2 -DCMAKE_BUILD_TYPE=Release -DCMAKE_C_COMPILER=/usr/bin/gcc

It tells me TensorRT is being built for version 8.2.0:

Building for TensorRT version: 8.2.0, library version: 8

Just to double check I ran a "dpkg -l" command and saw that my TensorRT version is actually 7.1.3. Even so, the cmake compiles with no errors and the build files get written succesfully. So I tried running make:

make -j3

And got the following error:

[  2%] Built target third_party.protobuf
[  3%] Built target third_party.protobuf_aarch64
[  4%] Built target gen_onnx_proto
[  4%] Built target caffe_proto
[  5%] Running gen_proto.py on onnx/onnx-data.in.proto
Consolidate compiler generated dependencies of target nvinfer_plugin
Consolidate compiler generated dependencies of target nvinfer_plugin_static
[  5%] Building CXX object plugin/CMakeFiles/nvinfer_plugin.dir/tfliteNMSPlugin/tfliteNMSPlugin.cpp.o
[  6%] Building CXX object plugin/CMakeFiles/nvinfer_plugin_static.dir/tfliteNMSPlugin/tfliteNMSPlugin.cpp.o
Processing /home/joao/tensorrt-examples/TensorRT/parsers/onnx/third_party/onnx/onnx/onnx-data.in.proto
Writing /home/joao/tensorrt-examples/TensorRT/build/parsers/onnx/third_party/onnx/onnx/onnx-data_onnx2trt_onnx.proto
Writing /home/joao/tensorrt-examples/TensorRT/build/parsers/onnx/third_party/onnx/onnx/onnx-data_onnx2trt_onnx.proto3
Writing /home/joao/tensorrt-examples/TensorRT/build/parsers/onnx/third_party/onnx/onnx/onnx-data.pb.h
generating /home/joao/tensorrt-examples/TensorRT/build/parsers/onnx/third_party/onnx/onnx/onnx_data_pb.py
[  6%] Running gen_proto.py on onnx/onnx-operators.in.proto
Processing /home/joao/tensorrt-examples/TensorRT/parsers/onnx/third_party/onnx/onnx/onnx-operators.in.proto
Writing /home/joao/tensorrt-examples/TensorRT/build/parsers/onnx/third_party/onnx/onnx/onnx-operators_onnx2trt_onnx-ml.proto
Writing /home/joao/tensorrt-examples/TensorRT/build/parsers/onnx/third_party/onnx/onnx/onnx-operators_onnx2trt_onnx-ml.proto3
Writing /home/joao/tensorrt-examples/TensorRT/build/parsers/onnx/third_party/onnx/onnx/onnx-operators-ml.pb.h
generating /home/joao/tensorrt-examples/TensorRT/build/parsers/onnx/third_party/onnx/onnx/onnx_operators_pb.py
[  6%] Running C++ protocol buffer compiler on /home/joao/tensorrt-examples/TensorRT/build/parsers/onnx/third_party/onnx/onnx/onnx-data_onnx2trt_onnx.proto
[  6%] Running C++ protocol buffer compiler on /home/joao/tensorrt-examples/TensorRT/build/parsers/onnx/third_party/onnx/onnx/onnx-operators_onnx2trt_onnx-ml.proto
[  6%] Building CXX object parsers/onnx/third_party/onnx/CMakeFiles/onnx_proto.dir/onnx/onnx_onnx2trt_onnx-ml.pb.cc.o
plugin/CMakeFiles/nvinfer_plugin.dir/build.make:439: recipe for target 'plugin/CMakeFiles/nvinfer_plugin.dir/tfliteNMSPlugin/tfliteNMSPlugin.cpp.o' failed
CMakeFiles/Makefile2:1361: recipe for target 'plugin/CMakeFiles/nvinfer_plugin.dir/all' failed
[  7%] Building CXX object plugin/CMakeFiles/nvinfer_plugin_static.dir/bertQKVToContextPlugin/fused_multihead_attention/src/fused_multihead_attention_fp16_64_64_kernel.sm75.cpp.o
plugin/CMakeFiles/nvinfer_plugin_static.dir/build.make:439: recipe for target 'plugin/CMakeFiles/nvinfer_plugin_static.dir/tfliteNMSPlugin/tfliteNMSPlugin.cpp.o' failed
[  7%] Building CXX object parsers/onnx/third_party/onnx/CMakeFiles/onnx_proto.dir/onnx/onnx-operators_onnx2trt_onnx-ml.pb.cc.o
CMakeFiles/Makefile2:1387: recipe for target 'plugin/CMakeFiles/nvinfer_plugin_static.dir/all' failed
[  7%] Building CXX object parsers/onnx/third_party/onnx/CMakeFiles/onnx_proto.dir/onnx/onnx-data_onnx2trt_onnx.pb.cc.o
[  7%] Linking CXX static library libonnx_proto.a
[  8%] Built target onnx_proto
Makefile:155: recipe for target 'all' failed

Would you have any other specific instructions in order to reproduce your object detection examples? I need to run tensorrt optimized object detection inference from tensorflow models and this has been the best guide I have found so far.

Thank you in advance!

I have built the project on Jetpack 4.5.1 following the instructions from: https://github.com/NobuoTsukamoto/tensorrt-examples/issues/3#issuecomment-1062732055tensor

Then, created ONNX GraphSurgeon for ssd_mobilenet_v1_fpn_coco using the Add_TFLiteNMS_Plugin notebook, by replacing "ssdlite_mobilenet_v2_coco_2018_05_09" model with "ssd_mobilenet_v1_fpn_shared_box_predictor_640x640_coco14_sync_2018_07_03" and input shapes from "input_shapes=1,300,300,3" to "input_shapes=1,640,640,3". Here is my TFLiteNMS_Plugin notebook.

When trying to convert the model from onnx to trt on Jetson Nano with convert_onnxgs2trt.py, got the following error:

Beginning ONNX file parsing
[TensorRT] WARNING: onnx2trt_utils.cpp:220: Your ONNX model has been generated with INT64 weights, while TensorRT does not natively support INT64. Attempting to cast down to INT32.
[TensorRT] INTERNAL ERROR: Assertion failed: inputs[0].nbDims == 3
/home/joao/tensorrt-examples/TensorRT/plugin/tfliteNMSPlugin/tfliteNMSPlugin.cpp:74
Aborting...

Looking deeper into tfliteNMSPlugin.cpp, this assertion comes from TFLiteNMSPlugin::getOutputDimensions, at line 74. Just to check, I tried printing the number of dimensions of the inputs from the input vector:

Dims TFLiteNMSPlugin::getOutputDimensions(int index, const Dims* inputs, int nbInputDims) noexcept
{
    printf("inputs[0] dimensions: %i\n", inputs[0].nbDims);
    printf("inputs[1] dimensions: %i\n", inputs[1].nbDims);
    printf("inputs[2] dimensions: %i\n", inputs[2].nbDims);
    ASSERT(nbInputDims == 3);
    ASSERT(index >= 0 && index < this->getNbOutputs());
    ASSERT(inputs[0].nbDims == 3);
    ASSERT(inputs[1].nbDims == 2 || (inputs[1].nbDims == 3 && inputs[1].d[2] == 1));
    ASSERT(inputs[2].nbDims == 2);
    // boxesSize: number of box coordinates for one sample
    boxesSize = inputs[0].d[0] * inputs[0].d[1] * inputs[0].d[2];
    // scoresSize: number of scores for one sample
    scoresSize = inputs[1].d[0] * inputs[1].d[1];
    // anchorSize: number of anchors for one sample
    anchorsSize = inputs[2].d[0] * inputs[2].d[2];

    // num_detections
    if (index == 0)
    {
        Dims dim0{};
        dim0.nbDims = 0;
        return dim0;
    }
    // nmsed_boxes
    if (index == 1)
    {
        return DimsHW(param.max_detections, 4);
    }
    // nmsed_scores or nmsed_classes
    Dims dim1{};
    dim1.nbDims = 1;
    dim1.d[0] = param.max_detections;
    return dim1;
}

The result was:

inputs[0] dimensions: 2
inputs[1] dimensions: 2
inputs[2] dimensions: 2
[TensorRT] INTERNAL ERROR: Assertion failed: inputs[0].nbDims == 3

So my questions are:

1. Have you been able to use the NMS Plugin for any other networks from Tensorflow 1 Model Zoo than SSDLite MobileNetv2? If yes, how did you do it?
2. If not, do you know what might be causing this input dimensions errors?

Here is my network ONNX GraphSurgeon file.

EDIT: Sharing also an image of the network generated from Netron.