Detectron2 for Document Layout Analysis

Hi,

I am trying to predict titles, tables and text in an input image.

I used the command :

python demo/demo.py --config-file configs/DLA_mask_rcnn_R_101_FPN_3x.yaml --input "1.png" --output "./testt.png" --confidence-threshold 0.5 --opts MODEL.WEIGHTS "detectron2://COCO-InstanceSegmentation/mask_rcnn_R_101_FPN_3x/138205316/model_final_a3ec72.pkl" MODEL.DEVICE cpu

The command runs with the following logs and returns true at the end.

Config './configs/DLA_mask_rcnn_R_101_FPN_3x.yaml' has no VERSION. Assuming it to be compatible with latest v2. 'roi_heads.box_predictor.cls_score.weight' has shape (81, 1024) in the checkpoint but (6, 1024) in the model! Skipped. 'roi_heads.box_predictor.cls_score.bias' has shape (81,) in the checkpoint but (6,) in the model! Skipped. 'roi_heads.box_predictor.bbox_pred.weight' has shape (320, 1024) in the checkpoint but (20, 1024) in the model! Skipped. 'roi_heads.box_predictor.bbox_pred.bias' has shape (320,) in the checkpoint but (20,) in the model! Skipped. 'roi_heads.mask_head.predictor.weight' has shape (80, 256, 1, 1) in the checkpoint but (5, 256, 1, 1) in the model! Skipped. 'roi_heads.mask_head.predictor.bias' has shape (80,) in the checkpoint but (5,) in the model! Skipped.

However, the output file remains unchanged with no bounding boxes or labels getting predicted. Can anyone guide why it's happening so.

Kindly help.

Thanks!

Hi,

I want to fine-tune the model with an existing class (table in my case). How should I set the category id of the new data? What should be its value, in a sense should it be the same as in the pre-trained model?

Thanks in advance!

Hi, I'm currently using the pre-trained model weights supplied in the repo. But, I need to fine-tune the model further to suit my requirements. So, I ran a sample training with only one image and one instance/bbox in that image, but the training script says it needs 1 day 18 hrs to complete training on this minuscule training set. Is this an issue with ETA estimation or does the model really take that long to train on one image ??

I've gone through https://github.com/hpanwar08/detectron2/blob/master/tools/train_net_dla.py training script. But, I'm bit confused on how to structure my custom data (both JSON and IMG-data).

If you can share a sample JSON and folder structure for placing training images, it would be great !!

Hi @hpanwar08

Can you please confirm, what is the LR (learning rate you used), also did you used same learning rate through out your training process or was it scheduled at different learning rates ?

Unfortunately, I always run into an error when fine-tuning on my own dataset (in coco format). The detectron2 documentation on datasets was of no help either (maybe I overlooked something though).

After registering the data via register_coco_instances (as seen in train_net_dla.py) the following code

train data (train_data = DatasetCatalog.get("dla_train")
train data[0]

gives me:

{'file_name': 'cyber_train/public_imgs_sampled/file01.png',
 'height': 1754,
 'width': 1241,
 'image_id': 0,
 'annotations': [{'iscrowd': 0,
   'bbox': [78, 600, 977, 33],
   'category_id': 1,
   'segmentation': [[83, 606, 83, 633, 1050, 633, 1050, 606]],
   'bbox_mode': <BoxMode.XYWH_ABS: 1>}
...

which does look good to me telling from the COCO formatting style.

I can even visualize the data using Visualizer:

for d in random.sample(train_data, 1):
    img = cv2.imread(d["file_name"])
    visualizer = Visualizer(img[:, :, :], scale=0.5)
    out = visualizer.draw_dataset_dict(d)
    plt.imshow(out.get_image()[:,:,::-1])

which gives me the expected output:

Creating the trainer is also possible using trainer = DefaultTrainer(cfg), but calling trainer.train() gives me a stack trace for the AttributeError: Cannot find field 'gt_masks' in the given Instances!' in DataLoader. I am very confused about the error, because 'segmentation' is present in the train_data and the specified polygons are also valid telling from the output of Visualizer.

Anyway, thank you very much for your great work @hpanwar08 !

Hi, when I got the error by calling python demo/demo.py --config-file configs/DLA_mask_rcnn_X_101_32x8d_FPN_3x.yaml --input “/home/jupyter/layout/detectron2/input.jpg” --output “/home/jupyter/layout/” --confidence-threshold 0.5 --opts MODEL.WEIGHTS /home/jupyter/layout/detectron2/model_final_trimmed.pth MODEL.DEVICE cpu Here is the error, not sure why the .yaml doesn't have VERSION. Thank you

WARNING [03/26 04:43:12 d2.config.compat]: Config 'configs/DLA_mask_rcnn_X_101_32x8d_FPN_3x.yaml' has no VERSION. Assuming it to be compatible with latest v2.
Traceback (most recent call last):
  File "demo/demo.py", line 76, in <module>
    demo = VisualizationDemo(cfg)
  File "/home/jupyter/layout/detectron2/demo/predictor.py", line 35, in __init__
    self.predictor = DefaultPredictor(cfg)
  File "/opt/conda/lib/python3.7/site-packages/detectron2/engine/defaults.py", line 168, in __init__
    checkpointer.load(cfg.MODEL.WEIGHTS)
  File "/opt/conda/lib/python3.7/site-packages/fvcore/common/checkpoint.py", line 103, in load
    checkpoint = self._load_file(path)
  File "/opt/conda/lib/python3.7/site-packages/detectron2/checkpoint/detection_checkpoint.py", line 42, in _load_file
    loaded = super()._load_file(filename)  # load native pth checkpoint
  File "/opt/conda/lib/python3.7/site-packages/fvcore/common/checkpoint.py", line 189, in _load_file
    return torch.load(f, map_location=torch.device("cpu"))
  File "/opt/conda/lib/python3.7/site-packages/torch/serialization.py", line 529, in load
    return _legacy_load(opened_file, map_location, pickle_module, **pickle_load_args)
  File "/opt/conda/lib/python3.7/site-packages/torch/serialization.py", line 709, in _legacy_load
    deserialized_objects[key]._set_from_file(f, offset, f_should_read_directly)
RuntimeError: unexpected EOF, expected 16489904 more bytes. The file might be corrupted.
terminate called after throwing an instance of 'c10::Error'
  what():  owning_ptr == NullType::singleton() || owning_ptr->refcount_.load() > 0 INTERNAL ASSERT FAILED at /pytorch/c10/util/intrusive_ptr.h:348, please report a bug to PyTorch. intrusive_ptr: Can only intrusive_ptr::reclaim() owning pointers that were created using intrusive_ptr::release(). (reclaim at /pytorch/c10/util/intrusive_ptr.h:348)
frame #0: c10::Error::Error(c10::SourceLocation, std::string const&) + 0x33 (0x7f545aa70193 in /opt/conda/lib/python3.7/site-packages/torch/lib/libc10.so)
frame #1: <unknown function> + 0x186b6af (0x7f545c72e6af in /opt/conda/lib/python3.7/site-packages/torch/lib/libtorch.so)
frame #2: THStorage_free + 0x17 (0x7f545cef6cb7 in /opt/conda/lib/python3.7/site-packages/torch/lib/libtorch.so)
frame #3: <unknown function> + 0x55d23d (0x7f54a377223d in /opt/conda/lib/python3.7/site-packages/torch/lib/libtorch_python.so)
<omitting python frames>
frame #22: __libc_start_main + 0xf1 (0x7f54c6f032e1 in /lib/x86_64-linux-gnu/libc.so.6)

Aborted

Hello,

In PDF documents am currently working on text is coming along with tables and figures. I am only concerned with tables and figures (each one essential). Is there any parameter that I could set to avoid appearance of text? Thanks

Hi, while I am trying to finetune the publaynet model in colab I keep running into errors. What would the process of this be? Should I start with this package or the normal detectron2 from facebookreserch? I have tried to manually add the publaynet model to my google drive but it does not seem to work that way. Some steps to achieve this would be of great help.

Thanks in advance

Hi @hpanwar08 , I was trying to use your pre-trained models for prediction by following your instructions and using the below command:

python demo/demo.py --config-file configs/DLA_mask_rcnn_X_101_32x8d_FPN_3x.yaml --input "<path to image.jpg>" --output <path to save the predicted image> --confidence-threshold 0.5 --opts MODEL.WEIGHTS <path to model_final_trimmed.pth> MODEL.DEVICE cpu

I am using Google Colab for the work and I am getting this error:

Traceback (most recent call last):
  File "demo/demo.py", line 73, in <module>
    demo = VisualizationDemo(cfg)
  File "/content/drive/My Drive/PublayNet/detectron2/demo/predictor.py", line 35, in __init__
    self.predictor = DefaultPredictor(cfg)
  File "/usr/local/lib/python3.6/dist-packages/detectron2/engine/defaults.py", line 187, in __init__
    checkpointer.load(cfg.MODEL.WEIGHTS)
  File "/usr/local/lib/python3.6/dist-packages/fvcore/common/checkpoint.py", line 117, in load
    checkpoint = self._load_file(path)
  File "/usr/local/lib/python3.6/dist-packages/detectron2/checkpoint/detection_checkpoint.py", line 42, in _load_file
    loaded = super()._load_file(filename)  # load native pth checkpoint
  File "/usr/local/lib/python3.6/dist-packages/fvcore/common/checkpoint.py", line 213, in _load_file
    return torch.load(f, map_location=torch.device("cpu"))
  File "/usr/local/lib/python3.6/dist-packages/torch/serialization.py", line 586, in load
    with _open_zipfile_reader(f) as opened_zipfile:
  File "/usr/local/lib/python3.6/dist-packages/torch/serialization.py", line 246, in __init__
    super(_open_zipfile_reader, self).__init__(torch._C.PyTorchFileReader(name_or_buffer))
RuntimeError: [enforce fail at inline_container.cc:208] . file not found: /version

the output i got is shown below; 00:44 detectron2]: Arguments: Namespace(confidence_threshold=0.2, config_file='configs/DLA_mask_rcnn_X_101_32x8d_FPN_3x.yaml', input=['/content/gdrive/My Drive/Untitled folder/PMC4527132_00004.jpg'], opts=['MODEL.WEIGHTS', '/content/gdrive/My Drive/Unt/model_final.pth', 'MODEL.DEVICE', 'cpu'], output='/content/gdrive/My Drive/Unt', video_input=None, webcam=False) WARNING [03/27 12:00:44 d2.config.compat]: Config 'configs/DLA_mask_rcnn_X_101_32x8d_FPN_3x.yaml' has no VERSION. Assuming it to be compatible with latest v2. 'backbone.bottom_up.res2.0.conv1.weight' has shape (64, 64, 1, 1) in the checkpoint but (256, 64, 1, 1) in the model! Skipped. 'backbone.bottom_up.res2.0.conv1.norm.weight' has shape (64,) in the checkpoint but (256,) in the model! Skipped. 'backbone.bottom_up.res2.0.conv1.norm.bias' has shape (64,) in the checkpoint but (256,) in the model! Skipped. 'backbone.bottom_up.res2.0.conv1.norm.running_mean' has shape (64,) in the checkpoint but (256,) in the model! Skipped. 'backbone.bottom_up.res2.0.conv1.norm.running_var' has shape (64,) in the checkpoint but (256,) in the model! Skipped. 'backbone.bottom_up.res2.0.conv2.weight' has shape (64, 64, 3, 3) in the checkpoint but (256, 8, 3, 3) in the model! Skipped. 'backbone.bottom_up.res2.0.conv2.norm.weight' has shape (64,) in the checkpoint but (256,) in the model! Skipped. 'backbone.bottom_up.res2.0.conv2.norm.bias' has shape (64,) in the checkpoint but (256,) in the model! Skipped. 'backbone.bottom_up.res2.0.conv2.norm.running_mean' has shape (64,) in the checkpoint but (256,) in the model! Skipped. 'backbone.bottom_up.res2.0.conv2.norm.running_var' has shape (64,) in the checkpoint but (256,) in the model! Skipped. 'backbone.bottom_up.res2.0.conv3.weight' has shape (256, 64, 1, 1) in the checkpoint but (256, 256, 1, 1) in the model! Skipped. 'backbone.bottom_up.res2.1.conv1.weight' has shape (64, 256, 1, 1) in the checkpoint but (256, 256, 1, 1) in the model! Skipped. 'backbone.bottom_up.res2.1.conv1.norm.weight' has shape (64,) in the checkpoint but (256,) in the model! Skipped. 'backbone.bottom_up.res2.1.conv1.norm.bias' has shape (64,) in the checkpoint but (256,) in the model! Skipped. 'backbone.bottom_up.res2.1.conv1.norm.running_mean' has shape (64,) in the checkpoint but (256,) in the model! Skipped. 'backbone.bottom_up.res2.1.conv1.norm.running_var' has shape (64,) in the checkpoint but (256,) in the model! Skipped. 'backbone.bottom_up.res2.1.conv2.weight' has shape (64, 64, 3, 3) in the checkpoint but (256, 8, 3, 3) in the model! Skipped. 'backbone.bottom_up.res2.1.conv2.norm.weight' has shape (64,) in the checkpoint but (256,) in the model! Skipped. 'backbone.bottom_up.res2.1.conv2.norm.bias' has shape (64,) in the checkpoint but (256,) in the model! Skipped. 'backbone.bottom_up.res2.1.conv2.norm.running_mean' has shape (64,) in the checkpoint but (256,) in the model! Skipped. 'backbone.bottom_up.res2.1.conv2.norm.running_var' has shape (64,) in the checkpoint but (256,) in the model! Skipped. 'backbone.bottom_up.res2.1.conv3.weight' has shape (256, 64, 1, 1) in the checkpoint but (256, 256, 1, 1) in the model! Skipped. 'backbone.bottom_up.res2.2.conv1.weight' has shape (64, 256, 1, 1) in the checkpoint but (256, 256, 1, 1) in the model! Skipped. 'backbone.bottom_up.res2.2.conv1.norm.weight' has shape (64,) in the checkpoint but (256,) in the model! Skipped. 'backbone.bottom_up.res2.2.conv1.norm.bias' has shape (64,) in the checkpoint but (256,) in the model! Skipped. 'backbone.bottom_up.res2.2.conv1.norm.running_mean' has shape (64,) in the checkpoint but (256,) in the model! Skipped. 'backbone.bottom_up.res2.2.conv1.norm.running_var' has shape (64,) in the checkpoint but (256,) in the model! Skipped. 'backbone.bottom_up.res2.2.conv2.weight' has shape (64, 64, 3, 3) in the checkpoint but (256, 8, 3, 3) in the model! Skipped. 'backbone.bottom_up.res2.2.conv2.norm.weight' has shape (64,) in the checkpoint but (256,) in the model! Skipped. 'backbone.bottom_up.res2.2.conv2.norm.bias' has shape (64,) in the checkpoint but (256,) in the model! Skipped. 'backbone.bottom_up.res2.2.conv2.norm.running_mean' has shape (64,) in the checkpoint but (256,) in the model! Skipped. 'backbone.bottom_up.res2.2.conv2.norm.running_var' has shape (64,) in the checkpoint but (256,) in the model! Skipped. 'backbone.bottom_up.res2.2.conv3.weight' has shape (256, 64, 1, 1) in the checkpoint but (256, 256, 1, 1) in the model! Skipped. 'backbone.bottom_up.res3.0.conv1.weight' has shape (128, 256, 1, 1) in the checkpoint but (512, 256, 1, 1) in the model! Skipped. 'backbone.bottom_up.res3.0.conv1.norm.weight' has shape (128,) in the checkpoint but (512,) in the model! Skipped. 'backbone.bottom_up.res3.0.conv1.norm.bias' has shape (128,) in the checkpoint but (512,) in the model! Skipped. 'backbone.bottom_up.res3.0.conv1.norm.running_mean' has shape (128,) in the checkpoint but (512,) in the model! Skipped. 'backbone.bottom_up.res3.0.conv1.norm.running_var' has shape (128,) in the checkpoint but (512,) in the model! Skipped. 'backbone.bottom_up.res3.0.conv2.weight' has shape (128, 128, 3, 3) in the checkpoint but (512, 16, 3, 3) in the model! Skipped. 'backbone.bottom_up.res3.0.conv2.norm.weight' has shape (128,) in the checkpoint but (512,) in the model! Skipped. 'backbone.bottom_up.res3.0.conv2.norm.bias' has shape (128,) in the checkpoint but (512,) in the model! Skipped. 'backbone.bottom_up.res3.0.conv2.norm.running_mean' has shape (128,) in the checkpoint but (512,) in the model! Skipped. 'backbone.bottom_up.res3.0.conv2.norm.running_var' has shape (128,) in the checkpoint but (512,) in the model! Skipped. 'backbone.bottom_up.res3.0.conv3.weight' has shape (512, 128, 1, 1) in the checkpoint but (512, 512, 1, 1) in the model! Skipped. 'backbone.bottom_up.res3.1.conv1.weight' has shape (128, 512, 1, 1) in the checkpoint but (512, 512, 1, 1) in the model! Skipped. 'backbone.bottom_up.res3.1.conv1.norm.weight' has shape (128,) in the checkpoint but (512,) in the model! Skipped. 'backbone.bottom_up.res3.1.conv1.norm.bias' has shape (128,) in the checkpoint but (512,) in the model! Skipped. 'backbone.bottom_up.res3.1.conv1.norm.running_mean' has shape (128,) in the checkpoint but (512,) in the model! Skipped. 'backbone.bottom_up.res3.1.conv1.norm.running_var' has shape (128,) in the checkpoint but (512,) in the model! Skipped. 'backbone.bottom_up.res3.1.conv2.weight' has shape (128, 128, 3, 3) in the checkpoint but (512, 16, 3, 3) in the model! Skipped. 'backbone.bottom_up.res3.1.conv2.norm.weight' has shape (128,) in the checkpoint but (512,) in the model! Skipped. 'backbone.bottom_up.res3.1.conv2.norm.bias' has shape (128,) in the checkpoint but (512,) in the model! Skipped. 'backbone.bottom_up.res3.1.conv2.norm.running_mean' has shape (128,) in the checkpoint but (512,) in the model! Skipped. 'backbone.bottom_up.res3.1.conv2.norm.running_var' has shape (128,) in the checkpoint but (512,) in the model! Skipped. 'backbone.bottom_up.res3.1.conv3.weight' has shape (512, 128, 1, 1) in the checkpoint but (512, 512, 1, 1) in the model! Skipped. 'backbone.bottom_up.res3.2.conv1.weight' has shape (128, 512, 1, 1) in the checkpoint but (512, 512, 1, 1) in the model! Skipped. 'backbone.bottom_up.res3.2.conv1.norm.weight' has shape (128,) in the checkpoint but (512,) in the model! Skipped. 'backbone.bottom_up.res3.2.conv1.norm.bias' has shape (128,) in the checkpoint but (512,) in the model! Skipped. 'backbone.bottom_up.res3.2.conv1.norm.running_mean' has shape (128,) in the checkpoint but (512,) in the model! Skipped. 'backbone.bottom_up.res3.2.conv1.norm.running_var' has shape (128,) in the checkpoint but (512,) in the model! Skipped. 'backbone.bottom_up.res3.2.conv2.weight' has shape (128, 128, 3, 3) in the checkpoint but (512, 16, 3, 3) in the model! Skipped. 'backbone.bottom_up.res3.2.conv2.norm.weight' has shape (128,) in the checkpoint but (512,) in the model! Skipped. 'backbone.bottom_up.res3.2.conv2.norm.bias' has shape (128,) in the checkpoint but (512,) in the model! Skipped. 'backbone.bottom_up.res3.2.conv2.norm.running_mean' has shape (128,) in the checkpoint but (512,) in the model! Skipped. 'backbone.bottom_up.res3.2.conv2.norm.running_var' has shape (128,) in the checkpoint but (512,) in the model! Skipped. 'backbone.bottom_up.res3.2.conv3.weight' has shape (512, 128, 1, 1) in the checkpoint but (512, 512, 1, 1) in the model! Skipped. 'backbone.bottom_up.res3.3.conv1.weight' has shape (128, 512, 1, 1) in the checkpoint but (512, 512, 1, 1) in the model! Skipped. 'backbone.bottom_up.res3.3.conv1.norm.weight' has shape (128,) in the checkpoint but (512,) in the model! Skipped. 'backbone.bottom_up.res3.3.conv1.norm.bias' has shape (128,) in the checkpoint but (512,) in the model! Skipped. 'backbone.bottom_up.res3.3.conv1.norm.running_mean' has shape (128,) in the checkpoint but (512,) in the model! Skipped. 'backbone.bottom_up.res3.3.conv1.norm.running_var' has shape (128,) in the checkpoint but (512,) in the model! Skipped. 'backbone.bottom_up.res3.3.conv2.weight' has shape (128, 128, 3, 3) in the checkpoint but (512, 16, 3, 3) in the model! Skipped. 'backbone.bottom_up.res3.3.conv2.norm.weight' has shape (128,) in the checkpoint but (512,) in the model! Skipped. 'backbone.bottom_up.res3.3.conv2.norm.bias' has shape (128,) in the checkpoint but (512,) in the model! Skipped. 'backbone.bottom_up.res3.3.conv2.norm.running_mean' has shape (128,) in the checkpoint but (512,) in the model! Skipped. 'backbone.bottom_up.res3.3.conv2.norm.running_var' has shape (128,) in the checkpoint but (512,) in the model! Skipped. 'backbone.bottom_up.res3.3.conv3.weight' has shape (512, 128, 1, 1) in the checkpoint but (512, 512, 1, 1) in the model! Skipped. 'backbone.bottom_up.res4.0.conv1.weight' has shape (256, 512, 1, 1) in the checkpoint but (1024, 512, 1, 1) in the model! Skipped. 'backbone.bottom_up.res4.0.conv1.norm.weight' has shape (256,) in the checkpoint but (1024,) in the model! Skipped. 'backbone.bottom_up.res4.0.conv1.norm.bias' has shape (256,) in the checkpoint but (1024,) in the model! Skipped. 'backbone.bottom_up.res4.0.conv1.norm.running_mean' has shape (256,) in the checkpoint but (1024,) in the model! Skipped. 'backbone.bottom_up.res4.0.conv1.norm.running_var' has shape (256,) in the checkpoint but (1024,) in the model! Skipped. 'backbone.bottom_up.res4.0.conv2.weight' has shape (256, 256, 3, 3) in the checkpoint but (1024, 32, 3, 3) in the model! Skipped. 'backbone.bottom_up.res4.0.conv2.norm.weight' has shape (256,) in the checkpoint but (1024,) in the model! Skipped. 'backbone.bottom_up.res4.0.conv2.norm.bias' has shape (256,) in the checkpoint but (1024,) in the model! Skipped. 'backbone.bottom_up.res4.0.conv2.norm.running_mean' has shape (256,) in the checkpoint but (1024,) in the model! Skipped. 'backbone.bottom_up.res4.0.conv2.norm.running_var' has shape (256,) in the checkpoint but (1024,) in the model! Skipped. 'backbone.bottom_up.res4.0.conv3.weight' has shape (1024, 256, 1, 1) in the checkpoint but (1024, 1024, 1, 1) in the model! Skipped. 'backbone.bottom_up.res4.1.conv1.weight' has shape (256, 1024, 1, 1) in the checkpoint but (1024, 1024, 1, 1) in the model! Skipped. 'backbone.bottom_up.res4.1.conv1.norm.weight' has shape (256,) in the checkpoint but (1024,) in the model! Skipped. 'backbone.bottom_up.res4.1.conv1.norm.bias' has shape (256,) in the checkpoint but (1024,) in the model! Skipped. 'backbone.bottom_up.res4.1.conv1.norm.running_mean' has shape (256,) in the checkpoint but (1024,) in the model! Skipped. 'backbone.bottom_up.res4.1.conv1.norm.running_var' has shape (256,) in the checkpoint but (1024,) in the model! Skipped. 'backbone.bottom_up.res4.1.conv2.weight' has shape (256, 256, 3, 3) in the checkpoint but (1024, 32, 3, 3) in the model! Skipped. 'backbone.bottom_up.res4.1.conv2.norm.weight' has shape (256,) in the checkpoint but (1024,) in the model! Skipped. 'backbone.bottom_up.res4.1.conv2.norm.bias' has shape (256,) in the checkpoint but (1024,) in the model! Skipped. 'backbone.bottom_up.res4.1.conv2.norm.running_mean' has shape (256,) in the checkpoint but (1024,) in the model! Skipped. 'backbone.bottom_up.res4.1.conv2.norm.running_var' has shape (256,) in the checkpoint but (1024,) in the model! Skipped. 'backbone.bottom_up.res4.1.conv3.weight' has shape (1024, 256, 1, 1) in the checkpoint but (1024, 1024, 1, 1) in the model! Skipped. 'backbone.bottom_up.res4.2.conv1.weight' has shape (256, 1024, 1, 1) in the checkpoint but (1024, 1024, 1, 1) in the model! Skipped. 'backbone.bottom_up.res4.2.conv1.norm.weight' has shape (256,) in the checkpoint but (1024,) in the model! Skipped. 'backbone.bottom_up.res4.2.conv1.norm.bias' has shape (256,) in the checkpoint but (1024,) in the model! Skipped. 'backbone.bottom_up.res4.2.conv1.norm.running_mean' has shape (256,) in the checkpoint but (1024,) in the model! Skipped. 'backbone.bottom_up.res4.2.conv1.norm.running_var' has shape (256,) in the checkpoint but (1024,) in the model! Skipped. 'backbone.bottom_up.res4.2.conv2.weight' has shape (256, 256, 3, 3) in the checkpoint but (1024, 32, 3, 3) in the model! Skipped. 'backbone.bottom_up.res4.2.conv2.norm.weight' has shape (256,) in the checkpoint but (1024,) in the model! Skipped. 'backbone.bottom_up.res4.2.conv2.norm.bias' has shape (256,) in the checkpoint but (1024,) in the model! Skipped. 'backbone.bottom_up.res4.2.conv2.norm.running_mean' has shape (256,) in the checkpoint but (1024,) in the model! Skipped. 'backbone.bottom_up.res4.2.conv2.norm.running_var' has shape (256,) in the checkpoint but (1024,) in the model! Skipped. 'backbone.bottom_up.res4.2.conv3.weight' has shape (1024, 256, 1, 1) in the checkpoint but (1024, 1024, 1, 1) in the model! Skipped. 'backbone.bottom_up.res4.3.conv1.weight' has shape (256, 1024, 1, 1) in the checkpoint but (1024, 1024, 1, 1) in the model! Skipped. 'backbone.bottom_up.res4.3.conv1.norm.weight' has shape (256,) in the checkpoint but (1024,) in the model! Skipped. 'backbone.bottom_up.res4.3.conv1.norm.bias' has shape (256,) in the checkpoint but (1024,) in the model! Skipped. 'backbone.bottom_up.res4.3.conv1.norm.running_mean' has shape (256,) in the checkpoint but (1024,) in the model! Skipped. 'backbone.bottom_up.res4.3.conv1.norm.running_var' has shape (256,) in the checkpoint but (1024,) in the model! Skipped. 'backbone.bottom_up.res4.3.conv2.weight' has shape (256, 256, 3, 3) in the checkpoint but (1024, 32, 3, 3) in the model! Skipped. 'backbone.bottom_up.res4.3.conv2.norm.weight' has shape (256,) in the checkpoint but (1024,) in the model! Skipped. 'backbone.bottom_up.res4.3.conv2.norm.bias' has shape (256,) in the checkpoint but (1024,) in the model! Skipped. 'backbone.bottom_up.res4.3.conv2.norm.running_mean' has shape (256,) in the checkpoint but (1024,) in the model! Skipped. 'backbone.bottom_up.res4.3.conv2.norm.running_var' has shape (256,) in the checkpoint but (1024,) in the model! Skipped. 'backbone.bottom_up.res4.3.conv3.weight' has shape (1024, 256, 1, 1) in the checkpoint but (1024, 1024, 1, 1) in the model! Skipped. 'backbone.bottom_up.res4.4.conv1.weight' has shape (256, 1024, 1, 1) in the checkpoint but (1024, 1024, 1, 1) in the model! Skipped. 'backbone.bottom_up.res4.4.conv1.norm.weight' has shape (256,) in the checkpoint but (1024,) in the model! Skipped. 'backbone.bottom_up.res4.4.conv1.norm.bias' has shape (256,) in the checkpoint but (1024,) in the model! Skipped. 'backbone.bottom_up.res4.4.conv1.norm.running_mean' has shape (256,) in the checkpoint but (1024,) in the model! Skipped. 'backbone.bottom_up.res4.4.conv1.norm.running_var' has shape (256,) in the checkpoint but (1024,) in the model! Skipped. 'backbone.bottom_up.res4.4.conv2.weight' has shape (256, 256, 3, 3) in the checkpoint but (1024, 32, 3, 3) in the model! Skipped. 'backbone.bottom_up.res4.4.conv2.norm.weight' has shape (256,) in the checkpoint but (1024,) in the model! Skipped. 'backbone.bottom_up.res4.4.conv2.norm.bias' has shape (256,) in the checkpoint but (1024,) in the model! Skipped. 'backbone.bottom_up.res4.4.conv2.norm.running_mean' has shape (256,) in the checkpoint but (1024,) in the model! Skipped. 'backbone.bottom_up.res4.4.conv2.norm.running_var' has shape (256,) in the checkpoint but (1024,) in the model! Skipped. 'backbone.bottom_up.res4.4.conv3.weight' has shape (1024, 256, 1, 1) in the checkpoint but (1024, 1024, 1, 1) in the model! Skipped. 'backbone.bottom_up.res4.5.conv1.weight' has shape (256, 1024, 1, 1) in the checkpoint but (1024, 1024, 1, 1) in the model! Skipped. 'backbone.bottom_up.res4.5.conv1.norm.weight' has shape (256,) in the checkpoint but (1024,) in the model! Skipped. 'backbone.bottom_up.res4.5.conv1.norm.bias' has shape (256,) in the checkpoint but (1024,) in the model! Skipped. 'backbone.bottom_up.res4.5.conv1.norm.running_mean' has shape (256,) in the checkpoint but (1024,) in the model! Skipped. 'backbone.bottom_up.res4.5.conv1.norm.running_var' has shape (256,) in the checkpoint but (1024,) in the model! Skipped. 'backbone.bottom_up.res4.5.conv2.weight' has shape (256, 256, 3, 3) in the checkpoint but (1024, 32, 3, 3) in the model! Skipped. 'backbone.bottom_up.res4.5.conv2.norm.weight' has shape (256,) in the checkpoint but (1024,) in the model! Skipped. 'backbone.bottom_up.res4.5.conv2.norm.bias' has shape (256,) in the checkpoint but (1024,) in the model! Skipped. 'backbone.bottom_up.res4.5.conv2.norm.running_mean' has shape (256,) in the checkpoint but (1024,) in the model! Skipped. 'backbone.bottom_up.res4.5.conv2.norm.running_var' has shape (256,) in the checkpoint but (1024,) in the model! Skipped. 'backbone.bottom_up.res4.5.conv3.weight' has shape (1024, 256, 1, 1) in the checkpoint but (1024, 1024, 1, 1) in the model! Skipped. 'backbone.bottom_up.res4.6.conv1.weight' has shape (256, 1024, 1, 1) in the checkpoint but (1024, 1024, 1, 1) in the model! Skipped. 'backbone.bottom_up.res4.6.conv1.norm.weight' has shape (256,) in the checkpoint but (1024,) in the model! Skipped. 'backbone.bottom_up.res4.6.conv1.norm.bias' has shape (256,) in the checkpoint but (1024,) in the model! Skipped. 'backbone.bottom_up.res4.6.conv1.norm.running_mean' has shape (256,) in the checkpoint but (1024,) in the model! Skipped. 'backbone.bottom_up.res4.6.conv1.norm.running_var' has shape (256,) in the checkpoint but (1024,) in the model! Skipped. 'backbone.bottom_up.res4.6.conv2.weight' has shape (256, 256, 3, 3) in the checkpoint but (1024, 32, 3, 3) in the model! Skipped. 'backbone.bottom_up.res4.6.conv2.norm.weight' has shape (256,) in the checkpoint but (1024,) in the model! Skipped. 'backbone.bottom_up.res4.6.conv2.norm.bias' has shape (256,) in the checkpoint but (1024,) in the model! Skipped. 'backbone.bottom_up.res4.6.conv2.norm.running_mean' has shape (256,) in the checkpoint but (1024,) in the model! Skipped. 'backbone.bottom_up.res4.6.conv2.norm.running_var' has shape (256,) in the checkpoint but (1024,) in the model! Skipped. 'backbone.bottom_up.res4.6.conv3.weight' has shape (1024, 256, 1, 1) in the checkpoint but (1024, 1024, 1, 1) in the model! Skipped. 'backbone.bottom_up.res4.7.conv1.weight' has shape (256, 1024, 1, 1) in the checkpoint but (1024, 1024, 1, 1) in the model! Skipped. 'backbone.bottom_up.res4.7.conv1.norm.weight' has shape (256,) in the checkpoint but (1024,) in the model! Skipped. 'backbone.bottom_up.res4.7.conv1.norm.bias' has shape (256,) in the checkpoint but (1024,) in the model! Skipped. 'backbone.bottom_up.res4.7.conv1.norm.running_mean' has shape (256,) in the checkpoint but (1024,) in the model! Skipped. 'backbone.bottom_up.res4.7.conv1.norm.running_var' has shape (256,) in the checkpoint but (1024,) in the model! Skipped. 'backbone.bottom_up.res4.7.conv2.weight' has shape (256, 256, 3, 3) in the checkpoint but (1024, 32, 3, 3) in the model! Skipped. 'backbone.bottom_up.res4.7.conv2.norm.weight' has shape (256,) in the checkpoint but (1024,) in the model! Skipped. 'backbone.bottom_up.res4.7.conv2.norm.bias' has shape (256,) in the checkpoint but (1024,) in the model! Skipped. 'backbone.bottom_up.res4.7.conv2.norm.running_mean' has shape (256,) in the checkpoint but (1024,) in the model! Skipped. 'backbone.bottom_up.res4.7.conv2.norm.running_var' has shape (256,) in the checkpoint but (1024,) in the model! Skipped. 'backbone.bottom_up.res4.7.conv3.weight' has shape (1024, 256, 1, 1) in the checkpoint but (1024, 1024, 1, 1) in the model! Skipped. 'backbone.bottom_up.res4.8.conv1.weight' has shape (256, 1024, 1, 1) in the checkpoint but (1024, 1024, 1, 1) in the model! Skipped. 'backbone.bottom_up.res4.8.conv1.norm.weight' has shape (256,) in the checkpoint but (1024,) in the model! Skipped. 'backbone.bottom_up.res4.8.conv1.norm.bias' has shape (256,) in the checkpoint but (1024,) in the model! Skipped. 'backbone.bottom_up.res4.8.conv1.norm.running_mean' has shape (256,) in the checkpoint but (1024,) in the model! Skipped. 'backbone.bottom_up.res4.8.conv1.norm.running_var' has shape (256,) in the checkpoint but (1024,) in the model! Skipped. 'backbone.bottom_up.res4.8.conv2.weight' has shape (256, 256, 3, 3) in the checkpoint but (1024, 32, 3, 3) in the model! Skipped. 'backbone.bottom_up.res4.8.conv2.norm.weight' has shape (256,) in the checkpoint but (1024,) in the model! Skipped. 'backbone.bottom_up.res4.8.conv2.norm.bias' has shape (256,) in the checkpoint but (1024,) in the model! Skipped. 'backbone.bottom_up.res4.8.conv2.norm.running_mean' has shape (256,) in the checkpoint but (1024,) in the model! Skipped. 'backbone.bottom_up.res4.8.conv2.norm.running_var' has shape (256,) in the checkpoint but (1024,) in the model! Skipped. 'backbone.bottom_up.res4.8.conv3.weight' has shape (1024, 256, 1, 1) in the checkpoint but (1024, 1024, 1, 1) in the model! Skipped. 'backbone.bottom_up.res4.9.conv1.weight' has shape (256, 1024, 1, 1) in the checkpoint but (1024, 1024, 1, 1) in the model! Skipped. 'backbone.bottom_up.res4.9.conv1.norm.weight' has shape (256,) in the checkpoint but (1024,) in the model! Skipped. 'backbone.bottom_up.res4.9.conv1.norm.bias' has shape (256,) in the checkpoint but (1024,) in the model! Skipped. 'backbone.bottom_up.res4.9.conv1.norm.running_mean' has shape (256,) in the checkpoint but (1024,) in the model! Skipped. 'backbone.bottom_up.res4.9.conv1.norm.running_var' has shape (256,) in the checkpoint but (1024,) in the model! Skipped. 'backbone.bottom_up.res4.9.conv2.weight' has shape (256, 256, 3, 3) in the checkpoint but (1024, 32, 3, 3) in the model! Skipped. 'backbone.bottom_up.res4.9.conv2.norm.weight' has shape (256,) in the checkpoint but (1024,) in the model! Skipped. 'backbone.bottom_up.res4.9.conv2.norm.bias' has shape (256,) in the checkpoint but (1024,) in the model! Skipped. 'backbone.bottom_up.res4.9.conv2.norm.running_mean' has shape (256,) in the checkpoint but (1024,) in the model! Skipped. 'backbone.bottom_up.res4.9.conv2.norm.running_var' has shape (256,) in the checkpoint but (1024,) in the model! Skipped. 'backbone.bottom_up.res4.9.conv3.weight' has shape (1024, 256, 1, 1) in the checkpoint but (1024, 1024, 1, 1) in the model! Skipped. 'backbone.bottom_up.res4.10.conv1.weight' has shape (256, 1024, 1, 1) in the checkpoint but (1024, 1024, 1, 1) in the model! Skipped. 'backbone.bottom_up.res4.10.conv1.norm.weight' has shape (256,) in the checkpoint but (1024,) in the model! Skipped. 'backbone.bottom_up.res4.10.conv1.norm.bias' has shape (256,) in the checkpoint but (1024,) in the model! Skipped. 'backbone.bottom_up.res4.10.conv1.norm.running_mean' has shape (256,) in the checkpoint but (1024,) in the model! Skipped. 'backbone.bottom_up.res4.10.conv1.norm.running_var' has shape (256,) in the checkpoint but (1024,) in the model! Skipped. 'backbone.bottom_up.res4.10.conv2.weight' has shape (256, 256, 3, 3) in the checkpoint but (1024, 32, 3, 3) in the model! Skipped. 'backbone.bottom_up.res4.10.conv2.norm.weight' has shape (256,) in the checkpoint but (1024,) in the model! Skipped. 'backbone.bottom_up.res4.10.conv2.norm.bias' has shape (256,) in the checkpoint but (1024,) in the model! Skipped. 'backbone.bottom_up.res4.10.conv2.norm.running_mean' has shape (256,) in the checkpoint but (1024,) in the model! Skipped. 'backbone.bottom_up.res4.10.conv2.norm.running_var' has shape (256,) in the checkpoint but (1024,) in the model! Skipped. 'backbone.bottom_up.res4.10.conv3.weight' has shape (1024, 256, 1, 1) in the checkpoint but (1024, 1024, 1, 1) in the model! Skipped. 'backbone.bottom_up.res4.11.conv1.weight' has shape (256, 1024, 1, 1) in the checkpoint but (1024, 1024, 1, 1) in the model! Skipped. 'backbone.bottom_up.res4.11.conv1.norm.weight' has shape (256,) in the checkpoint but (1024,) in the model! Skipped. 'backbone.bottom_up.res4.11.conv1.norm.bias' has shape (256,) in the checkpoint but (1024,) in the model! Skipped. 'backbone.bottom_up.res4.11.conv1.norm.running_mean' has shape (256,) in the checkpoint but (1024,) in the model! Skipped. 'backbone.bottom_up.res4.11.conv1.norm.running_var' has shape (256,) in the checkpoint but (1024,) in the model! Skipped. 'backbone.bottom_up.res4.11.conv2.weight' has shape (256, 256, 3, 3) in the checkpoint but (1024, 32, 3, 3) in the model! Skipped. 'backbone.bottom_up.res4.11.conv2.norm.weight' has shape (256,) in the checkpoint but (1024,) in the model! Skipped. 'backbone.bottom_up.res4.11.conv2.norm.bias' has shape (256,) in the checkpoint but (1024,) in the model! Skipped. 'backbone.bottom_up.res4.11.conv2.norm.running_mean' has shape (256,) in the checkpoint but (1024,) in the model! Skipped. 'backbone.bottom_up.res4.11.conv2.norm.running_var' has shape (256,) in the checkpoint but (1024,) in the model! Skipped. 'backbone.bottom_up.res4.11.conv3.weight' has shape (1024, 256, 1, 1) in the checkpoint but (1024, 1024, 1, 1) in the model! Skipped. 'backbone.bottom_up.res4.12.conv1.weight' has shape (256, 1024, 1, 1) in the checkpoint but (1024, 1024, 1, 1) in the model! Skipped. 'backbone.bottom_up.res4.12.conv1.norm.weight' has shape (256,) in the checkpoint but (1024,) in the model! Skipped. 'backbone.bottom_up.res4.12.conv1.norm.bias' has shape (256,) in the checkpoint but (1024,) in the model! Skipped. 'backbone.bottom_up.res4.12.conv1.norm.running_mean' has shape (256,) in the checkpoint but (1024,) in the model! Skipped. 'backbone.bottom_up.res4.12.conv1.norm.running_var' has shape (256,) in the checkpoint but (1024,) in the model! Skipped. 'backbone.bottom_up.res4.12.conv2.weight' has shape (256, 256, 3, 3) in the checkpoint but (1024, 32, 3, 3) in the model! Skipped. 'backbone.bottom_up.res4.12.conv2.norm.weight' has shape (256,) in the checkpoint but (1024,) in the model! Skipped. 'backbone.bottom_up.res4.12.conv2.norm.bias' has shape (256,) in the checkpoint but (1024,) in the model! Skipped. 'backbone.bottom_up.res4.12.conv2.norm.running_mean' has shape (256,) in the checkpoint but (1024,) in the model! Skipped. 'backbone.bottom_up.res4.12.conv2.norm.running_var' has shape (256,) in the checkpoint but (1024,) in the model! Skipped. 'backbone.bottom_up.res4.12.conv3.weight' has shape (1024, 256, 1, 1) in the checkpoint but (1024, 1024, 1, 1) in the model! Skipped. 'backbone.bottom_up.res4.13.conv1.weight' has shape (256, 1024, 1, 1) in the checkpoint but (1024, 1024, 1, 1) in the model! Skipped. 'backbone.bottom_up.res4.13.conv1.norm.weight' has shape (256,) in the checkpoint but (1024,) in the model! Skipped. 'backbone.bottom_up.res4.13.conv1.norm.bias' has shape (256,) in the checkpoint but (1024,) in the model! Skipped. 'backbone.bottom_up.res4.13.conv1.norm.running_mean' has shape (256,) in the checkpoint but (1024,) in the model! Skipped. 'backbone.bottom_up.res4.13.conv1.norm.running_var' has shape (256,) in the checkpoint but (1024,) in the model! Skipped. 'backbone.bottom_up.res4.13.conv2.weight' has shape (256, 256, 3, 3) in the checkpoint but (1024, 32, 3, 3) in the model! Skipped. 'backbone.bottom_up.res4.13.conv2.norm.weight' has shape (256,) in the checkpoint but (1024,) in the model! Skipped. 'backbone.bottom_up.res4.13.conv2.norm.bias' has shape (256,) in the checkpoint but (1024,) in the model! Skipped. 'backbone.bottom_up.res4.13.conv2.norm.running_mean' has shape (256,) in the checkpoint but (1024,) in the model! Skipped. 'backbone.bottom_up.res4.13.conv2.norm.running_var' has shape (256,) in the checkpoint but (1024,) in the model! Skipped. 'backbone.bottom_up.res4.13.conv3.weight' has shape (1024, 256, 1, 1) in the checkpoint but (1024, 1024, 1, 1) in the model! Skipped. 'backbone.bottom_up.res4.14.conv1.weight' has shape (256, 1024, 1, 1) in the checkpoint but (1024, 1024, 1, 1) in the model! Skipped. 'backbone.bottom_up.res4.14.conv1.norm.weight' has shape (256,) in the checkpoint but (1024,) in the model! Skipped. 'backbone.bottom_up.res4.14.conv1.norm.bias' has shape (256,) in the checkpoint but (1024,) in the model! Skipped. 'backbone.bottom_up.res4.14.conv1.norm.running_mean' has shape (256,) in the checkpoint but (1024,) in the model! Skipped. 'backbone.bottom_up.res4.14.conv1.norm.running_var' has shape (256,) in the checkpoint but (1024,) in the model! Skipped. 'backbone.bottom_up.res4.14.conv2.weight' has shape (256, 256, 3, 3) in the checkpoint but (1024, 32, 3, 3) in the model! Skipped. 'backbone.bottom_up.res4.14.conv2.norm.weight' has shape (256,) in the checkpoint but (1024,) in the model! Skipped. 'backbone.bottom_up.res4.14.conv2.norm.bias' has shape (256,) in the checkpoint but (1024,) in the model! Skipped. 'backbone.bottom_up.res4.14.conv2.norm.running_mean' has shape (256,) in the checkpoint but (1024,) in the model! Skipped. 'backbone.bottom_up.res4.14.conv2.norm.running_var' has shape (256,) in the checkpoint but (1024,) in the model! Skipped. 'backbone.bottom_up.res4.14.conv3.weight' has shape (1024, 256, 1, 1) in the checkpoint but (1024, 1024, 1, 1) in the model! Skipped. 'backbone.bottom_up.res4.15.conv1.weight' has shape (256, 1024, 1, 1) in the checkpoint but (1024, 1024, 1, 1) in the model! Skipped. 'backbone.bottom_up.res4.15.conv1.norm.weight' has shape (256,) in the checkpoint but (1024,) in the model! Skipped. 'backbone.bottom_up.res4.15.conv1.norm.bias' has shape (256,) in the checkpoint but (1024,) in the model! Skipped. 'backbone.bottom_up.res4.15.conv1.norm.running_mean' has shape (256,) in the checkpoint but (1024,) in the model! Skipped. 'backbone.bottom_up.res4.15.conv1.norm.running_var' has shape (256,) in the checkpoint but (1024,) in the model! Skipped. 'backbone.bottom_up.res4.15.conv2.weight' has shape (256, 256, 3, 3) in the checkpoint but (1024, 32, 3, 3) in the model! Skipped. 'backbone.bottom_up.res4.15.conv2.norm.weight' has shape (256,) in the checkpoint but (1024,) in the model! Skipped. 'backbone.bottom_up.res4.15.conv2.norm.bias' has shape (256,) in the checkpoint but (1024,) in the model! Skipped. 'backbone.bottom_up.res4.15.conv2.norm.running_mean' has shape (256,) in the checkpoint but (1024,) in the model! Skipped. 'backbone.bottom_up.res4.15.conv2.norm.running_var' has shape (256,) in the checkpoint but (1024,) in the model! Skipped. 'backbone.bottom_up.res4.15.conv3.weight' has shape (1024, 256, 1, 1) in the checkpoint but (1024, 1024, 1, 1) in the model! Skipped. 'backbone.bottom_up.res4.16.conv1.weight' has shape (256, 1024, 1, 1) in the checkpoint but (1024, 1024, 1, 1) in the model! Skipped. 'backbone.bottom_up.res4.16.conv1.norm.weight' has shape (256,) in the checkpoint but (1024,) in the model! Skipped. 'backbone.bottom_up.res4.16.conv1.norm.bias' has shape (256,) in the checkpoint but (1024,) in the model! Skipped. 'backbone.bottom_up.res4.16.conv1.norm.running_mean' has shape (256,) in the checkpoint but (1024,) in the model! Skipped. 'backbone.bottom_up.res4.16.conv1.norm.running_var' has shape (256,) in the checkpoint but (1024,) in the model! Skipped. 'backbone.bottom_up.res4.16.conv2.weight' has shape (256, 256, 3, 3) in the checkpoint but (1024, 32, 3, 3) in the model! Skipped. 'backbone.bottom_up.res4.16.conv2.norm.weight' has shape (256,) in the checkpoint but (1024,) in the model! Skipped. 'backbone.bottom_up.res4.16.conv2.norm.bias' has shape (256,) in the checkpoint but (1024,) in the model! Skipped. 'backbone.bottom_up.res4.16.conv2.norm.running_mean' has shape (256,) in the checkpoint but (1024,) in the model! Skipped. 'backbone.bottom_up.res4.16.conv2.norm.running_var' has shape (256,) in the checkpoint but (1024,) in the model! Skipped. 'backbone.bottom_up.res4.16.conv3.weight' has shape (1024, 256, 1, 1) in the checkpoint but (1024, 1024, 1, 1) in the model! Skipped. 'backbone.bottom_up.res4.17.conv1.weight' has shape (256, 1024, 1, 1) in the checkpoint but (1024, 1024, 1, 1) in the model! Skipped. 'backbone.bottom_up.res4.17.conv1.norm.weight' has shape (256,) in the checkpoint but (1024,) in the model! Skipped. 'backbone.bottom_up.res4.17.conv1.norm.bias' has shape (256,) in the checkpoint but (1024,) in the model! Skipped. 'backbone.bottom_up.res4.17.conv1.norm.running_mean' has shape (256,) in the checkpoint but (1024,) in the model! Skipped. 'backbone.bottom_up.res4.17.conv1.norm.running_var' has shape (256,) in the checkpoint but (1024,) in the model! Skipped. 'backbone.bottom_up.res4.17.conv2.weight' has shape (256, 256, 3, 3) in the checkpoint but (1024, 32, 3, 3) in the model! Skipped. 'backbone.bottom_up.res4.17.conv2.norm.weight' has shape (256,) in the checkpoint but (1024,) in the model! Skipped. 'backbone.bottom_up.res4.17.conv2.norm.bias' has shape (256,) in the checkpoint but (1024,) in the model! Skipped. 'backbone.bottom_up.res4.17.conv2.norm.running_mean' has shape (256,) in the checkpoint but (1024,) in the model! Skipped. 'backbone.bottom_up.res4.17.conv2.norm.running_var' has shape (256,) in the checkpoint but (1024,) in the model! Skipped. 'backbone.bottom_up.res4.17.conv3.weight' has shape (1024, 256, 1, 1) in the checkpoint but (1024, 1024, 1, 1) in the model! Skipped. 'backbone.bottom_up.res4.18.conv1.weight' has shape (256, 1024, 1, 1) in the checkpoint but (1024, 1024, 1, 1) in the model! Skipped. 'backbone.bottom_up.res4.18.conv1.norm.weight' has shape (256,) in the checkpoint but (1024,) in the model! Skipped. 'backbone.bottom_up.res4.18.conv1.norm.bias' has shape (256,) in the checkpoint but (1024,) in the model! Skipped. 'backbone.bottom_up.res4.18.conv1.norm.running_mean' has shape (256,) in the checkpoint but (1024,) in the model! Skipped. 'backbone.bottom_up.res4.18.conv1.norm.running_var' has shape (256,) in the checkpoint but (1024,) in the model! Skipped. 'backbone.bottom_up.res4.18.conv2.weight' has shape (256, 256, 3, 3) in the checkpoint but (1024, 32, 3, 3) in the model! Skipped. 'backbone.bottom_up.res4.18.conv2.norm.weight' has shape (256,) in the checkpoint but (1024,) in the model! Skipped. 'backbone.bottom_up.res4.18.conv2.norm.bias' has shape (256,) in the checkpoint but (1024,) in the model! Skipped. 'backbone.bottom_up.res4.18.conv2.norm.running_mean' has shape (256,) in the checkpoint but (1024,) in the model! Skipped. 'backbone.bottom_up.res4.18.conv2.norm.running_var' has shape (256,) in the checkpoint but (1024,) in the model! Skipped. 'backbone.bottom_up.res4.18.conv3.weight' has shape (1024, 256, 1, 1) in the checkpoint but (1024, 1024, 1, 1) in the model! Skipped. 'backbone.bottom_up.res4.19.conv1.weight' has shape (256, 1024, 1, 1) in the checkpoint but (1024, 1024, 1, 1) in the model! Skipped. 'backbone.bottom_up.res4.19.conv1.norm.weight' has shape (256,) in the checkpoint but (1024,) in the model! Skipped. 'backbone.bottom_up.res4.19.conv1.norm.bias' has shape (256,) in the checkpoint but (1024,) in the model! Skipped. 'backbone.bottom_up.res4.19.conv1.norm.running_mean' has shape (256,) in the checkpoint but (1024,) in the model! Skipped. 'backbone.bottom_up.res4.19.conv1.norm.running_var' has shape (256,) in the checkpoint but (1024,) in the model! Skipped. 'backbone.bottom_up.res4.19.conv2.weight' has shape (256, 256, 3, 3) in the checkpoint but (1024, 32, 3, 3) in the model! Skipped. 'backbone.bottom_up.res4.19.conv2.norm.weight' has shape (256,) in the checkpoint but (1024,) in the model! Skipped. 'backbone.bottom_up.res4.19.conv2.norm.bias' has shape (256,) in the checkpoint but (1024,) in the model! Skipped. 'backbone.bottom_up.res4.19.conv2.norm.running_mean' has shape (256,) in the checkpoint but (1024,) in the model! Skipped. 'backbone.bottom_up.res4.19.conv2.norm.running_var' has shape (256,) in the checkpoint but (1024,) in the model! Skipped. 'backbone.bottom_up.res4.19.conv3.weight' has shape (1024, 256, 1, 1) in the checkpoint but (1024, 1024, 1, 1) in the model! Skipped. 'backbone.bottom_up.res4.20.conv1.weight' has shape (256, 1024, 1, 1) in the checkpoint but (1024, 1024, 1, 1) in the model! Skipped. 'backbone.bottom_up.res4.20.conv1.norm.weight' has shape (256,) in the checkpoint but (1024,) in the model! Skipped. 'backbone.bottom_up.res4.20.conv1.norm.bias' has shape (256,) in the checkpoint but (1024,) in the model! Skipped. 'backbone.bottom_up.res4.20.conv1.norm.running_mean' has shape (256,) in the checkpoint but (1024,) in the model! Skipped. 'backbone.bottom_up.res4.20.conv1.norm.running_var' has shape (256,) in the checkpoint but (1024,) in the model! Skipped. 'backbone.bottom_up.res4.20.conv2.weight' has shape (256, 256, 3, 3) in the checkpoint but (1024, 32, 3, 3) in the model! Skipped. 'backbone.bottom_up.res4.20.conv2.norm.weight' has shape (256,) in the checkpoint but (1024,) in the model! Skipped. 'backbone.bottom_up.res4.20.conv2.norm.bias' has shape (256,) in the checkpoint but (1024,) in the model! Skipped. 'backbone.bottom_up.res4.20.conv2.norm.running_mean' has shape (256,) in the checkpoint but (1024,) in the model! Skipped. 'backbone.bottom_up.res4.20.conv2.norm.running_var' has shape (256,) in the checkpoint but (1024,) in the model! Skipped. 'backbone.bottom_up.res4.20.conv3.weight' has shape (1024, 256, 1, 1) in the checkpoint but (1024, 1024, 1, 1) in the model! Skipped. 'backbone.bottom_up.res4.21.conv1.weight' has shape (256, 1024, 1, 1) in the checkpoint but (1024, 1024, 1, 1) in the model! Skipped. 'backbone.bottom_up.res4.21.conv1.norm.weight' has shape (256,) in the checkpoint but (1024,) in the model! Skipped. 'backbone.bottom_up.res4.21.conv1.norm.bias' has shape (256,) in the checkpoint but (1024,) in the model! Skipped. 'backbone.bottom_up.res4.21.conv1.norm.running_mean' has shape (256,) in the checkpoint but (1024,) in the model! Skipped. 'backbone.bottom_up.res4.21.conv1.norm.running_var' has shape (256,) in the checkpoint but (1024,) in the model! Skipped. 'backbone.bottom_up.res4.21.conv2.weight' has shape (256, 256, 3, 3) in the checkpoint but (1024, 32, 3, 3) in the model! Skipped. 'backbone.bottom_up.res4.21.conv2.norm.weight' has shape (256,) in the checkpoint but (1024,) in the model! Skipped. 'backbone.bottom_up.res4.21.conv2.norm.bias' has shape (256,) in the checkpoint but (1024,) in the model! Skipped. 'backbone.bottom_up.res4.21.conv2.norm.running_mean' has shape (256,) in the checkpoint but (1024,) in the model! Skipped. 'backbone.bottom_up.res4.21.conv2.norm.running_var' has shape (256,) in the checkpoint but (1024,) in the model! Skipped. 'backbone.bottom_up.res4.21.conv3.weight' has shape (1024, 256, 1, 1) in the checkpoint but (1024, 1024, 1, 1) in the model! Skipped. 'backbone.bottom_up.res4.22.conv1.weight' has shape (256, 1024, 1, 1) in the checkpoint but (1024, 1024, 1, 1) in the model! Skipped. 'backbone.bottom_up.res4.22.conv1.norm.weight' has shape (256,) in the checkpoint but (1024,) in the model! Skipped. 'backbone.bottom_up.res4.22.conv1.norm.bias' has shape (256,) in the checkpoint but (1024,) in the model! Skipped. 'backbone.bottom_up.res4.22.conv1.norm.running_mean' has shape (256,) in the checkpoint but (1024,) in the model! Skipped. 'backbone.bottom_up.res4.22.conv1.norm.running_var' has shape (256,) in the checkpoint but (1024,) in the model! Skipped. 'backbone.bottom_up.res4.22.conv2.weight' has shape (256, 256, 3, 3) in the checkpoint but (1024, 32, 3, 3) in the model! Skipped. 'backbone.bottom_up.res4.22.conv2.norm.weight' has shape (256,) in the checkpoint but (1024,) in the model! Skipped. 'backbone.bottom_up.res4.22.conv2.norm.bias' has shape (256,) in the checkpoint but (1024,) in the model! Skipped. 'backbone.bottom_up.res4.22.conv2.norm.running_mean' has shape (256,) in the checkpoint but (1024,) in the model! Skipped. 'backbone.bottom_up.res4.22.conv2.norm.running_var' has shape (256,) in the checkpoint but (1024,) in the model! Skipped. 'backbone.bottom_up.res4.22.conv3.weight' has shape (1024, 256, 1, 1) in the checkpoint but (1024, 1024, 1, 1) in the model! Skipped. 'backbone.bottom_up.res5.0.conv1.weight' has shape (512, 1024, 1, 1) in the checkpoint but (2048, 1024, 1, 1) in the model! Skipped. 'backbone.bottom_up.res5.0.conv1.norm.weight' has shape (512,) in the checkpoint but (2048,) in the model! Skipped. 'backbone.bottom_up.res5.0.conv1.norm.bias' has shape (512,) in the checkpoint but (2048,) in the model! Skipped. 'backbone.bottom_up.res5.0.conv1.norm.running_mean' has shape (512,) in the checkpoint but (2048,) in the model! Skipped. 'backbone.bottom_up.res5.0.conv1.norm.running_var' has shape (512,) in the checkpoint but (2048,) in the model! Skipped. 'backbone.bottom_up.res5.0.conv2.weight' has shape (512, 512, 3, 3) in the checkpoint but (2048, 64, 3, 3) in the model! Skipped. 'backbone.bottom_up.res5.0.conv2.norm.weight' has shape (512,) in the checkpoint but (2048,) in the model! Skipped. 'backbone.bottom_up.res5.0.conv2.norm.bias' has shape (512,) in the checkpoint but (2048,) in the model! Skipped. 'backbone.bottom_up.res5.0.conv2.norm.running_mean' has shape (512,) in the checkpoint but (2048,) in the model! Skipped. 'backbone.bottom_up.res5.0.conv2.norm.running_var' has shape (512,) in the checkpoint but (2048,) in the model! Skipped. 'backbone.bottom_up.res5.0.conv3.weight' has shape (2048, 512, 1, 1) in the checkpoint but (2048, 2048, 1, 1) in the model! Skipped. 'backbone.bottom_up.res5.1.conv1.weight' has shape (512, 2048, 1, 1) in the checkpoint but (2048, 2048, 1, 1) in the model! Skipped. 'backbone.bottom_up.res5.1.conv1.norm.weight' has shape (512,) in the checkpoint but (2048,) in the model! Skipped. 'backbone.bottom_up.res5.1.conv1.norm.bias' has shape (512,) in the checkpoint but (2048,) in the model! Skipped. 'backbone.bottom_up.res5.1.conv1.norm.running_mean' has shape (512,) in the checkpoint but (2048,) in the model! Skipped. 'backbone.bottom_up.res5.1.conv1.norm.running_var' has shape (512,) in the checkpoint but (2048,) in the model! Skipped. 'backbone.bottom_up.res5.1.conv2.weight' has shape (512, 512, 3, 3) in the checkpoint but (2048, 64, 3, 3) in the model! Skipped. 'backbone.bottom_up.res5.1.conv2.norm.weight' has shape (512,) in the checkpoint but (2048,) in the model! Skipped. 'backbone.bottom_up.res5.1.conv2.norm.bias' has shape (512,) in the checkpoint but (2048,) in the model! Skipped. 'backbone.bottom_up.res5.1.conv2.norm.running_mean' has shape (512,) in the checkpoint but (2048,) in the model! Skipped. 'backbone.bottom_up.res5.1.conv2.norm.running_var' has shape (512,) in the checkpoint but (2048,) in the model! Skipped. 'backbone.bottom_up.res5.1.conv3.weight' has shape (2048, 512, 1, 1) in the checkpoint but (2048, 2048, 1, 1) in the model! Skipped. 'backbone.bottom_up.res5.2.conv1.weight' has shape (512, 2048, 1, 1) in the checkpoint but (2048, 2048, 1, 1) in the model! Skipped. 'backbone.bottom_up.res5.2.conv1.norm.weight' has shape (512,) in the checkpoint but (2048,) in the model! Skipped. 'backbone.bottom_up.res5.2.conv1.norm.bias' has shape (512,) in the checkpoint but (2048,) in the model! Skipped. 'backbone.bottom_up.res5.2.conv1.norm.running_mean' has shape (512,) in the checkpoint but (2048,) in the model! Skipped. 'backbone.bottom_up.res5.2.conv1.norm.running_var' has shape (512,) in the checkpoint but (2048,) in the model! Skipped. 'backbone.bottom_up.res5.2.conv2.weight' has shape (512, 512, 3, 3) in the checkpoint but (2048, 64, 3, 3) in the model! Skipped. 'backbone.bottom_up.res5.2.conv2.norm.weight' has shape (512,) in the checkpoint but (2048,) in the model! Skipped. 'backbone.bottom_up.res5.2.conv2.norm.bias' has shape (512,) in the checkpoint but (2048,) in the model! Skipped. 'backbone.bottom_up.res5.2.conv2.norm.running_mean' has shape (512,) in the checkpoint but (2048,) in the model! Skipped. 'backbone.bottom_up.res5.2.conv2.norm.running_var' has shape (512,) in the checkpoint but (2048,) in the model! Skipped. 'backbone.bottom_up.res5.2.conv3.weight' has shape (2048, 512, 1, 1) in the checkpoint but (2048, 2048, 1, 1) in the model! Skipped. 0% 0/1 [00:00<?, ?it/s][03/27 12:01:02 detectron2]: /content/gdrive/My Drive/Untitled folder/PMC4527132_00004.jpg: detected 0 instances in 15.44s 100% 1/1 [00:15<00:00, 15.52s/it] i use google colab and i use the pre trained weights given in repository

Hi,

I prepared a custom dataset in COCO format having 3 classes like below: But there is no samples related to 'Resumes' class. Only 'heading' and 'text' classes are present there in my sample.

"categories": [
        {
            "id": 0,
            "name": "Resumes",
            "supercategory": "none"
        },
        {
            "id": 1,
            "name": "heading",
            "supercategory": "Resumes"
        },
        {
            "id": 2,
            "name": "text",
            "supercategory": "Resumes"
        }
    ],
"images":[{
            "id": 159,
            "license": 1,
            "file_name": "outfile_Lavanya-4_10---2--docx-pdf_1.rf.bfcd3081e0b5399829ae9b1bdc1e67d4.jpg",
            "height": 842,
            "width": 596,
            "date_captured": "2021-09-09T13:55:03+00:00"
        },
    ],
    "annotations": [
        {
            "id": 0,
            "image_id": 0,
            "category_id": 2,
            "bbox": [
                7,
                6,
                266.66666666666663,
                101.28205128205127
            ],
            "area": 27008.547008547,
            "segmentation": [
                [
                    7,
                    6,
                    266.66666666666663,
                    6,
                    266.66666666666663,
                    101.28205128205127,
                    7,
                    101.28205128205127
                ]
            ],
            "iscrowd": 0
        },
    ]

I'm using this config file for finetuning

_BASE_: "Base-RCNN-FPN.yaml"
MODEL:
  MASK_ON: True
#  WEIGHTS: "detectron2://ImageNetPretrained/FAIR/X-101-32x8d.pkl"
  #WEIGHTS: "detectron2://COCO-InstanceSegmentation/mask_rcnn_X_101_32x8d_FPN_3x/139653917/model_final_2d9806.pkl"
  PIXEL_STD: [57.375, 57.120, 58.395]
  ROI_HEADS:
    NUM_CLASSES: 5
  RESNETS:
    STRIDE_IN_1X1: False  # this is a C2 model
    NUM_GROUPS: 32
    WIDTH_PER_GROUP: 8
    DEPTH: 101
DATASETS:
  TRAIN: ("dla_train",)
  TEST: ("dla_val",)
SOLVER:
  STEPS: (210000, 250000)
  MAX_ITER: 125500
  IMS_PER_BATCH: 2
  #BASE_LR: 0.0009
  BASE_LR: 0.00005
DATALOADER:
  NUM_WORKERS: 1

Got following error:-

WARNING [09/09 20:18:55 d2.data.datasets.coco]: 
Category ids in annotations are not in [1, #categories]! We'll apply a mapping for you.

[09/09 20:18:55 d2.data.datasets.coco]: Loaded 638 images in COCO format from ./data/train/annotations.json
[09/09 20:18:55 d2.data.build]: Removed 320 images with no usable annotations. 318 images left.
[09/09 20:18:55 d2.data.build]: Distribution of instances among all 3 categories:
|  category  | #instances   |  category  | #instances   |  category  | #instances   |
|:----------:|:-------------|:----------:|:-------------|:----------:|:-------------|
|  Resumes   | 0            |  heading   | 1028         |    text    | 1951         |
|            |              |            |              |            |              |
|   total    | 2979         |            |              |            |              |
[09/09 20:18:55 d2.data.detection_utils]: TransformGens used in training: [ResizeShortestEdge(short_edge_length=(640, 672, 704, 736, 768, 800), max_size=1333, sample_style='choice'), RandomFlip()]
[09/09 20:18:55 d2.data.build]: Using training sampler TrainingSampler
[09/09 20:18:56 d2.engine.train_loop]: Starting training from iteration 75500
ERROR [09/09 20:18:57 d2.engine.train_loop]: Exception during training:
Traceback (most recent call last):
  File "/home/ujjawal/miniconda2/envs/caffe2/lib/python3.7/site-packages/detectron2/engine/train_loop.py", line 132, in train
    self.run_step()
  File "/home/ujjawal/miniconda2/envs/caffe2/lib/python3.7/site-packages/detectron2/engine/train_loop.py", line 216, in run_step
    self._detect_anomaly(losses, loss_dict)
  File "/home/ujjawal/miniconda2/envs/caffe2/lib/python3.7/site-packages/detectron2/engine/train_loop.py", line 239, in _detect_anomaly
    self.iter, loss_dict
FloatingPointError: Loss became infinite or NaN at iteration=75501!
loss_dict = {'loss_cls': tensor(nan, device='cuda:0', grad_fn=<NllLossBackward>), 'loss_box_reg': tensor(nan, device='cuda:0', grad_fn=<DivBackward0>), 'loss_mask': tensor(0.7118, device='cuda:0', grad_fn=<BinaryCrossEntropyWithLogitsBackward>), 'loss_rpn_cls': tensor(0.6949, device='cuda:0', grad_fn=<MulBackward0>), 'loss_rpn_loc': tensor(0.4812, device='cuda:0', grad_fn=<MulBackward0>)}

I tried to change the NUM_CLASSES: 5 to 3 but no luck. Some suggested to reduce the LR still no luck. Can anyone please suggest a way to tackle this issue?

Firstly, thank you so much @hpanwar08 for your contributions! I was trying to use Mask_RCNN but I observed that whenever data is a little bit spreaded like in a table, then it fails to record it. There is no bbox in that case. Consider the following picture as a reference.

Can you help me with this? Thank you

Hi @hpanwar08 Thank you very much for your contributions.

Currently, I trained detectron2 (use RCNN-FPN network) with PubLayNet. However, I want to use dilated convolution for this network (RCNN-FPN network, not DC5 network).

Could you please give me the guideline to training?

Thank you very much.

🚀 Feature

Hi, is there an internal feature which lets each classed be saved as a seperate segment, or image? I am trying to identify tables, seperate and then run through a tabular data analyzer and ocr - so far am able to get the image predictions with your code, but not the actual annotations/segmented fields for further analysis/ocr.

Motivation

Pitch

Note that you can implement many features by extending detectron2. See projects for some examples. We would only consider adding new features if they are relevant to many users.