mmdetection version of TinyBenchmark.

Last update: Aug 27, 2022

Related tags

Deep Learning TOV_mmdetection

Overview

introduction

This project is an mmdetection version of TinyBenchmark.

TODO list:

add TinyPerson dataset and evaluation
add crop and merge for image during inference
implement RetinaNet and Faster-FPN baseline on TinyPerson
add SM/MSM experiment support
add visDronePerson dataset support and baseline performance
add point localization task for TinyPerson
add point localization task for visDronePerson
add point localization task for COCO

install and setup

download project

git clone https://github.com/ucas-vg/TOV_mmdetection --recursive

install mmdetection

conda create -n open-mmlab python=3.7 -y
conda activate open-mmlab
conda install -c pytorch pytorch=1.5.0 cudatoolkit=10.2 torchvision -y  # (recommand)
# install latest pytorch prebuilt with the default prebuilt CUDA version (usually the latest)
# conda install -c pytorch pytorch torchvision -y

# install the latest mmcv
pip install mmcv-full --user
# install mmdetection

cd TOV_mmdetection
pip uninstall pycocotools
pip install -r requirements/build.txt
pip install -v -e . --user  # or "python setup.py develop"

For more detail, please refer mmdetection install to install mmdetecion.

Quickly Start

to train baseline of TinyPerson, download the mini_annotation of all annotation is enough, which can be downloaded as tiny_set/mini_annotations.tar.gz in Baidu Yun(password:pmcq) / Google Driver.

mkdir data
ln -s $Path of TinyPerson$ data/tiny_set
tar -zxvf data/tiny_set/mini_annotations.tar.gz && mv mini_annotations data/tiny_set/

# run experiment, for other config run, see exp/Baseline_TinyPerson.sh
export GPU=4 && LR=02 && CUDA_VISIBLE_DEVICES=0,1,2,3 PORT=10000 tools/dist_train.sh configs2/TinyPerson/base/faster_rcnn_r50_fpn_1x_TinyPerson640.py $GPU \
  --work-dir ../TOV_mmdetection_cache/work_dir/TinyPerson/Base/faster_rcnn_r50_fpn_1x_TinyPerson640/old640x512_lr0${LR}_1x_${GPU}g/ \
  --cfg-options optimizer.lr=0.${LR}

performance

All train and test on 2080Ti,

CUDA10.1/10.2
python3.7, cudatookit=10.2, pytorch=1.5, torchvision=0.6

for Faster-FPN, we think the gain compare to TinyBenchmark may come from the cut and merge during inference running time and multi-gpu training.

performance 43.80(2) where 2 means the performance is mean result of running such setting for 2 time.

detector	num_gpu	$AP_{50}^{tiny}$	script
Faster-FPN	4	48.63(1)	exp/Baseline_TinyPerson.sh:exp1.1
Adap RetainaNet	1	43.80(2)	exp/Baseline_TinyPerson.sh:exp2.1
Adap RetainaNet	4	44.94(1)	exp/Baseline_TinyPerson.sh:exp2.2(clip grad)

Comments

COCO val $mmap$

Sorry, I don't know which precision this COCO val $mmap$ represents. Which of AP0.5, AP0.75, AP0.5-0.95, APs, APm and APl is obtained by the network pre-training on the coco data set?

opened by sxy1122 7
Run foveabox
Hi, I also use foveabox to run this dataset, but foveabox does not support test-time augmentations; instead, I use the function simple_test_bboxes (in dense_test_mixins.py), but the type of input does not match, so I rewrite the function, but when testing, it shows null results，I would like to know what I have to do to get this algorithm up and running and have results, I hope you can give me some advice, thanks!

I first simply bbox the elements in the tuples in turn, but the test results are empty

results_list = [] for i, feat in enumerate(feats): outs = self.forward(feat) results_list += self.get_bboxes(*outs, img_metas[i], rescale=rescale) return results_list

Then I modified the code again, but it will report the following error error：get_bboxes() got multiple values for argument 'rescale'

outs = [] for feat in feats: outs.append(self.forward(feat)) results_list = self.get_bboxes(*outs, img_metas, rescale=rescale) return results_list

Finally I solved the problem with the following code changes，but also reported a new error, I do not know if it makes sense to continue to modify， I don't know what's wrong with it. ：error： File "/home/xxxx/TOV_mmdetection-main/mmdet/models/dense_heads/fovea_head.py", line 277, in featmap_sizes = [featmap.size()[-2:] for featmap in cls_scores] AttributeError: 'list' object has no attribute 'size'

cls_scores = [] bbox_preds = [] for feat in feats: cls_score, bbox_pred = self.forward(feat) cls_scores.append(cls_score) bbox_preds.append(bbox_pred) results_list = self.get_bboxes(cls_scores, bbox_preds, img_metas, rescale=rescale) return results_list
opened by Hshuqin 7

Run fcos with the tiny person dataset

hello，I modified the fcos profile based on three existing profiles (retinanet, faster rcnn, reppoints), but the final result is very low performance( Or, is that how it performs ? ), I don't know where the problem exists, can you give me some advice?

My configuration file is as follows.

`_base_ = [
    '../../_base_/datasets/TinyPerson/TinyPerson_detection_640x512.py',
    '../../../configs/_base_/schedules/schedule_1x.py', '../../../configs/_base_/default_runtime.py'
]
optimizer = dict(type='SGD', lr=0.005, momentum=0.9, weight_decay=0.0001)  # single GPU
model = dict(
    type='FCOS',
    pretrained='open-mmlab://detectron/resnet50_caffe',
    backbone=dict(
        type='ResNet',
        depth=50,
        num_stages=4,
        out_indices=(0, 1, 2, 3),
        frozen_stages=1,
        norm_cfg=dict(type='BN', requires_grad=False),
        norm_eval=True,
        style='caffe'),
    neck=dict(
        type='FPN',
        in_channels=[256, 512, 1024, 2048],
        out_channels=256,
        start_level=1,
        add_extra_convs=True,
        extra_convs_on_inputs=False,  # use P5
        num_outs=5,
        relu_before_extra_convs=True),
    bbox_head=dict(
        type='FCOSHead',
        num_classes=1, # 80
        in_channels=256,
        stacked_convs=4,
        feat_channels=256,
        strides=[8, 16, 32, 64, 128],
        loss_cls=dict(
            type='FocalLoss',
            use_sigmoid=True,
            gamma=2.0,
            alpha=0.25,
            loss_weight=1.0),
        loss_bbox=dict(type='IoULoss', loss_weight=1.0),
        loss_centerness=dict(
            type='CrossEntropyLoss', use_sigmoid=True, loss_weight=1.0)),
 
    train_cfg=dict(
        assigner=dict(
            type='MaxIoUAssigner',
            pos_iou_thr=0.5,
            neg_iou_thr=0.4,
            min_pos_iou=0,
            ignore_iof_thr=-1),
        allowed_border=-1,
        pos_weight=-1,
        debug=False),
    test_cfg=dict(
        nms_pre=1000,
        min_bbox_size=0,
        score_thr=0.05,
        nms=dict(type='nms', iou_threshold=0.5),
        max_per_img=200))  # 100
img_norm_cfg = dict(
    mean=[102.9801, 115.9465, 122.7717], std=[1.0, 1.0, 1.0], to_rgb=False)
optimizer_config = dict(
    _delete_=True, grad_clip=dict(max_norm=35, norm_type=2))
runner = dict(type='EpochBasedRunner', max_epochs=12)
`
My results are as follows：

2021-07-04 14:09:28,657 - mmdet - INFO -  Average Precision  (AP) @[ IoU=0.25      | area=   all | maxDets=200 ] = 0.0387
2021-07-04 14:09:28,657 - mmdet - INFO -  Average Precision  (AP) @[ IoU=0.25      | area=  tiny | maxDets=200 ] = 0.0419
2021-07-04 14:09:28,657 - mmdet - INFO -  Average Precision  (AP) @[ IoU=0.25      | area= tiny1 | maxDets=200 ] = 0.0000
2021-07-04 14:09:28,658 - mmdet - INFO -  Average Precision  (AP) @[ IoU=0.25      | area= tiny2 | maxDets=200 ] = 0.0198
2021-07-04 14:09:28,658 - mmdet - INFO -  Average Precision  (AP) @[ IoU=0.25      | area= tiny3 | maxDets=200 ] = 0.0898
2021-07-04 14:09:28,658 - mmdet - INFO -  Average Precision  (AP) @[ IoU=0.25      | area= small | maxDets=200 ] = 0.0872
2021-07-04 14:09:28,658 - mmdet - INFO -  Average Precision  (AP) @[ IoU=0.25      | area=reasonable | maxDets=200 ] = 0.1180
2021-07-04 14:09:28,658 - mmdet - INFO -  Average Precision  (AP) @[ IoU=0.50      | area=   all | maxDets=200 ] = 0.0083
2021-07-04 14:09:28,658 - mmdet - INFO -  Average Precision  (AP) @[ IoU=0.50      | area=  tiny | maxDets=200 ] = 0.0099
2021-07-04 14:09:28,659 - mmdet - INFO -  Average Precision  (AP) @[ IoU=0.50      | area= tiny1 | maxDets=200 ] = 0.0000
2021-07-04 14:09:28,659 - mmdet - INFO -  Average Precision  (AP) @[ IoU=0.50      | area= tiny2 | maxDets=200 ] = 0.0000
2021-07-04 14:09:28,659 - mmdet - INFO -  Average Precision  (AP) @[ IoU=0.50      | area= tiny3 | maxDets=200 ] = 0.0102
2021-07-04 14:09:28,659 - mmdet - INFO -  Average Precision  (AP) @[ IoU=0.50      | area= small | maxDets=200 ] = 0.0051
2021-07-04 14:09:28,659 - mmdet - INFO -  Average Precision  (AP) @[ IoU=0.50      | area=reasonable | maxDets=200 ] = 0.0333
2021-07-04 14:09:28,659 - mmdet - INFO -  Average Precision  (AP) @[ IoU=0.75      | area=   all | maxDets=200 ] = 0.0012
2021-07-04 14:09:28,659 - mmdet - INFO -  Average Precision  (AP) @[ IoU=0.75      | area=  tiny | maxDets=200 ] = 0.0002
2021-07-04 14:09:28,660 - mmdet - INFO -  Average Precision  (AP) @[ IoU=0.75      | area= tiny1 | maxDets=200 ] = 0.0000
2021-07-04 14:09:28,660 - mmdet - INFO -  Average Precision  (AP) @[ IoU=0.75      | area= tiny2 | maxDets=200 ] = 0.0000
2021-07-04 14:09:28,660 - mmdet - INFO -  Average Precision  (AP) @[ IoU=0.75      | area= tiny3 | maxDets=200 ] = 0.0002
2021-07-04 14:09:28,660 - mmdet - INFO -  Average Precision  (AP) @[ IoU=0.75      | area= small | maxDets=200 ] = 0.0000
2021-07-04 14:09:28,660 - mmdet - INFO -  Average Precision  (AP) @[ IoU=0.75      | area=reasonable | maxDets=200 ] = 0.0014
2021-07-04 14:09:28,660 - mmdet - INFO -  Average Recall     (AR) @[ IoU=0.25      | area=   all | maxDets=200 ] = 0.2699
2021-07-04 14:09:28,661 - mmdet - INFO -  Average Recall     (AR) @[ IoU=0.25      | area=  tiny | maxDets=200 ] = 0.0731
2021-07-04 14:09:28,661 - mmdet - INFO -  Average Recall     (AR) @[ IoU=0.25      | area= tiny1 | maxDets=200 ] = 0.0000
2021-07-04 14:09:28,661 - mmdet - INFO -  Average Recall     (AR) @[ IoU=0.25      | area= tiny2 | maxDets=200 ] = 0.0104
2021-07-04 14:09:28,661 - mmdet - INFO -  Average Recall     (AR) @[ IoU=0.25      | area= tiny3 | maxDets=200 ] = 0.1646
2021-07-04 14:09:28,661 - mmdet - INFO -  Average Recall     (AR) @[ IoU=0.25      | area= small | maxDets=200 ] = 0.5433
2021-07-04 14:09:28,661 - mmdet - INFO -  Average Recall     (AR) @[ IoU=0.25      | area=reasonable | maxDets=200 ] = 0.7332
2021-07-04 14:09:28,661 - mmdet - INFO -  Average Recall     (AR) @[ IoU=0.50      | area=   all | maxDets=200 ] = 0.1233
2021-07-04 14:09:28,662 - mmdet - INFO -  Average Recall     (AR) @[ IoU=0.50      | area=  tiny | maxDets=200 ] = 0.0081
2021-07-04 14:09:28,662 - mmdet - INFO -  Average Recall     (AR) @[ IoU=0.50      | area= tiny1 | maxDets=200 ] = 0.0000
2021-07-04 14:09:28,662 - mmdet - INFO -  Average Recall     (AR) @[ IoU=0.50      | area= tiny2 | maxDets=200 ] = 0.0000
2021-07-04 14:09:28,662 - mmdet - INFO -  Average Recall     (AR) @[ IoU=0.50      | area= tiny3 | maxDets=200 ] = 0.0191
2021-07-04 14:09:28,662 - mmdet - INFO -  Average Recall     (AR) @[ IoU=0.50      | area= small | maxDets=200 ] = 0.2072
2021-07-04 14:09:28,662 - mmdet - INFO -  Average Recall     (AR) @[ IoU=0.50      | area=reasonable | maxDets=200 ] = 0.4733
2021-07-04 14:09:28,662 - mmdet - INFO -  Average Recall     (AR) @[ IoU=0.75      | area=   all | maxDets=200 ] = 0.0084
2021-07-04 14:09:28,663 - mmdet - INFO -  Average Recall     (AR) @[ IoU=0.75      | area=  tiny | maxDets=200 ] = 0.0002
2021-07-04 14:09:28,663 - mmdet - INFO -  Average Recall     (AR) @[ IoU=0.75      | area= tiny1 | maxDets=200 ] = 0.0000
2021-07-04 14:09:28,663 - mmdet - INFO -  Average Recall     (AR) @[ IoU=0.75      | area= tiny2 | maxDets=200 ] = 0.0000
2021-07-04 14:09:28,663 - mmdet - INFO -  Average Recall     (AR) @[ IoU=0.75      | area= tiny3 | maxDets=200 ] = 0.0005
2021-07-04 14:09:28,663 - mmdet - INFO -  Average Recall     (AR) @[ IoU=0.75      | area= small | maxDets=200 ] = 0.0080
2021-07-04 14:09:28,663 - mmdet - INFO -  Average Recall     (AR) @[ IoU=0.75      | area=reasonable | maxDets=200 ] = 0.0400

opened by Hshuqin 3

The result of Faster RCNN mAP 49.81

Hello, I'm doing some research on small object detection. When I use your mmdetection based code, I follow "configs2/TinyPerson/base/Baseline_TinyPerson.sh:exp1.2 "can never achieve better results than map48. Can you give me a detailed experimental configuration and experimental log, or give me some suggestions?

I used two Titan V GPUs and tried a variety of combinations of batch and LR. The best result was only map0.4745 at one time. The following is my experiment log.

model = dict(
    type='FasterRCNN',
    backbone=dict(
        type='ResNet',
        depth=50,
        num_stages=4,
        out_indices=(0, 1, 2, 3),
        frozen_stages=1,
        norm_cfg=dict(type='BN', requires_grad=False),
        norm_eval=True,
        style='pytorch',
        init_cfg=dict(type='Pretrained', checkpoint='torchvision://resnet50')),
    neck=dict(
        type='FPN',
        in_channels=[256, 512, 1024, 2048],
        out_channels=256,
        num_outs=5),
    rpn_head=dict(
        type='RPNHead',
        in_channels=256,
        feat_channels=256,
        anchor_generator=dict(
            type='AnchorGenerator',
            scales=[2],
            ratios=[0.5, 1.0, 2.0],
            strides=[4, 8, 16, 32, 64]),
        bbox_coder=dict(
            type='DeltaXYWHBBoxCoder',
            target_means=[0.0, 0.0, 0.0, 0.0],
            target_stds=[1.0, 1.0, 1.0, 1.0]),
        loss_cls=dict(
            type='CrossEntropyLoss', use_sigmoid=True, loss_weight=1.0),
        loss_bbox=dict(type='L1Loss', loss_weight=1.0)),
    roi_head=dict(
        type='StandardRoIHead',
        bbox_roi_extractor=dict(
            type='SingleRoIExtractor',
            roi_layer=dict(type='RoIAlign', output_size=7, sampling_ratio=0),
            out_channels=256,
            featmap_strides=[4, 8, 16, 32]),
        bbox_head=dict(
            type='Shared2FCBBoxHead',
            in_channels=256,
            fc_out_channels=1024,
            roi_feat_size=7,
            num_classes=1,
            bbox_coder=dict(
                type='DeltaXYWHBBoxCoder',
                target_means=[0.0, 0.0, 0.0, 0.0],
                target_stds=[0.1, 0.1, 0.2, 0.2]),
            reg_class_agnostic=False,
            loss_cls=dict(
                type='CrossEntropyLoss', use_sigmoid=False, loss_weight=1.0),
            loss_bbox=dict(type='L1Loss', loss_weight=1.0))),
    train_cfg=dict(
        rpn=dict(
            assigner=dict(
                type='MaxIoUAssigner',
                pos_iou_thr=0.7,
                neg_iou_thr=0.3,
                min_pos_iou=0.3,
                match_low_quality=True,
                ignore_iof_thr=-1),
            sampler=dict(
                type='RandomSampler',
                num=256,
                pos_fraction=0.5,
                neg_pos_ub=-1,
                add_gt_as_proposals=False),
            allowed_border=-1,
            pos_weight=-1,
            debug=False),
        rpn_proposal=dict(
            nms_pre=2000,
            max_per_img=1000,
            nms=dict(type='nms', iou_threshold=0.7),
            min_bbox_size=0,
            nms_across_levels=False,
            nms_post=1000,
            max_num=1000),
        rcnn=dict(
            assigner=dict(
                type='MaxIoUAssigner',
                pos_iou_thr=0.5,
                neg_iou_thr=0.5,
                min_pos_iou=0.5,
                match_low_quality=False,
                ignore_iof_thr=-1),
            sampler=dict(
                type='RandomSampler',
                num=512,
                pos_fraction=0.25,
                neg_pos_ub=-1,
                add_gt_as_proposals=True),
            pos_weight=-1,
            debug=False)),
    test_cfg=dict(
        rpn=dict(
            nms_pre=1000,
            max_per_img=1000,
            nms=dict(type='nms', iou_threshold=0.7),
            min_bbox_size=0,
            nms_across_levels=False,
            nms_post=1000,
            max_num=1000),
        rcnn=dict(
            score_thr=0.05,
            nms=dict(type='nms', iou_threshold=0.5),
            max_per_img=-1,
            do_tile_as_aug=False)))
dataset_type = 'CocoFmtDataset'
data_root = './data/tiny_set/'
img_norm_cfg = dict(
    mean=[123.675, 116.28, 103.53], std=[58.395, 57.12, 57.375], to_rgb=True)
train_pipeline = [
    dict(type='LoadImageFromFile'),
    dict(type='LoadAnnotations', with_bbox=True),
    dict(type='Resize', scale_factor=[1.0], keep_ratio=True),
    dict(type='RandomFlip', flip_ratio=0.5),
    dict(
        type='Normalize',
        mean=[123.675, 116.28, 103.53],
        std=[58.395, 57.12, 57.375],
        to_rgb=True),
    dict(type='Pad', size_divisor=32),
    dict(type='DefaultFormatBundle'),
    dict(
        type='Collect',
        keys=['img', 'gt_bboxes', 'gt_labels', 'gt_bboxes_ignore'])
]
test_pipeline = [
    dict(type='LoadImageFromFile'),
    dict(
        type='CroppedTilesFlipAug',
        tile_shape=(640, 512),
        tile_overlap=(100, 100),
        scale_factor=[1.0],
        flip=False,
        transforms=[
            dict(type='Resize', keep_ratio=True),
            dict(type='RandomFlip'),
            dict(
                type='Normalize',
                mean=[123.675, 116.28, 103.53],
                std=[58.395, 57.12, 57.375],
                to_rgb=True),
            dict(type='Pad', size_divisor=32),
            dict(type='ImageToTensor', keys=['img']),
            dict(type='Collect', keys=['img'])
        ])
]
data = dict(
    samples_per_gpu=1,
    workers_per_gpu=1,
    train=dict(
        type='CocoFmtDataset',
        ann_file=
        './data/tiny_set/mini_annotations/tiny_set_train_sw640_sh512_all_erase.json',
        img_prefix=
        './data/tiny_set/erase_with_uncertain_dataset/train/',
        pipeline=[
            dict(type='LoadImageFromFile'),
            dict(type='LoadAnnotations', with_bbox=True),
            dict(type='Resize', scale_factor=[1.0], keep_ratio=True),
            dict(type='RandomFlip', flip_ratio=0.5),
            dict(
                type='Normalize',
                mean=[123.675, 116.28, 103.53],
                std=[58.395, 57.12, 57.375],
                to_rgb=True),
            dict(type='Pad', size_divisor=32),
            dict(type='DefaultFormatBundle'),
            dict(
                type='Collect',
                keys=['img', 'gt_bboxes', 'gt_labels', 'gt_bboxes_ignore'])
        ]),
    val=dict(
        type='CocoFmtDataset',
        ann_file=
        './data/tiny_set/mini_annotations/tiny_set_test_all.json',
        img_prefix='./data/tiny_set/test/',
        pipeline=[
            dict(type='LoadImageFromFile'),
            dict(
                type='CroppedTilesFlipAug',
                tile_shape=(640, 512),
                tile_overlap=(100, 100),
                scale_factor=[1.0],
                flip=False,
                transforms=[
                    dict(type='Resize', keep_ratio=True),
                    dict(type='RandomFlip'),
                    dict(
                        type='Normalize',
                        mean=[123.675, 116.28, 103.53],
                        std=[58.395, 57.12, 57.375],
                        to_rgb=True),
                    dict(type='Pad', size_divisor=32),
                    dict(type='ImageToTensor', keys=['img']),
                    dict(type='Collect', keys=['img'])
                ])
        ]),
    test=dict(
        type='CocoFmtDataset',
        ann_file=
        './data/tiny_set/mini_annotations/tiny_set_test_all.json',
        img_prefix='./data/tiny_set/test/',
        pipeline=[
            dict(type='LoadImageFromFile'),
            dict(
                type='CroppedTilesFlipAug',
                tile_shape=(640, 512),
                tile_overlap=(100, 100),
                scale_factor=[1.0],
                flip=False,
                transforms=[
                    dict(type='Resize', keep_ratio=True),
                    dict(type='RandomFlip'),
                    dict(
                        type='Normalize',
                        mean=[123.675, 116.28, 103.53],
                        std=[58.395, 57.12, 57.375],
                        to_rgb=True),
                    dict(type='Pad', size_divisor=32),
                    dict(type='ImageToTensor', keys=['img']),
                    dict(type='Collect', keys=['img'])
                ])
        ]))
check = dict(stop_while_nan=True)
evaluation = dict(
    interval=4,
    metric='bbox',
    iou_thrs=[0.25, 0.5, 0.75],
    proposal_nums=[200],
    cocofmt_kwargs=dict(
        ignore_uncertain=True,
        use_ignore_attr=True,
        use_iod_for_ignore=True,
        iod_th_of_iou_f='lambda iou: iou',
        cocofmt_param=dict(evaluate_standard='tiny')))
optimizer = dict(type='SGD', lr=0.01, momentum=0.9, weight_decay=0.0001)
optimizer_config = dict(grad_clip=None)
lr_config = dict(
    policy='step',
    warmup='linear',
    warmup_iters=500,
    warmup_ratio=0.001,
    step=[6, 10])
runner = dict(type='EpochBasedRunner', max_epochs=12)
checkpoint_config = dict(interval=1)
log_config = dict(interval=50, hooks=[dict(type='TextLoggerHook')])
custom_hooks = [dict(type='NumClassCheckHook')]
dist_params = dict(backend='nccl')
log_level = 'INFO'
load_from = None
resume_from = None
workflow = [('train', 1)]
work_dir = './checkpoints/TinyPerson/faster_rcnn_r50_fpn_1x_TinyPerson640/old640x512_lr0.01_1x_2g_1IMGpGPU/'
gpu_ids = range(0, 2)

Here are the results of my experiment

2021-08-23 15:39:36,557 - mmdet - INFO -  Average Precision  (AP) @[ IoU=0.25      | area=   all | maxDets=200 ] = 0.7075
2021-08-23 15:39:36,558 - mmdet - INFO -  Average Precision  (AP) @[ IoU=0.25      | area=  tiny | maxDets=200 ] = 0.6759
2021-08-23 15:39:36,558 - mmdet - INFO -  Average Precision  (AP) @[ IoU=0.25      | area= tiny1 | maxDets=200 ] = 0.4847
2021-08-23 15:39:36,558 - mmdet - INFO -  Average Precision  (AP) @[ IoU=0.25      | area= tiny2 | maxDets=200 ] = 0.7375
2021-08-23 15:39:36,558 - mmdet - INFO -  Average Precision  (AP) @[ IoU=0.25      | area= tiny3 | maxDets=200 ] = 0.7801
2021-08-23 15:39:36,558 - mmdet - INFO -  Average Precision  (AP) @[ IoU=0.25      | area= small | maxDets=200 ] = 0.8247
2021-08-23 15:39:36,558 - mmdet - INFO -  Average Precision  (AP) @[ IoU=0.25      | area=reasonable | maxDets=200 ] = 0.8164
2021-08-23 15:39:36,558 - mmdet - INFO -  Average Precision  (AP) @[ IoU=0.50      | area=   all | maxDets=200 ] = 0.5080
2021-08-23 15:39:36,558 - mmdet - INFO -  Average Precision  (AP) @[ IoU=0.50      | area=  tiny | maxDets=200 ] = 0.4745
2021-08-23 15:39:36,558 - mmdet - INFO -  Average Precision  (AP) @[ IoU=0.50      | area= tiny1 | maxDets=200 ] = 0.3062
2021-08-23 15:39:36,558 - mmdet - INFO -  Average Precision  (AP) @[ IoU=0.50      | area= tiny2 | maxDets=200 ] = 0.5281
2021-08-23 15:39:36,558 - mmdet - INFO -  Average Precision  (AP) @[ IoU=0.50      | area= tiny3 | maxDets=200 ] = 0.5844
2021-08-23 15:39:36,558 - mmdet - INFO -  Average Precision  (AP) @[ IoU=0.50      | area= small | maxDets=200 ] = 0.6339
2021-08-23 15:39:36,559 - mmdet - INFO -  Average Precision  (AP) @[ IoU=0.50      | area=reasonable | maxDets=200 ] = 0.6218
2021-08-23 15:39:36,559 - mmdet - INFO -  Average Precision  (AP) @[ IoU=0.75      | area=   all | maxDets=200 ] = 0.0759
2021-08-23 15:39:36,559 - mmdet - INFO -  Average Precision  (AP) @[ IoU=0.75      | area=  tiny | maxDets=200 ] = 0.0617
2021-08-23 15:39:36,559 - mmdet - INFO -  Average Precision  (AP) @[ IoU=0.75      | area= tiny1 | maxDets=200 ] = 0.0251
2021-08-23 15:39:36,559 - mmdet - INFO -  Average Precision  (AP) @[ IoU=0.75      | area= tiny2 | maxDets=200 ] = 0.0579
2021-08-23 15:39:36,559 - mmdet - INFO -  Average Precision  (AP) @[ IoU=0.75      | area= tiny3 | maxDets=200 ] = 0.0926
2021-08-23 15:39:36,559 - mmdet - INFO -  Average Precision  (AP) @[ IoU=0.75      | area= small | maxDets=200 ] = 0.1167
2021-08-23 15:39:36,559 - mmdet - INFO -  Average Precision  (AP) @[ IoU=0.75      | area=reasonable | maxDets=200 ] = 0.1183
2021-08-23 15:39:36,559 - mmdet - INFO -  Average Recall     (AR) @[ IoU=0.25      | area=   all | maxDets=200 ] = 0.8626
2021-08-23 15:39:36,559 - mmdet - INFO -  Average Recall     (AR) @[ IoU=0.25      | area=  tiny | maxDets=200 ] = 0.8344
2021-08-23 15:39:36,559 - mmdet - INFO -  Average Recall     (AR) @[ IoU=0.25      | area= tiny1 | maxDets=200 ] = 0.7085
2021-08-23 15:39:36,559 - mmdet - INFO -  Average Recall     (AR) @[ IoU=0.25      | area= tiny2 | maxDets=200 ] = 0.8701
2021-08-23 15:39:36,559 - mmdet - INFO -  Average Recall     (AR) @[ IoU=0.25      | area= tiny3 | maxDets=200 ] = 0.8925
2021-08-23 15:39:36,559 - mmdet - INFO -  Average Recall     (AR) @[ IoU=0.25      | area= small | maxDets=200 ] = 0.9151
2021-08-23 15:39:36,560 - mmdet - INFO -  Average Recall     (AR) @[ IoU=0.25      | area=reasonable | maxDets=200 ] = 0.9277
2021-08-23 15:39:36,560 - mmdet - INFO -  Average Recall     (AR) @[ IoU=0.50      | area=   all | maxDets=200 ] = 0.7154
2021-08-23 15:39:36,560 - mmdet - INFO -  Average Recall     (AR) @[ IoU=0.50      | area=  tiny | maxDets=200 ] = 0.6776
2021-08-23 15:39:36,560 - mmdet - INFO -  Average Recall     (AR) @[ IoU=0.50      | area= tiny1 | maxDets=200 ] = 0.5109
2021-08-23 15:39:36,560 - mmdet - INFO -  Average Recall     (AR) @[ IoU=0.50      | area= tiny2 | maxDets=200 ] = 0.7081
2021-08-23 15:39:36,560 - mmdet - INFO -  Average Recall     (AR) @[ IoU=0.50      | area= tiny3 | maxDets=200 ] = 0.7590
2021-08-23 15:39:36,560 - mmdet - INFO -  Average Recall     (AR) @[ IoU=0.50      | area= small | maxDets=200 ] = 0.7827
2021-08-23 15:39:36,560 - mmdet - INFO -  Average Recall     (AR) @[ IoU=0.50      | area=reasonable | maxDets=200 ] = 0.7900
2021-08-23 15:39:36,560 - mmdet - INFO -  Average Recall     (AR) @[ IoU=0.75      | area=   all | maxDets=200 ] = 0.2084
2021-08-23 15:39:36,560 - mmdet - INFO -  Average Recall     (AR) @[ IoU=0.75      | area=  tiny | maxDets=200 ] = 0.1829
2021-08-23 15:39:36,560 - mmdet - INFO -  Average Recall     (AR) @[ IoU=0.75      | area= tiny1 | maxDets=200 ] = 0.1020
2021-08-23 15:39:36,560 - mmdet - INFO -  Average Recall     (AR) @[ IoU=0.75      | area= tiny2 | maxDets=200 ] = 0.1839
2021-08-23 15:39:36,560 - mmdet - INFO -  Average Recall     (AR) @[ IoU=0.75      | area= tiny3 | maxDets=200 ] = 0.2323
2021-08-23 15:39:36,561 - mmdet - INFO -  Average Recall     (AR) @[ IoU=0.75      | area= small | maxDets=200 ] = 0.2447
2021-08-23 15:39:36,561 - mmdet - INFO -  Average Recall     (AR) @[ IoU=0.75      | area=reasonable | maxDets=200 ] = 0.2676
2021-08-23 15:39:36,606 - mmdet - INFO - Exp name: faster_rcnn_r50_fpn_1x_TinyPerson640.py
2021-08-23 15:39:36,607 - mmdet - INFO - Epoch(val) [12][393]	bbox_mAP: 0.7080, bbox_mAP_50: 0.6760, bbox_mAP_75: 0.4850, bbox_mAP_s: 0.7380, bbox_mAP_m: 0.7800, bbox_mAP_l: 0.8250, bbox_mAP_copypaste: 0.708 0.676 0.485 0.738 0.780 0.825

opened by range428 2

Weird ranges of bbox during evaluation.

The command I run is:

export GPU=1 && CONFIG="fcos_standard_r50_caffe_fpns4_gn-head_1x_TinyPerson640" && CUDA_VISIBLE_DEVICES=6 tools/dist_train.sh configs2/TinyPerson/base/${CONFIG}.py $GPU

During the evaluation every four epochs, part of the messages are shown like:

2021-08-05 12:27:13,251 - mmdet - INFO - Evaluating bbox... INFO:mmdet:Evaluating bbox... Loading and preparing results... DONE (t=1.22s) creating index... index created! {'cocoGt': <mmdet.datasets.api_wrappers.coco_api.COCO object at 0x7f65ba763050>, 'cocoDt': <pycocotools.coco.COCO object at 0x7f667c403b50>, 'evalImgs': defaultdict(<class 'list'>, {}), 'eval': {}, '_gts': defaultdict(<class 'list'>, {}), '_dts': defaultdict(<class 'list'>, {}), 'params': <huicv.evaluation.expand_cocofmt_eval.ExpandParam object at 0x7f65bbde3610>, '_paramsEval': {}, 'stats': [], 'ious': {}, 'use_ignore_attr': True, 'use_iod_for_ignore': True, 'ignore_uncertain': True, 'iod_th_of_iou_f': <function at 0x7f65bc762200>} {'evaluate_standard': 'tiny', 'catIds': [1], 'iouThrs': array([0.25, 0.5 , 0.75]), 'recThrs': array([0. , 0.01, 0.02, 0.03, 0.04, 0.05, 0.06, 0.07, 0.08, 0.09, 0.1 , 0.11, 0.12, 0.13, 0.14, 0.15, 0.16, 0.17, 0.18, 0.19, 0.2 , 0.21, 0.22, 0.23, 0.24, 0.25, 0.26, 0.27, 0.28, 0.29, 0.3 , 0.31, 0.32, 0.33, 0.34, 0.35, 0.36, 0.37, 0.38, 0.39, 0.4 , 0.41, 0.42, 0.43, 0.44, 0.45, 0.46, 0.47, 0.48, 0.49, 0.5 , 0.51, 0.52, 0.53, 0.54, 0.55, 0.56, 0.57, 0.58, 0.59, 0.6 , 0.61, 0.62, 0.63, 0.64, 0.65, 0.66, 0.67, 0.68, 0.69, 0.7 , 0.71, 0.72, 0.73, 0.74, 0.75, 0.76, 0.77, 0.78, 0.79, 0.8 , 0.81, 0.82, 0.83, 0.84, 0.85, 0.86, 0.87, 0.88, 0.89, 0.9 , 0.91, 0.92, 0.93, 0.94, 0.95, 0.96, 0.97, 0.98, 0.99, 1. ]), 'maxDets': [200], 'areaRng': [[1, 10000000000.0], [1, 400], [1, 64], [64, 144], [144, 400], [400, 1024], [1024, 10000000000.0]], 'areaRngLbl': ['all', 'tiny', 'tiny1', 'tiny2', 'tiny3', 'small', 'reasonable'], 'useCats': 1, 'iouType': 'bbox', 'useSegm': None}

I am wondering are the ranges listed in 'areaRng' correctly? It seems the ranges do not align with the labels in 'areaRngLbl'. From my understanding, the ranges should be [[1,inf],[2,20],[2,8],[8,12],[12,20],[20,32],[32,inf]]. Is there anything I misunderstand?

opened by freshn 2
details about evaluation listed on README.md

Excellent work👍👍👍 Could you please tell me does the evaluation results listed on README.md run on the tiny_set_test_all.json, i.e., using only 1 category person instead of splitting it into sea_person and earth_person while evaluating? Thank you for your release again! Looking forward to your reply! :)

opened by shaunyuan22 2
ModuleNotFoundError: No module named 'huicv' and MR

ModuleNotFoundError: No module named 'huicv'

After the training, I eval it, and I also imported the huicv package into the project. Why did he make an error and say that there was no module.

python huicv/evaluation/evaluate_tiny.py --res exp/latest_result.json --gt data/tiny_set/mini_annotations/tiny_set_test_all.json --detail

In addition, there is another question. The verification accuracy has been obtained during training. What do you mean in your project is to send the. JSON file obtained from training to the huicv module for verification again?

opened by sxy1122 1
Could you please tell me how to get the./exp/latest_result.json by myself?

In my understand,the json file was created at every epoch end,with running evaluation.the format like this: {"image_id": 794, "bbox": [1493.8309326171875, 1502.0306396484375, 49.6331787109375, 63.1834716796875], "score": 0.9902538657188416, "category_id": 1} but when I run with tools/test.py by myself,the json file I got like this: [1493.8309326171875, 1502.0306396484375, 1543.464111328125, 1565.214111328125, 0.9902538657188416] what I want to know is,what should I do to get the json file like latest_result.json with trained weights file?Much thanks.

opened by JustWeZero 1
How would we understand the folder 'exp'?

@yinglang I have no idea what is saved in the folder 'exp'. The '.json' file in that folder can be evaluated by huicv/evaluatation/evaluate_tiny.py but not the one in 'work_dir/.../.json'. Could you explain what is this folder used for? How can I evaluate the results based on the training log under folder 'work_dir' instead of 'exp'? Thank you.

opened by freshn 1

ModuleNotFoundError: No module named 'huicv'

I have run pip install -v -e . then I run ./tools/dist_train.sh configs2/mydataset/fcos_r50_caffe_fpn_gn-head_1x_mydataset_msm_tinyperson.py 2

error:

*****************************************
Setting OMP_NUM_THREADS environment variable for each process to be 1 in default, to avoid your system being overloaded, please further tune the variable for optimal performance in your application as needed. 
*****************************************
Traceback (most recent call last):
  File "./tools/train.py", line 15, in <module>
    from mmdet.apis import set_random_seed, train_detector
Traceback (most recent call last):
  File "/home/host/mounted2/lsy/PointTinyBenchmark/TOV_mmdetection/mmdet/apis/__init__.py", line 1, in <module>
  File "./tools/train.py", line 15, in <module>
    from .inference import (async_inference_detector, inference_detector,
  File "/home/host/mounted2/lsy/PointTinyBenchmark/TOV_mmdetection/mmdet/apis/inference.py", line 11, in <module>
        from mmdet.apis import set_random_seed, train_detectorfrom mmdet.datasets import replace_ImageToTensor
  File "/home/host/mounted2/lsy/PointTinyBenchmark/TOV_mmdetection/mmdet/datasets/__init__.py", line 15, in <module>

  File "/home/host/mounted2/lsy/PointTinyBenchmark/TOV_mmdetection/mmdet/apis/__init__.py", line 1, in <module>
    from .cocofmt import CocoFmtDataset  # add by hui
  File "/home/host/mounted2/lsy/PointTinyBenchmark/TOV_mmdetection/mmdet/datasets/cocofmt.py", line 12, in <module>
    from .inference import (async_inference_detector, inference_detector,
  File "/home/host/mounted2/lsy/PointTinyBenchmark/TOV_mmdetection/mmdet/apis/inference.py", line 11, in <module>
    from huicv.evaluation.expand_cocofmt_eval import COCOExpandEval
ModuleNotFoundError: No module named 'huicv'
    from mmdet.datasets import replace_ImageToTensor
  File "/home/host/mounted2/lsy/PointTinyBenchmark/TOV_mmdetection/mmdet/datasets/__init__.py", line 15, in <module>
    from .cocofmt import CocoFmtDataset  # add by hui
  File "/home/host/mounted2/lsy/PointTinyBenchmark/TOV_mmdetection/mmdet/datasets/cocofmt.py", line 12, in <module>
    from huicv.evaluation.expand_cocofmt_eval import COCOExpandEval
ModuleNotFoundError: No module named 'huicv'
Killing subprocess 3674929
Killing subprocess 3674930
Traceback (most recent call last):
  File "/home/host/mounted2/anaconda3/envs/MMDetection/lib/python3.7/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "/home/host/mounted2/anaconda3/envs/MMDetection/lib/python3.7/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/home/host/mounted2/anaconda3/envs/MMDetection/lib/python3.7/site-packages/torch/distributed/launch.py", line 340, in <module>
    main()
  File "/home/host/mounted2/anaconda3/envs/MMDetection/lib/python3.7/site-packages/torch/distributed/launch.py", line 326, in main
    sigkill_handler(signal.SIGTERM, None)  # not coming back
  File "/home/host/mounted2/anaconda3/envs/MMDetection/lib/python3.7/site-packages/torch/distributed/launch.py", line 301, in sigkill_handler
    raise subprocess.CalledProcessError(returncode=last_return_code, cmd=cmd)
subprocess.CalledProcessError: Command '['/home/host/mounted2/anaconda3/envs/MMDetection/bin/python', '-u', './tools/train.py', '--local_rank=1', 'configs2/mydataset/fcos_r50_caffe_fpn_gn-head_1x_mydataset_msm_tinyperson.py', '--launcher', 'pytorch']' returned non-zero exit status 1.

and then I run pip install huicv

but :

ERROR: Could not find a version that satisfies the requirement huicv (from versions: none)
ERROR: No matching distribution found for huicv

How can I solve this problem? Thank you!!

opened by little-spoon 0

Where is the AP50 of RepPoint？

Hello，I found that results(AP50tiny) of some experiments are not provided here, like exp3.4. So where can I find the results to judge whether my training is right or not? Or maybe is there any papers I can reference to? The result(AP50tiny) of exp3.4 in epoch 12 is only 34.13, which is very low.

opened by missyoudaisy 0
Reproduce SM on tinyperson dataset
Hello, I tried to run a few algorithms under that repository, but the performance of retinanet and fcos is better than sm(iou_thrs=[0.25, 0.5, 0.75]), I don't know if there is a problem with my configuration file, can you help me look at it? Or which parameters are not modified correctly？ configs file: TOV_mmdetection-main/configs2/TinyPerson/scale_match/retinanet_r50_fpns4_1x_coco_sm_tinyperson.py

A few important changes are as follows. Data： data = dict( samples_per_gpu=4, # 2 workers_per_gpu=1, train=dict( type=dataset_type, # ann_file=data_root + 'erase_with_uncertain_dataset/annotations/corner/task/tiny_set_train_sw640_sh512_all.json', ann_file=data_root + 'mini_annotations/tiny_set_train_sw640_sh512_all_erase.json', # same as last line img_prefix=data_root + 'erase_with_uncertain_dataset/train/', pipeline=train_pipeline, # train_ignore_as_bg=False, ), val=dict( type=dataset_type, # ann_file=data_root + 'annotations/corner/task/tiny_set_test_sw640_sh512_all.json', ann_file=data_root + 'mini_annotations/tiny_set_test_all.json',

img_prefix=data_root + 'test/', pipeline=test_pipeline), test=dict( type=dataset_type, # ann_file=data_root + 'annotations/corner/task/tiny_set_test_sw640_sh512_all.json', ann_file=data_root + 'mini_annotations/tiny_set_test_all.json', img_prefix=data_root + 'test/', pipeline=test_pipeline) )

Evaluation： evaluation = dict(interval=1, metric='bbox', iou_thrs=[0.25, 0.5, 0.75], # set None mean use 0.5:1.0::0.05 proposal_nums=[200], cocofmt_kwargs=dict( ignore_uncertain=True, use_ignore_attr=True, use_iod_for_ignore=True, iod_th_of_iou_f="lambda iou: iou", #"lambda iou: (2*iou)/(1+iou)", cocofmt_param=dict( evaluate_standard='tiny', # or 'coco' # iouThrs=[0.25, 0.5, 0.75], # set this same as set evaluation.iou_thrs # maxDets=[200], # set this same as set evaluation.proposal_nums ) ))

In the test pipeline, the img_scale was modified: # img_scale=(333, 200), img_scale=(640, 512),

In the train pipline, anno_file was modified: anno_file="/home/xxxxx/data/tiny_set/mini_annotations/tiny_set_train_all_erase.json",

Other configurations follow the original settings. Looking forward to your suggestions
opened by Hshuqin 15
can you show the result of detection on the picture ? the result is bad

I run test.py to show the result,finding the result is bad.

python tools/test.py configs2/TinyPreson/base/faster_rcnn_r50_fpn_1x_TinyPerson640.py ./work_dirs/...../lastest.pth --show

opened by JXQI 3

Owner

GitHub https://github.com/ucas-vg/TinyBenchmark

The official implementation of Equalization Loss v1 & v2 (CVPR 2020, 2021) based on MMDetection.

The Equalization Losses for Long-tailed Object Detection and Instance Segmentation This repo is official implementation CVPR 2021 paper: Equalization

129 Dec 16, 2022

Convert openmmlab (not only mmdetection) series model to tensorrt

MMDet to TensorRT This project aims to convert the mmdetection model to TensorRT model end2end. Focus on object detection for now. Mask support is exp

4 Dec 17, 2021

OBBDetection: an oriented object detection toolbox modified from MMdetection

OBBDetection note: If you have questions or good suggestions, feel free to propose issues and contact me. introduction OBBDetection is an oriented obj

3 Nov 11, 2022

RetinaNet-PyTorch - A RetinaNet Pytorch Implementation on remote sensing images and has the similar mAP result with RetinaNet in MMdetection

?? RetinaNet Horizontal Detector Based PyTorch This is a horizontal detector Ret

13 Nov 19, 2022

Hand gesture recognition based whiteboard that allows you to write on live webcam. This is the first version and has features like 4 different colors, eraser and a recording option that records your session and saves it in a "recordings" folder. Use index finger to draw and two or more fingers to move around and select items. Future version will contain more functionalities like changeable thickness, color palette, integration with zoom and google meet etc.

hand-write Hand gesture recognition based whiteboard that allows you to write on live webcam. This is the first version and has features like 4 differ

27 Dec 16, 2022

A PaddlePaddle version of Neural Renderer, refer to its PyTorch version

Neural 3D Mesh Renderer in PadddlePaddle A PaddlePaddle version of Neural Renderer, refer to its PyTorch version Install Run: pip install neural-rende

13 Jul 12, 2022

PyTorch version of Stable Baselines, reliable implementations of reinforcement learning algorithms.

4.7k Jan 1, 2023

Boundary IoU API (Beta version)

Boundary IoU API (Beta version) Bowen Cheng, Ross Girshick, Piotr Dollár, Alexander C. Berg, Alexander Kirillov [arXiv] [Project] [BibTeX] This API is

177 Dec 29, 2022

[Open Source]. The improved version of AnimeGAN. Landscape photos/videos to anime

4.4k Dec 27, 2022

YOLTv4 builds upon YOLT and SIMRDWN, and updates these frameworks to use the most performant version of YOLO, YOLOv4

YOLTv4 builds upon YOLT and SIMRDWN, and updates these frameworks to use the most performant version of YOLO, YOLOv4. YOLTv4 is designed to detect objects in aerial or satellite imagery in arbitrarily large images that far exceed the ~600×600 pixel size typically ingested by deep learning object detection frameworks.

161 Jan 6, 2023

A PaddlePaddle version image model zoo.

Paddle-Image-Models English | 简体中文 A PaddlePaddle version image model zoo. Install Package Install by pip： $ pip install ppim Install by wheel package

131 Dec 7, 2022

Notspot robot simulation - Python version

Notspot robot simulation - Python version This repository contains all the files and code needed to simulate the notspot quadrupedal robot using Gazeb

50 Sep 26, 2022

Official implementation of FCL-taco2: Fast, Controllable and Lightweight version of Tacotron2 @ ICASSP 2021

FCL-Taco2: Towards Fast, Controllable and Lightweight Text-to-Speech synthesis (ICASSP 2021) Paper | Demo Block diagram of FCL-taco2, where the decode

39 Sep 28, 2022

Pytorch version of VidLanKD: Improving Language Understanding viaVideo-Distilled Knowledge Transfer

VidLanKD Implementation of VidLanKD: Improving Language Understanding via Video-Distilled Knowledge Transfer by Zineng Tang, Jaemin Cho, Hao Tan, Mohi

54 Dec 20, 2022

The PyTorch improved version of TPAMI 2017 paper: Face Alignment in Full Pose Range: A 3D Total Solution.

Face Alignment in Full Pose Range: A 3D Total Solution By Jianzhu Guo. [Updates] 2020.8.30: The pre-trained model and code of ECCV-20 are made public

3.4k Jan 2, 2023

A modified version of DeepMind's Alphafold2 to divide CPU part (MSA and template searching) and GPU part (prediction model)

ParallelFold Author: Bozitao Zhong This is a modified version of DeepMind's Alphafold2 to divide CPU part (MSA and template searching) and GPU part (p

77 Dec 22, 2022

mmdetection version of TinyBenchmark.

Related tags

Overview

introduction

install and setup

Quickly Start

performance

Comments

COCO val $mmap$

Run foveabox

Run fcos with the tiny person dataset

The result of Faster RCNN mAP 49.81

Weird ranges of bbox during evaluation.

details about evaluation listed on README.md

ModuleNotFoundError: No module named 'huicv' and MR

Could you please tell me how to get the./exp/latest_result.json by myself?

How would we understand the folder 'exp'?