Alpha-IoU: A Family of Power Intersection over Union Losses for Bounding Box Regression

Overview

Alpha-IoU: A Family of Power Intersection over Union Losses for Bounding Box Regression

YOLOv5 with alpha-IoU losses implemented in PyTorch.

Example results on the test set of PASCAL VOC 2007 using YOLOv5s trained by the vanilla IoU loss (top row) and the alpha-IoU loss with alpha=3 (bottom row). The alpha-IoU loss performs better than the vanilla IoU loss because it can localize objects more accurately (image 1 and 2), thus can detect more true positive objects (image 3 to 5) and fewer false positive objects (image 6 and 7).

Example results on the val set of MS COCO 2017 using YOLOv5s trained by the vanilla IoU loss (top row) and the alpha-IoU loss with alpha=3 (bottom row). The alpha-IoU loss performs better than the vanilla IoU loss because it can localize objects more accurately (image 1), thus can detect more true positive objects (image 2 to 5) and fewer false positive objects (image 4 to 7). Note that image 4 and 5 detect both more true positive and fewer false positive objects.

Citation

If you use our method, please consider citing:

@inproceedings{Jiabo_Alpha-IoU,
  author    = {He, Jiabo and Erfani, Sarah and Ma, Xingjun and Bailey, James and Chi, Ying and Hua, Xian-Sheng},
  title     = {Alpha-IoU: A Family of Power Intersection over Union Losses for Bounding Box Regression},
  booktitle = {NeurIPS},
  year      = {2021},
}

Modifications

This repository is a fork of ultralytics/yolov5, with an implementation of alpha-IoU losses while keeping the code as close to the original as possible.

Alpha-IoU Losses

Alpha-IoU losses can be configured in Line 131 of utils/loss.py, functionesd as 'bbox_alpha_iou'. The alpha values and types of losses (e.g., IoU, GIoU, DIoU, CIoU) can be selected in this function, which are defined in utils/general.py. Note that we should use a small constant epsilon to avoid torch.pow(0, alpha) or denominator=0.

Install

Python>=3.6.0 is required with all requirements.txt installed including PyTorch>=1.7:

$ git clone https://github.com/Jacobi93/Alpha-IoU
$ cd Alpha-IoU
$ pip install -r requirements.txt

Configurations

Configuration files can be found in data. We do not change either 'voc.yaml' or 'coco.yaml' used in the original repository. However, we could do more experiments. E.g.,

voc25.yaml # randomly use 25% PASCAL VOC as the training set
voc50.yaml # randomly use 50% PASCAL VOC as the training set

Code for generating different small training sets is in generate_small_sets.py. Code for generating different noisy labels is in generate_noisy_labels.py, and we should change the 'img2label_paths' function in utils/datasets.py accordingly.

Implementation Commands

For detailed installation instruction and network training options, please take a look at the README file or issue of ultralytics/yolov5. Following are sample commands we used for training and testing YOLOv5 with alpha-IoU, with more samples in instruction.txt.

python train.py --data voc.yaml --hyp hyp.scratch.yaml --cfg yolov5s.yaml --batch-size 64 --epochs 300 --device '0'
python test.py --data voc.yaml --img 640 --conf 0.001 --weights 'runs/train/voc_yolov5s_iou/weights/best.pt' --device '0'
python detect.py --source ../VOC/images/detect500 --weights 'runs/train/voc_yolov5s_iou/weights/best.pt' --conf 0.25

We can also randomly generate some images for detection and visualization results in generate_detect_images.py.

Pretrained Weights

Here are some pretrained models using the configurations in this repository, with alpha=3 in all experiments. Details of these pretrained models can be found in runs/train. All results are tested using 'weights/best.pt' for each experiment. It is a very simple yet effective method so that people is able to quickly apply our method to existing models following the 'bbox_alpha_iou' function in utils/general.py. Note that YOLOv5 has been updated for many versions and all pretrained models in this repository are obtained based on the YOLOv5 version 4.0, where details of all versions for YOLOv5 can be found. Researchers are also welcome to apply our method to other object detection models, e.g., Faster R-CNN, DETR, etc.

Comments
  • question

    question

    Yolov5's original loss function is ciou. When a = 3, the accuracy of using a-ciou will decrease by 1.2%. Is it true that a-ciou is not as effective as ciou?

    opened by weizhao-11 5
  • AttributeError: Can't get attribute 'SPPF' on <module 'models.common' from '/home/zkx/graduation/Alpha-IoU/models/common.py'>

    AttributeError: Can't get attribute 'SPPF' on

    image Thanks for your excellent project. When I test the trained model, I find this error. In the file models/common.py, I can't find the class SSPF. Could you please tell me the cause of this problem? Thank you very much !

    opened by LoveYueForever 5
  • What is the loss weight of Faster R-CNN with alpha-iou?

    What is the loss weight of Faster R-CNN with alpha-iou?

    Hi, I have tried to train alpha-iou with the Faster R-CNN using mmdetection. And the results confused me, cuz I couldn't got the same result as you described in paper. And I guss it maybe the different of loss weight you defined. Can you share your loss weight of your alpha-iou pre-defined ? My config is as below:

    _base_ = './faster_rcnn_r50_fpn_1x_coco.py'
    model = dict(
        roi_head=dict(
            bbox_head=dict(
                reg_decoded_bbox=True,
                loss_bbox=dict(type='AlphaIoULoss', loss_weight=1.0)
            )
        )
    )
    

    Hope to get your reply as soon as possible :) Thanks a lot 🤖

    opened by CCODING04 4
  • question about the result.png in run/train and the result run by myself

    question about the result.png in run/train and the result run by myself

    Dear author, Thanks for your work. I have a question about the result.png in runs/train.The loss of objectness and val objectness rise in the initial stage in voc_yolov5s_3diou result.png,I don't know how to explain this phenomenon.Besides,I cloned your code and run on my computer, i set batch-size=8 and set the funtcion bbox_iou function(DIoU=true , alpha=1) .And i got the [email protected] = 75.04, [email protected]:0.95=47.95. Could you please give me some suggestion on it. Thanks.

    opened by tengxiao24 4
  •  How should EIOU be combined with Alpha-IOU?

    How should EIOU be combined with Alpha-IOU?

    Hi, I have applied alpha IOU to yolov5, but I also applied EIOU to the algorithm. It seems that the function in your file named "general.py" code does not contain EIOU. Would you like to know how to modify the code if you want to add it? The following is the code written by a blogger I saw on CSDN, but when I train the model, the effect is not as good as the alpha IOU or eiou provided by you. The training effect is reduced by combining these two items. I think it may be a code problem, (Maybe it is the problem of dataset that I used.) so I want to ask you how you can combine eiou and alpha IOU?

    # 参考:https://mp.weixin.qq.com/s/l22GJtA7Vd11dpY9QG4k2A
    def bbox_alpha_iou(box1, box2, x1y1x2y2=False, GIoU=False, DIoU=False, CIoU=False, EIoU=False, alpha=3, eps=1e-9):
        # Returns tsqrt_he IoU of box1 to box2. box1 is 4, box2 is nx4
        box2 = box2.T
    
        # Get the coordinates of bounding boxes
        if x1y1x2y2:  # x1, y1, x2, y2 = box1
            b1_x1, b1_y1, b1_x2, b1_y2 = box1[0], box1[1], box1[2], box1[3]
            b2_x1, b2_y1, b2_x2, b2_y2 = box2[0], box2[1], box2[2], box2[3]
        else:  # transform from xywh to xyxy
            b1_x1, b1_x2 = box1[0] - box1[2] / 2, box1[0] + box1[2] / 2
            b1_y1, b1_y2 = box1[1] - box1[3] / 2, box1[1] + box1[3] / 2
            b2_x1, b2_x2 = box2[0] - box2[2] / 2, box2[0] + box2[2] / 2
            b2_y1, b2_y2 = box2[1] - box2[3] / 2, box2[1] + box2[3] / 2
    
        # Intersection area
        inter = (torch.min(b1_x2, b2_x2) - torch.max(b1_x1, b2_x1)).clamp(0) * \
                (torch.min(b1_y2, b2_y2) - torch.max(b1_y1, b2_y1)).clamp(0)
    
        # Union Area
        w1, h1 = b1_x2 - b1_x1, b1_y2 - b1_y1 + eps
        w2, h2 = b2_x2 - b2_x1, b2_y2 - b2_y1 + eps
        union = w1 * h1 + w2 * h2 - inter + eps
    
        # change iou into pow(iou+eps) 加入α次幂
        # alpha iou
        iou = torch.pow(inter / union + eps, alpha)
        beta = 2 * alpha
        if GIoU or DIoU or CIoU or EIoU:
            # 两个框的最小闭包区域的width和height
            cw = torch.max(b1_x2, b2_x2) - torch.min(b1_x1, b2_x1)  # convex (smallest enclosing box) width
            ch = torch.max(b1_y2, b2_y2) - torch.min(b1_y1, b2_y1)  # convex height
    
            if CIoU or DIoU or EIoU:  # Distance or Complete IoU https://arxiv.org/abs/1911.08287v1
                # 最小外接矩形 对角线的长度平方
                c2 = cw ** beta + ch ** beta + eps  # convex diagonal
                rho_x = torch.abs(b2_x1 + b2_x2 - b1_x1 - b1_x2)
                rho_y = torch.abs(b2_y1 + b2_y2 - b1_y1 - b1_y2)
                # 两个框中心点之间距离的平方
                rho2 = (rho_x ** beta + rho_y ** beta) / (2 ** beta)  # center distance
                if DIoU:
                    return iou - rho2 / c2  # DIoU
    
                elif CIoU:  # https://github.com/Zzh-tju/DIoU-SSD-pytorch/blob/master/utils/box/box_utils.py#L47
                    v = (4 / math.pi ** 2) * torch.pow(torch.atan(w2 / h2) - torch.atan(w1 / h1), 2)
                    with torch.no_grad():
                        alpha_ciou = v / ((1 + eps) - inter / union + v)
                    # return iou - (rho2 / c2 + v * alpha_ciou)  # CIoU
                    return iou - (rho2 / c2 + torch.pow(v * alpha_ciou + eps, alpha))  # CIoU
    
                # EIoU 在CIoU的基础上
                # 将预测框宽高的纵横比损失项 拆分成预测框的宽高分别与最小外接框宽高的差值
                # 加速了收敛提高了回归精度
                elif EIoU:
                    rho_w2 = ((b2_x2 - b2_x1) - (b1_x2 - b1_x1)) ** beta
                    rho_h2 = ((b2_y2 - b2_y1) - (b1_y2 - b1_y1)) ** beta
                    cw2 = cw ** beta + eps
                    ch2 = ch ** beta + eps
                    return iou - (rho2 / c2 + rho_w2 / cw2 + rho_h2 / ch2)
    
                # GIoU https://arxiv.org/pdf/1902.09630.pdf
                c_area = torch.max(cw * ch + eps, union)  # convex area
                return iou - torch.pow((c_area - union) / c_area + eps, alpha)  # GIoU
        else:
            return iou  # torch.log(iou+eps) or iou
    

    Hope to get your reply as soon as possible. Thank you so much!!!

    opened by OYsJcHar 2
  • question about the formula of alpha-DIoU loss in Eq. (4)

    question about the formula of alpha-DIoU loss in Eq. (4)

    Dear author, Thanks for your work. I have a question about the the formula of alpha-DIoU loss in Eq. (4) from the paper. The alpha-version of the penalty term computed based on B and Bgt is ρ^(2alpha) / c^(2alpha), where ρ(·) being the Euclidean distance and c being the diagonal length of the smallest enclosing box. In your code, utils/general.py, line 405: c2 = cw ** beta + ch ** beta + eps # convex diagonal and line 408: rho2 = (rho_x ** beta + rho_y ** beta) / (2 ** beta) # center distance, it calculates the alpha version of penalty term as (w^(2alpha) + h^(2alpha)) / ( ρx^(2alpha) + ρy^(2alpha)) [type 1]. Why not ((w^2 + h^2) / (ρx^2 + ρy^2))^alpha [type 2]? It seems type 2 is reasonable because it follows the form (term)^alpha, such as IoU^alpha, (BV)^alpha and keeps the representation of Euclidean distance. In type 1, I don't understand the meaning of w^(2alpha) + h^(2alpha), does it still mean distance? Could you please explain more details about your design and why it is better than type 2? I don't find the theoretical analysis about this part in the paper. Thanks

    opened by wangren97 2
  • noautoanchor

    noautoanchor

    Hello, when I used the command line of the code you published, the accuracy was very low, and the accuracy was normal after adding "noAutoAnchor" in the command line. May I ask whether you used "NoAutoAnchor" in your original experiment?

    opened by dandervis 1
  • _pickle.UnpicklingError: STACK_GLOBAL requires str

    _pickle.UnpicklingError: STACK_GLOBAL requires str

    Traceback (most recent call last):
      File "train.py", line 539, in <module>
        train(hyp, opt, device, tb_writer, wandb)
      File "train.py", line 199, in train
        image_weights=opt.image_weights, quad=opt.quad, prefix=colorstr('train: '))
      File "/data/yolo/Alpha-IoU/utils/datasets.py", line 72, in create_dataloader
        prefix=prefix)
      File "/data/yolo/Alpha-IoU/utils/datasets.py", line 399, in __init__
        cache, exists = torch.load(cache_path), True  # load
      File "/data/Anaconda3/envs/torch17/lib/python3.7/site-packages/torch/serialization.py", line 595, in load
        return _legacy_load(opened_file, map_location, pickle_module, **pickle_load_args)
      File "/data/Anaconda3/envs/torch17/lib/python3.7/site-packages/torch/serialization.py", line 764, in _legacy_load
        magic_number = pickle_module.load(f, **pickle_load_args)
    _pickle.UnpicklingError: STACK_GLOBAL requires str
    
    
    opened by fanweiya 0
Owner
Jacobi(Jiabo He)
Jacobi(Jiabo He)
Fast algorithms to compute an approximation of the minimal volume oriented bounding box of a point cloud in 3D.

ApproxMVBB Status Build UnitTests Homepage Fast algorithms to compute an approximation of the minimal volume oriented bounding box of a point cloud in

Gabriel Nützi 390 Dec 31, 2022
Tools to create pixel-wise object masks, bounding box labels (2D and 3D) and 3D object model (PLY triangle mesh) for object sequences filmed with an RGB-D camera.

Tools to create pixel-wise object masks, bounding box labels (2D and 3D) and 3D object model (PLY triangle mesh) for object sequences filmed with an RGB-D camera. This project prepares training and testing data for various deep learning projects such as 6D object pose estimation projects singleshotpose, as well as object detection and instance segmentation projects.

null 305 Dec 16, 2022
Improving Object Detection by Estimating Bounding Box Quality Accurately

Improving Object Detection by Estimating Bounding Box Quality Accurately Abstrac

null 2 Apr 14, 2022
LQM - Improving Object Detection by Estimating Bounding Box Quality Accurately

Improving Object Detection by Estimating Bounding Box Quality Accurately Abstract Object detection aims to locate and classify object instances in ima

IM Lab., POSTECH 0 Sep 28, 2022
UNION: An Unreferenced Metric for Evaluating Open-ended Story Generation

UNION Automatic Evaluation Metric described in the paper UNION: An UNreferenced MetrIc for Evaluating Open-eNded Story Generation (EMNLP 2020). Please

null 50 Dec 30, 2022
Plotting points that lie on the intersection of the given curves using gradient descent.

Plotting intersection of curves using gradient descent Webapp Link ---> What's the app about Why this app Plotting functions and their intersection. A

Divakar Verma 2 Jan 9, 2022
A Robust Non-IoU Alternative to Non-Maxima Suppression in Object Detection

Confluence: A Robust Non-IoU Alternative to Non-Maxima Suppression in Object Detection 1. 介绍 用以替代 NMS,在所有 bbox 中挑选出最优的集合。 NMS 仅考虑了 bbox 的得分,然后根据 IOU 来

null 44 Sep 15, 2022
Boundary IoU API (Beta version)

Boundary IoU API (Beta version) Bowen Cheng, Ross Girshick, Piotr Dollár, Alexander C. Berg, Alexander Kirillov [arXiv] [Project] [BibTeX] This API is

Bowen Cheng 177 Dec 29, 2022
Location-Sensitive Visual Recognition with Cross-IOU Loss

The trained models are temporarily unavailable, but you can train the code using reasonable computational resource. Location-Sensitive Visual Recognit

Kaiwen Duan 146 Dec 25, 2022
Official PyTorch Implementation of Mask-aware IoU and maYOLACT Detector [BMVC2021]

The official implementation of Mask-aware IoU and maYOLACT detector. Our implementation is based on mmdetection. Mask-aware IoU for Anchor Assignment

Kemal Oksuz 11 Oct 21, 2021
Complete-IoU (CIoU) Loss and Cluster-NMS for Object Detection and Instance Segmentation (YOLACT)

Complete-IoU Loss and Cluster-NMS for Improving Object Detection and Instance Segmentation. Our paper is accepted by IEEE Transactions on Cybernetics

null 290 Dec 25, 2022
Black-Box-Tuning - Black-Box Tuning for Language-Model-as-a-Service

Black-Box-Tuning Source code for paper "Black-Box Tuning for Language-Model-as-a

Tianxiang Sun 149 Jan 4, 2023
Calculates carbon footprint based on fuel mix and discharge profile at the utility selected. Can create graphs and tabular output for fuel mix based on input file of series of power drawn over a period of time.

carbon-footprint-calculator Conda distribution ~/anaconda3/bin/conda install anaconda-client conda-build ~/anaconda3/bin/conda config --set anaconda_u

Seattle university Renewable energy research 7 Sep 26, 2022
Seach Losses of our paper 'Loss Function Discovery for Object Detection via Convergence-Simulation Driven Search', accepted by ICLR 2021.

CSE-Autoloss Designing proper loss functions for vision tasks has been a long-standing research direction to advance the capability of existing models

Peidong Liu(刘沛东) 54 Dec 17, 2022
Dogs classification with Deep Metric Learning using some popular losses

Tsinghua Dogs classification with Deep Metric Learning 1. Introduction Tsinghua Dogs dataset Tsinghua Dogs is a fine-grained classification dataset fo

QuocThangNguyen 45 Nov 9, 2022
Hypersearch weight debugging and losses tutorial

tutorial Activate tensorboard option Running TensorBoard remotely When working on a remote server, you can use SSH tunneling to forward the port of th

null 1 Dec 11, 2021
Quantile Regression DQN a Minimal Working Example, Distributional Reinforcement Learning with Quantile Regression

Quantile Regression DQN Quantile Regression DQN a Minimal Working Example, Distributional Reinforcement Learning with Quantile Regression (https://arx

Arsenii Senya Ashukha 80 Sep 17, 2022
Hitters Linear Regression - Hitters Linear Regression With Python

Hitters_Linear_Regression Kullanacağımız veri seti Carnegie Mellon Üniversitesi'

AyseBuyukcelik 2 Jan 26, 2022