COD-Rank-Localize-and-Segment (CVPR2021)

JingZhang

Last update: Dec 20, 2022

Related tags

Deep Learning COD-Rank-Localize-and-Segment

Overview

COD-Rank-Localize-and-Segment (CVPR2021)

Simultaneously Localize, Segment and Rank the Camouflaged Objects

Full camouflage fixation training dataset is available!

The full camouflage fixation training dataset is available with the full fixation maps for the COD10K training dataset, which can be downloaded from: https://drive.google.com/file/d/1inb5iNTDswFPDm4SpzBbVgZdI4puAv_3/view?usp=sharing

Camouflage Localization and Ranking dataset

We labeled the COD10K training dataset with eye tracker to localize the camouflaged objects, and generate 2000 images with localization and ranking label (We are generating fixation and label for all the existing training and testing dataset, and will release the dataset very soon.). The training dataset is as:

https://drive.google.com/file/d/12kSU6QrPAiumWpSkMqi5nPMo1awBW0_N/view?usp=sharing

which include 2000 images, with the corresponding fixation label, ranking label and instance level labels.

The testing dataset is as:

https://drive.google.com/file/d/1Gz5GzL9eeW13aZjlzaisrJFGO-HmhLxS/view?usp=sharing

which include 280 images with fixation, ranking and instance level labels.

Our Results

We train our triple-task learning framework with the above 2000 image training dataset and show the results in Table 1 of the main paper. The resulted camouflage maps are as:

https://drive.google.com/file/d/1ahu77JP-hzjgup20fNIftCB_cHanE323/view?usp=sharing

We also train our camouflaged object detection task along with the original COD10K training dataset, and show the performance in Table 4. The resulted camouflage maps are as:

https://drive.google.com/file/d/10sr2lX38FEgSXL3k27gidlaPKo5VQyjv/view?usp=sharing

Note that, we re-train our models, and the resulted performance is slightly difference from our reported numbers.

Benchmark results:

Please download the benchmark results (camoudlage maps) for your convienience. All the benchmark methods are trained with the COD10K training dataset (of size 4040):

https://drive.google.com/drive/folders/1sdly_TFW2WVqSm-hzuVXYKnu3DxkF-0F?usp=sharing

Or the computed evaluation metrics:

https://drive.google.com/file/d/17SyikbvnNF6g0_2BteyplQLid2o0KZTc/view?usp=sharing

New dataset: NC4K

Please download our newly collected camouflaged object detection testing dataset, namely NC4K, in the link below (with image, ground truth map, and instance level annotation): https://drive.google.com/file/d/1kzpX_U3gbgO9MuwZIWTuRVpiB7V6yrAQ/view?usp=sharing

or please download it from BaiduNetDisk: 链接:https://pan.baidu.com/s/1bG4F2KJ_4UJG_7XG6ZNBHA 密码:d581

Our Bib:

Please cite our paper if necessary:

@inproceedings{yunqiu_cod21,
  title={Simultaneously Localize, Segment and Rank the Camouflaged Objects},
  author={Lyu, Yunqiu and Zhang, Jing and Dai, Yuchao and Li, Aixuan and Liu, Bowen and Barnes, Nick and Fan, Deng-Ping},
  booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
  year={2021}
}

Contact

Please drop me an email for further problems or discussion: [email protected]

You might also like...

[ICLR 2021] Rank the Episodes: A Simple Approach for Exploration in Procedurally-Generated Environments.

[ICLR 2021] RAPID: A Simple Approach for Exploration in Reinforcement Learning This is the Tensorflow implementation of ICLR 2021 paper Rank the Episo

48 Nov 21, 2022

Rank 1st in the public leaderboard of ScanRefer (2021-03-18)

InstanceRefer InstanceRefer: Cooperative Holistic Understanding for Visual Grounding on Point Clouds through Instance Multi-level Contextual Referring

63 Dec 7, 2022

code for `Look Closer to Segment Better: Boundary Patch Refinement for Instance Segmentation`

Look Closer to Segment Better: Boundary Patch Refinement for Instance Segmentation (CVPR 2021) Introduction PBR is a conceptually simple yet effective

143 Jan 5, 2023

Code for "LoRA: Low-Rank Adaptation of Large Language Models"

LoRA: Low-Rank Adaptation of Large Language Models This repo contains the implementation of LoRA in GPT-2 and steps to replicate the results in our re

394 Jan 8, 2023

LETR: Line Segment Detection Using Transformers without Edges

LETR: Line Segment Detection Using Transformers without Edges Introduction This repository contains the official code and pretrained models for Line S

157 Jan 6, 2023

Official PyTorch Implementation of Rank & Sort Loss [ICCV2021]

Rank & Sort Loss for Object Detection and Instance Segmentation The official implementation of Rank & Sort Loss. Our implementation is based on mmdete

229 Dec 20, 2022

TensorFlow Ranking is a library for Learning-to-Rank (LTR) techniques on the TensorFlow platform

2.6k Jan 4, 2023

This is the pytorch implementation for the paper: Generalizable Mixed-Precision Quantization via Attribution Rank Preservation, which is accepted to ICCV2021.

GMPQ: Generalizable Mixed-Precision Quantization via Attribution Rank Preservation This is the pytorch implementation for the paper: Generalizable Mix

18 Sep 2, 2022

Open source implementation of AceNAS: Learning to Rank Ace Neural Architectures with Weak Supervision of Weight Sharing

AceNAS This repo is the experiment code of AceNAS, and is not considered as an official release. We are working on integrating AceNAS as a built-in st

6 Sep 7, 2022

Comments

Image and mask have different sizes.

I find there are 77 images that have different size with their corresponding masks:

image (370, 700) and mask (370, 702) have different size: ../NC4K/GT/1008.png
image (553, 369) and mask (555, 371) have different size: ../NC4K/GT/1017.png
image (576, 1024) and mask (577, 1024) have different size: ../NC4K/GT/1021.png
image (367, 550) and mask (368, 550) have different size: ../NC4K/GT/1072.png
image (600, 900) and mask (604, 904) have different size: ../NC4K/GT/1130.png
image (375, 500) and mask (378, 502) have different size: ../NC4K/GT/1142.png
image (640, 640) and mask (641, 643) have different size: ../NC4K/GT/1195.png
image (750, 750) and mask (681, 688) have different size: ../NC4K/GT/1214.png
image (600, 800) and mask (603, 805) have different size: ../NC4K/GT/1232.png
image (400, 600) and mask (401, 603) have different size: ../NC4K/GT/1301.png
image (520, 342) and mask (520, 346) have different size: ../NC4K/GT/1302.png
image (300, 400) and mask (300, 403) have different size: ../NC4K/GT/1331.png
image (400, 400) and mask (403, 405) have different size: ../NC4K/GT/1688.png
image (638, 992) and mask (640, 993) have different size: ../NC4K/GT/1731.png
image (911, 796) and mask (915, 798) have different size: ../NC4K/GT/1758.png
image (619, 560) and mask (620, 562) have different size: ../NC4K/GT/1838.png
image (740, 472) and mask (742, 473) have different size: ../NC4K/GT/1865.png
image (300, 400) and mask (302, 402) have different size: ../NC4K/GT/2002.png
image (517, 799) and mask (518, 800) have different size: ../NC4K/GT/2032.png
image (529, 799) and mask (530, 802) have different size: ../NC4K/GT/2093.png
image (640, 640) and mask (639, 640) have different size: ../NC4K/GT/2118.png
image (640, 640) and mask (643, 642) have different size: ../NC4K/GT/2186.png
image (640, 640) and mask (641, 645) have different size: ../NC4K/GT/219.png
image (480, 480) and mask (479, 485) have different size: ../NC4K/GT/2195.png
image (640, 640) and mask (641, 642) have different size: ../NC4K/GT/2240.png
image (490, 800) and mask (493, 804) have different size: ../NC4K/GT/2255.png
image (466, 681) and mask (470, 682) have different size: ../NC4K/GT/2289.png
image (425, 513) and mask (428, 519) have different size: ../NC4K/GT/232.png
image (360, 640) and mask (361, 642) have different size: ../NC4K/GT/2324.png
image (612, 612) and mask (613, 614) have different size: ../NC4K/GT/2393.png
image (716, 1080) and mask (718, 1082) have different size: ../NC4K/GT/2396.png
image (640, 640) and mask (641, 640) have different size: ../NC4K/GT/2401.png
image (436, 708) and mask (439, 709) have different size: ../NC4K/GT/2434.png
image (418, 624) and mask (420, 626) have different size: ../NC4K/GT/2451.png
image (362, 700) and mask (363, 701) have different size: ../NC4K/GT/2456.png
image (640, 640) and mask (640, 641) have different size: ../NC4K/GT/2483.png
image (640, 640) and mask (642, 640) have different size: ../NC4K/GT/2495.png
image (480, 852) and mask (376, 667) have different size: ../NC4K/GT/2615.png
image (576, 1024) and mask (579, 1025) have different size: ../NC4K/GT/2817.png
image (617, 822) and mask (617, 823) have different size: ../NC4K/GT/3005.png
image (640, 640) and mask (644, 645) have different size: ../NC4K/GT/305.png
image (640, 640) and mask (641, 640) have different size: ../NC4K/GT/3173.png
image (576, 720) and mask (578, 721) have different size: ../NC4K/GT/3221.png
image (640, 640) and mask (640, 641) have different size: ../NC4K/GT/3309.png
image (609, 609) and mask (608, 612) have different size: ../NC4K/GT/3333.png
image (613, 900) and mask (614, 902) have different size: ../NC4K/GT/3345.png
image (640, 640) and mask (640, 641) have different size: ../NC4K/GT/3367.png
image (748, 750) and mask (681, 682) have different size: ../NC4K/GT/3376.png
image (593, 850) and mask (593, 851) have different size: ../NC4K/GT/3549.png
image (452, 640) and mask (451, 639) have different size: ../NC4K/GT/3557.png
image (640, 640) and mask (642, 641) have different size: ../NC4K/GT/3627.png
image (506, 900) and mask (507, 902) have different size: ../NC4K/GT/3956.png
image (640, 640) and mask (642, 640) have different size: ../NC4K/GT/4291.png
image (640, 640) and mask (454, 455) have different size: ../NC4K/GT/4349.png
image (640, 640) and mask (453, 454) have different size: ../NC4K/GT/4355.png
image (640, 640) and mask (456, 454) have different size: ../NC4K/GT/4391.png
image (640, 640) and mask (454, 455) have different size: ../NC4K/GT/4465.png
image (608, 608) and mask (454, 454) have different size: ../NC4K/GT/4508.png
image (640, 640) and mask (454, 454) have different size: ../NC4K/GT/4518.png
image (640, 640) and mask (453, 455) have different size: ../NC4K/GT/4524.png
image (640, 640) and mask (454, 453) have different size: ../NC4K/GT/4530.png
image (480, 852) and mask (376, 668) have different size: ../NC4K/GT/4576.png
image (398, 598) and mask (400, 600) have different size: ../NC4K/GT/4580.png
image (480, 852) and mask (377, 669) have different size: ../NC4K/GT/4582.png
image (480, 852) and mask (376, 668) have different size: ../NC4K/GT/4615.png
image (404, 786) and mask (344, 669) have different size: ../NC4K/GT/4679.png
image (480, 852) and mask (376, 667) have different size: ../NC4K/GT/4814.png
image (640, 640) and mask (454, 453) have different size: ../NC4K/GT/5149.png
image (400, 320) and mask (401, 321) have different size: ../NC4K/GT/5187.png
image (720, 1080) and mask (446, 668) have different size: ../NC4K/GT/5310.png
image (640, 640) and mask (455, 454) have different size: ../NC4K/GT/5316.png
image (640, 640) and mask (453, 453) have different size: ../NC4K/GT/5417.png
image (640, 640) and mask (453, 453) have different size: ../NC4K/GT/5428.png
image (249, 603) and mask (250, 604) have different size: ../NC4K/GT/743.png
image (778, 900) and mask (779, 902) have different size: ../NC4K/GT/884.png
image (387, 579) and mask (388, 581) have different size: ../NC4K/GT/906.png
image (403, 620) and mask (404, 621) have different size: ../NC4K/GT/907.png

For a more accurate evaluation, how does the predicted result with the same size as the original image be aligned with the mask?

opened by lartpang 2

Some errors in NC4K

If we read the images using the opencv lib, we will find that:

Some images (jpeg) can not be displayed in my computer. The question can be solved by reading and re-writing these images using the opencv lib.
2287.jpg is None.
4299.jpg is None.
1214.jpg is not valid jpeg (image file is truncated (3 bytes not processed)), it will mask the opencv lib raise the warning Premature end of JPEG file, and we can use the PIL lib to catch it.

# -*- coding: utf-8 -*-
# @Time    : 2021/3/9
# @Author  : Lart Pang
# @GitHub  : https://github.com/lartpang

import os

import cv2
from PIL import Image


def check_pic(path):
    """
    https://www.zhihu.com/question/30372655/answer/157762216
    """
    try:
        Image.open(path).load()
    except Exception as e:
        print(e)
        return False
    else:
        return True


source_root = ".../NC4K/Imgs"
target_root = ".../NC4K/FixedImages"


image_names = os.listdir(source_root)
for image_name in image_names:
    s_image_path = os.path.join(source_root, image_name)
    t_image_path = os.path.join(target_root, image_name)

    if not check_pic(s_image_path):
        print(f"{s_image_path} is not valid jpeg. But we still read it and rewrite.")

    s_image_array = cv2.imread(s_image_path, cv2.IMREAD_COLOR)
    if not os.path.exists(s_image_path):
        raise FileNotFoundError(f"{s_image_path} is not exist")
    if s_image_array is None:
        print(f"{s_image_path} is None.")
    else:
        cv2.imwrite(t_image_path, s_image_array)

opened by lartpang 1

COD-Rank-Localize-and-Segment (CVPR2021)

Related tags

Overview

COD-Rank-Localize-and-Segment (CVPR2021)

Full camouflage fixation training dataset is available!

Camouflage Localization and Ranking dataset

Our Results

Benchmark results:

New dataset: NC4K

Our Bib:

Contact

You might also like...

[ICLR 2021] Rank the Episodes: A Simple Approach for Exploration in Procedurally-Generated Environments.

Rank 1st in the public leaderboard of ScanRefer (2021-03-18)

code for `Look Closer to Segment Better: Boundary Patch Refinement for Instance Segmentation`

Code for "LoRA: Low-Rank Adaptation of Large Language Models"

LETR: Line Segment Detection Using Transformers without Edges

Official PyTorch Implementation of Rank & Sort Loss [ICCV2021]

TensorFlow Ranking is a library for Learning-to-Rank (LTR) techniques on the TensorFlow platform

This is the pytorch implementation for the paper: Generalizable Mixed-Precision Quantization via Attribution Rank Preservation, which is accepted to ICCV2021.

Open source implementation of AceNAS: Learning to Rank Ace Neural Architectures with Weak Supervision of Weight Sharing

Comments

Image and mask have different sizes.

Some errors in NC4K

Owner

JingZhang

A pure PyTorch implementation of the loss described in "Online Segment to Segment Neural Transduction"

[CVPR2021] DoDNet: Learning to segment multi-organ and tumors from multiple partially labeled datasets

An efficient and effective learning to rank algorithm by mining information across ranking candidates. This repository contains the tensorflow implementation of SERank model. The code is developed based on TF-Ranking.

Source Code for DialogBERT: Discourse-Aware Response Generation via Learning to Recover and Rank Utterances (https://arxiv.org/pdf/2012.01775.pdf)

Official Tensorflow implementation of "M-LSD: Towards Light-weight and Real-time Line Segment Detection"

Pytorch implementation of "M-LSD: Towards Light-weight and Real-time Line Segment Detection"

Pytorch implementation of paper "Learning Co-segmentation by Segment Swapping for Retrieval and Discovery"

some classic model used to segment the medical images like CT、X-ray and so on

An e-commerce company wants to segment its customers and determine marketing strategies according to these segments.

Code for "Learning to Segment Rigid Motions from Two Frames".