This project provides the code and datasets for 'CapSal: Leveraging Captioning to Boost Semantics for Salient Object Detection', CVPR 2019.

lu zhang

Last update: Aug 19, 2022

Overview

Code-and-Dataset-for-CapSal

This project provides the code and datasets for 'CapSal: Leveraging Captioning to Boost Semantics for Salient Object Detection', CVPR 2019. Paper link

Our code is implemented based on the Mask RCNN in Tensorflow and Keras. You can first install the maskrcnn according to the instruction or INSTALL.md.

COCO-CapSal Dataset

The COCO-CapSal dataset provides the saliency ground truth as well as the image captions for each image. It contains 5265 images for training and 1459 ones for validation. The annotations can be downloaded at BaiduYun or GoogleDrive. The folder 'capsal' contains the images, ground truth maps as well as the caprions (json file) of both training and validation sets.

Evaluation

For testing the CapSal model, first download the trained model at BaiduYun or Google ) and put it under the ./model. Run test_capsal.py to obtain the saliency maps of different datasets. The saliency map is avaliable at Google or BaiduYun.

Train

Run 'train.py'.

Citation

    @InProceedings{Zhang_2019_CVPR,
            author = {Zhang, Lu and Zhang, Jianming and Lin, Zhe and Lu, Huchuan and He, You},
            title = {CapSal: Leveraging Captioning to Boost Semantics for Salient Object Detection},
            booktitle = CVPR,
            year = {2019}}

This repo is the code release of EMNLP 2021 conference paper "Connect-the-Dots: Bridging Semantics between Words and Definitions via Aligning Word Sense Inventories".

Connect-the-Dots: Bridging Semantics between Words and Definitions via Aligning Word Sense Inventories This repo is the code release of EMNLP 2021 con

12 Nov 22, 2022

Code for SentiBERT: A Transferable Transformer-Based Architecture for Compositional Sentiment Semantics (ACL'2020).

Comments

coco evaluation tool

Hi, thank you for sharing your work,

I'm trying to get it working but I am running into a few issues. One of which is that I can not downlaod from baidu, is there any way you could share the file linked in eval_cap.py on google drive or with me directly?

Many thanks

opened by DStickley 0
some problem about Evaluation

Hello，When I run test_Capsal.py,the program has been working on the first image and don't have any results.It seems like stuck but no display error.Do you know where I went wrong?Thank you

''' Running COCO evaluation on 1459 images. coco 0 Backend TkAgg is interactive backend. Turning interactive mode on. Processing 1 images image shape: (480, 640, 3) min: 0.00000 max: 255.00000 uint8 molded_images shape: (1, 1024, 1024, 3) min: -123.70000 max: 151.10000 float64 image_metas shape: (1, 14) min: 0.00000 max: 1024.00000 int64 anchors shape: (1, 261888, 4) min: -0.35390 max: 1.29134 float32 '''

opened by JingJLiu 1
Can you provide evaluation criteria class?
I am very interested in your work.But i have some problems on it.

can you provide your evaluation criteria code about F-measure?I can't get your result by myself function. 2.which did you train your model in DUTS-train or COCO-capsal? 3.hou to train on DUTS-train dataset? is it this,firstly,we train ICN using caption data of coco-capsal.In second stage, we will fixed ICN and train LGPN using DUTS-train because of lack of caption data of DUTS-train? I hope to hear from you.
opened by zhuguanglueying 0

This project provides the code and datasets for 'CapSal: Leveraging Captioning to Boost Semantics for Salient Object Detection', CVPR 2019.

Related tags

Overview

Code-and-Dataset-for-CapSal

COCO-CapSal Dataset

Evaluation

Train

Citation

You might also like...

This repo is the code release of EMNLP 2021 conference paper "Connect-the-Dots: Bridging Semantics between Words and Definitions via Aligning Word Sense Inventories".

Code for SentiBERT: A Transferable Transformer-Based Architecture for Compositional Sentiment Semantics (ACL'2020).

Code for CVPR2021 paper "Learning Salient Boundary Feature for Anchor-free Temporal Action Localization"

Codes for paper "Towards Diverse Paragraph Captioning for Untrimmed Videos". CVPR 2021

Boost learning for GNNs from the graph structure under challenging heterophily settings. (NeurIPS'20)

Implementation for "Seamless Manga Inpainting with Semantics Awareness" (SIGGRAPH 2021 issue)

Official Implementation of LARGE: Latent-Based Regression through GAN Semantics

A Protein-RNA Interface Predictor Based on Semantics of Sequences

An algorithm study of the 6th iOS 10 set of Boost Camp Web Mobile

Comments

coco evaluation tool

some problem about Evaluation

Can you provide evaluation criteria class?

Owner

lu zhang

Code for ACM MM2021 paper "Complementary Trilateral Decoder for Fast and Accurate Salient Object Detection"

Code for the TIP 2021 Paper "Salient Object Detection with Purificatory Mechanism and Structural Similarity Loss"

Code for the ICME 2021 paper "Exploring Driving-Aware Salient Object Detection via Knowledge Transfer"

U2-Net: Going Deeper with Nested U-Structure for Salient Object Detection

Simple image captioning model - CLIP prefix captioning.

PointRCNN: 3D Object Proposal Generation and Detection from Point Cloud, CVPR 2019.

This project aim to create multi-label classification annotation tool to boost annotation speed and make it more easier.

An official implementation of "SFNet: Learning Object-aware Semantic Correspondence" (CVPR 2019, TPAMI 2020) in PyTorch.

A object detecting neural network powered by the yolo architecture and leveraging the PyTorch framework and associated libraries.

Ready-to-use code and tutorial notebooks to boost your way into few-shot image classification.