TransCD: Scene Change Detection via Transformer-based Architecture

wangzhixue

Last update: Dec 11, 2022

Related tags

Deep Learning TransCD

Overview

TransCD: Scene Change Detection via Transformer-based Architecture

Requirements

Python 3.7.0  
Pytorch 1.6.0  
Visdom 0.1.8.9  
Torchvision 0.7.0

Datasets

CD2014 dataset
- paper: changedetection.net: A new change detection benchmark dataset
- paper: CDnet 2014: An Expanded Change Detection Benchmark Dataset
- dataset: http://changedetection.net/
VL-CMU-CD
- paper: Street-view change detection with deconvolutional networks
- dataset: https://ghsi.github.io/proj/RSS2016.html

Pretrained Model

Pretrained models for CDNet-2014 and VL-CMU-CD are available. You can download them from the following link.

CDNet-2014: [Baiduyun] the password is 78cp. [GoogleDrive].
- We uploaded six models trained on CDNet-2014 dataset, they are SViT_E1_D1_16, SViT_E1_D1_32, SViT_E4_D4_16, SViT_E4_D4_32, Res_SViT_E1_D1_16 and Res_SViT_E4_D4_16.
VL-CMU-CD: [Baiduyun] the password is ydzl. [GoogleDrive].
- We uploaded four models trained on VL-CMU-CD dataset, ther are SViT_E1_D1_16, SViT_E1_D1_32, Res_SViT_E1_D1_16 and Res_SViT_E1_D1_32.

Test

Before test, please download datasets and predtrained models. Copy pretrained models to folder './dataset_name/outputs/best_weights', and run the following command:

cd TransCD_ROOT
python test.py --net_cfg 
   
     --train_cfg

Use --save_changemap True to save predicted changemaps. For example:

python test.py --net_cfg SVit_E1_D1_32 --train_cfg CDNet_2014 --save_changemap True

Training

Before training, please download datasets and revise dataset path in configs.py to your path. CD TransCD_ROOT

python -m visdom.server
python train.py --net_cfg 
   
     --train_cfg

For example:

python -m visdom.server
python train.py --net_cfg Res_SViT_E1_D1_16 --train_cfg VL_CMU_CD

To display training processing, copy 'http://localhost:8097' to your browser.

Citing TransCD

If you use this repository or would like to refer the paper, please use the following BibTex entry.

@inproceddings{TransCD,
title={TransCD: Scene Change Detection via Transformer-based Architecture},
author={ZHIXUE WANG, YU ZHANG*, LIN LUO, NAN WANG},
journal={Optics Express},
yera={2021},
organization={The Optical Society},
}

Reference

-Akcay, Samet, Amir Atapour-Abarghouei, and Toby P. Breckon. "Ganomaly: Semi-supervised anomaly detection via adversarial training." Asian conference on computer vision. Springer, Cham, 2018.
-Chen, Jieneng, et al. "Transunet: Transformers make strong encoders for medical image segmentation." arXiv preprint arXiv:2102.04306 (2021).

Comments

RuntimeError:The size of tensor a (512) must match The size of tensor B (3) at non-singleton Dimension 4

https://github.com/wangle53/TransCD/blob/be511a1d2e39a8906dc44023e35d310c991de75b/utils/make_dataset.py#L45

Thank you for sharing the code.

I have a question I'd like to answer from you.

At make_dataset.py, line 45, gt adds a dimension, which causes The error"RuntimeError: The size of tensor a (512) must match The size of tensor B (3) at non-singleton Dimension 4". What are the considerations for this step?

Best regards

opened by neil-jinsh-public 7
How to generate maskGT in VL_CMU_CD dataset?

In offered train.txt, the name of label is mask**.png, be the name of label in original VL_CMU_CD is gt**.png, so how to generate maskGT? can I just change their name?

opened by WangDeyu 4
test.py can not find test.txt

using arguments: '--net_cfg SViT_E1_D1_32 --train_cfg CDNet_2014 --save_changemap True' test.py returns "No such file or directory: './CDNet_2014/data/test.txt' "

opened by huntkao 4
precision, oa, recall, etc... are all the same of every category

The metrics of every category are all the same in test_score.txt. My arguments: --net_cfg SViT_E1_D1_32 --train_cfg CDNet_2014 --save_changemap True

I cropped some below: Category:badWeather,precision:0.9654355468223322,oa:0.8092331883428242,recall:0.1367026792675585,f1:0.23949377961693205,kappa:0.19565049499554038 Category:baseline,precision:0.9654355468223322,oa:0.8092331883428242,recall:0.1367026792675585,f1:0.23949377961693205,kappa:0.19565049499554038 Category:cameraJitter,precision:0.9654355468223322,oa:0.8092331883428242,recall:0.1367026792675585,f1:0.23949377961693205,kappa:0.19565049499554038 Category:dynamicBackground,precision:0.9654355468223322,oa:0.8092331883428242,recall:0.1367026792675585,f1:0.23949377961693205,kappa:0.19565049499554038 Category:intermittentObjectMotion,precision:0.9654355468223322,oa:0.8092331883428242,recall:0.1367026792675585,f1:0.23949377961693205,kappa: 0.19565049499554038

opened by huntkao 2
Miss-mach dataset format

CDNet-2014 dataset format is not the same as train.py reading format which is based on t1, t2 and gt. Is there any other resources for CDNet-2014 dataset which is match to your code?

opened by sajadalipour 1
Model Application Transfering

Do you consider transferring the model to medical image recognition or segment? Medical images always include multiple targets or structures, especially in longitudinal monitoring in vivo, can your model be applied to track or swiftly extract different structures from these images?

Lucas

opened by Luchsyanfeng 1
how to train model on vl_cmu_cd dataset?

thanks for sharing your great work.when i use this code train my model like vl_cmu_cd format,but it did't work,the error is the following:
return _VF.broadcast_tensors(tensors) # type: ignore[attr-defined] RuntimeError: The size of tensor a (512) must match the size of tensor b (3) at non-singleton dimension 4 look forward for your reply

opened by jcluo1994 24

Owner

wangzhixue

GitHub

Change is Everywhere: Single-Temporal Supervised Object Change Detection in Remote Sensing Imagery (ICCV 2021)

Change is Everywhere Single-Temporal Supervised Object Change Detection in Remote Sensing Imagery by Zhuo Zheng, Ailong Ma, Liangpei Zhang and Yanfei

125 Dec 13, 2022

Episodic Transformer (E.T.) is a novel attention-based architecture for vision-and-language navigation. E.T. is based on a multimodal transformer that encodes language inputs and the full episode history of visual observations and actions.

Episodic Transformers (E.T.) Episodic Transformer for Vision-and-Language Navigation Alexander Pashevich, Cordelia Schmid, Chen Sun Episodic Transform

62 Dec 24, 2022

Very simple NCHW and NHWC conversion tool for ONNX. Change to the specified input order for each and every input OP. Also, change the channel order of RGB and BGR. Simple Channel Converter for ONNX.

scc4onnx Very simple NCHW and NHWC conversion tool for ONNX. Change to the specified input order for each and every input OP. Also, change the channel

16 Dec 22, 2022

Automatic number plate recognition using tech: Yolo, OCR, Scene text detection, scene text recognation, flask, torch

Automatic Number Plate Recognition Automatic Number Plate Recognition (ANPR) is the process of reading the characters on the plate with various optica

52 Dec 22, 2022

Official PyTorch code of DeepPanoContext: Panoramic 3D Scene Understanding with Holistic Scene Context Graph and Relation-based Optimization (ICCV 2021 Oral).

DeepPanoContext (DPC) [Project Page (with interactive results)][Paper] DeepPanoContext: Panoramic 3D Scene Understanding with Holistic Scene Context G

66 Nov 16, 2022

Pytorch implementation of Make-A-Scene: Scene-Based Text-to-Image Generation with Human Priors

Make-A-Scene - PyTorch Pytorch implementation (inofficial) of Make-A-Scene: Scene-Based Text-to-Image Generation with Human Priors (https://arxiv.org/

259 Dec 28, 2022

code for paper "Does Unsupervised Architecture Representation Learning Help Neural Architecture Search?"

Does Unsupervised Architecture Representation Learning Help Neural Architecture Search? Code for paper: Does Unsupervised Architecture Representation

39 Dec 17, 2022

Pytorch re-implementation of Paper: SwinTextSpotter: Scene Text Spotting via Better Synergy between Text Detection and Text Recognition (CVPR 2022)

SwinTextSpotter This is the pytorch implementation of Paper: SwinTextSpotter: Scene Text Spotting via Better Synergy between Text Detection and Text R

183 Jan 3, 2023

Code for CVPR 2021 oral paper "Exploring Data-Efficient 3D Scene Understanding with Contrastive Scene Contexts"

Exploring Data-Efficient 3D Scene Understanding with Contrastive Scene Contexts The rapid progress in 3D scene understanding has come with growing dem

182 Dec 30, 2022

Neural Scene Graphs for Dynamic Scene (CVPR 2021)

Implementation of Neural Scene Graphs, that optimizes multiple radiance fields to represent different objects and a static scene background. Learned representations can be rendered with novel object compositions and views.

151 Dec 26, 2022

A weakly-supervised scene graph generation codebase. The implementation of our CVPR2021 paper ``Linguistic Structures as Weak Supervision for Visual Scene Graph Generation''

README.md shall be finished soon. WSSGG 0 Overview 1 Installation 1.1 Faster-RCNN 1.2 Language Parser 1.3 GloVe Embeddings 2 Settings 2.1 VG-GT-Graph

35 Nov 20, 2022

Remote sensing change detection tool based on PaddlePaddle

PdRSCD PdRSCD（PaddlePaddle Remote Sensing Change Detection）是一个基于飞桨PaddlePaddle的遥感变化检测的项目，pypi包名为ppcd。目前0.2版本，最新支持图像列表输入的训练和预测，如多期影像、多源影像甚至多期多源影像。可以快速完

38 Aug 31, 2022

Code for SentiBERT: A Transferable Transformer-Based Architecture for Compositional Sentiment Semantics (ACL'2020).

SentiBERT Code for SentiBERT: A Transferable Transformer-Based Architecture for Compositional Sentiment Semantics (ACL'2020). https://arxiv.org/abs/20

66 Aug 13, 2022

VSR-Transformer - This paper proposes a new Transformer for video super-resolution (called VSR-Transformer).

VSR-Transformer By Jiezhang Cao, Yawei Li, Kai Zhang, Luc Van Gool This paper proposes a new Transformer for video super-resolution (called VSR-Transf

225 Nov 13, 2022

Official implement of Paper：A deeply supervised image fusion network for change detection in high resolution bi-temporal remote sening images

A deeply supervised image fusion network for change detection in high resolution bi-temporal remote sensing images 深度监督影像融合网络DSIFN用于高分辨率双时相遥感影像变化检测 Of

135 Dec 19, 2022

Official Pytorch Implementation of Adversarial Instance Augmentation for Building Change Detection in Remote Sensing Images.

IAug_CDNet Official Implementation of Adversarial Instance Augmentation for Building Change Detection in Remote Sensing Images. Overview We propose a

53 Dec 2, 2022

TransCD: Scene Change Detection via Transformer-based Architecture

Related tags

Overview

TransCD: Scene Change Detection via Transformer-based Architecture

Requirements

Datasets

Pretrained Model

Test

Training

Citing TransCD

Reference

Comments

RuntimeError:The size of tensor a (512) must match The size of tensor B (3) at non-singleton Dimension 4

How to generate maskGT in VL_CMU_CD dataset?

test.py can not find test.txt

precision, oa, recall, etc... are all the same of every category

Miss-mach dataset format

Model Application Transfering

how to train model on vl_cmu_cd dataset?

Owner

wangzhixue

Change is Everywhere: Single-Temporal Supervised Object Change Detection in Remote Sensing Imagery (ICCV 2021)

Episodic Transformer (E.T.) is a novel attention-based architecture for vision-and-language navigation. E.T. is based on a multimodal transformer that encodes language inputs and the full episode history of visual observations and actions.

Very simple NCHW and NHWC conversion tool for ONNX. Change to the specified input order for each and every input OP. Also, change the channel order of RGB and BGR. Simple Channel Converter for ONNX.

Automatic number plate recognition using tech: Yolo, OCR, Scene text detection, scene text recognation, flask, torch

Official PyTorch code of DeepPanoContext: Panoramic 3D Scene Understanding with Holistic Scene Context Graph and Relation-based Optimization (ICCV 2021 Oral).

Pytorch implementation of Make-A-Scene: Scene-Based Text-to-Image Generation with Human Priors

code for paper "Does Unsupervised Architecture Representation Learning Help Neural Architecture Search?"

Pytorch re-implementation of Paper: SwinTextSpotter: Scene Text Spotting via Better Synergy between Text Detection and Text Recognition (CVPR 2022)

Code for CVPR 2021 oral paper "Exploring Data-Efficient 3D Scene Understanding with Contrastive Scene Contexts"

Neural Scene Graphs for Dynamic Scene (CVPR 2021)

A weakly-supervised scene graph generation codebase. The implementation of our CVPR2021 paper ``Linguistic Structures as Weak Supervision for Visual Scene Graph Generation''

Remote sensing change detection tool based on PaddlePaddle

Code for SentiBERT: A Transferable Transformer-Based Architecture for Compositional Sentiment Semantics (ACL'2020).

VSR-Transformer - This paper proposes a new Transformer for video super-resolution (called VSR-Transformer).

Official implement of Paper：A deeply supervised image fusion network for change detection in high resolution bi-temporal remote sening images

Official Pytorch Implementation of Adversarial Instance Augmentation for Building Change Detection in Remote Sensing Images.

1st place solution to the Satellite Image Change Detection Challenge hosted by SenseTime

Deep learning models for change detection of remote sensing images

From this paper "SESNet: A Semantically Enhanced Siamese Network for Remote Sensing Change Detection"