TransCD: Scene Change Detection via Transformer-based Architecture

Overview

TransCD: Scene Change Detection via Transformer-based Architecture

image

Requirements

Python 3.7.0  
Pytorch 1.6.0  
Visdom 0.1.8.9  
Torchvision 0.7.0

Datasets

Pretrained Model

Pretrained models for CDNet-2014 and VL-CMU-CD are available. You can download them from the following link.

  • CDNet-2014: [Baiduyun] the password is 78cp. [GoogleDrive].
    • We uploaded six models trained on CDNet-2014 dataset, they are SViT_E1_D1_16, SViT_E1_D1_32, SViT_E4_D4_16, SViT_E4_D4_32, Res_SViT_E1_D1_16 and Res_SViT_E4_D4_16.
  • VL-CMU-CD: [Baiduyun] the password is ydzl. [GoogleDrive].
    • We uploaded four models trained on VL-CMU-CD dataset, ther are SViT_E1_D1_16, SViT_E1_D1_32, Res_SViT_E1_D1_16 and Res_SViT_E1_D1_32.

Test

Before test, please download datasets and predtrained models. Copy pretrained models to folder './dataset_name/outputs/best_weights', and run the following command:

cd TransCD_ROOT
python test.py --net_cfg 
   
     --train_cfg 
    

    
   

Use --save_changemap True to save predicted changemaps. For example:

python test.py --net_cfg SVit_E1_D1_32 --train_cfg CDNet_2014 --save_changemap True

Training

Before training, please download datasets and revise dataset path in configs.py to your path. CD TransCD_ROOT

python -m visdom.server
python train.py --net_cfg 
   
     --train_cfg 
    

    
   

For example:

python -m visdom.server
python train.py --net_cfg Res_SViT_E1_D1_16 --train_cfg VL_CMU_CD

To display training processing, copy 'http://localhost:8097' to your browser.

Citing TransCD

If you use this repository or would like to refer the paper, please use the following BibTex entry.

@inproceddings{TransCD,
title={TransCD: Scene Change Detection via Transformer-based Architecture},
author={ZHIXUE WANG, YU ZHANG*, LIN LUO, NAN WANG},
journal={Optics Express},
yera={2021},
organization={The Optical Society},
}

Reference

-Akcay, Samet, Amir Atapour-Abarghouei, and Toby P. Breckon. "Ganomaly: Semi-supervised anomaly detection via adversarial training." Asian conference on computer vision. Springer, Cham, 2018.
-Chen, Jieneng, et al. "Transunet: Transformers make strong encoders for medical image segmentation." arXiv preprint arXiv:2102.04306 (2021).
Comments
  • RuntimeError:The size of tensor a (512) must match The size of tensor B (3) at non-singleton Dimension 4

    RuntimeError:The size of tensor a (512) must match The size of tensor B (3) at non-singleton Dimension 4

    https://github.com/wangle53/TransCD/blob/be511a1d2e39a8906dc44023e35d310c991de75b/utils/make_dataset.py#L45

    Thank you for sharing the code.

    I have a question I'd like to answer from you.

    At make_dataset.py, line 45, gt adds a dimension, which causes The error"RuntimeError: The size of tensor a (512) must match The size of tensor B (3) at non-singleton Dimension 4". What are the considerations for this step?

    Best regards

    opened by neil-jinsh-public 7
  • How to generate maskGT in VL_CMU_CD dataset?

    How to generate maskGT in VL_CMU_CD dataset?

    In offered train.txt, the name of label is mask**.png, be the name of label in original VL_CMU_CD is gt**.png, so how to generate maskGT? can I just change their name?

    opened by WangDeyu 4
  • test.py can not find test.txt

    test.py can not find test.txt

    using arguments: '--net_cfg SViT_E1_D1_32 --train_cfg CDNet_2014 --save_changemap True' test.py returns "No such file or directory: './CDNet_2014/data/test.txt' "

    opened by huntkao 4
  • precision, oa, recall, etc... are all the same of every category

    precision, oa, recall, etc... are all the same of every category

    The metrics of every category are all the same in test_score.txt. My arguments: --net_cfg SViT_E1_D1_32 --train_cfg CDNet_2014 --save_changemap True

    I cropped some below: Category:badWeather,precision:0.9654355468223322,oa:0.8092331883428242,recall:0.1367026792675585,f1:0.23949377961693205,kappa:0.19565049499554038 Category:baseline,precision:0.9654355468223322,oa:0.8092331883428242,recall:0.1367026792675585,f1:0.23949377961693205,kappa:0.19565049499554038 Category:cameraJitter,precision:0.9654355468223322,oa:0.8092331883428242,recall:0.1367026792675585,f1:0.23949377961693205,kappa:0.19565049499554038 Category:dynamicBackground,precision:0.9654355468223322,oa:0.8092331883428242,recall:0.1367026792675585,f1:0.23949377961693205,kappa:0.19565049499554038 Category:intermittentObjectMotion,precision:0.9654355468223322,oa:0.8092331883428242,recall:0.1367026792675585,f1:0.23949377961693205,kappa: 0.19565049499554038

    opened by huntkao 2
  • Miss-mach dataset format

    Miss-mach dataset format

    CDNet-2014 dataset format is not the same as train.py reading format which is based on t1, t2 and gt. Is there any other resources for CDNet-2014 dataset which is match to your code?

    opened by sajadalipour 1
  • Model Application Transfering

    Model Application Transfering

    Do you consider transferring the model to medical image recognition or segment? Medical images always include multiple targets or structures, especially in longitudinal monitoring in vivo, can your model be applied to track or swiftly extract different structures from these images?

    Lucas

    opened by Luchsyanfeng 1
  • how to train model on vl_cmu_cd dataset?

    how to train model on vl_cmu_cd dataset?

    thanks for sharing your great work.when i use this code train my model like vl_cmu_cd format,but it did't work,the error is the following:
    return _VF.broadcast_tensors(tensors) # type: ignore[attr-defined] RuntimeError: The size of tensor a (512) must match the size of tensor b (3) at non-singleton dimension 4 look forward for your reply

    opened by jcluo1994 24
Owner
wangzhixue
wangzhixue
Change is Everywhere: Single-Temporal Supervised Object Change Detection in Remote Sensing Imagery (ICCV 2021)

Change is Everywhere Single-Temporal Supervised Object Change Detection in Remote Sensing Imagery by Zhuo Zheng, Ailong Ma, Liangpei Zhang and Yanfei

Zhuo Zheng 125 Dec 13, 2022
Alex Pashevich 62 Dec 24, 2022
Very simple NCHW and NHWC conversion tool for ONNX. Change to the specified input order for each and every input OP. Also, change the channel order of RGB and BGR. Simple Channel Converter for ONNX.

scc4onnx Very simple NCHW and NHWC conversion tool for ONNX. Change to the specified input order for each and every input OP. Also, change the channel

Katsuya Hyodo 16 Dec 22, 2022
Automatic number plate recognition using tech: Yolo, OCR, Scene text detection, scene text recognation, flask, torch

Automatic Number Plate Recognition Automatic Number Plate Recognition (ANPR) is the process of reading the characters on the plate with various optica

Meftun AKARSU 52 Dec 22, 2022
Official PyTorch code of DeepPanoContext: Panoramic 3D Scene Understanding with Holistic Scene Context Graph and Relation-based Optimization (ICCV 2021 Oral).

DeepPanoContext (DPC) [Project Page (with interactive results)][Paper] DeepPanoContext: Panoramic 3D Scene Understanding with Holistic Scene Context G

Cheng Zhang 66 Nov 16, 2022
Pytorch implementation of Make-A-Scene: Scene-Based Text-to-Image Generation with Human Priors

Make-A-Scene - PyTorch Pytorch implementation (inofficial) of Make-A-Scene: Scene-Based Text-to-Image Generation with Human Priors (https://arxiv.org/

Casual GAN Papers 259 Dec 28, 2022
code for paper "Does Unsupervised Architecture Representation Learning Help Neural Architecture Search?"

Does Unsupervised Architecture Representation Learning Help Neural Architecture Search? Code for paper: Does Unsupervised Architecture Representation

null 39 Dec 17, 2022
Pytorch re-implementation of Paper: SwinTextSpotter: Scene Text Spotting via Better Synergy between Text Detection and Text Recognition (CVPR 2022)

SwinTextSpotter This is the pytorch implementation of Paper: SwinTextSpotter: Scene Text Spotting via Better Synergy between Text Detection and Text R

mxin262 183 Jan 3, 2023
Code for CVPR 2021 oral paper "Exploring Data-Efficient 3D Scene Understanding with Contrastive Scene Contexts"

Exploring Data-Efficient 3D Scene Understanding with Contrastive Scene Contexts The rapid progress in 3D scene understanding has come with growing dem

Facebook Research 182 Dec 30, 2022
Neural Scene Graphs for Dynamic Scene (CVPR 2021)

Implementation of Neural Scene Graphs, that optimizes multiple radiance fields to represent different objects and a static scene background. Learned representations can be rendered with novel object compositions and views.

null 151 Dec 26, 2022
A weakly-supervised scene graph generation codebase. The implementation of our CVPR2021 paper ``Linguistic Structures as Weak Supervision for Visual Scene Graph Generation''

README.md shall be finished soon. WSSGG 0 Overview 1 Installation 1.1 Faster-RCNN 1.2 Language Parser 1.3 GloVe Embeddings 2 Settings 2.1 VG-GT-Graph

Keren Ye 35 Nov 20, 2022
Remote sensing change detection tool based on PaddlePaddle

PdRSCD PdRSCD(PaddlePaddle Remote Sensing Change Detection)是一个基于飞桨PaddlePaddle的遥感变化检测的项目,pypi包名为ppcd。目前0.2版本,最新支持图像列表输入的训练和预测,如多期影像、多源影像甚至多期多源影像。可以快速完

null 38 Aug 31, 2022
Code for SentiBERT: A Transferable Transformer-Based Architecture for Compositional Sentiment Semantics (ACL'2020).

SentiBERT Code for SentiBERT: A Transferable Transformer-Based Architecture for Compositional Sentiment Semantics (ACL'2020). https://arxiv.org/abs/20

Da Yin 66 Aug 13, 2022
VSR-Transformer - This paper proposes a new Transformer for video super-resolution (called VSR-Transformer).

VSR-Transformer By Jiezhang Cao, Yawei Li, Kai Zhang, Luc Van Gool This paper proposes a new Transformer for video super-resolution (called VSR-Transf

Jiezhang Cao 225 Nov 13, 2022
Official implement of Paper:A deeply supervised image fusion network for change detection in high resolution bi-temporal remote sening images

A deeply supervised image fusion network for change detection in high resolution bi-temporal remote sensing images 深度监督影像融合网络DSIFN用于高分辨率双时相遥感影像变化检测 Of

Chenxiao Zhang 135 Dec 19, 2022
Official Pytorch Implementation of Adversarial Instance Augmentation for Building Change Detection in Remote Sensing Images.

IAug_CDNet Official Implementation of Adversarial Instance Augmentation for Building Change Detection in Remote Sensing Images. Overview We propose a

null 53 Dec 2, 2022
1st place solution to the Satellite Image Change Detection Challenge hosted by SenseTime

1st place solution to the Satellite Image Change Detection Challenge hosted by SenseTime

Lihe Yang 209 Jan 1, 2023
Deep learning models for change detection of remote sensing images

Change Detection Models (Remote Sensing) Python library with Neural Networks for Change Detection based on PyTorch. ⚡ ⚡ ⚡ I am trying to build this pr

Kaiyu Li 176 Dec 24, 2022
From this paper "SESNet: A Semantically Enhanced Siamese Network for Remote Sensing Change Detection"

SESNet for remote sensing image change detection It is the implementation of the paper: "SESNet: A Semantically Enhanced Siamese Network for Remote Se

null 1 May 24, 2022