[cvpr22] Perturbed and Strict Mean Teachers for Semi-supervised Semantic Segmentation

Overview

PS-MT

[cvpr22] Perturbed and Strict Mean Teachers for Semi-supervised Semantic Segmentation

by Yuyuan Liu, Yu Tian, Yuanhong Chen, Fengbei Liu, Vasileios Belagiannis and Gustavo Carneiro

Computer Vision and Pattern Recognition Conference (CVPR), 2022

image

Installation

Please install the dependencies and dataset based on this installation document.

Getting start

Please follow this instruction document to reproduce our results.

Results

Pascal VOC12 dataset

  1. augmented set

    Backbone 1/16 (662) 1/8 (1323) 1/4 (2646) 1/2 (5291)
    50 72.83 75.70 76.43 77.88
    101 75.50 78.20 78.72 79.76
  2. high quality set (based on res101)

    1/16 (92) 1/8 (183) 1/4 (366) 1/2 (732) full (1464)
    65.80 69.58 76.57 78.42 80.01

CityScape dataset

  1. following the setting of CAC (720x720, CE supervised loss)

    Backbone slid. eval 1/8 (372) 1/4 (744) 1/2 (1488)
    50 74.37 75.15 76.02
    50 75.76 76.92 77.64
    101 76.89 77.60 79.09
  2. following the setting of CPS (800x800, OHEM supervised loss)

    Backbone slid. eval 1/8 (372) 1/4 (744) 1/2 (1488)
    50 77.12 78.38 79.22

Training details

Some examples of training details, including:

  1. VOC12 dataset in this wandb link.
  2. Cityscapes dataset in this wandb link (w/ 1-teacher inference).

In details, after clicking the run, you can checkout:

  1. overall information (e.g., training command line, hardware information and training time).
  2. training details (e.g., loss curves, validation results and visualization)
  3. output logs (well, sometimes might crash ...)

Acknowledgement & Citation

The code is highly based on the CCT. Many thanks for their great work.

Please consider citing this project in your publications if it helps your research.

@article{liu2021perturbed,
  title={Perturbed and Strict Mean Teachers for Semi-supervised Semantic Segmentation},
  author={Liu, Yuyuan and Tian, Yu and Chen, Yuanhong and Liu, Fengbei and Belagiannis, Vasileios and Carneiro, Gustavo},
  journal={arXiv preprint arXiv:2111.12903},
  year={2021}
}

TODO

  • Code of deeplabv3+ for voc12
  • Code of deeplabv3+ for cityscapes
  • Code of pspnet for voc12
Comments
  • Performance on the testing data

    Performance on the testing data

    Hi, I got some models and results by running './scripts/train_voc_aug.sh -l 1323 -g 4 -b 50'. How can I get the testing results on Pascal VOC? Is the valid_Mean_IoU (0.7005) same as testing result? Run summary: global_step 23119 learning_rate_0 1e-05 learning_rate_1 1e-05 loss_sup 0.05151 loss_unsup 0.0 mIoU_labeled 0.932 mIoU_unlabeled 0.619 pixel_acc_labeled 0.98 pixel_acc_unlabeled 0.886 ramp_up 1.0 valid_Mean_IoU 0.7005 valid_Pixel_Accuracy 0.9316

    opened by LPXTT 8
  • How to download pretrained model?  bucket_namespace

    How to download pretrained model? bucket_namespace

    Hello author, could you provide the download link of ResNet50 and ResNet101? I don't know the bucket_namespace and bucket_name in the download script. I downloaded two pretrained models from other place but the size dosen't match. Thanks.

    opened by LPXTT 7
  • About training

    About training

    I run with ./scripts/train_voc_aug.sh -l 1323 -g 4 -b 101 but get error:

    ID 3 Warm (4) | Ls 0.51 |: 98%|█████████████████████████████████████████████████████████████████████▎ | 40/41 [01:04<00:00, 1.18it/s] ID 3 Warm (4) | Ls 0.51 |: 98%|█████████████████████████████████████████████████████████████████████▎ | 40/41 [01:14<00:00, 1.18it/s] ID 3 Warm (4) | Ls 0.51 |: 100%|███████████████████████████████████████████████████████████████████████| 41/41 [01:14<00:00, 3.65s/it] ID 3 Warm (4) | Ls 0.51 |: 100%|███████████████████████████████████████████████████████████████████████| 41/41 [01:14<00:00, 1.81s/it]

    0%| | 0/289 [00:00<?, ?it/s]wandb: Network error (ConnectionError), entering retry loop.

    How can I solve this problem?Can I run without wandb?

    opened by Yan1026 5
  • sliding evaluation

    sliding evaluation

    Hello, thank you for your excellent work, I read your paper, I have a place is not very clear, is the experiment mentioned in the sliding evaluation is what it means, hope to get your reply, thank you!

    opened by NJNUCS 4
  • Training on Cityscape

    Training on Cityscape

    Sorry to bother you. I train with bash ./scripts/train_city.sh -l 372 -g 4 -b 50,but get error:

    availble_gpus= [0, 1, 2, 3]
      0%|                                                                                                           | 0/93 [00:00<?, ?it/s]
      0%|                                                                                                           | 0/93 [00:05<?, ?it/s]
    wandb: Waiting for W&B process to finish... (failed 1).
    wandb: - 0.000 MB of 0.000 MB uploaded (0.000 MB deduped)
    wandb: \ 0.000 MB of 0.000 MB uploaded (0.000 MB deduped)
    wandb: | 0.000 MB of 0.000 MB uploaded (0.000 MB deduped)
    wandb:                                                                                
    wandb: You can sync this run to the cloud by running:
    wandb: wandb sync /homedjy/PS-MT/wandb/offline-run-20220908_195002-2iykpb0m
    wandb: Find logs at: ./wandb/offline-run-20220908_195002-2iykpb0m/logs
    Traceback (most recent call last):
      File "CityCode/main.py", line 199, in <module>
        main(-1, 1, config, args)
      File "CityCode/main.py", line 116, in main
        trainer.train()
      File "/home/PS-MT/CityCode/Base/base_trainer.py", line 145, in train
        _ = self._warm_up(epoch, id=1)
      File "/homedjy/PS-MT/CityCode/train.py", line 173, in _warm_up
        curr_iter=batch_idx, epoch=epoch-1, id=id, warm_up=True)
      File "/home/.conda/envs/ps-mt/lib/python3.7/site-packages/torch/nn/modules/module.py", line 889, in _call_impl
        result = self.forward(*input, **kwargs)
      File "/home/.conda/envs/ps-mt/lib/python3.7/site-packages/torch/nn/parallel/data_parallel.py", line 155, in forward
        "them on device: {}".format(self.src_device_obj, t.device))
    RuntimeError: module must have its parameters and buffers on device cuda:0 (device_ids[0]) but found one of them on device: cpu
    
    

    I try to fix it but no effect.I want use GPU5,6,7,8,because GPU0123 is occupied.But when print availble_gpus,it's still [0, 1, 2, 3]. I can train model on VOC in same case. Do you have any ideas?

    opened by Yan1026 4
  • Training with multi-GPUs

    Training with multi-GPUs

    The code works fine when I train with one gpu. The _warm_up process works fine when using multi-gpus distributed training,but the _train_epoch process gets stuck. Gpus and cpus are still running normally. Have you encountered the same problem?

    opened by GuGuLL123 4
  • Results of supervised baseline

    Results of supervised baseline

    Hi,

    Did you apply any extra techniques to your supervised baseline, such as setting output stride as 8, auxiliary loss, or OHEM? Since your reported baseline results are very high on the Pascal dataset, according to Figure 3.

    opened by LiheYoung 4
  • About batch_size

    About batch_size

    HI! I train with ./scripts/train_voc_aug.sh -l 1323 -g 2 -b 101,and I use 4*V100(16G),batchsize=4,lr=0.0025. In your code,it's 2*V100(32G),batchsize=8,lr=0.0025.But miou in result is just75.29%.When I set lr=0.00125 ,the miou=76.33%,lower than you 78.20%.

    I set batchsize in scripts/train_voc_aug.sh line 49:

    nohup python3 VocCode/main.py --labeled_examples="${labelled}" --gpus=${gpus} --backbone=${backbone} --warm_up=5 --batch_size=4 --semi_p_th=.6 --semi_n_th=.0 --learning-rate=1.25e-3 \
    --epochs=${max_epochs} --unsup_weight=${unsup_weight} > voc_aug_"${labelled}"_"${backbone}".out &
    

    In addition, I have done experiments on Cityscapes(372), and only with batchsize set to 2 does not get error:out of GUP memory.(gpu=4*v100(16G),lr=0.0045).In your code it's gpu=2*v100(32G),lr=0.0045,batchsize=8

    Can you give me some advice?

    opened by Yan1026 3
  • About training config

    About training config

    Hi,thank you for your great work! If I use 4GPU,and batchsize=16 learning-rate=1e-2 ,will it be the best configuration?

    And why you said 2GPU can get better performance rather than 4GPU?

    opened by Yan1026 3
  • CutMix for Cityscapes

    CutMix for Cityscapes

    Hi, thank you for your work. I have a question about the CuTMIx.

    Since the batch size for each GPU is 1 when using 8GPUs (conventional setting), I wonder how to perform CutMix when only one unlabeled image is in a GPU? Can you give me some advice?

    opened by JoyHuYY1412 3
  • questions about Conf-CE loss

    questions about Conf-CE loss

    I recently read your paper and it's a great job. I have a question about the Conf-CE loss in the paper. Can you explain what the c(w) formula means? I confused y~ and y^ in the c(w), formula(4), figure 2. question1 question2

    opened by Darcy103 3
Owner
Yuyuan Liu
Yuyuan Liu
[CVPR22] Official codebase of Semantic Segmentation by Early Region Proxy.

RegionProxy Figure 2. Performance vs. GFLOPs on ADE20K val split. Semantic Segmentation by Early Region Proxy Yifan Zhang, Bo Pang, Cewu Lu CVPR 2022

Yifan 54 Nov 29, 2022
Official code for "Mean Shift for Self-Supervised Learning"

MSF Official code for "Mean Shift for Self-Supervised Learning" Requirements Python >= 3.7.6 PyTorch >= 1.4 torchvision >= 0.5.0 faiss-gpu >= 1.6.1 In

UMBC Vision 44 Nov 21, 2022
Anti-Adversarially Manipulated Attributions for Weakly and Semi-Supervised Semantic Segmentation (CVPR 2021)

Anti-Adversarially Manipulated Attributions for Weakly and Semi-Supervised Semantic Segmentation Input Image Initial CAM Successive Maps with adversar

Jungbeom Lee 110 Dec 7, 2022
Shape-aware Semi-supervised 3D Semantic Segmentation for Medical Images

SASSnet Code for paper: Shape-aware Semi-supervised 3D Semantic Segmentation for Medical Images(MICCAI 2020) Our code is origin from UA-MT You can fin

klein 125 Jan 3, 2023
Semi-supervised Semantic Segmentation with Directional Context-aware Consistency (CVPR 2021)

Semi-supervised Semantic Segmentation with Directional Context-aware Consistency (CAC) Xin Lai*, Zhuotao Tian*, Li Jiang, Shu Liu, Hengshuang Zhao, Li

Jia Research Lab 137 Dec 14, 2022
Semi-supervised Semantic Segmentation with Directional Context-aware Consistency (CVPR 2021)

Semi-supervised Semantic Segmentation with Directional Context-aware Consistency (CAC) Xin Lai*, Zhuotao Tian*, Li Jiang, Shu Liu, Hengshuang Zhao, Li

DV Lab 137 Dec 14, 2022
[CVPR 2021] Semi-Supervised Semantic Segmentation with Cross Pseudo Supervision

TorchSemiSeg [CVPR 2021] Semi-Supervised Semantic Segmentation with Cross Pseudo Supervision by Xiaokang Chen1, Yuhui Yuan2, Gang Zeng1, Jingdong Wang

Chen XiaoKang 387 Jan 8, 2023
ST++: Make Self-training Work Better for Semi-supervised Semantic Segmentation

ST++ This is the official PyTorch implementation of our paper: ST++: Make Self-training Work Better for Semi-supervised Semantic Segmentation. Lihe Ya

Lihe Yang 147 Jan 3, 2023
[ICCV 2021] A Simple Baseline for Semi-supervised Semantic Segmentation with Strong Data Augmentation

[ICCV 2021] A Simple Baseline for Semi-supervised Semantic Segmentation with Strong Data Augmentation

CodingMan 45 Dec 12, 2022
[CVPR 2022] Semi-Supervised Semantic Segmentation Using Unreliable Pseudo-Labels

Using Unreliable Pseudo Labels Official PyTorch implementation of Semi-Supervised Semantic Segmentation Using Unreliable Pseudo Labels, CVPR 2022. Ple

Haochen Wang 268 Dec 24, 2022
Unet network with mean teacher for altrasound image segmentation

Unet network with mean teacher for altrasound image segmentation

null 5 Nov 21, 2022
Learning Pixel-level Semantic Affinity with Image-level Supervision for Weakly Supervised Semantic Segmentation, CVPR 2018

Learning Pixel-level Semantic Affinity with Image-level Supervision This code is deprecated. Please see https://github.com/jiwoon-ahn/irn instead. Int

Jiwoon Ahn 337 Dec 15, 2022
[CVPR 2021] Teachers Do More Than Teach: Compressing Image-to-Image Models (CAT)

CAT arXiv Pytorch implementation of our method for compressing image-to-image models. Teachers Do More Than Teach: Compressing Image-to-Image Models Q

Snap Research 160 Dec 9, 2022
Recall Loss for Semantic Segmentation (This repo implements the paper: Recall Loss for Semantic Segmentation)

Recall Loss for Semantic Segmentation (This repo implements the paper: Recall Loss for Semantic Segmentation) Download Synthia dataset The model uses

null 32 Sep 21, 2022
UniMoCo: Unsupervised, Semi-Supervised and Full-Supervised Visual Representation Learning

UniMoCo: Unsupervised, Semi-Supervised and Full-Supervised Visual Representation Learning This is the official PyTorch implementation for UniMoCo pape

dddzg 49 Jan 2, 2023
Project looking into use of autoencoder for semi-supervised learning and comparing data requirements compared to supervised learning.

Project looking into use of autoencoder for semi-supervised learning and comparing data requirements compared to supervised learning.

Tom-R.T.Kvalvaag 2 Dec 17, 2021
A pytorch implementation of MBNET: MOS PREDICTION FOR SYNTHESIZED SPEECH WITH MEAN-BIAS NETWORK

Pytorch-MBNet A pytorch implementation of MBNET: MOS PREDICTION FOR SYNTHESIZED SPEECH WITH MEAN-BIAS NETWORK Training To train a new model, please ru

null 46 Dec 28, 2022
Hybrid CenterNet - Hybrid-supervised object detection / Weakly semi-supervised object detection

Hybrid-Supervised Object Detection System Object detection system trained by hybrid-supervision/weakly semi-supervision (HSOD/WSSOD): This project is

null 5 Dec 10, 2022
[CVPR 2021] MiVOS - Mask Propagation module. Reproduced STM (and better) with training code :star2:. Semi-supervised video object segmentation evaluation.

MiVOS (CVPR 2021) - Mask Propagation Ho Kei Cheng, Yu-Wing Tai, Chi-Keung Tang [arXiv] [Paper PDF] [Project Page] [Papers with Code] This repo impleme

Rex Cheng 106 Jan 3, 2023