This is the unofficial code of Deep Dual-resolution Networks for Real-time and Accurate Semantic Segmentation of Road Scenes. which achieve state-of-the-art trade-off between accuracy and speed on cityscapes and camvid, without using inference acceleration and extra data

Last update: Dec 23, 2022

Overview

Deep Dual-resolution Networks for Real-time and Accurate Semantic Segmentation of Road Scenes

Introduction

This is the unofficial code of Deep Dual-resolution Networks for Real-time and Accurate Semantic Segmentation of Road Scenes. which achieve state-of-the-art trade-off between accuracy and speed on cityscapes and camvid, without using inference acceleration and extra data!on single 2080Ti GPU, DDRNet-23-slim yields 77.4% mIoU at 109 FPS on Cityscapes test set and 74.4% mIoU at 230 FPS on CamVid test set.

The code mainly borrows from HRNet-Semantic-Segmentation OCR and the official repository, thanks for their work.

requirements

Here I list the software and hardware used in my experiment

pytorch==1.7.0
3080*2
cuda==11.1

Quick start

0. Data preparation

You need to download the Cityscapesdatasets. and rename the folder cityscapes, then put the data under data folder.

└── data
  ├── cityscapes
  └── list

1. Pretrained model

download the pretrained model on imagenet or the segmentation model from the official，and put the files in ${PROJECT}/pretrained_models folder

VAL

use the official pretrained model and our eval.py code. with ydhongHIT's advice now can reach the same accuracy in the paper. Thanks.

cd ${PROJECT}
python tools/eval.py --cfg experiments/cityscapes/ddrnet23_slim.yaml

model	Train Set	Test Set	OHEM	Multi-scale	Flip	mIoU	Link
DDRNet23_slim	unknown	eval	Yes	No	No	77.83	official
DDRNet23_slim	unknown	eval	Yes	No	Yes	78.42	official
DDRNet23	unknown	eval	Yes	No	No	79.51	official
DDRNet23	unknown	eval	Yes	No	Yes	79.98	official

Note

with the ALIGN_CORNERS: false in ***.yaml will reach higher accuracy.

TRAIN

download the imagenet pretrained model, and then train the model with 2 nvidia-3080

cd ${PROJECT}
python -m torch.distributed.launch --nproc_per_node=2 tools/train.py --cfg experiments/cityscapes/ddrnet23_slim.yaml

the own trained model coming soon

OWN model

model	Train Set	Test Set	OHEM	Multi-scale	Flip	mIoU	Link
DDRNet23_slim	train	eval	Yes	No	Yes	77.77	Baidu/password:it2s
DDRNet23_slim	train	eval	Yes	Yes	Yes	79.57	Baidu/password:it2s
DDRNet23	train	eval	Yes	No	Yes	~	None
DDRNet39	train	eval	Yes	No	Yes	~	None

Note

set the ALIGN_CORNERS: true in ***.yaml, because i use the default setting in HRNet-Semantic-Segmentation OCR.
Multi-scale with scales: 0.5,0.75,1.0,1.25,1.5,1.75. it runs too slow.
from ydhongHIT, can change the align_corners=True with better performance, the default option is False

Reference

[1] HRNet-Semantic-Segmentation OCR branch

[2] the official repository

Comments

DDRNet-23-slim, cannot reproduce results.
Hello, Thank you for the great work!

I am trying to reproduce the results of ddrnet-23_slim on Cityscapes val (77.83 mIoU) but could not do it.

I tried training for more than 484 epochs - 600 epochs and 1500 epochs with 2, 5 and 8 GPUs.

Evaluating with both heads model index 0 and model index 1

The best performance is around 74.2 mIoU.

I also noticed, the results do not match when you use different number of gpus but train for the same number of epochs.

Here are the training logs, 600epochs_5gpus_ddrnet23_slim_2021-06-02-20-21_train.txt 600epochs_8gpu_ddrnet23_slim_2021-06-03-09-32_train.txt 1500epochs_2gus_ddrnet23_slim_2021-06-04-18-36_train.txt

I would really appreciate your help. Thank you.
opened by rawalkhirodkar 6
Unable to reproduce the inference on my images from pre-trained models

I am trying to reproduce the model results on my own dataset from the pre-trained models on cityscapes but tools/eval.py is throwing errors each time I am trying and fixing the details. Can you just give me a gist of steps that I would need to take in order to get inference on my images from the pretrained model of ddrnet23_slim?

opened by Gourav2K 3
How to train / eval on only one GPU

I would highly appreciate if You were kind and tell me, how to eval and train this model on one GPU ?

I was changed GPUS: (0, 1) to GPUS: (0,) and not working.

opened by MateKristof 1
Could you share your training log?

Could you share your training log? I used the ddrnet23_slim's default setting (experiments/cityscapes/ddrnet23_slim.yaml) with 1000 Epochs but got only 0.6223 mIoU. thanks.

=> saving checkpoint to output/cityscapes/ddrnet23_slimcheckpoint.pth.tar Loss: 0.396, MeanIU: 0.6223, Best_mIoU: 0.6223 [0.96390848 0.76018533 0.88509226 0.32266355 0.43711438 0.54837026 0.62688693 0.70588259 0.91061873 0.5774142 0.93025264 0.7288157 0.41738881 0.89958299 0.27927647 0.50435703 0.23100624 0.38669031 0.70799136] Epoch: [999/1000] Iter:[0/92], Time: 4.41, lr: [1.995262314968881e-05], Loss: 0.317782, Acc:0.815917 Epoch: [999/1000] Iter:[10/92], Time: 1.06, lr: [1.798967935132047e-05], Loss: 0.330938, Acc:0.799646 Epoch: [999/1000] Iter:[20/92], Time: 0.91, lr: [1.6002587191915764e-05], Loss: 0.326241, Acc:0.789909 Epoch: [999/1000] Iter:[30/92], Time: 0.85, lr: [1.3987608745552127e-05], Loss: 0.331387, Acc:0.794377 Epoch: [999/1000] Iter:[40/92], Time: 0.80, lr: [1.1939715118663247e-05], Loss: 0.329101, Acc:0.795018 Epoch: [999/1000] Iter:[50/92], Time: 0.77, lr: [9.851793876099303e-06], Loss: 0.327667, Acc:0.797766 Epoch: [999/1000] Iter:[60/92], Time: 0.76, lr: [7.71304625589395e-06], Loss: 0.324819, Acc:0.797479 Epoch: [999/1000] Iter:[70/92], Time: 0.75, lr: [5.505178031253932e-06], Loss: 0.327130, Acc:0.796518 Epoch: [999/1000] Iter:[80/92], Time: 0.74, lr: [3.190465676883869e-06], Loss: 0.324793, Acc:0.797461 Epoch: [999/1000] Iter:[90/92], Time: 0.73, lr: [6.36089096925948e-07], Loss: 0.324365, Acc:0.794952

opened by polor1010 1
> > 同问

同问

你好，我也有这个疑问，请问您调换权重值实验了吗？我指标只有77，离79差的有点多，方便交流一下吗

我跑了slim版本的是75多，按照论文设置权重的话精度还会降低一些，你呢

Originally posted by @J0kerrr164 in https://github.com/chenjun2hao/DDRNet.pytorch/issues/22#issuecomment-1188930457

opened by scl666 0
image, label, _, name, *border_padding = batch

Hi, I cannot find the defined variable "border _padding " in function testval in file function.py. could you pls. tell me ? thanks !

image, label, _, name, *border_padding = batch

opened by Yaoxingtian 1
how can I reach the same accuracy in the paper?

Hi, bro, I train DDRNet-23 without pretrained model using 1024x1024 resolution. While it get 0.71mIoU metric. I want to know how can I reach the same accuracy in the paper? what is your trick? thanks

opened by coallar 0
bug

https://github.com/chenjun2hao/DDRNet.pytorch/blob/bc0e193e87ead839dbc715c48e6bfb059cf21b27/lib/utils/utils.py#L35 "and" should be "or"

opened by laocan 0

This is the unofficial code of Deep Dual-resolution Networks for Real-time and Accurate Semantic Segmentation of Road Scenes. which achieve state-of-the-art trade-off between accuracy and speed on cityscapes and camvid, without using inference acceleration and extra data

Related tags

Overview

Deep Dual-resolution Networks for Real-time and Accurate Semantic Segmentation of Road Scenes

Introduction

requirements

Quick start

0. Data preparation

1. Pretrained model

VAL

TRAIN

OWN model

Reference

Comments

Owner

chenjun

We evaluate our method on different datasets (including ShapeNet, CUB-200-2011, and Pascal3D+) and achieve state-of-the-art results, outperforming all the other supervised and unsupervised methods and 3D representations, all in terms of performance, accuracy, and training time.

SpeechNAS Better Trade off between Latency and Accuracy for Large Scale Speaker Verification

PyTorch implementation of Memory-based semantic segmentation for off-road unstructured natural environments.

Code repository for Semantic Terrain Classification for Off-Road Autonomous Driving

Mind the Trade-off: Debiasing NLU Models without Degrading the In-distribution Performance

DFFNet: An IoT-perceptive Dual Feature Fusion Network for General Real-time Semantic Segmentation

Propose a principled and practically effective framework for unsupervised accuracy estimation and error detection tasks with theoretical analysis and state-of-the-art performance.

Implementation of CVPR 2020 Dual Super-Resolution Learning for Semantic Segmentation

Semantic Segmentation for Real Point Cloud Scenes via Bilateral Augmentation and Adaptive Fusion (CVPR 2021)

Instance Segmentation in 3D Scenes using Semantic Superpoint Tree Networks

A complete, self-contained example for training ImageNet at state-of-the-art speed with FFCV

deep-table implements various state-of-the-art deep learning and self-supervised learning algorithms for tabular data using PyTorch.

FADNet++: Real-Time and Accurate Disparity Estimation with Configurable Networks

MASA-SR: Matching Acceleration and Spatial Adaptation for Reference-Based Image Super-Resolution (CVPR2021)

Speed-Test - You can check your intenet speed using this tool

tsai is an open-source deep learning package built on top of Pytorch & fastai focused on state-of-the-art techniques for time series classification, regression and forecasting.

State of the Art Neural Networks for Deep Learning

Implementation for the paper 'YOLO-ReT: Towards High Accuracy Real-time Object Detection on Edge GPUs'

TLDR; Train custom adaptive filter optimizers without hand tuning or extra labels.