LightNet++: Boosted Light-weighted Networks for Real-time Semantic Segmentation

Related tags

Deep Learning pytorch apex attention-mechanism semantic-segmentation mixnet cityscapes mobilenetv2 mobilenetv2plus unsharp-masking aspp deeplab-v3-plus deepdrive shufflenetv2 lightnet shufflenetv2plus inplaceabn dense-aspp light-weighted-network efficientnet bifpn
Overview

LightNet++

!!!New Repo.!!! ⇒ EfficientNet.PyTorch: Concise, Modular, Human-friendly PyTorch implementation of EfficientNet with Pre-trained Weights

!!!New Repo.!!! ⇒ MixNet-Pytorch: Concise, Modular, Human-friendly PyTorch implementation of MixNet with Pre-trained Weights

This repository contains the code (PyTorch-1.0+, W.I.P.) for: "LightNet++: Boosted Light-weighted Networks for Real-time Semantic Segmentation" by Huijun Liu.
LightNet++ is an advanced version of LightNet, which purpose to get more concise model design, smaller models, and better performance.

  • MobileNetV2Plus: Modified MobileNetV2 (backbone)[1,8] + DSASPPInPlaceABNBlock[2,3] + Parallel Bottleneck Channel-Spatial Attention Block (PBCSABlock)[6] + UnSharp Masking (USM) + Encoder-Decoder Arch.[3] + InplaceABN[4].

  • ShuffleNetV2Plus: Modified ShuffleNetV2 (backbone)[1,8] + DSASPPInPlaceABNBlock[2,3] + Parallel Bottleneck Channel-Spatial Attention Block (PBCSABlock)[6]+ UnSharp Masking (USM) + Encoder-Decoder Arch.[3] + InplaceABN[4].

  • MixSeg-MixBiFPN: Modified MixNet (backbone)[1,8] + MixBiFPNBlock[2,3] + Encoder-Decoder Arch.[3]

More about USM(Unsharp Mask)-Operator Block see Repo: SharpPeleeNet

Dependencies

Datasets for Autonomous Driving

Results

Results on Cityscapes (Pixel-level/Semantic Segmentation)

Model mIoU (S.S* Mixed Precision) Model Weight
MobileNetV2Plus X1.0 71.5314 (WIP) cityscapes_mobilenetv2plus_x1.0.pkl (14.3 MB)
ShuffleNetV2Plus X1.0 69.0885-72.5255 (WIP) cityscapes_shufflenetv2plus_x1.0.pkl (8.59 MB)
MixSeg+MixBiFPN ArchS 72.2321 (WIP) cityscapes_mixseg_archs_mixbifpn.pkl (16.4 MB)
  • S.S.: Single Scale (1024x2048)

Feature Visualization

Comments

  • "RuntimeError: CUDA out of memory."

    Thanks for your job!!! I got this error when using "train_shuffle.py" to train model on citysacpes dataset

    Traceback (most recent call last):    
  File "/home/wcgu/code/RTSeg/LightNetPlusPlus/scripts/train_shuffle.py", line 382, in <module>    
    train(train_args, data_path, save_path)    
  File "/home/wcgu/code/RTSeg/LightNetPlusPlus/scripts/train_shuffle.py", line 290, in train    
    val_loss = loss_fn(input=net_out, target=labels_val, K=topk, weight=None)    
  File "/home/wcgu/code/RTSeg/LightNetPlusPlus/scripts/loss.py", line 111, in bootstrapped_cross_entropy2d    
    size_average=size_average)    
  File "/home/wcgu/code/RTSeg/LightNetPlusPlus/scripts/loss.py", line 88, in _bootstrap_xentropy_single    
    log_p = log_p[target.view(n * h * w, 1).repeat(1, c) >= 0]   
RuntimeError: CUDA out of memory. Tried to allocate 304.00 MiB (GPU 0; 10.72 GiB total capacity; 9.30 GiB already allocated; 226.62 MiB free; 59.54 MiB cached)

    The batch_size is 1, and the training step is ok. Could you help me to solve it?

    opened by IOEvan 1
  • Missing keys in state dict

    Missing keys in state dict

    Only the deepdrive checkpoint file seems to have the usm.norm_act.weight

    ...
> # ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ #
> 1. Setting up Model...
Traceback (most recent call last):
  File "deploy/cityscapes/evaluation/ss.py", line 253, in <module>
    model.load_state_dict(pre_weight)
  File "/home/ben/LightNetPlusPlus/env/lib/python3.6/site-packages/torch/nn/modules/module.py", line 777, in load_state_dict
    self.__class__.__name__, "\n\t".join(error_msgs)))
RuntimeError: Error(s) in loading state_dict for DataParallel:
	Missing key(s) in state_dict: "module.usm.norm_act.weight", "module.usm.norm_act.bias", "module.usm.norm_act.running_mean", "module.usm.norm_act.running_var". 

(env) ben@mercury:~/LightNetPlusPlus$ grep -r norm_act.weight ./*
Binary file ./checkpoint/cityscapes_mobilenetv2plus_x1.0.pkl matches
Binary file ./checkpoint/cityscapes_shufflenetv2plus_x0.5.pkl matches
Binary file ./checkpoint/cityscapes_shufflenetv2plus_x1.0.pkl matches
Binary file ./checkpoint/deepdrive_shufflenetv2plus_x1.0.pkl matches

(env) ben@mercury:~/LightNetPlusPlus$ grep -r usm.norm_act.weight ./*
Binary file ./checkpoint/deepdrive_shufflenetv2plus_x1.0.pkl matches
    opened by barnjamin 0
  • Training setting

    Training setting

    Hi, @ansleliu , thanks for your work, can you share the training protocal, lr, batchsize, optimizer, epochs etc., for training on Cityscapes? Thanks a lot.

    opened by wangq95 0
  • Have you published the paper

    Have you published the paper

    Hi, thanks for sharing the code. Is the paper "LightNet++: Boosted Light-weighted Networks for Real-time Semantic Segmentation" published? Because of I couldn't find it anywhere!

    opened by adavoudi 4
  • train

    train

    Hey, @ansleliu First of all I thank you for good job. How I can start to training this models? Usually repository has been file 'train.py' If I need to take this file in thr old repository of LightNet then what needs to be changed in it.

    opened by thepuremind 0
Owner
linksense
LinkSense is a leading computer vision core technology solution provider in the field of industrial artificial intelligence.
linksense
GitHub https://ansleliu.github.io/LightNet.html
Official code of "R2RNet: Low-light Image Enhancement via Real-low to Real-normal Network."

R2RNet Official code of "R2RNet: Low-light Image Enhancement via Real-low to Real-normal Network." Jiang Hai, Zhu Xuan, Ren Yang, Yutong Hao, Fengzhu

null 77 Dec 24, 2022
Real-Time-Student-Attendence-System - Real Time Student Attendence System

Real-Time-Student-Attendence-System The Student Attendance Management System Pro

Rounak Das 1 Feb 15, 2022
HyperSeg: Patch-wise Hypernetwork for Real-time Semantic Segmentation Official PyTorch Implementation

: We present a novel, real-time, semantic segmentation network in which the encoder both encodes and generates the parameters (weights) of the decoder. Furthermore, to allow maximal adaptivity, the weights at each decoder block vary spatially. For this purpose, we design a new type of hypernetwork, composed of a nested U-Net for drawing higher level context features

Yuval Nirkin 182 Dec 14, 2022
FANet - Real-time Semantic Segmentation with Fast Attention

FANet Real-time Semantic Segmentation with Fast Attention Ping Hu, Federico Perazzi, Fabian Caba Heilbron, Oliver Wang, Zhe Lin, Kate Saenko , Stan Sc

Ping Hu 42 Nov 30, 2022
DFFNet: An IoT-perceptive Dual Feature Fusion Network for General Real-time Semantic Segmentation

DFFNet Paper DFFNet: An IoT-perceptive Dual Feature Fusion Network for General Real-time Semantic Segmentation. Xiangyan Tang, Wenxuan Tu, Keqiu Li, J

null 4 Sep 23, 2022
Recall Loss for Semantic Segmentation (This repo implements the paper: Recall Loss for Semantic Segmentation)

Recall Loss for Semantic Segmentation (This repo implements the paper: Recall Loss for Semantic Segmentation) Download Synthia dataset The model uses

null 32 Sep 21, 2022
Official Tensorflow implementation of "M-LSD: Towards Light-weight and Real-time Line Segment Detection"

M-LSD: Towards Light-weight and Real-time Line Segment Detection Official Tensorflow implementation of "M-LSD: Towards Light-weight and Real-time Line

NAVER/LINE Vision 357 Jan 4, 2023
Pytorch implementation of "M-LSD: Towards Light-weight and Real-time Line Segment Detection"

M-LSD: Towards Light-weight and Real-time Line Segment Detection Pytorch implementation of "M-LSD: Towards Light-weight and Real-time Line Segment Det

null 123 Jan 4, 2023
Light-weight network, depth estimation, knowledge distillation, real-time depth estimation, auxiliary data.

light-weight-depth-estimation Boosting Light-Weight Depth Estimation Via Knowledge Distillation, https://arxiv.org/abs/2105.06143 Junjie Hu, Chenyou F

Junjie Hu 13 Dec 10, 2022
Boosted neural network for tabular data

XBNet - Xtremely Boosted Network Boosted neural network for tabular data XBNet is an open source project which is built with PyTorch which tries to co

Tushar Sarkar 175 Jan 4, 2023
Multivariate Boosted TRee

Multivariate Boosted TRee What is MBTR MBTR is a python package for multivariate boosted tree regressors trained in parameter space. The package can h

SUPSI-DACD-ISAAC 61 Dec 19, 2022
Boosted CVaR Classification (NeurIPS 2021)

Boosted CVaR Classification Runtian Zhai, Chen Dan, Arun Sai Suggala, Zico Kolter, Pradeep Ravikumar NeurIPS 2021 Table of Contents Quick Start Train

Runtian Zhai 4 Feb 15, 2022
Code for paper: Group-CAM: Group Score-Weighted Visual Explanations for Deep Convolutional Networks

Group-CAM By Zhang, Qinglong and Rao, Lu and Yang, Yubin [State Key Laboratory for Novel Software Technology at Nanjing University] This repo is the o

zhql 98 Nov 16, 2022
Semantic Segmentation for Real Point Cloud Scenes via Bilateral Augmentation and Adaptive Fusion (CVPR 2021)

Semantic Segmentation for Real Point Cloud Scenes via Bilateral Augmentation and Adaptive Fusion (CVPR 2021) This repository is for BAAF-Net introduce

null 90 Dec 29, 2022
Learning Pixel-level Semantic Affinity with Image-level Supervision for Weakly Supervised Semantic Segmentation, CVPR 2018

Learning Pixel-level Semantic Affinity with Image-level Supervision This code is deprecated. Please see https://github.com/jiwoon-ahn/irn instead. Int

Jiwoon Ahn 337 Dec 15, 2022
Siamese-nn-semantic-text-similarity - A repository containing comprehensive Neural Networks based PyTorch implementations for the semantic text similarity task

Siamese Deep Neural Networks for Semantic Text Similarity PyTorch A repository c

Shahrukh Khan 32 Dec 15, 2022
Applications using the GTN library and code to reproduce experiments in "Differentiable Weighted Finite-State Transducers"

gtn_applications An applications library using GTN. Current examples include: Offline handwriting recognition Automatic speech recognition Installing

Facebook Research 68 Dec 29, 2022
Monocular Depth Estimation - Weighted-average prediction from multiple pre-trained depth estimation models

merged_depth runs (1) AdaBins, (2) DiverseDepth, (3) MiDaS, (4) SGDepth, and (5) Monodepth2, and calculates a weighted-average per-pixel absolute dept

Pranav 39 Nov 21, 2022
Automatic differentiation with weighted finite-state transducers.

GTN: Automatic Differentiation with WFSTs Quickstart | Installation | Documentation What is GTN? GTN is a framework for automatic differentiation with

null 100 Dec 29, 2022