CondLaneNet: a Top-to-down Lane Detection Framework Based on Conditional Convolution

Overview

CondLaneNet: a Top-to-down Lane Detection Framework Based on Conditional Convolution

This is the official implementation code of the paper "CondLaneNet: a Top-to-down Lane Detection Framework Based on ConditionalConvolution". (Link: https://arxiv.org/abs/2105.05003) We achieve state-of-the-art performance on multiple lane detection benchmarks.

Architecture,

Installation

This implementation is based on mmdetection(v2.0.0). Please refer to install.md for installation.

Datasets

We conducted experiments on CurveLanes, CULane and TuSimple. Please refer to dataset.md for installation.

Models

For your convenience, we provide the following trained models on Curvelanes, CULane, and TuSimple datasets

Model Speed F1 Link
curvelanes_small 154FPS 85.09 download
curvelanes_medium 109FPS 85.92 download
curvelanes_large 48FPS 86.10 download
culane_small 220FPS 78.14 download
culane_medium 152FPS 78.74 download
culane_large 58FPS 79.48 download
tusimple_small 220FPS 97.01 download
tusimple_medium 152FPS 96.98 download
tusimple_large 58FPS 97.24 download

Testing

CurveLanes 1 Edit the "data_root" in the config file to your Curvelanes dataset path. For example, for the small version, open "configs/curvelanes/curvelanes_small_test.py" and set "data_root" to "[your-data-path]/curvelanes".

2 run the test script

cd [project-root]
python tools/condlanenet/curvelanes/test_curvelanes.py configs/condlanenet/curvelanes/curvelanes_small_test.py [model-path] --evaluate

If "--evaluate" is added, the evaluation results will be printed. If you want to save the visualization results, you can add "--show" and add "--show_dst" to specify the save path.

CULane

1 Edit the "data_root" in the config file to your CULane dataset path. For example,for the small version, you should open "configs/culane/culane_small_test.py" and set the "data_root" to "[your-data-path]/culane".

2 run the test script

cd [project-root]
python tools/condlanenet/culane/test_culane.py configs/condlanenet/culane/culane_small_test.py [model-path]
  • you can add "--show" and add "--show_dst" to specify the save path.
  • you can add "--results_dst" to specify the result saving path.

3 We use the official evaluation tools of SCNN to evaluate the results.

TuSimple

1 Edit the "data_root" in the config file to your TuSimple dataset path. For example,for the small version, you should open "configs/tusimple/tusimple_small_test.py" and set the "data_root" to "[your-data-path]/tuSimple".

2 run the test script

cd [project-root]
python tools/condlanenet/tusimple/test_tusimple.py configs/condlanenet/tusimple/tusimple_small_test.py [model-path]
  • you can add "--show" and add "--show_dst" to specify the save path.
  • you can add "--results_dst" to specify the result saving path.

3 We use the official evaluation tools of TuSimple to evaluate the results.

Speed Test

cd [project-root]
python tools/condlanenet/speed_test.py configs/condlanenet/culane/culane_small_test.py [model-path]

Training

For example, train CULane using 4 gpus:

cd [project-root]
CUDA_VISIBLE_DEVICES=0,1,2,3 PORT=29001 tools/dist_train.sh configs/condlanenet/culane/culane_small_train.py 4 --no-validate 

Results

CurveLanes

Model F1 Speed GFLOPS
Small(ResNet-18) 85.09 154FPS 10.3
Medium(ResNet-34) 85.92 109FPS 19.7
Large(ResNet-101) 86.10 48FPS 44.9

CULane

Model F1 Speed GFLOPS
Small(ResNet-18) 78.14 220FPS 10.2
Medium(ResNet-34) 78.74 152FPS 19.6
Large(ResNet-101) 79.48 58FPS 44.8

TuSimple

Model F1 Speed GFLOPS
Small(ResNet-18) 97.01 220FPS 10.2
Medium(ResNet-34) 96.98 152FPS 19.6
Large(ResNet-101) 97.24 58FPS 44.8

Visualization results

Results

Comments
  • Could you please tell me how to correct this error?Thank you in advance!!!

    Could you please tell me how to correct this error?Thank you in advance!!!

    ~/projects/condlanenet/conditional-lane-detection$ python tools/condlanenet/culane/test_culane.py Traceback (most recent call last): File "tools/condlanenet/culane/test_culane.py", line 21, in from mmdet.models.detectors.condlanenet import CondLanePostProcessor File "/home/yjx/projects/condlanenet/conditional-lane-detection/mmdet/models/init.py", line 1, in from .backbones import * # noqa: F401,F403 File "/home/yjx/projects/condlanenet/conditional-lane-detection/mmdet/models/backbones/init.py", line 1, in from .hrnet import HRNet File "/home/yjx/projects/condlanenet/conditional-lane-detection/mmdet/models/backbones/hrnet.py", line 7, in from mmdet.utils import get_root_logger ImportError: cannot import name 'get_root_logger' from 'mmdet.utils' (/home/yjx/projects/condlanenet/conditional-lane-detection/mmdet/utils/init.py)

    opened by Starboy-at-earth 16
  • export to onnx

    export to onnx

    Hi, thank you for your great work! I have run it on my own datasets (different view with culane), and it predicts very well. but I have problem in how to export .pth to onnx.

    I have tried : python tools/pytorch2onnx.py configs/condlanenet/culane/culane_small_test.py ./culane_small.pth --out ./condlanenet.onnx --shape 320 800

    and it did save a ONNX file(44.7MB) in my path. but when I open ONNX,I don't understand the output.

    image

    image

    can anyone here help me out? thanks!

    opened by lucky-xu-1994 8
  • results do not agree with the author

    results do not agree with the author

    I tested the model provided by the authors and the results were a bit different from what the authors described. Do you have the same experience? model: large. result: [{"name":"Accuracy","value":0.9636818107904509,"order":"desc"},{"name":"FP","value":0.021141864366163457,"order":"asc"},{"name":"FN","value":0.03633477114785525,"order":"asc"},{"name":"F1","value":0.9712022686934042,"order":"asc"}]

    opened by CongerW 4
  • Running inference

    Running inference

    Hi, great work !! I am trying to run inference on some of my own images, but the output is incorrect. Here are the steps I followed:-

    1. Loaded the small model trained on curvelanes dataset
    2. Resized the image to (800,320) [I want the output size to be (800,320) only].
    3. Got the predictions
    4. Supplied the predictions to the appropriate post-processor with downscale = 8.
    5. Supplied the output of the post-processor to adjust_results function with the following parameters :- crop_shape = (320,800) img_shape = (320,800) crop_offset = (0,0) ori_shape = (320,800)
    6. The output I am getting is this - https://drive.google.com/file/d/1BwTaZ1I5VVWAEoQKjx2q6-Wg8UFzv6fT/view?usp=sharing

    It would be great if you can tell me where I am going wrong.

    Thank You

    opened by Tanay0101 3
  • 'CurvelanesDataset is not in the dataset registry'

    'CurvelanesDataset is not in the dataset registry'

    Exception has occurred: KeyError 'CurvelanesDataset is not in the dataset registry' File "/home/rx/ADAS/Lane/conditional-lane-detection/mmdetection/mmdet/datasets/builder.py", line 59, in build_dataset dataset = build_from_cfg(cfg, DATASETS, default_args) File "/home/rx/ADAS/Lane/conditional-lane-detection/tools/condlanenet/curvelanes/test_curvelanes.py", line 218, in main dataset = build_dataset(cfg.data.test) File "/home/rx/ADAS/Lane/conditional-lane-detection/tools/condlanenet/curvelanes/test_curvelanes.py", line 248, in main() 报这个错,请帮忙看下什么原因?

    opened by ldol31627 2
  • RuntimeError: one of the variables needed for gradient computation has been modified by an inplace operation

    RuntimeError: one of the variables needed for gradient computation has been modified by an inplace operation

    Hi all,

    when training the model, I get the following error when Pytorch's anomaly detection is on

    RuntimeError: one of the variables needed for gradient computation has been modified by an inplace operation: [torch.cuda.FloatTensor [4, 64, 10, 25]], which is output 0 of ReluBackward0, is at version 1; expected version 0 instead. Hint: the backtrace further above shows the operation that failed to compute its gradient. The variable in question was changed in there or anywhere later. Good luck!
    

    The traceback is the following:

    [W python_anomaly_mode.cpp:104] Warning: Error detected in ReluBackward0. Traceback of forward call that caused the error:
      File "tools/train.py", line 159, in <module>
        main()
      File "tools/train.py", line 155, in main
        meta=meta)
      File "/home/<user>/Code/conditional-lane-detection/mmdet/apis/train.py", line 167, in train_detector
        runner.run(data_loaders, cfg.workflow, cfg.total_epochs)
      File "/home/<user>/.conda/envs/conditional-lane-detection/lib/python3.7/site-packages/mmcv/runner/runner.py", line 383, in run
        epoch_runner(data_loaders[i], **kwargs)
      File "/home/<user>/.conda/envs/conditional-lane-detection/lib/python3.7/site-packages/mmcv/runner/runner.py", line 282, in train
        self.model, data_batch, train_mode=True, **kwargs)
      File "/home/<user>/Code/conditional-lane-detection/mmdet/apis/train.py", line 74, in batch_processor
        losses = model(**data)
      File "/home/<user>/.conda/envs/conditional-lane-detection/lib/python3.7/site-packages/torch/nn/modules/module.py", line 1102, in _call_impl
        return forward_call(*input, **kwargs)
      File "/home/<user>/.conda/envs/conditional-lane-detection/lib/python3.7/site-packages/torch/nn/parallel/data_parallel.py", line 166, in forward
        return self.module(*inputs[0], **kwargs[0])
      File "/home/<user>/.conda/envs/conditional-lane-detection/lib/python3.7/site-packages/torch/nn/modules/module.py", line 1102, in _call_impl
        return forward_call(*input, **kwargs)
      File "/home/<user>/Code/conditional-lane-detection/mmdet/models/detectors/condlanenet.py", line 327, in forward
        return self.forward_train(img, img_metas, **kwargs)
      File "/home/<user>/Code/conditional-lane-detection/mmdet/models/detectors/condlanenet.py", line 344, in forward_train
        output, memory = self.neck(output)
      File "/home/<user>/.conda/envs/conditional-lane-detection/lib/python3.7/site-packages/torch/nn/modules/module.py", line 1102, in _call_impl
        return forward_call(*input, **kwargs)
      File "/home/<user>/Code/conditional-lane-detection/mmdet/core/fp16/decorators.py", line 49, in new_func
        return old_func(*args, **kwargs)
      File "/home/<user>/Code/conditional-lane-detection/mmdet/models/necks/trans_fpn.py", line 257, in forward
        trans_feat = self.trans_head(src[self.trans_idx])
      File "/home/<user>/.conda/envs/conditional-lane-detection/lib/python3.7/site-packages/torch/nn/modules/module.py", line 1102, in _call_impl
        return forward_call(*input, **kwargs)
      File "/home/<user>/Code/conditional-lane-detection/mmdet/models/necks/trans_fpn.py", line 153, in forward
        src = layer(src, pos.to(src.device))
      File "/home/<user>/.conda/envs/conditional-lane-detection/lib/python3.7/site-packages/torch/nn/modules/module.py", line 1102, in _call_impl
        return forward_call(*input, **kwargs)
      File "/home/<user>/Code/conditional-lane-detection/mmdet/models/necks/trans_fpn.py", line 105, in forward
        x = self.pre_conv(x)
      File "/home/<user>/.conda/envs/conditional-lane-detection/lib/python3.7/site-packages/torch/nn/modules/module.py", line 1102, in _call_impl
        return forward_call(*input, **kwargs)
      File "/home/<user>/.conda/envs/conditional-lane-detection/lib/python3.7/site-packages/mmcv/cnn/bricks/conv_module.py", line 181, in forward
        x = self.activate(x)
      File "/home/<user>/.conda/envs/conditional-lane-detection/lib/python3.7/site-packages/torch/nn/modules/module.py", line 1102, in _call_impl
        return forward_call(*input, **kwargs)
      File "/home/<user>/.conda/envs/conditional-lane-detection/lib/python3.7/site-packages/torch/nn/modules/activation.py", line 98, in forward
        return F.relu(input, inplace=self.inplace)
      File "/home/<user>/.conda/envs/conditional-lane-detection/lib/python3.7/site-packages/torch/nn/functional.py", line 1299, in relu
        result = torch.relu(input)
     (function _print_stack)
    

    Anybody knows how to fix this?

    opened by andy-96 1
  • TypeError: 'DataContainer' object is not subscriptable

    TypeError: 'DataContainer' object is not subscriptable

    请教大佬 ,用推荐的环境配置

    MMCV: 0.5.6
    MMDetection: 2.0.0+73c2043
    

    train的时候没问题,但是test的时候报如下error,检查了数据集是没问题的,还可能是哪里的问题呢

    (openmm) [wangjinsheng@HOST-10-198-32-69 tools]$ sh test_slurm.sh 
    [                                                  ] 0/2782, elapsed: 0s, ETA:[                                                  ] 0/2782, elapsed: 0s, ETA:Traceback (most recent call last):
    Traceback (most recent call last):
      File "test.py", line 149, in <module>
      File "test.py", line 149, in <module>
        main()
      File "test.py", line 127, in main
        args.show_score_thr)
      File "/mnt/lustre/wangjinsheng/project/lane-detection/conditional-lane-detection/mmdet/apis/test.py", line 28, in single_gpu_test
        main()
      File "test.py", line 127, in main
        args.show_score_thr)
      File "/mnt/lustre/wangjinsheng/project/lane-detection/conditional-lane-detection/mmdet/apis/test.py", line 28, in single_gpu_test
        img_tensor = data['img'][0]
    TypeError: 'DataContainer' object is not subscriptable
        img_tensor = data['img'][0]
    TypeError: 'DataContainer' object is not subscriptable
    
    opened by Wolfwjs 1
  • MMCV version

    MMCV version

    Hey! I tried to test on the tusimple large. I ran this code

    python tools/condlanenet/tusimple/test_tusimple.py configs/condlanenet/tusimple/tusimple_large_test.py D:/conditional-lane-detection-master/tusimple_large.pth

    and it gives error like this

    File "tools/condlanenet/tusimple/test_tusimple.py", line 16, in from mmdet.datasets import build_dataloader, build_dataset File "d:\conditional-lane-detection-master\mmdetection\mmdet_init_.py", line 25, in f'MMCV=={mmcv.version} is used but incompatible. '
    AssertionError: MMCV==0.5.6 is used but incompatible. Please install mmcv>=1.3.2, <=1.4.0.

    What should I do?

    opened by parthdave1210 1
  • The model and loaded state dict do not match exactly.

    The model and loaded state dict do not match exactly.

    Hi, I tried to use the given pretrained model for testing, however, I got the following results:

    python tools/condlanenet/culane/test_culane.py configs/condlanenet/culane/culane_small_test.py history/official/culane_small.pth The model and loaded state dict do not match exactly

    unexpected key in source state_dict: bbox_head.reg_branch.0.conv.weight, bbox_head.reg_branch.0.bn.weight, bbox_head.reg_branch.0.bn.bias, bbox_head.reg_branch.0.bn.running_mean, bbox_head.reg_branch.0.bn.running_var, bbox_head.reg_branch.0.bn.num_batches_tracked, bbox_head.reg_branch.1.conv.weight, bbox_head.reg_branch.1.bn.weight, bbox_head.reg_branch.1.bn.bias, bbox_head.reg_branch.1.bn.running_mean, bbox_head.reg_branch.1.bn.running_var, bbox_head.reg_branch.1.bn.num_batches_tracked, bbox_head.reg_branch.2.conv.weight, bbox_head.reg_branch.2.bn.weight, bbox_head.reg_branch.2.bn.bias, bbox_head.reg_branch.2.bn.running_mean, bbox_head.reg_branch.2.bn.running_var, bbox_head.reg_branch.2.bn.num_batches_tracked

    completed: 0, elapsed: 0s% Is there something wrong with the pretrained models or the codes?

    opened by dleam 1
  • row_loss not going down

    row_loss not going down

    I am training culane_small with only two images to overfit. It's a crop-row image. But the row_loss is not going down. It's around 22. Is this normal? How can I tune to reduce the loss? Also after training, if I run culane_test, test output has no line drawn on them. Here is a ground truth image:

    0_d03caef42cc4688a2ee02217f26c46e 00000 jpg gt

    opened by sudokhan112 0
  • How can l solve this problem?

    How can l solve this problem?

    when l try to train my dataset(Curvelanenet), l use the instructions as follows: python tools/train.py configs/condlanenet/curvelanes/curvelanes_large_train.py then the problems occurs: RuntimeError: one of the variables needed for gradient computation has been modified by an inplace operation: [torch.cuda.FloatTensor [1, 256, 10, 25]], which is output 0 of ReluBackward0, is at version 1; expected version 0 instead. Hint: enable anomaly detection to find the operation that failed to compute its gradient, with torch.autograd.set_detect_anomaly(True). How can l solve this problem?

    opened by lovelydjj 1
  • Question about img_norm_cfg

    Question about img_norm_cfg

    Hi, the mean and std in img_norm_cfg is "mean=[75.3, 76.6, 77.6], std=[50.5, 53.8, 54.3], to_rgb=False". Why didn't you use the origin value mean=[123.675, 116.28, 103.53], std=[58.395, 57.12, 57.375], to_rgb=True? Thank you !

    opened by notabigfish 0
  • tools.condlanenet.common Issue

    tools.condlanenet.common Issue

    Getting Error:

    /content/conditional-lane-detection No CUDA runtime is found, using CUDA_HOME='/usr/local/cuda' Traceback (most recent call last): File "tools/condlanenet/tusimple/test_tusimple.py", line 19, in from tools.condlanenet.common import tusimple_convert_formal, COLORS ModuleNotFoundError: No module named 'tools'

    Any solution??

    opened by imadalishah 0
  • aboult  albumentation

    aboult albumentation

    train_pipeline = [ dict(type='albumentation', pipelines=train_al_pipeline),

    I can't seem to find this albumentation in MMDET,Could you please provide this one?

    opened by Ehangiun 0
Owner
Alibaba Cloud
More Than Just Cloud
Alibaba Cloud
LaneDetectionAndLaneKeeping - Lane Detection And Lane Keeping

LaneDetectionAndLaneKeeping This project is part of my bachelor's thesis. The go

null 5 Jun 27, 2022
Lane assist for ETS2, built with the ultra-fast-lane-detection model.

Euro-Truck-Simulator-2-Lane-Assist Lane assist for ETS2, built with the ultra-fast-lane-detection model. This project was made possible by the amazing

null 36 Jan 5, 2023
Lane follower: Lane-detector (OpenCV) + Object-detector (YOLO5) + CAN-bus

Lane Follower This code is for the lane follower, including perception and control, as shown below. Environment Hardware Industrial Camera Intel-NUC(1

Siqi Fan 3 Jul 7, 2022
Find-Lane-Line - Use openCV library and Python to detect the road-lane-line

Find-Lane-Line This project is to use openCV library and Python to detect the road-lane-line. Data Pipeline Step one : Color Selection Step two : Cann

Kenny Cheng 3 Aug 17, 2022
An attempt at the implementation of Glom, Geoffrey Hinton's new idea that integrates neural fields, predictive coding, top-down-bottom-up, and attention (consensus between columns)

GLOM - Pytorch (wip) An attempt at the implementation of Glom, Geoffrey Hinton's new idea that integrates neural fields, predictive coding,

Phil Wang 173 Dec 14, 2022
Implementation of Bidirectional Recurrent Independent Mechanisms (Learning to Combine Top-Down and Bottom-Up Signals in Recurrent Neural Networks with Attention over Modules)

BRIMs Bidirectional Recurrent Independent Mechanisms Implementation of the paper Learning to Combine Top-Down and Bottom-Up Signals in Recurrent Neura

Sarthak Mittal 26 May 26, 2022
3D Multi-Person Pose Estimation by Integrating Top-Down and Bottom-Up Networks

3D Multi-Person Pose Estimation by Integrating Top-Down and Bottom-Up Networks Introduction This repository contains the code and models for the follo

null 124 Jan 6, 2023
Official PyTorch implementation of "Camera Distance-aware Top-down Approach for 3D Multi-person Pose Estimation from a Single RGB Image", ICCV 2019

PoseNet of "Camera Distance-aware Top-down Approach for 3D Multi-person Pose Estimation from a Single RGB Image" Introduction This repo is official Py

Gyeongsik Moon 677 Dec 25, 2022
TransFGU: A Top-down Approach to Fine-Grained Unsupervised Semantic Segmentation

TransFGU: A Top-down Approach to Fine-Grained Unsupervised Semantic Segmentation Zhaoyun Yin, Pichao Wang, Fan Wang, Xianzhe Xu, Hanling Zhang, Hao Li

DamoCV 25 Dec 16, 2022
A Data Annotation Tool for Semantic Segmentation, Object Detection and Lane Line Detection.(In Development Stage)

Data-Annotation-Tool How to Run this Tool? To run this software, follow the steps: git clone https://github.com/Autonomous-Car-Project/Data-Annotation

TiVRA AI 13 Aug 18, 2022
Example scripts for the detection of lanes using the ultra fast lane detection model in ONNX.

Example scripts for the detection of lanes using the ultra fast lane detection model in ONNX.

Ibai Gorordo 35 Sep 7, 2022
Example scripts for the detection of lanes using the ultra fast lane detection model in Tensorflow Lite.

TFlite Ultra Fast Lane Detection Inference Example scripts for the detection of lanes using the ultra fast lane detection model in Tensorflow Lite. So

Ibai Gorordo 12 Aug 27, 2022
A lane detection integrated Real-time Instance Segmentation based on YOLACT (You Only Look At CoefficienTs)

Real-time Instance Segmentation and Lane Detection This is a lane detection integrated Real-time Instance Segmentation based on YOLACT (You Only Look

Jin 4 Dec 30, 2022
Implementation of our paper 'RESA: Recurrent Feature-Shift Aggregator for Lane Detection' in AAAI2021.

RESA PyTorch implementation of the paper "RESA: Recurrent Feature-Shift Aggregator for Lane Detection". Our paper has been accepted by AAAI2021. Intro

null 137 Jan 2, 2023
LaneAF: Robust Multi-Lane Detection with Affinity Fields

LaneAF: Robust Multi-Lane Detection with Affinity Fields This repository contains Pytorch code for training and testing LaneAF lane detection models i

null 155 Dec 17, 2022
VIL-100: A New Dataset and A Baseline Model for Video Instance Lane Detection (ICCV 2021)

Preparation Please see dataset/README.md to get more details about our datasets-VIL100 Please see INSTALL.md to install environment and evaluation too

null 82 Dec 15, 2022
Code for the IJCAI 2021 paper "Structure Guided Lane Detection"

SGNet Project for the IJCAI 2021 paper "Structure Guided Lane Detection" Abstract Recently, lane detection has made great progress with the rapid deve

Jinming Su 27 Dec 8, 2022
Python scripts for performing lane detection using the LSTR model in ONNX

ONNX LSTR Lane Detection Python scripts for performing lane detection using the Lane Shape Prediction with Transformers (LSTR) model in ONNX. Requirem

Ibai Gorordo 29 Aug 30, 2022
PyTorch implementation of 'Gen-LaneNet: a generalized and scalable approach for 3D lane detection'

(pytorch) Gen-LaneNet: a generalized and scalable approach for 3D lane detection Introduction This is a pytorch implementation of Gen-LaneNet, which p

Yuliang Guo 233 Jan 6, 2023