Official code for "Eigenlanes: Data-Driven Lane Descriptors for Structurally Diverse Lanes", CVPR2022

Overview

Python 3.6

[CVPR 2022] Eigenlanes: Data-Driven Lane Descriptors for Structurally Diverse Lanes

Dongkwon Jin, Wonhui Park, Seong-Gyun Jeong, Heeyeon Kwon, and Chang-Su Kim

overview

Official implementation for "Eigenlanes: Data-Driven Lane Descriptors for Structurally Diverse Lanes" [paper] [supp] [video].

We construct a new dataset called "SDLane". SDLane is available at here. Now, only test set is provided due to privacy issues. All dataset will be provided soon.

Video

Video

Related work

We wil also present another paper, "Eigencontours: Novel Contour Descriptors Based on Low-Rank Approximation", accepted to CVPR 2022 (oral) [github] [video].

Requirements

  • PyTorch >= 1.6
  • CUDA >= 10.0
  • CuDNN >= 7.6.5
  • python >= 3.6

Installation

  1. Download repository. We call this directory as ROOT:
$ git clone https://github.com/dongkwonjin/Eigenlanes.git
  1. Download pre-trained model parameters and preprocessed data in ROOT:
$ cd ROOT
$ unzip pretrained.zip
$ unzip preprocessed.zip
  1. Create conda environment:
$ conda create -n eigenlanes python=3.7 anaconda
$ conda activate eigenlanes
  1. Install dependencies:
$ conda install pytorch torchvision torchaudio cudatoolkit=10.2 -c pytorch
$ pip install -r requirements.txt

Directory structure

.                           # ROOT
├── Preprocessing           # directory for data preprocessing
│   ├── culane              # dataset name (culane, tusimple)
|   |   ├── P00             # preprocessing step 1
|   |   |   ├── code
|   |   ├── P01             # preprocessing step 2
|   |   |   ├── code
|   │   └── ...
│   └── ...                 # etc.
├── Modeling                # directory for modeling
│   ├── culane              # dataset name (culane, tusimple)
|   |   ├── code
│   ├── tusimple           
|   |   ├── code
│   └── ...                 # etc.
├── pretrained              # pretrained model parameters 
│   ├── culane              
│   ├── tusimple            
│   └── ...                 # etc.
├── preprocessed            # preprocessed data
│   ├── culane              # dataset name (culane, tusimple)
|   |   ├── P03             
|   |   |   ├── output
|   |   ├── P04             
|   |   |   ├── output
|   │   └── ...
│   └── ...
.

Evaluation (for CULane)

To test on CULane, you need to install official CULane evaluation tools. The official metric implementation is available here. Please downloads the tools into ROOT/Modeling/culane/code/evaluation/culane/. The tools require OpenCV C++. Please follow here to install OpenCV C++. Then, you compile the evaluation tools. We recommend to see an installation guideline

$ cd ROOT/Modeling/culane/code/evaluation/culane/
$ make

Train

  1. Set the dataset you want to train (DATASET_NAME)
  2. Parse your dataset path into the -dataset_dir argument.
  3. Edit config.py if you want to control the training process in detail
$ cd ROOT/Modeling/DATASET_NAME/code/
$ python main.py --run_mode train --pre_dir ROOT/preprocessed/DATASET_NAME/ --dataset_dir /where/is/your/dataset/path/ 

Test

  1. Set the dataset you want to test (DATASET_NAME)
  2. Parse your dataset path into the -dataset_dir argument.
  3. If you want to get the performances of our work,
$ cd ROOT/Modeling/DATASET_NAME/code/
$ python main.py --run_mode test_paper --pre_dir ROOT/preprocessed/DATASET_NAME/ --paper_weight_dir ROOT/pretrained/DATASET_NAME/ --dataset_dir /where/is/your/dataset/path/
  1. If you want to evaluate a model you trained,
$ cd ROOT/Modeling/DATASET_NAME/code/
$ python main.py --run_mode test --pre_dir ROOT/preprocessed/DATASET_NAME/ --dataset_dir /where/is/your/dataset/path/

Preprocessing

example

Data preprocessing is divided into five steps, which are P00, P01, P02, P03, and P04. Below we describe each step in detail.

  1. In P00, the type of ground-truth lanes in a dataset is converted to pickle format.
  2. In P01, each lane in a training set is represented by 2D points sampled uniformly in the vertical direction.
  3. In P02, lane matrix is constructed and SVD is performed. Then, each lane is transformed to its coefficient vector.
  4. In P03, clustering is performed to obtain lane candidates.
  5. In P04, training labels are generated to train the SI module in the proposed SIIC-Net.

If you want to get the preproessed data, please run the preprocessing codes in order. Also, you can download the preprocessed data.

$ cd ROOT/Preprocessing/DATASET_NAME/PXX_each_preprocessing_step/code/
$ python main.py --dataset_dir /where/is/your/dataset/path/

Reference

@Inproceedings{
    Jin2022eigenlanes,
    title={Eigenlanes: Data-Driven Lane Descriptors for Structurally Diverse Lanes},
    author={Jin, Dongkwon and Park, Wonhui and Jeong, Seong-Gyun and Kwon, Heeyeon and Kim, Chang-Su},
    booktitle={CVPR},
    year={2022}
}
Comments
  • can't find the file

    can't find the file "lane_candidates_1000.pickle",but it exists

    First I'd like to thank you for your great work and your open source. I download the code today and want to test the performance on tusimple, here is my command: python main.py --run_mode test_paper --pre_dir ROOT/preprocessed/tusimple/ --paper_weight_dir ROOT/pretrained/tusimple/ --dataset_dir ROOT/dataset/ however,it has some problems like this: `Traceback (most recent call last): File "main.py", line 51, in main() File "main.py", line 35, in main dict_DB = prepare_visualization(cfg, dict_DB) File "/home/gdc/gong/lane-detection/EigenlanesCVPR2022/ROOT/Modeling/tusimple/code/libs/prepare.py", line 41, in prepare_visualization dict_DB['visualize'] = Visualize_cv(cfg=cfg) File "/home/gdc/gong/lane-detection/EigenlanesCVPR2022/ROOT/Modeling/tusimple/code/visualizes/visualize.py", line 24, in init self.candidates = load_pickle(self.cfg.dir['pre3'] + 'lane_candidates_' + str(self.cfg.n_clusters)) File "/home/gdc/gong/lane-detection/EigenlanesCVPR2022/ROOT/Modeling/tusimple/code/libs/utils.py", line 76, in load_pickle with open(file_path + '.pickle', 'rb') as f: FileNotFoundError: [Errno 2] No such file or directory: 'ROOT/preprocessed/tusimple/P03_clustering/output_train_set/pickle/lane_candidates_1000.pickle'

    ` I'm sure there is that file in that directory。And,If I want to test some pictures,what should I do?Put all the pictures in one folder will work? Any advice would be greatly appreciated

    opened by GongDianCheng 8
  • ABOUT THE SPEED

    ABOUT THE SPEED

    Thanks for your great work. I'm puzzled at the speed cause I just ran 14 fps on my pc.Here are the configurations of my pc.

    • CPU-------Intel(R) Core(TM) i7-7700HQ CPU @ 2.80GHz
    • GPU-------NVIDIA GTX 1050 4GB
    • Memory-- 24GB

    Could you please show me the configurations of your machine where you ran 101 fps?

    opened by ITBoy-China 4
  • curve result

    curve result

    This paper mentioned that the model proposed in this paper is more effective in curve scene testing. But from the results, on the culane dataset, the detection accuracy of curve category is not high, and it is lower than other models. How to explain this? Thank you very much.

    opened by miao02830 1
  • Error: image not found,but it does exist

    Error: image not found,but it does exist

    CUDA_VISIBLE_DEVICES=1 nohup python main.py --run_mode train --pre_dir /home/Disk-2T/chedao/Eigenlanes-main/preprocessed/tusimple/ --dataset_dir /home/Disk-2T/chedao/Eigenlanes-main/Tusimple > tisimpletrain1.log 2>&1 & [1] 28049 (chelane) caipeng@li-MS-7B96:/home/Disk-2T/caipeng_chedao/Eigenlanes-main/Modeling/tusimple/code$ tail -f tisimpletrain1.log train start /home/Disk-2Taipeng_chedao/Eigenlanes-main/Modeling/tusimple/code/networks/model.py:158: UserWarning: This overload of nonzero is deprecated: nonzero() Consider using one of the following signatures instead: nonzero(*, bool as_tuple) (Triggered internally at /pytorch/torch/csrc/utils/python_arg_parser.cpp:766.) area[i] = mask[i].nonzero().shape[0] /home/cp/anaconda3/envs/chelane/lib/python3.8/site-packages/torch/nn/_reduction.py:44: UserWarning: size_average and reduce args will be deprecated, please use reduction='mean' instead. warnings.warn(warning.format(ret)) /home/cp/anaconda3/envs/chelane/lib/python3.8/site-packages/torch/nn/_reduction.py:44: UserWarning: size_average and reduce args will be deprecated, please use reduction='none' instead. warnings.warn(warning.format(ret)) img iter 0 ==> 0313-1/50820/20 img iter 50 ==> 0313-1/43140/20 img iter 100 ==> 0313-1/6400/20 img iter 150 ==> 0313-2/42280/20 img iter 200 ==> 0313-1/5820/20 img iter 250 ==> 0313-2/8040/20

    Average Loss : 0.060205, 0.027879, 0.026334, 0.686977, 1.284911, 0.057571, 0.004878, 0.066173, 0.354894, label distribution [80.7 0. 0. 0. 0. 0. 0. 1.8 14.5 2.9 0. ]

    epoch 1

    train start /home/cp/anaconda3/envs/chelane/lib/python3.8/site-packages/torch/nn/functional.py:3118: UserWarning: Default upsampling behavior when mode=bilinear is changed to align_corners=False since 0.4.0. Please specify align_corners=True if the old behavior is desired. See the documentation of nn.Upsample for details. warnings.warn("Default upsampling behavior when mode={} is changed " /home/cp/anaconda3/envs/chelane/lib/python3.8/site-packages/torch/optim/lr_scheduler.py:143: UserWarning: The epoch parameter in scheduler.step() was not necessary and is being deprecated where possible. Please use scheduler.step() to step the scheduler. During the deprecation, if epoch is different from None, the closed form is used instead of the new chainable form, where available. Please open an issue if you are unable to replicate your use case: https://github.com/pytorch/pytorch/issues/new/choose. warnings.warn(EPOCH_DEPRECATION_WARNING, UserWarning) img iter 0 ==> 0313-1/16760/20 img iter 50 ==> 0313-1/13680/20 img iter 100 ==> 0313-2/25200/20 img iter 150 ==> 0313-2/1055/20 img iter 200 ==> 0313-2/615/20 img iter 250 ==> 0313-2/6360/20

    Average Loss : 0.050388, 0.021135, 0.007948, 0.073888, 0.403831, 0.031649, 0.004619, 0.039286, 0.174917, label distribution [79.8 0. 0. 0. 0. 0. 0. 2.5 14.9 2.8 0. ]

    Traceback (most recent call last): File "main.py", line 51, in main() File "main.py", line 46, in main main_train(cfg, dict_DB) File "main.py", line 23, in main_train train_process.run() File "/home/Disk-2T/chedao/Eigenlanes-main/Modeling/tusimple/code/trains/train.py", line 135, in run self.validation() File "/home/Disk-2T/chedao/Eigenlanes-main/Modeling/tusimple/code/trains/train.py", line 115, in validation metric = self.test_process.run(self.model, mode='val') File "/home/Disk-2T/chedao/Eigenlanes-main/Modeling/tusimple/code/tests/test.py", line 23, in run for i, self.batch in enumerate(self.testloader): # load batch data File "/home/cp/anaconda3/envs/chelane/lib/python3.8/site-packages/torch/utils/data/dataloader.py", line 363, in next data = self._next_data() File "/home/cp/anaconda3/envs/chelane/lib/python3.8/site-packages/torch/utils/data/dataloader.py", line 989, in _next_data return self._process_data(data) File "/home/cp/anaconda3/envs/chelane/lib/python3.8/site-packages/torch/utils/data/dataloader.py", line 1014, in _process_data data.reraise() File "/home/cp/anaconda3/envs/chelane/lib/python3.8/site-packages/torch/_utils.py", line 395, in reraise raise self.exc_type(msg) FileNotFoundError: Caught FileNotFoundError in DataLoader worker process 0. Original Traceback (most recent call last): File "/home/cp/anaconda3/envs/chelane/lib/python3.8/site-packages/torch/utils/data/_utils/worker.py", line 185, in _worker_loop data = fetcher.fetch(index) File "/home/cp/anaconda3/envs/chelane/lib/python3.8/site-packages/torch/utils/data/_utils/fetch.py", line 44, in fetch data = [self.dataset[idx] for idx in possibly_batched_index] File "/home/cp/anaconda3/envs/chelane/lib/python3.8/site-packages/torch/utils/data/_utils/fetch.py", line 44, in data = [self.dataset[idx] for idx in possibly_batched_index] File "/home/Disk-2T/chedao/Eigenlanes-main/Modeling/tusimple/code/datasets/dataset_tusimple.py", line 249, in getitem out.update(self.get_image(idx)) File "/home/Disk-2T/chedao/Eigenlanes-main/Modeling/tusimple/code/datasets/dataset_tusimple.py", line 200, in get_image img = Image.open(self.cfg.dir['dataset'] + '/test_set/clips/{}.jpg'.format(self.datalist[idx])).convert('RGB') File "/home/caipeng/anaconda3/envs/chelane/lib/python3.8/site-packages/PIL/Image.py", line 2766, in open fp = builtins.open(filename, "rb") FileNotFoundError: [Errno 2] No such file or directory: '/home/Disk-2T/chedao/Eigenlanes-main/Tusimple/test_set/clips/0530/1492626760788443246_0/20.jpg'

    opened by shuizaola 1
  • RuntimeError: Attempting to deserialize object on a CUDA device but torch.cuda.is_available() is False.

    RuntimeError: Attempting to deserialize object on a CUDA device but torch.cuda.is_available() is False.

    @dongkwonjin First I'd like to thank you for your great work and your open source. I download the code today and want to test the performance on CULane, here is my command

    python main.py --run_mode test_paper --pre_dir ../../../preprocessed/culane/ --paper_weight_dir ../../../pretrained/culane/ --dataset_dir /media/zdw/disc/dataset/CULane/extract/
    

    But I got the error

    Traceback (most recent call last):
      File "main.py", line 45, in <module>
        main()
      File "main.py", line 29, in main
        dict_DB = prepare_visualization(cfg, dict_DB)
      File "/media/zdw/disc/whu_selfdriving/drivable_area/eigen/Eigenlanes-main/Modeling/culane/code/libs/prepare.py", line 53, in prepare_visualization
        dict_DB['visualize'] = Visualize_cv(cfg=cfg)
      File "/media/zdw/disc/whu_selfdriving/drivable_area/eigen/Eigenlanes-main/Modeling/culane/code/visualizes/visualize.py", line 25, in __init__
        self.candidates = load_pickle(self.cfg.dir['pre3'] + 'lane_candidates_' + str(self.cfg.n_clusters))
      File "/media/zdw/disc/whu_selfdriving/drivable_area/eigen/Eigenlanes-main/Modeling/culane/code/libs/utils.py", line 77, in load_pickle
        data = pickle.load(f)
      File "/home/zdw/anaconda3/envs/eigen/lib/python3.7/site-packages/torch/storage.py", line 181, in _load_from_bytes
        return torch.load(io.BytesIO(b))
      File "/home/zdw/anaconda3/envs/eigen/lib/python3.7/site-packages/torch/serialization.py", line 713, in load
        return _legacy_load(opened_file, map_location, pickle_module, **pickle_load_args)
      File "/home/zdw/anaconda3/envs/eigen/lib/python3.7/site-packages/torch/serialization.py", line 930, in _legacy_load
        result = unpickler.load()
      File "/home/zdw/anaconda3/envs/eigen/lib/python3.7/site-packages/torch/serialization.py", line 876, in persistent_load
        wrap_storage=restore_location(obj, location),
      File "/home/zdw/anaconda3/envs/eigen/lib/python3.7/site-packages/torch/serialization.py", line 176, in default_restore_location
        result = fn(storage, location)
      File "/home/zdw/anaconda3/envs/eigen/lib/python3.7/site-packages/torch/serialization.py", line 152, in _cuda_deserialize
        device = validate_cuda_device(location)
      File "/home/zdw/anaconda3/envs/eigen/lib/python3.7/site-packages/torch/serialization.py", line 136, in validate_cuda_device
        raise RuntimeError('Attempting to deserialize object on a CUDA '
    RuntimeError: Attempting to deserialize object on a CUDA device but torch.cuda.is_available() is False. If you are running on a CPU-only machine, please use torch.load with map_location=torch.device('cpu') to map your storages to the CPU.
    
    

    It shows that torch.cuda.is_available() is False, but when i run this line in terminal, it outputs True:

    Python 3.7.13 (default, Mar 29 2022, 02:18:16) 
    [GCC 7.5.0] :: Anaconda, Inc. on linux
    Type "help", "copyright", "credits" or "license" for more information.
    >>> import torch
    >>> torch.cuda.is_available()
    True
    

    Could you help me with this? Thanks very much.

    opened by zdddw 1
Owner
Dongkwon Jin
BS: EE, Korea University Grad: EE, Korea University (Current)
Dongkwon Jin
Official code for CVPR2022 paper: Depth-Aware Generative Adversarial Network for Talking Head Video Generation

?? Depth-Aware Generative Adversarial Network for Talking Head Video Generation (CVPR 2022) ?? If DaGAN is helpful in your photos/projects, please hel

Fa-Ting Hong 503 Jan 4, 2023
This is an official implementation of the CVPR2022 paper "Blind2Unblind: Self-Supervised Image Denoising with Visible Blind Spots".

Blind2Unblind: Self-Supervised Image Denoising with Visible Blind Spots Blind2Unblind Citing Blind2Unblind @inproceedings{wang2022blind2unblind, tit

demonsjin 58 Dec 6, 2022
The official codes of our CVPR2022 paper: A Differentiable Two-stage Alignment Scheme for Burst Image Reconstruction with Large Shift

TwoStageAlign The official codes of our CVPR2022 paper: A Differentiable Two-stage Alignment Scheme for Burst Image Reconstruction with Large Shift Pa

Shi Guo 32 Dec 15, 2022
Code for the CVPR2022 paper "Frequency-driven Imperceptible Adversarial Attack on Semantic Similarity"

Introduction This is an official release of the paper "Frequency-driven Imperceptible Adversarial Attack on Semantic Similarity" (arxiv link). Abstrac

Leo 21 Nov 23, 2022
Source code for CVPR2022 paper "Abandoning the Bayer-Filter to See in the Dark"

Abandoning the Bayer-Filter to See in the Dark (CVPR 2022) Paper: https://arxiv.org/abs/2203.04042 (Arxiv version) This code includes the training and

null 74 Dec 15, 2022
PSTR: End-to-End One-Step Person Search With Transformers (CVPR2022)

PSTR (CVPR2022) This code is an official implementation of "PSTR: End-to-End One-Step Person Search With Transformers (CVPR2022)". End-to-end one-step

Jiale Cao 28 Dec 13, 2022
CVPR2022 paper "Dense Learning based Semi-Supervised Object Detection"

[CVPR2022] DSL: Dense Learning based Semi-Supervised Object Detection DSL is the first work on Anchor-Free detector for Semi-Supervised Object Detecti

Bhchen 69 Dec 8, 2022
[CVPR2022] Bridge-Prompt: Towards Ordinal Action Understanding in Instructional Videos

Bridge-Prompt: Towards Ordinal Action Understanding in Instructional Videos Created by Muheng Li, Lei Chen, Yueqi Duan, Zhilan Hu, Jianjiang Feng, Jie

null 58 Dec 23, 2022
Group R-CNN for Point-based Weakly Semi-supervised Object Detection (CVPR2022)

Group R-CNN for Point-based Weakly Semi-supervised Object Detection (CVPR2022) By Shilong Zhang*, Zhuoran Yu*, Liyang Liu*, Xinjiang Wang, Aojun Zhou,

Shilong Zhang 129 Dec 24, 2022
Multi-View Consistent Generative Adversarial Networks for 3D-aware Image Synthesis (CVPR2022)

Multi-View Consistent Generative Adversarial Networks for 3D-aware Image Synthesis Multi-View Consistent Generative Adversarial Networks for 3D-aware

Xuanmeng Zhang 78 Dec 10, 2022
TCTrack: Temporal Contexts for Aerial Tracking (CVPR2022)

TCTrack: Temporal Contexts for Aerial Tracking (CVPR2022) Ziang Cao and Ziyuan Huang and Liang Pan and Shiwei Zhang and Ziwei Liu and Changhong Fu In

Intelligent Vision for Robotics in Complex Environment 100 Dec 19, 2022
Incremental Transformer Structure Enhanced Image Inpainting with Masking Positional Encoding (CVPR2022)

Incremental Transformer Structure Enhanced Image Inpainting with Masking Positional Encoding by Qiaole Dong*, Chenjie Cao*, Yanwei Fu Paper and Supple

Qiaole Dong 190 Dec 27, 2022
FaceVerse: a Fine-grained and Detail-controllable 3D Face Morphable Model from a Hybrid Dataset (CVPR2022)

FaceVerse FaceVerse: a Fine-grained and Detail-controllable 3D Face Morphable Model from a Hybrid Dataset Lizhen Wang, Zhiyuan Chen, Tao Yu, Chenguang

Lizhen Wang 219 Dec 28, 2022
Towards Implicit Text-Guided 3D Shape Generation (CVPR2022)

Towards Implicit Text-Guided 3D Shape Generation Towards Implicit Text-Guided 3D Shape Generation (CVPR2022) Code for the paper [Towards Implicit Text

null 55 Dec 16, 2022
Video Frame Interpolation with Transformer (CVPR2022)

VFIformer Official PyTorch implementation of our CVPR2022 paper Video Frame Interpolation with Transformer Dependencies python >= 3.8 pytorch >= 1.8.0

DV Lab 63 Dec 16, 2022
[CVPR2022] Representation Compensation Networks for Continual Semantic Segmentation

RCIL [CVPR2022] Representation Compensation Networks for Continual Semantic Segmentation Chang-Bin Zhang1, Jia-Wen Xiao1, Xialei Liu1, Ying-Cong Chen2

Chang-Bin Zhang 71 Dec 28, 2022
A Text Attention Network for Spatial Deformation Robust Scene Text Image Super-resolution (CVPR2022)

A Text Attention Network for Spatial Deformation Robust Scene Text Image Super-resolution (CVPR2022) https://arxiv.org/abs/2203.09388 Jianqi Ma, Zheto

MA Jianqi, shiki 104 Jan 5, 2023
Unsupervised Domain Adaptation for Nighttime Aerial Tracking (CVPR2022)

Unsupervised Domain Adaptation for Nighttime Aerial Tracking (CVPR2022) Junjie Ye, Changhong Fu, Guangze Zheng, Danda Pani Paudel, and Guang Chen. Uns

Intelligent Vision for Robotics in Complex Environment 91 Dec 30, 2022
CVPR2022 (Oral) - Rethinking Semantic Segmentation: A Prototype View

Rethinking Semantic Segmentation: A Prototype View Rethinking Semantic Segmentation: A Prototype View, Tianfei Zhou, Wenguan Wang, Ender Konukoglu and

Tianfei Zhou 239 Dec 26, 2022