Source code of all the projects of Udacity Self-Driving Car Engineer Nanodegree.

Overview

self-driving-car

In this repository I will share the source code of all the projects of Udacity Self-Driving Car Engineer Nanodegree.

Hope this might be useful to someone! :-)

Overview

Projects

Overview
P1: Basic Lane Finding
(code)

Overview
P2: Traffic Signs
(code)

Overview
P3: Behavioral Cloning
(code)

Overview
P4: Adv. Lane Finding
(code)

Overview
P5: Vehicle Detection
(code)

Overview
P6: Ext. Kalman Filter
(code)

Overview
P7: Unsc. Kalman Filter
(code)

Overview
P8: Kidnapped Vehicle
(code)

Overview
P9: PID Controller
(code)

Overview
P10: MPC Controller
(code)

Overview
P11: Path Planning
(code)

Overview
P12: Road Segmentation
(code)

Capstone


Table of Contents

P1 - Detecting Lane Lines (basic)

  • Summary: Detected highway lane lines on a video stream. Used OpencV image analysis techniques to identify lines, including Hough Transforms and Canny edge detection.
  • Keywords: Computer Vision

P2 - Traffic Sign Classification

  • Summary: Built and trained a deep neural network to classify traffic signs, using TensorFlow. Experimented with different network architectures. Performed image pre-processing and validation to guard against overfitting.
  • Keywords: Deep Learning, TensorFlow, Computer Vision

P3 - Behavioral Cloning

  • Summary: Built and trained a convolutional neural network for end-to-end driving in a simulator, using TensorFlow and Keras. Used optimization techniques such as regularization and dropout to generalize the network for driving on multiple tracks.
  • Keywords: Deep Learning, Keras, Convolutional Neural Networks

P4 - Advanced Lane Finding

  • Summary: Built an advanced lane-finding algorithm using distortion correction, image rectification, color transforms, and gradient thresholding. Identified lane curvature and vehicle displacement. Overcame environmental challenges such as shadows and pavement changes.
  • Keywords: Computer Vision, OpenCV

P5 - Vehicle Detection and Tracking

  • Summary: Created a vehicle detection and tracking pipeline with OpenCV, histogram of oriented gradients (HOG), and support vector machines (SVM). Implemented the same pipeline using a deep network to perform detection. Optimized and evaluated the model on video data from a automotive camera taken during highway driving.
  • Keywords: Computer Vision, Deep Learning, OpenCV

P6 - Extended Kalman Filter

  • Summary: Implement the extended Kalman filter in C++. Simulated lidar and radar measurements are used to detect a bicycle that travels around your vehicle. Kalman filter, lidar measurements and radar measurements are used to track the bicycle's position and velocity.
  • Keywords: C++, Kalman Filter

P7 - Unscented Kalman Filter

  • Summary: Utilize an Unscented Kalman Filter to estimate the state of a moving object of interest with noisy lidar and radar measurements. Kalman filter, lidar measurements and radar measurements are used to track the bicycle's position and velocity.
  • Keywords: C++, Kalman Filter

P8 - Kidnapped Vehicle

  • Summary: Your robot has been kidnapped and transported to a new location! Luckily it has a map of this location, a (noisy) GPS estimate of its initial location, and lots of (noisy) sensor and control data. In this project you will implement a 2 dimensional particle filter in C++. Your particle filter will be given a map and some initial localization information (analogous to what a GPS would provide). At each time step your filter will also get observation and control data.
  • Keywords: C++, Particle Filter

P9 - PID Control

  • Summary: Implement a PID controller for keeping the car on track by appropriately adjusting the steering angle.
  • Keywords: C++, PID Controller

P10 - MPC Control

  • Summary: Implement an MPC controller for keeping the car on track by appropriately adjusting the steering angle. Differently from previously implemented PID controller, MPC controller has the ability to anticipate future events and can take control actions accordingly. Indeed, future time steps are taking into account while optimizing current time slot.
  • Keywords: C++, MPC Controller

P11 - Path Planning

  • Summary: The goal in this project is to build a path planner that is able to create smooth, safe trajectories for the car to follow. The highway track has other vehicles, all going different speeds, but approximately obeying the 50 MPH speed limit. The car transmits its location, along with its sensor fusion data, which estimates the location of all the vehicles on the same side of the road.
  • Keywords: C++, Path Planning

P12 - Road Segmentation

  • Summary: Implement the road segmentation using a fully-convolutional network.
  • Keywords: Python, TensorFlow, Semantic Segmentation

Comments
  • How to run project 3 with my own images and control

    How to run project 3 with my own images and control

    I would like to run Project 3 - Behavioral Cloning but with my own images and control. How can I do it? It was not very clear which is the training steep, and which is the inference step. Also, what is the best way to replace the data with my own images and control? My control data also have throttle data, but I can start with just steering.

    opened by ArtlyStyles 3
  • Question 1: Data augmentation

    Question 1: Data augmentation

    Hello sir, Here shouldn't be X_train instead of X_train_norm inside train_test_split()? X_train_norm, X_val_norm, y_train, y_val = train_test_split(X_train_norm, y_train, test_size=VAL_RATIO, random_state=0) Anyway, here img_rgb = X_train[0], you used X_train[0] which will always give me the first picture of the training dataset! will never be random. And here: plt.title('Example of RGB image (class = {})'.format(y_train[0])), you used the new training labels (after splitation).

    Not sure if you got my point, but it is not correct, I am new to python so I couldn't figure out how to fix the issue here, so any help?

    opened by AboHadi313 3
  • Where is the VGG pretrained model in Project 12

    Where is the VGG pretrained model in Project 12

    I got this error when I try to run the project 12

    Traceback (most recent call last): File "main_27.py", line 284, in run() File "main_27.py", line 257, in run image_input, keep_prob, vgg_layer3_out, vgg_layer4_out, vgg_layer7_out = load_vgg(sess, vgg_path) File "main_27.py", line 132, in load_vgg tf.saved_model.loader.load(sess, ['vgg16'], vgg_path) File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/saved_model/loader_impl.py", line 200, in load saved_model = _parse_saved_model(export_dir) File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/saved_model/loader_impl.py", line 78, in _parse_saved_model constants.SAVED_MODEL_FILENAME_PB)) IOError: SavedModel file does not exist at: /home/jack/code/self-driving-car/project_12_road_segmentation/data/vgg/{saved_model.pbtxt|saved_model.pb} `

    opened by yangruixuan 2
  • Advance Lane Detection - TypeError: 'NoneType' object is not subscriptable

    Advance Lane Detection - TypeError: 'NoneType' object is not subscriptable

    def draw_back_onto_the_road(img_undistorted, Minv, line_lt, line_rt, keep_state):
        height, width, _ = img_undistorted.shape
    
        left_fit = line_lt.average_fit if keep_state else line_lt.last_fit_pixel
        right_fit = line_rt.average_fit if keep_state else line_rt.last_fit_pixel
        left_fit.sort()
        right_fit.sort()
        # Generate x and y values for plotting
        ploty = np.linspace(0, height - 1, height)
        left_fitx = left_fit[0] * ploty ** 2 + left_fit[1] * ploty + left_fit[2]
        right_fitx = right_fit[0] * ploty ** 2 + right_fit[1] * ploty + right_fit[2]
    

    Error

    left_fitx = left_fit_pixel[0] * ploty ** 2 + left_fit_pixel[1] * ploty + left_fit_pixel[2]
    TypeError: 'NoneType' object is not subscriptable
    

    Someone please help me with this

    opened by vighc 1
  • Lane / road marking / vanishing point dataset

    Lane / road marking / vanishing point dataset

    VPGNet dataset is available. It contains around 21K images with human-annotated lane, road markings, and vanishing point labels. For more details, please check our project page.

    opened by SeokjuLee 1
  • project_5_vehicle_detection -- Got TypeError: 'module' object is not callable when run main_ssd.py

    project_5_vehicle_detection -- Got TypeError: 'module' object is not callable when run main_ssd.py

    When I run main_ssd.py I got this:

    Traceback (most recent call last):
      File "/home/binli/self-driving-car/project_5_vehicle_detection/main_ssd.py", line 10, in <module>
        ssd_model, bbox_helper, color_palette = get_SSD_model()
      File "/home/binli/self-driving-car/project_5_vehicle_detection/SSD.py", line 805, in get_SSD_model
        model_ssd = SSD300(input_shape=(300, 300, 3), num_classes=NUM_CLASSES, pretrained=True)
      File "/home/binli/self-driving-car/project_5_vehicle_detection/SSD.py", line 679, in SSD300
        mode='concat', concat_axis=1, name='mbox_loc')
    TypeError: 'module' object is not callable
    

    I tried a lot but I cannot fix this? Any suggestions about this? @ndrplz thanks a lot

    opened by binlidaily 1
  • project 4

    project 4

    i m unable to access video. error: from moviepy.editor import VideoFileClip ModuleNotFoundError: No module named 'moviepy'

    i have installed moviepy package but its still not working

    opened by ghost 1
  • project_4 : unsupported pickle protocol: %d

    project_4 : unsupported pickle protocol: %d" % proto

    Hello,

    I'm stuck in this issue when I run project 4:

    $ python ./main.py Loading cached camera calibration... Traceback (most recent call last): File "./main.py", line 135, in ret, mtx, dist, rvecs, tvecs = calibrate_camera(calib_images_dir='camera_cal') File "/home/nxnam/DigitalRace/hanson/self-driving-car/project_4_advanced_lane_finding/calibration_utils.py", line 20, in wrapper calibration = pickle.load(dump_file) File "/usr/lib/python2.7/pickle.py", line 1384, in load return Unpickler(file).load() File "/usr/lib/python2.7/pickle.py", line 864, in load dispatchkey File "/usr/lib/python2.7/pickle.py", line 892, in load_proto raise ValueError, "unsupported pickle protocol: %d" % proto ValueError: unsupported pickle protocol: 3

    How can I fix this?

    Thank you.

    opened by nxnam714 1
  • project_4 VideoClip actually generates RGB image

    project_4 VideoClip actually generates RGB image

    In project 4, VideoClip generates RGB image instead of BGR, which would take serious effect on extracting yellow color for thresh_frame_in_HSV() in binarization_utils.py.

    opened by juncaofish 1
  • Following datas are needed.

    Following datas are needed.

    You have used train.p and test.p. But I did not find your repository. Please, give these datasets.

    # Load pickled data
    train, test = load_traffic_sign_data('../traffic_signs_data/train.p', '../traffic_signs_data/test.p')
    
    opened by rezwanh001 1
  • ENet-Label-Torch is available now (a light-weight and effective lane detection model)

    ENet-Label-Torch is available now (a light-weight and effective lane detection model)

    Our ENet-Label-Torch has been released. More details can be found in my repo.

    Key features:

    (1) ENet-label is a light-weight lane detection model based on ENet and adopts self attention distillation (more details can be found in our paper which will be published soon).

    (2) It has 20 × fewer parameters and runs 10 × faster compared to the state-of-the-art SCNN, and achieves 72.0 (F1-measure) on CULane testing set (better than SCNN which achieves 71.6).

    (Do not hesitate to try our model!!!)

    Performance on CULane testing set (F1-measure):

    |Category|SCNN-Torch|SCNN-Tensorflow|ENet-Label-Torch| |:---:|:---:|:---:|:---:| |Normal|90.6|90.2|90.7| |Crowded|69.7|71.9|70.8| |Night|66.1|64.6|65.9| |No line|43.4|45.8|44.7| |Shadow|66.9|73.8|70.6| |Arrow|84.1|83.8|85.8| |Dazzle light|58.5|59.5|64.4| |Curve|64.4|63.4|65.4| |Crossroad|1990|4137|2729| |Total|71.6|71.3|72.0| |Runtime(ms)|133.5|--|13.4| |Parameter(M)|20.72|--|0.98|

    opened by cardwing 0
  • SystemError: Unknown opcode

    SystemError: Unknown opcode

    Try to run project 3 on ubuntu18: python3 drive.py got error: SystemError: Unknown opcode

    seems it is python version mismatch, ubuntu 18 is python3.6, what version of python do you use to train the model?

    Thanks

    opened by gms2009 0
  • How to make sure the loop time is 20ms in project_11?

    How to make sure the loop time is 20ms in project_11?

    Hello, I cannot find the exact detail to make sure we have a fix-time loop when computing future road points in Project 11 (Path Planning). Could you please tell me where the detail is. Thank you!

    opened by bryanibit 0
  • Advanced Lane Finding Lane Offset

    Advanced Lane Finding Lane Offset

    I have worked for hours and hours with your code. It is wonderful! Thanks for sharing it. However, one thing that makes it hard to implement in an actual bot is the fact that the road curvature and offset values are always positive. Could you please help me make them negative if the road curves, for example, left, and positive if it curves right? Same for the offset within the lane. My self-driving motercycle needs to know which way it is drifting. Thanks so much!

    opened by cdiener1 1
Owner
Andrea Palazzi
Senior Deep Learning Engineer @ Nomitri - Computer Vision PhD
Andrea Palazzi
Simulation of Self Driving Car

In this repository, the code to use Udacity's self driving car simulator as a testbed for training an autonomous car are provided.

Shyam Das Shrestha 1 Nov 21, 2021
Self-driving car env with PPO algorithm from stable baseline3

Self-driving car with RL stable baseline3 Most of the project develop from https://github.com/GerardMaggiolino/Gym-Medium-Post Please check it out! Th

Sornsiri.P 7 Dec 22, 2022
This script scrapes and stores the availability of timeslots for Car Driving Test at all RTA Serivce NSW centres in the state.

This script scrapes and stores the availability of timeslots for Car Driving Test at all RTA Serivce NSW centres in the state. Dependencies Account wi

Balamurugan Soundararaj 21 Dec 14, 2022
CRISCE: Automatically Generating Critical Driving Scenarios From Car Accident Sketches

CRISCE: Automatically Generating Critical Driving Scenarios From Car Accident Sketches This document describes how to install and use CRISCE (CRItical

Chair of Software Engineering II, Uni Passau 2 Feb 9, 2022
Udacity Suse Cloud Native Foundations Scholarship Course Walkthrough

SUSE Cloud Native Foundations Scholarship Udacity is collaborating with SUSE, a global leader in true open source solutions, to empower developers and

Shivansh Srivastava 34 Oct 18, 2022
Udacity's CS101: Intro to Computer Science - Building a Search Engine

Udacity's CS101: Intro to Computer Science - Building a Search Engine All soluti

Phillip 0 Feb 26, 2022
Roadmap to becoming a machine learning engineer in 2020

Roadmap to becoming a machine learning engineer in 2020, inspired by web-developer-roadmap.

Chris Hoyean Song 1.7k Dec 29, 2022
Reverse engineer your pytorch vision models, in style

?? Rover Reverse engineer your CNNs, in style Rover will help you break down your CNN and visualize the features from within the model. No need to wri

Mayukh Deb 32 Sep 24, 2022
Self-Supervised Pillar Motion Learning for Autonomous Driving (CVPR 2021)

Self-Supervised Pillar Motion Learning for Autonomous Driving Chenxu Luo, Xiaodong Yang, Alan Yuille Self-Supervised Pillar Motion Learning for Autono

QCraft 101 Dec 5, 2022
Graph Self-Attention Network for Learning Spatial-Temporal Interaction Representation in Autonomous Driving

GSAN Introduction Code for paper GSAN: Graph Self-Attention Network for Learning Spatial-Temporal Interaction Representation in Autonomous Driving, wh

YE Luyao 6 Oct 27, 2022
Reinforcement learning for self-driving in a 3D simulation

SelfDrive_AI Reinforcement learning for self-driving in a 3D simulation (Created using UNITY-3D) 1. Requirements for the SelfDrive_AI Gym You need Pyt

Surajit Saikia 17 Dec 14, 2021
Action Recognition for Self-Driving Cars

Action Recognition for Self-Driving Cars This repo contains the codes for the 2021 Fall semester project "Action Recognition for Self-Driving Cars" at

VITA lab at EPFL 3 Apr 7, 2022
(CVPR 2022) A minimalistic mapless end-to-end stack for joint perception, prediction, planning and control for self driving.

LAV Learning from All Vehicles Dian Chen, Philipp Krähenbühl CVPR 2022 (also arXiV 2203.11934) This repo contains code for paper Learning from all veh

Dian Chen 300 Dec 15, 2022
Code for Towards Streaming Perception (ECCV 2020) :car:

sAP — Code for Towards Streaming Perception ECCV Best Paper Honorable Mention Award Feb 2021: Announcing the Streaming Perception Challenge (CVPR 2021

Martin Li 85 Dec 22, 2022
All the essential resources and template code needed to understand and practice data structures and algorithms in python with few small projects to demonstrate their practical application.

Data Structures and Algorithms Python INDEX 1. Resources - Books Data Structures - Reema Thareja competitiveCoding Big-O Cheat Sheet DAA Syllabus Inte

Shushrut Kumar 129 Dec 15, 2022
An Artificial Intelligence trying to drive a car by itself on a user created map

An Artificial Intelligence trying to drive a car by itself on a user created map

Akhil Sahukaru 17 Jan 13, 2022
This project deploys a yolo fastest model in the form of tflite on raspberry 3b+. The model is from another repository of mine called -Trash-Classification-Car

Deploy-yolo-fastest-tflite-on-raspberry 觉得有用的话可以顺手点个star嗷 这个项目将垃圾分类小车中的tflite模型移植到了树莓派3b+上面。 该项目主要是为了记录在树莓派部署yolo fastest tflite的流程 (之后有时间会尝试用C++部署来提升

null 7 Aug 16, 2022
Indonesian Car License Plate Character Recognition using Tensorflow, Keras and OpenCV.

Monopol Indonesian Car License Plate (Indonesia Mobil Nomor Polisi) Character Recognition using Tensorflow, Keras and OpenCV. Background This applicat

Jayaku Briliantio 3 Apr 7, 2022
Car Parking Tracker Using OpenCv

Car Parking Vacancy Tracker Using OpenCv I used basic image processing methods i

Adwait Kelkar 30 Dec 3, 2022