Self-Driving Car Engineer Nanodegree Program

You can download the Term3 Simulator which contains the Path Planning Project from the [releases tab (https://github.com/udacity/self-driving-car-sim/releases/tag/T3_v1.2).

To run the simulator on Mac/Linux, first make the binary file executable with the following command:

sudo chmod u+x {simulator_file_name}

In this project your goal is to safely navigate around a virtual highway with other traffic that is driving +-10 MPH of the 50 MPH speed limit. You will be provided the car's localization and sensor fusion data, there is also a sparse map list of waypoints around the highway. The car should try to go as close as possible to the 50 MPH speed limit, which means passing slower traffic when possible, note that other cars will try to change lanes too. The car should avoid hitting other cars at all cost as well as driving inside of the marked road lanes at all times, unless going from one lane to another. The car should be able to make one complete loop around the 6946m highway. Since the car is trying to go 50 MPH, it should take a little over 5 minutes to complete 1 loop. Also the car should not experience total acceleration over 10 m/s^2 and jerk that is greater than 10 m/s^3.

Each waypoint in the list contains [x,y,s,dx,dy] values. x and y are the waypoint's map coordinate position, the s value is the distance along the road to get to that waypoint in meters, the dx and dy values define the unit normal vector pointing outward of the highway loop.

The highway's waypoints loop around so the frenet s value, distance along the road, goes from 0 to 6945.554.

1. Clone this repo.
2. Make a build directory: mkdir build && cd build
3. Compile: cmake .. && make
4. Run it: ./path_planning.

Here is the data provided from the Simulator to the C++ Program

["x"] The car's x position in map coordinates

["y"] The car's y position in map coordinates

["s"] The car's s position in frenet coordinates

["d"] The car's d position in frenet coordinates

["yaw"] The car's yaw angle in the map

["speed"] The car's speed in MPH

//Note: Return the previous list but with processed points removed, can be a nice tool to show how far along the path has processed since last time.

["previous_path_x"] The previous list of x points previously given to the simulator

["previous_path_y"] The previous list of y points previously given to the simulator

["end_path_s"] The previous list's last point's frenet s value

["end_path_d"] The previous list's last point's frenet d value

["sensor_fusion"] A 2d vector of cars and then that car's [car's unique ID, car's x position in map coordinates, car's y position in map coordinates, car's x velocity in m/s, car's y velocity in m/s, car's s position in frenet coordinates, car's d position in frenet coordinates.

1. The car uses a perfect controller and will visit every (x,y) point it recieves in the list every .02 seconds. The units for the (x,y) points are in meters and the spacing of the points determines the speed of the car. The vector going from a point to the next point in the list dictates the angle of the car. Acceleration both in the tangential and normal directions is measured along with the jerk, the rate of change of total Acceleration. The (x,y) point paths that the planner recieves should not have a total acceleration that goes over 10 m/s^2, also the jerk should not go over 50 m/s^3. (NOTE: As this is BETA, these requirements might change. Also currently jerk is over a .02 second interval, it would probably be better to average total acceleration over 1 second and measure jerk from that.

2. There will be some latency between the simulator running and the path planner returning a path, with optimized code usually its not very long maybe just 1-3 time steps. During this delay the simulator will continue using points that it was last given, because of this its a good idea to store the last points you have used so you can have a smooth transition. previous_path_x, and previous_path_y can be helpful for this transition since they show the last points given to the simulator controller with the processed points already removed. You would either return a path that extends this previous path or make sure to create a new path that has a smooth transition with this last path.

A really helpful resource for doing this project and creating smooth trajectories was using http://kluge.in-chemnitz.de/opensource/spline/, the spline function is in a single hearder file is really easy to use.

• cmake >= 3.5
  • All OSes: click here for installation instructions
• make >= 4.1
  • Linux: make is installed by default on most Linux distros
  • Mac: install Xcode command line tools to get make
  • Windows: Click here for installation instructions
• gcc/g++ >= 5.4
  • Linux: gcc / g++ is installed by default on most Linux distros
  • Mac: same deal as make - [install Xcode command line tools]((https://developer.apple.com/xcode/features/)
  • Windows: recommend using MinGW
• uWebSockets
  • Run either install-mac.sh or install-ubuntu.sh.
  • If you install from source, checkout to commit e94b6e1, i.e.

    git clone https://github.com/uWebSockets/uWebSockets 
    cd uWebSockets
    git checkout e94b6e1
    

We've purposefully kept editor configuration files out of this repo in order to keep it as simple and environment agnostic as possible. However, we recommend using the following settings:

• indent using spaces
• set tab width to 2 spaces (keeps the matrices in source code aligned)

Please (do your best to) stick to Google's C++ style guide.

Note: regardless of the changes you make, your project must be buildable using cmake and make!

Help your fellow students!

We decided to create Makefiles with cmake to keep this project as platform agnostic as possible. Similarly, we omitted IDE profiles in order to ensure that students don't feel pressured to use one IDE or another.

However! I'd love to help people get up and running with their IDEs of choice. If you've created a profile for an IDE that you think other students would appreciate, we'd love to have you add the requisite profile files and instructions to ide_profiles/. For example if you wanted to add a VS Code profile, you'd add:

• /ide_profiles/vscode/.vscode
• /ide_profiles/vscode/README.md

The README should explain what the profile does, how to take advantage of it, and how to install it.

Frankly, I've never been involved in a project with multiple IDE profiles before. I believe the best way to handle this would be to keep them out of the repo root to avoid clutter. My expectation is that most profiles will include instructions to copy files to a new location to get picked up by the IDE, but that's just a guess.

One last note here: regardless of the IDE used, every submitted project must still be compilable with cmake and make./

A well written README file can enhance your project and portfolio. Develop your abilities to create professional README files by completing this free course.

A class named Vehicle (which is defined in "vehicle.cpp" and "vehicle.h") is created to store all the useful information and define all the behaviors of the main vehicle.

Then in the main(), the following steps are taken:

• An Vehicle object agent is declared and the map_waypoints info is stored in agent： "main.cpp" line 57

Vehicle agent(map_waypoints_x, map_waypoints_y, map_waypoints_s, map_waypoints_dx, map_waypoints_dy);

• After the main car's localization and sensor fusion data are received, store these data in agent： "main.cpp" line 95

agent.setState(j[1]["x"], j[1]["y"], j[1]["s"], j[1]["d"], j[1]["yaw"], j[1]["speed"], j[1]["previous_path_x"], j[1]["previous_path_y"], j[1]["end_path_s"], j[1]["end_path_d"]);

• Using the sensor fusion data, the agent predicts the trajectory of other vehicles in the time horizon of 3s： "main.cpp" line 103

agent.generatePrediction(sensor_fusion);

• The agent generates the best trajectory in the time horizon = 3s ： "main.cpp" line 105

agent.FSMPlanner();

• The agent only outputs the first 0.5 second trajectory to the simulator and throws away the next 2.5 seconds motion in the best_trajectory： "main.cpp" line 108-109

vector<double> next_x_vals(agent.best_trajectory.next_x_vals.cbegin(), agent.best_trajectory.next_x_vals.cbegin() + 25);
vector<double> next_y_vals(agent.best_trajectory.next_y_vals.cbegin(), agent.best_trajectory.next_y_vals.cbegin() + 25);

A simple linear model (the vehicle's speed and lateral positon check_car_d are constant) is used to generate motion prediction predicted_s and predicted_d for the vehicles within 120m around the main vehicle. Meanwhile, check if there is a vehicle on the same lane with the main vehicle within the distance of 30m. If so, record its distance to the main vehicle as following_distance and its velocity as vel_ahead.

• Set max velocity max_vel to be 22m/s (equals to 49.2mph which is within the speed limit);
• If following_distance is less than 30m and the current trajectory is keeping lane, then set max_vel to be vel_ahead-2;
• If following_distance is less than 10m which indicates potential danger, then set max_vel to be vel_ahead-5;

2 and 3 are used to keep enough space between the main vehicle to the vehicle ahead.

To measure which lane is currently free of traffic, calculate the longitudinal distance of the nearest vehicle ahead on each lane (lane0, lane1, lane2) to the main vehicle and store these three distances as distances_ahead. These values are used in the cost calculation function.

A finite state machine of 5 states are used:

• KL: keep lane;
• LCL: lane change left;
• PLCL: prepare lane change left. the vehicle first changes to lane 1, then changes to lane 0. Therefore, PLCL can also be called lane change left twice;
• LCR: lane change right;
• PLCR: prepare lane change right. the vehicle first changes to lane 1, then changes to lane 2. Therefore, PLCR can also be called lane change right twice.

Two constraints on the states are used:

• The vehicle cannot choose actions that lead the vehicle out of the lanes: "vehicle.cpp" line 248-262

To be specific, when the main vehicle is on lane0, then it can only choose "KL","LCR","PLCR"; when the main vehicle is on lane1, then it can only choose "KL","LCR","LCL"; when the main vehicle is on lane2, then it can only choose "KL","LCL","PLCL";

• To avoid the accelerations and jerks of the planned trajectories getting to large, the following state transitions are not allowed: "vehicle.cpp" line 264-306

LCL <-> LCR, LCL <-> PLCR, PLCL <-> LCR, PLCL <-> PLCR

For each state in the successor_states:

• Select specific anchor points according to the state: "vehicle.cpp" line 163-206

If the vehicle's velocity ref_vel is lower than 5m/s, lane change at fixed distances is used:

If the vehicle's velocity ref_vel is larger than 5m/s, lane change at fixed time is used:

• Use the two points at the end of the previous path as the first two anchor points: "vehicle.cpp" line 317-361
• Transform the above anchor points to the vehicle's local coordinate system and generate the spline curve s: "vehicle.cpp" line 364-373
• Combine the previous_path points and horizon*50-prev_size new points generated by the spline curve (new points have been transformed back to the global coordinate system) to form the new candidate trajectory: "vehicle.cpp" line 375-403
• Convert x,y trajectories to s,d trajectories which are used in collision detection: "vehicle.cpp" line 406-419

• Cost_obstacle_avoidance: "vehicle.cpp" line 453-476

For time horizon = 3s, there are 3 * 50 time substeps. The collision detection is done in frenet coordinates between the main vehicle and any other vehicles in the prediction at each time substep. If the longitudinal distance between the two vehicles is below safe_longitudinal_distance = 6.0m, as well as the lateral distance between the two vehicles is below safe_lateral_distance = 3.0m, the two vehicles are regarded as having collision with each other.

If there is collision with any vehicle at any time substep for the main vehicle, then the cost_obstacle_avoidance = 99999.0 which is a large penalty for collision. Otherwise, the cost_obstacle_avoidance = 0.0.

• Cost_traffic_jam: "vehicle.cpp" line 478-491

The distances_ahead (a vector of 3 elements) which has already been calculated in the prediction serves as a good indication of the traffic condition on each lane. If the distances_ahead[i] is large, then the i-th lane is good because the vehicle ahead on that lane is quite far away. Therefore, it is obvious to select the lane with the largest value in the distances_ahead as the best lane. In terms of cost_traffic_jam, the trajectory which end lane's distance_ahead is largest should have the lowest cost value.

However, this cost function design can lead to rapid decision change when any two lanes has similar distance_ahead. For example, if the planner is given the following distances_ahead time series:

Then the main vehicle will drive to the 2-th lane, back to the 0-th lane, and then to the 2-th lane within 2 seconds. This behavior definitely will cause large acceleration and jerk which is not wanted.

Though an additional cost function for comfort or cost function for lane change can be added to punish the bahavior with quick decision change. Here, I propose another way to deal with problem. I introduce a fuzzyDistance function ("helpers.h" line 157-194) to filter the distance_ahead. If the difference between any two distances in distances_ahead is less than 20m, then they are regarded as belonging to the same distance group. The distances in the same distance group are then averaged. For example, {15, 12, 16}->{14.33, 14.33, 14.33}, {120, 110, 40}-{115, 115, 40}, {14, 40, 50}-{14, 45, 45}. I found this method can increase the robustness of the planning algorithm and avoid unnecessary lane changes. The philosophy behind this fuzzyDistance function is also similar to human driver's decision making process that the human drivers will wait and not do lane change until the advantage of changing lane is great enough.

After the distance_ahead is transformed into fuzzy_distance, a parabola cost function is used to calculate the cost_traffic_jam.

• The final weighted cost equals to weights[0] * cost_obstacle_avoidance + weights[1] * cost_traffic_jam.

For the trajectory to a specific state, if the weighted_cost is not smaller than 99999.0 which indicates collision, then this trajectory is not taken into account, except for that this trajectory is a keep-lane trajectory. Then select the trajectory with the lowest weighted cost as the best_trajectory.

If there is only one trajectory that can be selected but unfortunately has collision, then the max velocity max_vel is set to 5.0 m/s which is an emergency stop.

If the reference velocity ref_vel is lower than max velocity max_vel, then add 0.2m/s to the ref_vel; otherwise if the reference velocity ref_vel is larger than max velocity max_vel, then minus 0.2m/s from the ref_vel.

The above proposed planning algorithm achieved a best distance without incident of 61.27 miles on my local machine. The average distance between two incidents is about 22 miles. The screenshot of the best test is shown below:

• For now, the proposed method will fail if the vehicle ahead on the adjacent lane suddenly changes to the lane the main vehicle is on. One solution will be using more complex prediction models like AMM (autonomous multi modal), trajectory clustering or GNB (gaussian naive bayes) introduced in the lessons.
• The parameters in the algorithms can be further tuned and the speed regulation method can be improved.