In this paper, we use a low cost driving simulator to create a reinforcement learning environment for the training of a deep learning algorithm of self-driving cars. The states, action and reward according to the Markov Decision Process (MDP) are defined and implemented in the simulator. In the case of the states, the speed and heading, acceleration, brake, position, the state of the vehicle, and the images on the current screen are included. For actions, continuous values such as steering angle, acceleration, and brake are used for a driving vehicle. However, for simplicity in this study, discrete input values such as go, stop, turn left and turn right provided by a human operator are employed. In the case of rewards, if the vehicle goes well along the planned path, the bonus is given. On the other hand, if the vehicle invades the lane or hits surrounding objects, the penalty is given. At this time, waypoints are an important tool to calculate rewards. In this study, a certain region in the simulation scenario was selected as a test region, and the waypoints in the region were acquired for the path of interest. After virtual simulation environment creation, we use a Deep Q Network (DQN) algorithm for reinforcement learning by using images on screen for end-to-end self-driving algorithms. To verify the performance of the trained algorithm, driving scores of the algorithm are compared with those of auto-driver mode.

Keywords: driving simulator, reinforcement learning, training dataset, end-to-end self-driving