QPICAR Deep Learning

Abstract

The aim of the project is to train a smart tool kit named "Sunfounder Raspberry Pi Robot Car" to move without hitting the walls in a closed area. The goal is to maximize the driving time without crashing by reducing the number of hits. Ultrasonic sensor data collected from the vehicle are processed with reinforcement learning and deep reinforcement learning algorithms and the results are compared. In this study, Python programming language is used. In this study, firstly, the Q-Learning method, which is a reinforcement learning algorithm based on Markov decision processes, is used. The method basically relies on a memory table, Q-Table, in which the Q-values of the agent moving from one state to another are kept. This table is updated according to the results of the Bellman equation in every action of the agent, and as a result of this iterative process, it is optimized to provide that the agent moves to maximize its rewards. Deep Q-Learning (DQN) is used as a deep reinforcement learning algorithm. This algorithm was developed by the DeepMind Technologies team in 2013. Basically, it is based on the use of the Bellman equation, which is an element of the Q-Learning method, in combination with neural networks. This method is often used for training agents in complex and multidimensional environments such as video games. Due to the different type of the data used on the algorithm, minor changes were made to adapt it to the study. RElu and Softplus are used as activation functions. The results of the training process show that the DQN algorithm has an important advantage in terms of training the agent in a short time. At this point, the results are in accordance with other academic studies demonstrating the success of the DQN algorithm for complex environments. For future work, by differentiating the equipment that collects data on the vehicle, different data types such as image, temperature value, oxygen value can be collected and processed. At the same time, with changes to the reward setup in the algorithm, the agent can be trained to move to a specific target or to take actions to avoid a specific target.