Collision Avoidance Problem for Robots in Smart Warehouses Using a Focused Decentralized Reinforcement Learning Model

Abstract

The study uses a reinforcement learning model to address robot collision avoidance problems in intelligent warehouse environments. We simulate a virtual warehouse and optimize the robot control strategy to avoid collisions and maximize travel time simultaneously, aiming to optimize storage performance. In this virtual warehouse simulation, robots move on a grid map, performing tasks such as picking up and delivering items to designated locations. Throughout the research, we apply reinforcement learning methods, including Deep Q-Learning and Double Deep Q-Learning, comparing and evaluating them to achieve the best results. Additionally, we examine some heuristic algorithms to optimize performance. With these approaches, we have successfully optimized the movement performance of robots and reduced the risk of collisions, opening up the possibility of deployment in natural warehouse environments with the highest level of simulation to ensure the accuracy and practicality of the research findings. In our experiments, we use the PyTorch package to build models and settings for our particular environments. We tested our system in a grid environment with 5, 16, and 32 robots to evaluate the performance and stability of the proposed method. Evaluation results are based on criteria such as task completion rate, path length compared to actual path, and other factors. The results obtained from the tests were quite positive, with the average rate of each robot reaching its destination reaching a high level, up to 99.78%.