Development of Reinforcement Learning-based Obstacle Avoidance toward Autonomous Mobile Robots for an Industrial Environment

산업용 자율 주행 로봇에서의 격자 지도를 사용한 강화학습 기반 회피 경로 생성기 개발

Yang, Jeong-Yean

  • Received : 2018.12.28
  • Accepted : 2019.01.21
  • Published : 2019.03.28


Autonomous locomotion has two essential functionalities: mapping builds and updates maps by uncertain position information and measured sensor inputs, and localization is to find the positional information with the inaccurate map and the sensor information. In addition, obstacle detection, avoidance, and path designs are necessarily required for autonomous locomotion by combining the probabilistic methods based on uncertain locations. The sensory inputs, which are measured by a metric-based scanner, have difficulties of distinguishing moving obstacles like humans from static objects like walls in given environments. This paper proposes the low resolution grid map combined with reinforcement learning, which is compared with the conventional recognition method for detecting static and moving objects to generate obstacle avoiding path. Finally, the proposed method is verified with experimental results.


Obstacle Avoidance;Grid Map;Reinforcement Learning;Autonomous Locomotion;Path Generation


  1. S. Thrun, "A Probabilistic Approach to Concurrent Mapping and Localization for Mobile Robots," Machine Learning and Autonomous Robots, Vol.31, No.5, pp.1-25, 1998
  2. S. Thrun, W. Burgard, and D. Fox, Probabilistic Robotics, pp.171-172, MIT Press, 2005.
  3. D. Holz, D. Droeschel, S. Behnke, M. May, and H. Surmann, "Fast 3D perception for collision avoidance and SLAM in domestic environment," In Mobile robots navigation, In-Tech Education and Publishing, Vienna, Austria, 2010.
  4. G. Grisetti, C. Stachniss, and W. Burgard, "Improved Technique for Grid Mapping with Rao-Blackwellization Particle Filters," IEEE Trans. on Robotics (T-RO), Vol.23, pp.34-46, 2007.
  5. J. Civera, A. J. Davison, and J. M. M. Montiel, "Inverse depth parametrization for monocular SLAM," IEEE Trans. Robot., Vol.24, No.5, pp.932-945, 2008(10).
  6. A. J. Davison, I. D. Reid, N. D. Molton, and O. Stasse, "MonoSLAM: Real-time single camera SLAM," IEEE Trans. Pattern Anal. Mach. Intell., Vol.29, No.6, pp.1052-1067, 2007(6).
  7. D. Marzorati, M. Matteucci, D. Migliore, and D. G. Sorrenti, "Monocular SLAM with inverse scaling parametrization," In Proc. of the British machine vision conference, Leeds, pp.945-994, 2008.
  8. J. Sola, T. Vidal-Calleja, J. Civera, and J. M. M. Montiel, "Impact of landmark parametrization on monocular EKF-SLAM with points and lines," International Journal of Computer Vision, Vol.97, pp.339-368, 2012.
  9. R. S. Sutton and A. G. Barto, Reinforcement Learning: An Introduction, Cambridge Univ. Press, pp.157-159, 1998.
  10. J. Roberts, I. Manchester, and R. Tedrake, "Feedback controller parameterizations for reinforcementlearning," In 2011 IEEE Symposium on Adaptive Dynamic Programming and Reinforcement Learning (ADPRL), 2011.