Journal of the Korea Academia-Industrial cooperation Society (한국산학기술학회논문지)

The Korea Academia-Industrial cooperation Society (한국산학기술학회)
권호 표지
DOI QR코드

DOI QR Code

Traffic Control using Q-Learning Algorithm

Q 학습을 이용한 교통 제어 시스템

  • Zheng, Zhang (Mechanical Engineering Xian Jiaotong University) ;
  • Seung, Ji-Hoon (Electronics and Information Department, Chonbuk National University) ;
  • Kim, Tae-Yeong (Electronics and Information Department, Chonbuk National University) ;
  • Chong, Kil-To (Electronics and Information Department, Chonbuk National University)
  • Received : 2011.09.02
  • Accepted : 2011.11.10
  • Published : 2011.11.30
Download PDF
⟨ Previous Next ⟩

Abstract

A flexible mechanism is proposed in this paper to improve the dynamic response performance of a traffic flow control system in an urban area. The roads, vehicles, and traffic control systems are all modeled as intelligent systems, wherein a wireless communication network is used as the medium of communication between the vehicles and the roads. The necessary sensor networks are installed in the roads and on the roadside upon which reinforcement learning is adopted as the core algorithm for this mechanism. A traffic policy can be planned online according to the updated situations on the roads, based on all the information from the vehicles and the roads. This improves the flexibility of traffic flow and offers a much more efficient use of the roads over a traditional traffic control system. The optimum intersection signals can be learned automatically online. An intersection control system is studied as an example of the mechanism using Q-learning based algorithm, and simulation results showed that the proposed mechanism can improve the traffic efficiency and the waiting time at the signal light by more than 30% in various conditions compare to the traditional signaling system.

이 논문에서는 도심 지역의 교통 제어 시스템의 동적 응답 성능 향상을 위하여 적응형 Q-Learning 강화 학습 메커니즘을 설계 하였다. 도로, 자동차, 교통 제어 시스템을 지능 시스템으로 모델링 하고, 자동차와 도로 사이는 무선 통신을 이용한 네트워크가 구성된다. 도로와 대로변에 필요한 센터네트워크가 설치되고 Q-Learning 강화 학습은 제안한 메커니즘의 구현을 위해 핵심 알고리즘으로 채택하였다. 교통 신호 제어 규칙은 자동차와 도로에서 매 시간 업데이트된 정보에 따라서 결정되며, 이러한 방법은 기존의 교통 제어 시스템에 비하여 도로를 효율적으로 활용하며 결과적으로 교통 흐름을 개선 한다. 알고리즘을 활용한 최적의 신호 체계는 온라인상에서 자동으로 학습함으로서 구현된다. 시뮬레이션을 통하여 제안한 알고리즘이 기존 시스템에 비하여 효율성 개선과 차량의 대개 시간에 대한 성능 지수가 모두 30% 이상 향상되었다. 실험 결과를 통하여 제안한 시스템이 교통 흐름을 최적화함을 확인하였다.

Keywords

References

  1. http://www.ewh.ieee.org/tc/its/.
  2. Bong Keun Lee, Jae Du Chung, Keun Ho Ryu, "Multi-Agent Reinforcement Learning Model based on Fuzzy Inference", International Journal of Contents Vol. 9 No. 10, pp. 51-58, 2009.
  3. Wei Wu, Gong Shufeng, Liu Hongxiu, "A Coordinated Urban Traffic Signal Control Approach based on Multi Agent," Intelligent Engineering Systems, 2009. INES 2009. International Conference on, pp. 263-267, 2009
  4. Min Chee Choy, Dipti Srinivasan, Ruey Long Cheu. "Cooperative, Hybrid agent Architecture for Real-Time traffic Signal Control." IEEE Transaction on Systems, Man, and Cybernetics - Part A: Systems and Humans, Vol. 33, No. 5, pp. 597-607, Sep. 2003. https://doi.org/10.1109/TSMCA.2003.817394
  5. Kouvelas, A., Aboudolas, K., Papageorgiou, M., Kosmatopoulos, E. B., "A Hybrid Strategy for Real Time Traffic Signal Control of Urban Road Networks, Intelligent Transportation Systems," IEEE Transactions on, pp. 1-11, 2011.
  6. Srinivasan, D., Min Chee Choy, Cheu, R.L., "Neural Networks for Real-Time Traffic Signal Control, Intelligent Transportation Systems," IEEE Transactions on, pp. 261-272, 2006.
  7. Gokulan, B.P., Srinivasan, D., "Distributed Geometric Fuzzy Multiagent Urban Traffic Signal Control, Intelligent Transportation Systems," IEEE Transactions on, pp. 714-727, 2010.
  8. Balaji, P.G., German, X., Srinivasan, D.,"Urban Traffic Signal Control Using Reinforcement Learning Agents, Intelligent Transport Systems", IET, pp. 177-188, 2010.
  9. Robertson, D.I., Bretherton, R.D., "Optimizing Networks of Traffic Signals in Real Time-The SCOOT Method," Vehicular Technology, IEEE Transactions on, pp. 11-15, 1991.
  10. LIU Guang-ping, ZHAI Run-ping and PEI Yu-long. "A Calculating Method of Intersection Delay under Signal Control." Proceedings of the 2007 IEEE Intelligent Transportation Systems Conference, Seattle, WA, USA, Sept. 30 - Oct. 3, pp. 1114-1119, 2007.
  11. Weihua Bao, Quanlin Chen and Xiaoxia Xu. "An Adaptive Traffic Signal Timing Scheme for Bus Priority at Isolated Intersection." Proceedings of the 6th World Congress on Intelligent Control and Automation, June 21-23, Dalian, China, pp. 8712- 8716, 2006,
  12. Liao Yongquan, Cheng Xiangjun, "Study on Traffic Signal Control Based on Q-Learning," Fuzzy Systems and Knowledge Discovery, 2009. FSKD '09. Sixth International Conference on, pp. 581-585, 2009.
  13. D. Srinivasan and M.C. Choy. "Cooperative multi-agent system for coordinated traffic signal control." IEE Proc. Intell. Transp. Syst. Vol. 153, No. 1, pp. 41-50, March 2006. https://doi.org/10.1049/ip-its:20055011
  14. Lee, J.H., and Lee-Kwang, H. "Distributed and cooperative fuzzy controllers for traffic intersections group." IEEE Trans. Syst. Man Cybern. C, Appl. Rev. 29, (2), 1999.
  15. Michael Wooldridge. "An Introduction to Multi agent Systems." JOHN WILEY & SONS, LTD. Baffins Lane, Chichester, West Sussex PO191UD, England, 2002.
  16. Watkins, C., & Dayan, P. "Q-learning. Machine Learning," 8, pp. 279-292, 1997.
  17. A. D'Ambrogio et al., "Simulation model building of traffic intersections, Simulat. Modell. Pract. Theory", 2008.