The Journal of the Korea institute of electronic communication sciences (한국전자통신학회논문지)

Korea Institute of Electronic Communication Science (한국전자통신학회)
권호 표지
DOI QR코드

DOI QR Code

A Queue Management Mechanism for Service groups based on Deep Reinforcement Learning

심층강화학습 기반 서비스 그룹별 큐 관리 메커니즘

  • 정설령 (순천대학교 멀티미디어공학과) ;
  • 이성근 (순천대학교 멀티미디어공학과)
  • Received : 2020.10.26
  • Accepted : 2020.12.15
  • Published : 2020.12.31
Download PDF
⟨ Previous Next ⟩

Abstract

In order to provide various types of application services based on the Internet, it is ideal to guarantee the quality of service(QoS) for each flow. However, realizing these ideas is not an easy task.. It is effective to classify multiple flows having the same or similar service quality requirements into same group, and to provide service quality for each group. The queue management mechanism in the router plays a very important role in order to efficiently transmit data and to support differentiated quality of service for each service. In order to efficiently support various multimedia services, an intelligent and adaptive queue management mechanism is required. This paper proposes an intelligent queue management mechanism based on deep reinforcement learning that decides whether to deliver packets for each group based on the traffic information of each flow group flowing in for a certain period of time and the current network state information.

인터넷을 기반으로 다양한 종류의 응용 서비스들을 제공하기 위해서 각 흐름 별로 서비스 품질을 보장하는 것은 이상적이지만, 이를 실현하는 것은 매우 어려운 일이다. 서비스 품질 요구조건이 같거나 비슷한 여러 흐름들을 동일한 그룹으로 지정하고, 그룹별로 서비스 품질을 제공하는 방안이 효율적이다. 라우터에서 적용되는 큐 관리 메커니즘은 데이터의 효율적으로 전송하고, 서비스 별로 차별화된 서비스 품질을 지원하기 위하여 매우 중요한 역할을 수행한다. 다양한 멀티미디어 서비스를 효율적으로 지원하기 위해서 지능적이고 적응적인 큐 관리 메커니즘 기능이 필요하다. 본 논문은 일정 기간 유입되는 각 흐름 그룹의 트래픽 정보와 현재의 네트워크 상태 정보를 기반으로 그룹별 패킷의 전달 여부를 결정하는 심층강화학습 기반의 지능형 큐관리 메커니즘을 제안한다.

Keywords

References

  1. K. Zhu and E. Hossain, "Virtualization of 5G cellular networks as a hierarchical combinatorial auction," IEEE Transaction on Mobile Computing, vol. 15, no. 10, Oct. 2016, pp. 2640 - 2654. https://doi.org/10.1109/TMC.2015.2506578
  2. S. Lee, "Design and Application of LoRa-based Network Protocol in IoT Networks," J. of the Korea Institute of Electronic Communication Sciences, vol. 14, no. 6, pp. 1089 - 1095, 2020. https://doi.org/10.13067/JKIECS.2019.14.6.1089
  3. Z. Xu et al. ,"Experience-Driven Networking: A Deep Reinforcement Learning Based Approach," Proc. IEEE INFOCOM, Honolulu, HI, USA, Apr. 2018.
  4. S. Floyd and V. Jacobson, "Random early detection gateways for congestion avoidance, " IEEE/ACM Trans. Netw., vol. 1, no. 4, Aug. 1993, pp. 397-413. https://doi.org/10.1109/JAS.2014.7004667
  5. J. Kim, S. Lee, Koh, and J. Park, "Traffic Control Algorithm for Periodic Traffics in WSN," J. of the Korea Institute of Electronic Communication Sciences, vol. 5, no. 1, pp.44-50, 2010.
  6. J. Kim, S. Lee, Koh, and C. Jung, "A Marking Algorithm for QoS Provisioning in WMSN," J. of the Korea Institute of Electronic Communication Sciences, vol. 5, no. 2, pp.193-204, 2010.
  7. K. Nichols and V. Jacobson, "Controlling queue delay," ACM Queue, vol. 10, no. 5, May 2012, pp.1-14.
  8. K. Arulkumaran, M. Deisenroth, M. Brundage and A. Bharath, "Deep reinforcement learning : a brief survey," IEEE Signal Process. Magazine, vol. 34, no. 6, Nov. 2017, pp.26-38. https://doi.org/10.1109/MSP.2017.2743240
  9. X. Huang, T. Yuan, G. Qiao and Y. Ren, "Deep reinforcement learning for multimedia traffic control in software defined networking," IEEE Network, vol. 32, no. 6, Dec. 2018, pp. 35-41. https://doi.org/10.1109/MNET.2018.1800097
  10. B. Guo, X. Zhang, Y. Wang, and H. Yang, "Deep Q-network based multimedia multi-service QoS optimization for mobile edge computing systems," IEEE Access, vol. 7, Nov. 2019, pp. 160961-160972. https://doi.org/10.1109/access.2019.2951219
  11. K. Xiao, S. Mao, and J. K. Tugnait, "TCP-Drinc: Smart congestion control based on deep reinforcement learning,". IEEE Access, vol. 7, Jan. 2019, pp. 11892-11904. https://doi.org/10.1109/access.2019.2892046