A Development of Nurse Scheduling Model Based on Q-Learning Algorithm

  • JUNG, In-Chul (Department of Integrated IT center, Eulji University Medical Center) ;
  • KIM, Yeun-Su (Department of Integrated IT center, Eulji University Medical Center) ;
  • IM, Sae-Ran (Department of Integrated IT center, Eulji University Medical Center) ;
  • IHM, Chun-Hwa (Dept. of Laboratory Medicine, Eulji University Hospital)
  • Received : 2021.02.25
  • Accepted : 2021.06.05
  • Published : 2021.06.30


In this paper, We focused the issue of creating a socially problematic nurse schedule. The nurse schedule should be prepared in consideration of three shifts, appropriate placement of experienced workers, the fairness of work assignment, and legal work standards. Because of the complex structure of the nurse schedule, which must reflect various requirements, in most hospitals, the nurse in charge writes it by hand with a lot of time and effort. This study attempted to automatically create an optimized nurse schedule based on legal labor standards and fairness. We developed an I/O Q-Learning algorithm-based model based on Python and Web Application for automatic nurse schedule. The model was trained to converge to 100 by creating an Fairness Indicator Score(FIS) that considers Labor Standards Act, Work equity, Work preference. Manual nurse schedules and this model are compared with FIS. This model showed a higher work equity index of 13.31 points, work preference index of 1.52 points, and FIS of 16.38 points. This study was able to automatically generate nurse schedule based on reinforcement Learning. In addition, as a result of creating the nurse schedule of E hospital using this model, it was possible to reduce the time required from 88 hours to 3 hours. If additional supplementation of FIS and reinforcement Learning techniques such as DQN, CNN, Monte Carlo Simulation and AlphaZero additionally utilize a more an optimized model can be developed.


  1. Awadallah, M. A., Bolaji, A. L., & Al-Betar, M. A. (2015). A hybrid artificial bee colony for a nurse rostering problem. Applied Soft Computing, 35, 726-739.
  2. Kim, B. N., Oh, H. S., & Park, Y. S. (2011). A Study of Nurses' Resilience, Occupational Stress and Satisfaction. Korean Journal of Occupational Health Nursing, 20(1), 14-23.
  3. Kim, J. H., & Jo, H. S. (2013). A Comparative Study on Job Stress and Satisfaction between Ward Nurses and Outpatient Nurses. Korean J Occup Health Nurs, 22(2), 83-92.
  4. Lu, Z. P. (2012). Adaptive neighborhood search for nurse rostering. European Journal of Operational Research, 218, 865-876.
  5. Oh, Y. H. (2012). A nurse rostering problem with preference and fairness. Korea Society of IT Services Fall Conference, 190-193.
  6. Park, J. C., & Kim, H. C. (2018). Simple Q-Learning using heuristic strategies. Proceedings of the Korea Information Processing Society Conference, 25(2), 708-710.
  7. Song, M. Y. (2020). A Study on the Nurse Scheduling Optimization Model for Nurse Needs-Type Scheduling Automation System. Journal of the Korea Society of Computer and Information, 25(3), 57-64.
  8. Vaclavik, R., Sucha. P., & Hanzalek, Z. (2016). Roster Evaluation Based on Classifiers for the Nurse Rostering Problem, Journal of Heuristics 22(5), 667-697.
  9. Yoo, W. S., Seo, J. h., Kim, D. H., & Kim, K. H. (2019). Machine scheduling models based on reinforcement Learning for minimizing due date violation and setup change. The Journal of Society for e-Business Studies, 24(3), 19-33.
  10. Kong, Y. H., & Lee, W. C. (2017). Dynamic Obstacle Avoidance and Optimal Path Finding Algorithm for Mobile Robot Using Q-Learning. Journal of Korean Institute of Information Technology, 15(9), 57-62.
  11. Kang, H. G., Seo, D. S., Lee, B. S., & Kang, M. S. (2017). Applying CEE (CrossEntropyError) to improve performance of Q-Learning algorithm. Korean Journal of Artificial Intelligence, 5(1), 1-9.
  12. Park, J. G., Choi, E. S., Kim, B. J., & Zhang, P. (2017). A Study on the Development of DGA based on Deep Learning. Korean Journal of Artificial Intelligence, 5(1), 18-28.
  13. Lee, D. H., & Woo, C. W. (2005). Design and Implementation of Othello game Based on Reinforcement Learning. KIISE Fall Conference, 32(2), 778-780.