DOI QR코드

DOI QR Code

Measuring gameplay similarity between human and reinforcement learning artificial intelligence

사람과 강화학습 인공지능의 게임플레이 유사도 측정

  • 허민구 (호서대학교 게임학과) ;
  • 박창훈 (호서대학교 게임애니메이션융합학부)
  • Received : 2020.10.27
  • Accepted : 2020.12.16
  • Published : 2020.12.20

Abstract

Recently, research on automating game tests using artificial intelligence agents instead of humans is attracting attention. This paper aims to collect play data from human and artificial intelligence and analyze their similarity as a preliminary study for game balancing automation. At this time, constraints were added at the learning stage in order to create artificial intelligence that can play similar to humans. Play datas obtained 14 people and 60 artificial intelligence by playing Flippy bird games 10 times each. The collected datas compared and analyzed for movement trajectory, action position, and dead position using the cosine similarity method. As a result of the analysis, an artificial intelligence agent with a similarity of 0.9 or more with humans was found.

최근, 사람 대신 인공지능 에이전트를 이용하여 게임 테스트를 자동화하는 연구가 관심을 모으고 있다. 본 논문은 게임 밸런싱 자동화를 위한 선행 연구로써 사람과 인공지능으로부터 플레이 데이터를 수집하고 이들의 유사도를 분석하고자 한다. 이때, 사람과 유사한 플레이를 할 수 있는 인공지능의 생성을 위해 학습 단계에서 제약사항을 추가하였다. 플레이 데이터는 14명의 사람과 60개의 인공지능을 대상으로 플리피버드 게임을 각각 10회 실시하여 획득하였다. 수집한 데이터는 코사인 유사도 방법으로 이동 궤적, 액션 위치, 죽은 위치를 비교 분석하였다. 분석 결과 사람과의 유사도가 0.9 이상인 인공지능 에이전트를 찾을 수 있었다.

Keywords

References

  1. Hyung-ryul Son, "A game balance story that makes a fun game", pp31-45, 2014
  2. Luis Levy and Jeannie Novak, "Game development essentials: Game QA & testing. Cengage Learning", pp58-59, 2009.
  3. A. Drachen and A. Canossa. "Towards game play analysis via gameplay metrics", Proceedings of the 13th international MindTrek conference: Everyday life in the ubiquitous era, pp202-209, 2009.
  4. A.DRACHEN et al. "Game analytics: Maximizing the value of player data", Springer, London, pp205-253, 2013.
  5. Huizinga et al., "Automated defect prevention: best practices in software management". Wiley-IEEE Computer Society Press, pp74, 2007.
  6. Unreal Automation System Overview, https://docs.unrealengine.com/ko/Programming/Automation/index.html.
  7. Suk-ki Lee and Ho-Young Kwak., "Game Test Automation with Reinforce Learning", Proceedings of the Korean Society of Computer Information Conference, Vol 26, No.1, pp25-28, 2018.
  8. Oriol Vinyals et al., "Grandmaster level in StarCraft II using multi-agent reinforcement learning". Nature. Vol 575. pp350-572, 2019. https://doi.org/10.1038/s41586-019-1724-z
  9. Julian Togelius and G. N. Yannakakis., "General general game AI", IEEE Conference on Computational Intelligence and Games (CIG), pp20-23, 2016.
  10. Ahmed Khalifa et al., "Modifying MCTS for Human-like General Video Game Playing". IJCAI International Joint Conference on Artificial Intelligence. pp2514-2520, 2016
  11. Y. Burda et al., "Exploration by Random Network Distillation,", in Int. Conf. Learning Representations, New Orleans, LA, USA, pp1-17, 2019.
  12. BERNER, Christopher, et al. "Dota 2 with large scale deep reinforcement learning". arXiv preprint Vol 1912 No.06680, pp1-66, 2019.
  13. P. Hingston, "A new design for a turing test for bots", Computational Intelligence and Games (CIG) 2010 IEEE Symposium on, pp345-350, 2010.
  14. N. Shaker, J. Togelius, G. N. Yannakakis, L. Poovanna, V. S. Ethiraj, S. J. Johansson, et al., "The turing test track of the 2012 mario ai championship: entries and evaluation", Computational Intelligence in Games (CIG) 2013 IEEE Conference on, pp1-8, 2013.
  15. Alan Turing, "Turing 1950", pp433, 1950.
  16. Jiyoung Woo and Huy Kang Kim, "Trends of research on fraud detection in online games", Korea Institute Of Information Security And Cryptology, Vol 27, No. 4, pp14-21, 2017.
  17. Eun-Jo Lee et al, "A Study on Game Bot Detection Using Self-Similarity in MMORPGs", Journal of the Korea Institute of Information Security & Cryptology, Vol 26, No.1, pp93-107, 2016. https://doi.org/10.13089/JKIISC.2016.26.1.93
  18. Chang-wook Jeon and Taekyun Choi and Joonghyun Cho, "Natural language processing starting with TensorFlow and machine learning", pp122-123, 2019.
  19. David Silver et al., "Mastering the game of go without human knowledge", Vol 550, No. 7676, Nature, pp354-359, 2017 https://doi.org/10.1038/nature24270
  20. Unity ML-Agents Toolkit, https://github.com/Unity-Technologies/ml-agents, Unity-Technologies.
  21. Krizhevsky, Alex, Ilya Sutskever, and Geoffrey E. Hinton. "Imagenet classification with deep convolutional neural networks." Advances in neural information processing systems. pp1097-1105, 2012.
  22. Unity ML-Agents, https://github.com/UnityTechnologies/ml-agents/blob/master/docs/Learning-Environment-Design-Agents.md, Unity-Technologies.