Journal of Digital Contents Society (디지털콘텐츠학회 논문지)

Digital Contents Society (한국디지털콘텐츠학회)
권호 표지
DOI QR코드

DOI QR Code

Development of a Page Turner Application based on Eye Tracking Algorithm for the Performing Artists

연주자를 위한 시선 추적 기반 페이지 터너 애플리케이션 개발

  • Kim, Tae-Yu (Department of Computer Software Engineering, Soonchunhyang University) ;
  • Kim, Seokhoon (Department of Computer Software Engineering, Soonchunhyang University)
  • 김태유 (순천향대학교 컴퓨터소프트웨어공학과) ;
  • 김석훈 (순천향대학교 컴퓨터소프트웨어공학과)
  • Received : 2018.04.18
  • Accepted : 2018.04.28
  • Published : 2018.04.30
Download PDF
⟨ Previous Next ⟩

Abstract

Sheet music is one of the inevitable elements for successful melody interpretation, playing or rendering, and performance, most of performing artists usually utilize a paper sheet music in the cases. However, the paper sheet music can be a one of the reason to degrade the concentration of audiences and artists or entire performing flows. In addition, it might be a weakness to use a paper sheet music in an outside performing place. We propose an electronic sheet music page tuner application, which can overcome these problems, based on a Tablet PC. The proposed page tuner application, which uses the OpenCV to adapt an eye tracking and behavior recognition, can provide an automatical page pass function to the performing artists. We will expect that the proposed application can highly decrease the weakness of paper sheet music.

악보는 성공적인 곡 해석, 연주, 공연 등을 위해 필수적인 요소로 인식되고 있으며, 대부분의 연주자들은 이러한 상황에서 일반적으로 종이악보를 활용하고 있다. 그러나 종이악보는 페이지를 넘겨야 할 때 연주자 및 청중의 집중도를 떨어뜨리는 원인 중 하나가 될 뿐 아니라, 전체적인 연주의 흐름을 저해하는 요소로 작용하기도 한다. 또한 이러한 종이악보의 단점들은 공연장소 주변의 날씨나 환경 등으로 인해 더더욱 부각될 수밖에 없다. 본 논문에서는 이러한 종이악보의 단점들을 해결하기 위한 태블릿 PC기반의 전자악보 페이지 터너 애플리케이션을 제안한다. 제안하는 페이지 터너 애플리케이션은 원활한 연주 진행을 위해 연주자의 시선 또는 동작을 판독하여 전자악보를 다음페이지로 넘길 수 있도록 구현되어 있으며, 시선추적 및 동작 판독 알고리즘은 OpenCV를 통해 구현하였다. 제안하는 페이지 터너 애플리케이션을 통해 기존 종이악보가 갖고 있는 문제점을 상당부분 개선할 수 있을 것으로 기대한다.

Keywords

References

  1. S. Jang, S. Kim, C. Kim, "Attentional Processing of Emotional Faces in Schizophrenia: Evidence From Eye Tracking," Journal of abnormal psychology, Vol.125, No.7, 2016
  2. Y. Lin, K. Holmqvist, K. Miyoshi, K. Ashida, H., "Effects of detailed illustrations on science learning: an eye-tracking study," Instructional science, Vol.45, No.5, 2017
  3. B. Kortman, "Assessing for Hemi-Spatial Neglect Using Eye Tracking Glasses," Archives of physical medicine and rehabilitation, Vol.96, No.10, 2015
  4. D. Bolden, P. Barmby, S. Raine, M. Gardner, "How young children view mathematical representations: a study using eye-tracking technology," Educational research, Vol.57, No.1, 2015
  5. K. Brenton, N. Kate, "Assessing for Unilateral Spatial Neglect Using Eye-Tracking Glasses: A Feasibility Study," OCCUPATIONAL THERAPY IN HEALTH CARE, Vol.30, No.4, 2016
  6. R. K. Sungkur, M. A. Antoaroo, A. Beeharry, "Eye tracking system for enhanced learning experiences," EDUCATION AND INFORMATION TECHNOLOGIES, Vol.21, No.6, 2016
  7. M. Mehrube, N. Vuong, "Real-time eye tracking for password authentication," 2018 IEEE International Conference on Consumer Electronics (ICCE), pp. 1-4, 2017
  8. Byoung-jin Kim, Suk-ju Kang, "Reliability Measurement Technique of The Eye Tracking System Using Gaze Point Information," Journal of Digital Contents Society, Vol. 17, No. 5, pp. 367-373, 2016. https://doi.org/10.9728/dcs.2016.17.5.367
  9. R. Hosek, "Android eye detection and tracking with OpenCV". Available: http://romanhosek.cz/android-eye-detection-and-tracking-with-opencv/
  10. OpenCV 2.4.13.6 documentation Tutorials, "Template Matching". Available:https://docs.opencv.org/2.4/doc/tutorials/imgproc/histograms/template_matching/template_matching.html#theory
  11. J. Luan, Z. Yao, F. Zhao, L. Hao, "Search product and experience product online reviews: An eye-tracking study on consumers' review search behavior," Computers in human behavior, Vol.65, 2016
  12. T. Shechner, J. Jarcho, S. Wong, "Threats, rewards, and attention deployment in anxious youth and adults: An eye tracking study," Biological psychology, Vol.122, 2017
  13. K. Sturmer, T. Seidel, K. Muller, J. Hausler, "What is in the eye of preservice teachers while instructing: An eye-tracking study about attention processes in different teaching situations," ZEITSCHRIFT FUR ERZIEHUNGSWISSENSCHAFT, Vol.20, No.1, 2017
  14. R. Zheng, K. Nakano, H. Ishiko, K. Hagita, Kih, "Eye-Gaze Tracking Analysis of Driver Behavior While Interacting With Navigation Systems in an Urban Area," IEEE Transactions on Human-Machine Systems, Vol.46, No.4, 2016
  15. Y. Zhang, P. Zhibing, L. Chenhui, Xu, "Application Research of Gaze Tracking Technology in Battlefield Situation Visualization Assessment," LNEE, Vol.2015, No.356, 2015
  16. E. Skodras, V. G. Kanas, N. Fakotakis, "On visual gaze tracking based on a single low cost camera," Signal processing Image communication, Vol.36, 2015