Journal of the Korea Society of Computer and Information (한국컴퓨터정보학회논문지)

Korean Society of Computer Information (한국컴퓨터정보학회)
권호 표지
DOI QR코드

DOI QR Code

A System for Extraction of Audience Reaction Based on Neural Network

신경회로망 기반의 관객 반응 추출 시스템

  • 백영태 (김포대학교 스마트콘텐츠과) ;
  • 유은순 (단국대학교 미디어콘텐츠연구원) ;
  • 박승보 (인하대학교 IUT 사업단)
  • Received : 2015.02.11
  • Accepted : 2015.02.26
  • Published : 2015.02.28
Download PDF
⟨ Previous Next ⟩

Abstract

Emotional reaction of audience can be decided by observing reaction of audience for content. We can use a method to analyze visual data from video camera to detect reaction of audience fast and economically. This paper proposes the method and system to observe audience reaction from visual data of audience and define via neural network. Also we propose a new method to detect automatically an area for audience reaction with face detection to improve a fixed area assignment method which has a limitation not to adapt depending on audiences. Additionally, the evaluation is implemented to show that the proposed method and system is effective. The proposed method showed the performance elevation of 10.5 % (7.75 hit ration) compared to a fixed area assignment method.

관객의 감정 반응은 개별 관객들의 반응을 관측하여 판단할 수 있다. 영상 카메라를 이용한 시각적 데이터를 분석하여 활용하는 방법을 이용할 경우, 관객의 반응을 적은 비용으로 데이터를 추출하여 판단할 수 있는 것이 가능하다. 하지만, 영상 카메라를 이용할 경우 다수의 관객으로부터 발생하는 대규모의 영상 신호로부터 관객의 특징값들을 분석하여 반응을 판단하기 위한 방법과 시스템이 필요하다. 이를 위해 본 논문에서는 관객의 영상 데이터를 신경회로망 기법을 이용하여 관객의 반응을 관측하여 판단하는 방법과 시스템을 제안하고, 기존의 관객석을 고정적인 영역으로 분할하여 데이터를 취합하여 분석하기 때문에 발생하는 문제를 개선하기 위해 얼굴 검출을 통해 자동적으로 관객 영역을 분할하는 방법을 제안한다. 또한 실험을 통해 기존의 고정적으로 관객 영역을 지정하는 방법에 비해 10.5 %(7.75의 Hit Ratio)의 성능향상이 있음을 확인하였다. 이를 통해 제안하는 방법이 실제 관객 반응을 판단하는데 더 적합하다는 것을 알 수 있다.

Keywords

References

  1. S. G. Barsade, and D. E. Gibson, "Group emotion: A view from top and bottom," Research on Managing Groups and Teams, Stanford, CT: JAI Press, Vol.1, pp. 81-102, 1998.
  2. J. M. Ryu, S.-B. Park, J. K. Kim, "A Study of the Reactive Movement Synchronization for Analysis of Group Flow," Journal of Intelligent Information System, Vol. 19, No. 1, pp. 79-94, March 2013. https://doi.org/10.13088/jiis.2013.19.1.079
  3. N. Ambady, and R. Rosenthal, "Thin slices of expressive behavior as predictors of interpersonal consequences: A meta-analysis," Psychological Bulletin, Vol. 111, No. 2, pp. 256-274, 1992. https://doi.org/10.1037/0033-2909.111.2.256
  4. M. Coulson, "Attributing emotion to static body postures: recognition accuracy, confusions, and viewpoint dependence," Journal of Nonverbal Behavior, Vol. 28, No. 2, pp. 117-139, 1992. https://doi.org/10.1023/B:JONB.0000023655.25550.be
  5. R. Laban, "Modern Educational Dance," Macdonald & Evans, 1968.
  6. E. C. Ryoo, H. Ahn, J. K. Kim, "The Audience Behavior-based Emotion Prediction Model for Personalized Service," Journal of Intelligent Information System, Vol. 19, No. 2, pp. 73-85, June 2013.
  7. M. K. Leung, and Y. H. Yang, "Human Body Motion Segmentation in A Complex Scene," Pattern Recognition, Vol. 20, pp. 55-64, 1987. https://doi.org/10.1016/0031-3203(87)90017-3
  8. J. S. Lee, and M. G. Lee, "A Study on Pattern Recognition using DCT and Neural Network," The Korean Institute of Communications and Information Sciences, Vol. 22, pp. 481-492, 1997.
  9. J. W. Davis, "Appearance-Based Motion Recognition of Human Actions," Technical Report, MIT, 1996.
  10. A. P. Dempster, N. M. Laird, and D. B. Rubin, "Maximum-likelihood from incomplete data via the EM algorithm," Journal of the Royal Statistical Society, Series B, Vol. 39, No. 1, pp. 1-38, 1977.
  11. Y. Zhu, Y. Huang, G. Xu, H. Ren, and Z. Wen, "Vision-based Interpretation of Hand Gestures by Modeling Appearance Changes in Image Sequences," Proceedings of MVA'98 IAPR Workshop on Machine Vision Applications, pp. 573-576, 1998.
  12. M. Coulson, "Attributing emotion to static body postures: recognition accuracy, confusions, and viewpoint dependence," Journal of Nonverbal Behavior, Vol. 28, No. 2, pp. 117-139, 1992. https://doi.org/10.1023/B:JONB.0000023655.25550.be
  13. C. Lee, K. Um, and K. Cho, "Design of a Character's System to Express Mixed-Emotion by Gesture and Posture," The Proceedings of Spring on Korea Multimedia Society 2009, pp. 191-194, 2009.
  14. W. S. McCulloch, and W. Pitts, "A Logical Calculus of the Ideas Immanent in Nervous Activity," The Bulletin of Mathematical Biophysics, Vol. 5, No. 4, pp. 15-33, 1943. https://doi.org/10.1007/BF02478115
  15. S.-B. Park, "The Controller Design for Truck Backer-Upper By ART2 Neural Network," Master Thesis, Inha University, February 1997.
  16. D. Kim, "Theory and Appication of Neutal Network," Hitech Information, 1993.
  17. D. Hong, and W. Woo, "Emotional User Interface of Based on Augmented Reality for I2-NEXT," The Proceedings of Korea HCI Society 2005, pp. 96-101, 2005.
  18. J.-G. Jeong, S.-S. Park, and D.-S. Jang, "Real-Time face detection using the Skin color and Haar-like feature," Journal of the Korea society of computer and information, Vol. 10, No. 4, pp. 113-121, September 2005.
  19. Y. T. Baek, S.-B. Park, "Shot Type Detecting System using Face Detection," Journal of The Korea Society of Computer and Information, Vol. 17 No. 9, pp. 49-56, September 2012. https://doi.org/10.9708/jksci/2012.17.9.049
  20. S.-B. Park, J. M. Ryu, J. K. Kim, "A Group Arousal Analysis Based on the Movement Synchronization of Audiences," Multimedia Tools and Applications, Online published, 2014.