The Journal of the Institute of Internet, Broadcasting and Communication (한국인터넷방송통신학회논문지)

The Institute of Internet, Broadcasting and Communication (한국인터넷방송통신학회)
권호 표지
DOI QR코드

DOI QR Code

Video-based Facial Emotion Recognition using Active Shape Models and Statistical Pattern Recognizers

Active Shape Model과 통계적 패턴인식기를 이용한 얼굴 영상 기반 감정인식

  • 장길진 (경북대학교 전자공학부) ;
  • 조아라 (울산과학기술대학교 전기전자컴퓨터공학부) ;
  • 박정식 (영남대학교 정보통신공학과) ;
  • 서용호 (목원대학교 지능로봇공학과)
  • Received : 2014.05.21
  • Accepted : 2014.06.13
  • Published : 2014.06.30
Download PDF
⟨ Previous Next ⟩

Abstract

This paper proposes an efficient method for automatically distinguishing various facial expressions. To recognize the emotions from facial expressions, the facial images are obtained by digital cameras, and a number of feature points were extracted. The extracted feature points are then transformed to 49-dimensional feature vectors which are robust to scale and translational variations, and the facial emotions are recognized by statistical pattern classifiers such Naive Bayes, MLP (multi-layer perceptron), and SVM (support vector machine). Based on the experimental results with 5-fold cross validation, SVM was the best among the classifiers, whose performance was obtained by 50.8% for 6 emotion classification, and 78.0% for 3 emotions.

본 논문에서는 얼굴 영상으로부터 자동으로 사람의 감정을 인식하는 효과적인 방법을 제안한다. 얼굴 표정으로부터 감정을 파악하기 위해서는 카메라로부터 얼굴영상을 입력받고, ASM (active shape model)을 이용하여 얼굴의 영역 및 얼굴의 주요 특징점을 추출한다. 추출한 특징점으로부터 각 장면별로 49차의 크기 및 변이에 강인한 특징벡터를 추출한 후, 통계기반 패턴분류 방법을 사용하여 얼굴표정을 인식하였다. 사용된 패턴분류기는 Naive Bayes, 다중계층 신경회로망(MLP; multi-layer perceptron), 그리고 SVM (support vector machine)이며, 이중 SVM을 이용하였을 때 가장 높은 최종 성능을 얻을 수 있었으며, 6개의 감정분류에서 50.8%, 3개의 감정분류에서 78.0%의 인식결과를 보였다.

Keywords

References

  1. C. H. Ahn, J. H. Choi, S. J. Yang, W. T. Lim, and J.H. Cha, "Trends of Emotional Information Service," Electronics and Telecommunications Trends, Vol. 27, No. 6, December 2012.
  2. J. S. Park and J. W. Lee, "Trends of Emotional User Experience," Electronics and Telecommunications Trends, Vol. 26, No. 5, October 2011.
  3. WonBum Yoon, HeuiSeok Lim, and SungHyun Yoon, "Smart Phone Real Time Q&A System," in Proc.the KAIS Fall Conference, Vol. 1, pp. 12-15, November 2010.
  4. Yeong-Wook Yang, Jae-Choon Jo, and Heui-Seok Lim, "The Smart Phone Application Implement for Cognitive Ability Training," in Proc.the KAIS Fall Conference, Vol. 1, pp. 43-46, November 2010.
  5. Cheol-Ho Yoon, "Usability Study of Different Types of Mobile Phones Using KLM Model," Journal of the Korea Academia-Industrial cooperation Society Vol.14, No.5, pp. 2081-2085, 2013. https://doi.org/10.5762/KAIS.2013.14.5.2081
  6. Ji-Soo O, Jeong-Jin Kang, Myung-Jae Lim, and Ki-Young Lee, "Design and Implementation of an Emotion Recognition System using Physiological Signal," The Journal of The Institute of Internet, Broadcasting and Communication (JIIBC), Vol. 10, No. 1, pp.57-62, February 2010.
  7. Yeon-Soo Lee and Young B. Park, "An acoustic study of feeling information extracting method," The Journal of The Institute of Internet, Broadcasting and Communication (JIIBC), Vol. 10, No. 1, pp.51-55, February 2010.
  8. Hyunsup Yoon, Youngjoon Han, and Hernsoo Hahn, "Facial Expression Recognition based on Expression Space Feature," in Proc. Workshop on Speech Communication and Signal Processing, pp. 395-396, August 2009.
  9. Tae-Hun Yoo, Jong-Yong Lee, Jin-Soo Kim, and Sang-Hun Lee, "A Study on Detecting Salient Region using Frequency-Luminance of image," in Proc.the KAIS Spring Conference, Vol. 2, pp. 486-489, May 2012
  10. Paul Viola and Michael J. Jones, "Robust Real-Time Face Detection," International Journal of Computer Vision, Vol. 57, Issue 2, pp 137-154, May 2004. https://doi.org/10.1023/B:VISI.0000013087.49260.fb
  11. T.F. Cootes and C.J. Taylor and D.H. Cooper and J. Graham, "Active shape models - their training and application," Computer Vision and Image Understanding, Vol. 61, pp. 38-59, 1995. https://doi.org/10.1006/cviu.1995.1004
  12. Byungtae Ahn, Yudeog Han, and In So Kweon, "Real-Time Facial Landmarks Tracking using Active Shape Model and LK Optical Flow", in Proc. International Conference on Ubiquitous Robots and Ambient Intelligence (URAI 2012), pp. 541-543, November 2012.
  13. G. Antonio, M. Sorci, M. Bierlaire, and J. Thiran, "Discrete choice models for static facial expression recognition," in Proc. International Conference on Advanced Concepts for Intelligent Vision Systems, Vol. 4179, pp. 710-721, 2006.
  14. Y. Zhang and Q. Ji, "Facial expression understanding in image sequences using dynamic and active visual information fusion," in Proc. International Conference on Computer Vision, Vol. 2, pp. 1297-1304, 2003.
  15. D. Bimler, and G. Paramei, "Facial-expression affective attributes and their configural correlates: components and categories", Spanish Journal of Psychology, Vol. 9, pp. 19-31, 2006. https://doi.org/10.1017/S113874160000593X
  16. Pedro Domingos and Michael Pazzani, "On the optimality of the simple Bayesian classifier under zero-one loss," Machine Learning Vol. 29, pp. 103-137, 1997. https://doi.org/10.1023/A:1007413511361
  17. Dong-Hyung Lee, Man-Mo Kang, Young-Kee Kim, and Soo-Dong Lee, "Sign Language recognition Using Sequential Ram-based Cumulative Neural Networks," Journal of IWIT, Vol.9, No.5, pp. 205-211, October 2009.
  18. Keon-Jun Park, Yong-Kab Kim, Byun-Gon Kim, and Geun-Chang Hoang, "Design of Optimized Pattern Recognizer by Means of Fuzzy Neural Networks Based on Individual Input Space," Journal of IWIT, Vol.13, No.2, pp. 181-189, April 2013. https://doi.org/10.7236/JIIBC.2013.13.2.181
  19. Kang-Yun Hee and Park-Young, "Design and Implementation of Document Classifier for IT documents based on SVM," Journal of IWIT, Vol.4, No.1, pp. 71-78, October 2004.
  20. Ron Kohavi, "A study of cross-validation and bootstrap for accuracy estimation and model selection," in Proc. International Joint Conference on Artificial Intelligence, Vol. 2, No. 12, pp. 1137-1143, 1995.
  21. Richard Picard and Dennis Cook, "Cross-Validation of Regression Models," Journal of the American Statistical Association, Vol. 79, No. 387, pp. 575-583, 1984. https://doi.org/10.1080/01621459.1984.10478083
  22. Gyoo-Seok Choi, Jong-Jin Park, and Ha-Nam Nguyen, "Feature Selection Algorithm using Random Forest to Diagnose Cancer," International Journal of Advanced Smart Convergence (IJASC), Vol.1, No.1, pp.10-15, 2009.