Convergence Security Journal (융합보안논문지)

Korea convergence Security Association (한국융합보안학회)
권호 표지

Study of Facial Expression Recognition using Variable-sized Block

가변 크기 블록(Variable-sized Block)을 이용한 얼굴 표정 인식에 관한 연구

  • Received : 2019.02.15
  • Accepted : 2019.03.29
  • Published : 2019.03.31
Download PDF
⟨ Previous Next ⟩

Abstract

Most existing facial expression recognition methods use a uniform grid method that divides the entire facial image into uniform blocks when describing facial features. The problem of this method may include non-face backgrounds, which interferes with discrimination of facial expressions, and the feature of a face included in each block may vary depending on the position, size, and orientation of the face in the input image. In this paper, we propose a variable-size block method which determines the size and position of a block that best represents meaningful facial expression change. As a part of the effort, we propose the way to determine the optimal number, position and size of each block based on the facial feature points. For the evaluation of the proposed method, we generate the facial feature vectors using LDTP and construct a facial expression recognition system based on SVM. Experimental results show that the proposed method is superior to conventional uniform grid based method. Especially, it shows that the proposed method can adapt to the change of the input environment more effectively by showing relatively better performance than exiting methods in the images with large shape and orientation changes.

본 논문에서는 가변 크기 블록 기반의 새로운 얼굴 특징 표현 방법을 제안한다. 기존 외형 기반의 얼굴 표정 인식 방법들은 얼굴 특징을 표현하기 위해 얼굴 영상 전체를 균일한 블록으로 분할하는 uniform grid 방법을 사용하는데, 이는 다음 두가지 문제를 가지고 있다. 얼굴 이외의 배경이 포함될 수 있어 표정을 구분하는 데 방해 요소로 작용하고, 각 블록에 포함된 얼굴의 특징은 입력영상 내 얼굴의 위치, 크기 및 방위에 따라 달라질 수 있다. 본 논문에서는 이러한 문제를 해결하기 위해 유의미한 표정변화가 가장 잘 나타내는 블록의 크기와 위치를 결정하는 가변 크기 블록 방법을 제안한다. 이를 위해 얼굴의 특정점을 추출하여 표정인식에 기여도가 높은 얼굴부위에 대하여 블록 설정을 위한 기준점을 결정하고 AdaBoost 방법을 이용하여 각 얼굴부위에 대한 최적의 블록 크기를 결정하는 방법을 제시한다. 제안된 방법의 성능평가를 위해 LDTP를 이용하여 표정특징벡터를 생성하고 SVM 기반의 표정 인식 시스템을 구성하였다. 실험 결과 제안된 방법이 기존의 uniform grid 기반 방법보다 우수함을 확인하였다. 특히, 제안된 방법이 형태와 방위 등의 변화가 상대적으로 큰 MMI 데이터베이스에서 기존의 방법보다 상대적으로 우수한 성능을 보여줌으로써 입력 환경의 변화에 보다 효과적으로 적응할 수 있음을 확인하였다.

Keywords

References

  1. P. Weinberg, and W. Gottwald, "Impulsive consumer buying as a result of emotions", Journal of Business research, Vol. 10, No. 1, pp. 43-57, 1982. https://doi.org/10.1016/0148-2963(82)90016-9
  2. D. Consoli, "Emotions that influence purchase decisions and their electronic processing", Annales Universitatis Apulensis: Series Oeconomica, Vol. 11, No. 2, pp. 996, 2009.
  3. M. Kang, "A Study on Efficient Facial Expression Recognition System for Customer Satisfaction Feedback", Journal of Information and Security, Vol. 12, No. 4, pp. 75-84, Sept. 2012.
  4. S. Huh, D. Lee, K. Kim, "A Study on The Improvement of User Authentication using the Facial Recognition and OTP Technique in the Mobile Environment", Journal of Information and Security, Vol. 11, No. 3, pp. 75-84, 2011.
  5. Jai-Suk Lee, et al., "Research of video based Vibraimage technology stimulation examination KOCOSA", Journal of Information and Security, Vol. 15, Vo. 3, pp. 41-51, 2015.
  6. Tian, Ying-Li, Takeo Kanade, and Jeffrey F. Cohn, "Facial expression analysis", Handbook of face recognition. Springer, New York, NY, pp. 247-275, 2005.
  7. I. Kotsia, and I. Pitas, "Facial expression recognition in image sequences using geometric deformation features and support vector machines", IEEE trans. on image processing, Vol. 16, No. 1, pp. 172-187, 2007. https://doi.org/10.1109/TIP.2006.884954
  8. Pantic, Maja, and Marian Stewart Bartlett, "Machine analysis of facial expressions", Face recognition. InTech, 2007.
  9. M. A. Turk and A. P. Pentland, "Face recognition using eigenfaces", Computer Vision and Pattern Recognition, Proceedings CVPR'91., IEEE Computer Society Conference on. IEEE, pp. 586-591, 1991.
  10. P. Belhumeur and J. P. Hespanha and D. J. Kriegman, "Eigenfaces vs. sherfaces: Recognition using class specific linear projection", IEEE Trans. on Pattern Analysis and Machine Intelligence, Vol. 7, pp. 711-720, 1997.
  11. Z. Nenadic, "Information discriminant analysis: Feature extraction with an information-theoretic objective", IEEE Trans on Pattern Analysis and Machine Intelligence, Vol. 29, No. 8, pp. 1394-1407, 2007. https://doi.org/10.1109/TPAMI.2007.1156
  12. J. Yang, et al., "Two-dimensional PCA: a new approach to appearance-based face representation and recognition", IEEE trans. on pattern analysis and machine intelligence, Vol. 26, No. 1, pp. 131-137, 2004. https://doi.org/10.1109/TPAMI.2004.1261097
  13. T. Ahonen, A. Hadid, and M. Pietikainen, "Face description with local binary patterns: Application to face recognition", IEEE Trans. on Pattern Analysis & Machine Intelligence, Vol. 12, pp. 2037-2041, 2006.
  14. M. Lyons, et al., "Coding facial expressions with gabor wavelets", Automatic Face and Gesture Recognition, 1998. Proceedings. 3rd IEEE Int'l Conference on. 1998.
  15. Jabid, Taskeed, Md Hasanul Kabir, and Oksam Chae. "Robust facial expression recognition based on local directional pattern." ETRI journal, Vol. 32, No. 5 pp. 784-794, 2010. https://doi.org/10.4218/etrij.10.1510.0132
  16. C. Shan, S. Gong, and P. W. McOwan, "Facial expression recognition based on local binary patterns: A comprehensive study", Image and vision Computing, Vol. 27, No. 6, pp. 803-816, 2009. https://doi.org/10.1016/j.imavis.2008.08.005
  17. Z. Niu, et al., "2d cascaded adaboost for eye localization", Pattern Recognition, 2006. ICPR 2006. 18th International Conference on. Vol. 2. IEEE, pp. 1216-1219, 2006.
  18. Tzimiropoulos, Georgios, and Maja Pantic, "Fast algorithms for fitting active appearance models to unconstrained images", International journal of computer vision, Vol 122, No. 1, pp. 17-33, 2017. https://doi.org/10.1007/s11263-016-0950-1
  19. A. Asthana, et al., "Incremental face alignment in the wild", Proc. of the IEEE conf. on CVPR, pp. 1859-1866, 2014.
  20. Paul, Ekman, Wallace V. Friesen, and Joseph C. Hager, "Facial Action Coding System: the manual on CD ROM", Salt Lake City, A Human Face, 2002.
  21. P. Ekman, E. Rosenberg, and J. Hager, Facial action coding system affect interpretation dictionary (facsaid), 1998 (cit. on pp. 6, 29, 38, 55).
  22. B. Ryu, et al., "Local directional ternary pattern for facial expression recognition", IEEE Trans. on Image Processing, Vol. 26, No. 12, pp. 6006-6018, 2017. https://doi.org/10.1109/TIP.2017.2726010
  23. C. Cortes, and V. Vapnik, "Support-vector net works", Machine learning Vol. 20, No. 3, pp. 273-297, 1995. https://doi.org/10.1007/BF00994018
  24. Lucey, Patrick, et al., "The extended cohn-kanade dataset (ck+): A complete dataset for action unit and emotion-specified expression", Computer Vision and Pattern Recognition Workshops (CVPRW), 2010 IEEE Computer Society Conference on. IEEE, pp. 94-101, 2010.
  25. M. Valstar, and M. Pantic, "Induced disgust, happiness and surprise: an addition to the mmi facial expression database", Proc. 3rd Intern. Workshop on EMOTION (satellite of LREC): Corpora for Research on Emotion and Affect. pp. 65, 2010.
  26. S. W. Chew, et al., "Person-independent facial expression detection using constrained local models", Face and Gesture 2011. IEEE, pp. 915-920, 2011.
  27. L. A. Jeni, D. Takacs and A. Lorincz, "High quality facial expression recognition in video streams using shape related information only", 2011 IEEE International Conference on Computer Vision Workshops (ICCV Workshops), IEEE, pp. 2168-2174, 2011.
  28. S. Yang and B. Bhanu, "Understanding discrete facial expressions in video using an emotion avatar image", IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics), Vol. 42, No. 4, pp. 980-992, 2012. https://doi.org/10.1109/TSMCB.2012.2192269
  29. S. Lee, K. N. K. Plataniotis and Y. Ro, "Intra-class variation reduction using training expression images for sparse representation based facial expression recognition", IEEE Transactions on Affective Computing, Vol. 5, No. 3, pp. 340-351, 2014. https://doi.org/10.1109/TAFFC.2014.2346515
  30. A. Mollahosseini, D. Chan and M. H. Mahoor, "Going deeper in facial expression recognition using deep neural networks", 2016 IEEE winter conference on applications of computer vision (WACV), IEEE, pp. 1-10, 2016.
  31. M. S. Bartlett, et al., "Real Time Face Detection and Facial Expression Recognition: Development and Applications to Human Computer Interaction", 2003 Computer Vision and Pattern Recognition Workshop, Conference on, IEEE, Vol. 5, pp. 53-53, 2003.
  32. L. Zhong, et al., "Learning active facial patches for expression analysis", 2012 IEEE Conference on Computer Vision and Pattern Recognition, IEEE, pp. 2562-2569, 2012.
  33. J. Wang and L. Yin, "Static topographic modeling for facial expression recognition and analysis", Computer Vision and Image Understanding, Vol. 108, No.1-2, pp. 19-34, 2007. https://doi.org/10.1016/j.cviu.2006.10.011
  34. P. Burkert, et al., "Dexpression: Deep convolutional neural network for expression recognition", arXiv preprint arXiv:1509.05371, 2015.