Journal of the Korean Institute of Intelligent Systems (한국지능시스템학회논문지)

Korean Institute of Intelligent Systems (한국지능시스템학회)
권호 표지
DOI QR코드

DOI QR Code

Depth-Based Recognition System for Continuous Human Action Using Motion History Image and Histogram of Oriented Gradient with Spotter Model

모션 히스토리 영상 및 기울기 방향성 히스토그램과 적출 모델을 사용한 깊이 정보 기반의 연속적인 사람 행동 인식 시스템

  • Eum, Hyukmin (ADAS Camera Team, LG Electronics) ;
  • Lee, Heejin (Department of Electrical, Electronic and Control Engineering, Hankyong National University) ;
  • Yoon, Changyong (Department of Electrical Engineering, Suwon Science College)
  • 음혁민 (LG전자 ADAS카메라팀) ;
  • 이희진 (한경대학교 전기전자제어공학과) ;
  • 윤창용 (수원과학대학교 전기과)
  • Received : 2016.11.22
  • Accepted : 2016.12.19
  • Published : 2016.12.25
Download PDF
⟨ Previous Next ⟩

Abstract

In this paper, recognition system for continuous human action is explained by using motion history image and histogram of oriented gradient with spotter model based on depth information, and the spotter model which performs action spotting is proposed to improve recognition performance in the recognition system. The steps of this system are composed of pre-processing, human action and spotter modeling and continuous human action recognition. In pre-processing process, Depth-MHI-HOG is used to extract space-time template-based features after image segmentation, and human action and spotter modeling generates sequence by using the extracted feature. Human action models which are appropriate for each of defined action and a proposed spotter model are created by using these generated sequences and the hidden markov model. Continuous human action recognition performs action spotting to segment meaningful action and meaningless action by the spotter model in continuous action sequence, and continuously recognizes human action comparing probability values of model for meaningful action sequence. Experimental results demonstrate that the proposed model efficiently improves recognition performance in continuous action recognition system.

본 논문은 깊이 정보를 기반으로 모션 히스토리 영상 및 기울기 방향성 히스토그램과 적출 모델을 사용하여 연속적인 사람 행동들을 인식하는 시스템을 설명하고 연속적인 행동 인식 시스템에서 인식 성능을 개선하기 위해 행동 적출을 수행하는 적출 모델을 제안한다. 본 시스템의 구성은 전처리 과정, 사람 행동 및 적출 모델링 그리고 연속적인 사람 행동 인식으로 이루어져 있다. 전처리 과정에서는 영상 분할과 시공간 템플릿 기반의 특징을 추출하기 위하여 Depth-MHI-HOG 방법을 사용하였으며, 추출된 특징들은 사람 행동 및 적출 모델링 과정을 통해 시퀀스들로 생성된다. 이 생성된 시퀀스들과 은닉 마르코프 모델을 사용하여 정의된 각각의 행동에 적합한 사람 행동 모델과 제안된 적출 모델을 생성한다. 연속적인 사람 행동 인식은 연속적인 행동 시퀀스에서 적출 모델에 의해 의미 있는 행동과 의미 없는 행동을 분할하는 행동 적출과 의미 있는 행동 시퀀스에 대한 모델의 확률 값들을 비교하여 연속적으로 사람 행동들을 인식한다. 실험 결과를 통해 제안된 모델이 연속적인 행동 인식 시스템에서 인식 성능을 효과적으로 개선하는 것을 검증한다.

Keywords

References

  1. Z. Ren, J. Yuan, J. Meng, and Z. Zhang, "Robust part-based hand gesture recognition using kinect sensor," IEEE Trans. Multimed, vol. 15, no. 5, pp. 1110-1120, 2013. https://doi.org/10.1109/TMM.2013.2246148
  2. D. Cazzato, M. Leo, and C. Distante, "An Investigation on the Feasibility of Uncalibrated and Unconstrained Gaze Tracking for Human Assistive Applications by Using Head Pose Estimation," Sensors, vol. 14, no. 5, pp. 8363-8379, 2014. https://doi.org/10.3390/s140508363
  3. R. Poppe, "A survey on vision-based human action recognition,"Image and Vision computing, vol. 28, no. 6, pp. 976-990, 2010. https://doi.org/10.1016/j.imavis.2009.11.014
  4. O. D. Lara and M. A. Labrador, "A survey on human activity recognition using wearable sensors," Communications Surveys & Tutorials, IEEE, vol. 15, no. 3, pp. 1192-1209, 2013. https://doi.org/10.1109/SURV.2012.110112.00192
  5. M. Elmezain, A. Al-Hamadi, and B. Michaelis, "Hand trajectory-based gesture spotting and recognition using HMM," in Image Processing (ICIP), 2009 16th IEEE International Conference on, pp. 3577-3580, 2009.
  6. M. Elmezain, A. Al-Hamadi, and B. Michaelis, "Real-time capable system for hand gesture recognition using hidden markov models in stereo color image sequences," 2008.
  7. H. Eum, C. Yoon, H. Lee, and M. Park, "Continuous Human Action Recognition Using Depth-MHI-HOG and a Spotter Model," Sensors, vol. 15, no. 3, pp. 5197-5227, 2015. https://doi.org/10.3390/s150305197
  8. H. Eum, J. Bae, C. Yoon, and E. kim, "Ship Detection Using Edge-Based Segmentation and Histogram of Oriented Gradient with Ship Size Ratio," in International Journal of Fuzzy Logic and Intelligent Systems, vol. 15, no. 4, pp. 251-259, 2015. https://doi.org/10.5391/IJFIS.2015.15.4.251
  9. S-J. Kim, S-W. Kim, and J-Y. Choi, "Recognizing the Direction of Action using Generalized 4D Features," in Journal of Korean Institute of Intelligent Systems, vol. 24, no. 5, pp. 518-528, 2014. https://doi.org/10.5391/JKIIS.2014.24.5.518
  10. Y-H. Cho, "Shape Image Recognition by Using Histogram-based Correlation," in Journal of Korean Institute of Intelligent Systems, vol. 20, no. 4, pp. 548-553, 2010. https://doi.org/10.5391/JKIIS.2010.20.4.548
  11. J-H. Park, and J-K. Bae, "The Improved Evolutionary Programming with Direction Vectors," in Journal of Korean Institute of Intelligent Systems, vol. 10, no. 6, pp. 542-547, 2000.
  12. K-K. Kim, J-H. Kim, and H-J. Park, "A Segmentation-Based HMM and MLP Hybrid Classifier for English Legal Word Recognition," in Journal of Korean Institute of Intelligent Systems, vol. 11, no. 3, pp. 200-207, 2001.
  13. C. Ordonez, "Clustering binary data streams with K-means," in Proceedings of the 8th ACM SIGMOD workshop on Research issues in data mining and knowledge discovery, San Diego, pp. 12-19, 2003.
  14. D. Kim, J. Song, and D. Kim, "Simultaneous gesture segmentation and recognition based on forward spotting accumulative HMMs,"Pattern Recognition, vol. 40, no. 11, pp. 3012-3026, 2007. https://doi.org/10.1016/j.patcog.2007.02.010
  15. D. Choi, S-W. Bang, M. Min, and J-H. Lee, "The Weight-based k-NN Classification Algorithm with Feature Selection", Proceedings of Korean Institute of Intelligent Systems, pp. 255-256, 2010.