Journal of the Korea Academia-Industrial cooperation Society (한국산학기술학회논문지)

The Korea Academia-Industrial cooperation Society (한국산학기술학회)
권호 표지
DOI QR코드

DOI QR Code

Robust Hand Region Extraction Using a Joint-based Model

관절 기반의 모델을 활용한 강인한 손 영역 추출

  • 장석우 (안양대학교 소프트웨어학과) ;
  • 김설호 (숭실대학교 소프트웨어학부) ;
  • 김계영 (숭실대학교 소프트웨어학부)
  • Received : 2019.08.09
  • Accepted : 2019.09.06
  • Published : 2019.09.30
Download PDF
⟨ Previous Next ⟩

Abstract

Efforts to utilize human gestures to effectively implement a more natural and interactive interface between humans and computers have been ongoing in recent years. In this paper, we propose a new algorithm that accepts consecutive three-dimensional (3D) depth images, defines a hand model, and robustly extracts the human hand region based on six palm joints and 15 finger joints. Then, the 3D depth images are adaptively binarized to exclude non-interest areas, such as the background, and accurately extracts only the hand of the person, which is the area of interest. Experimental results show that the presented algorithm detects only the human hand region 2.4% more accurately than the existing method. The hand region extraction algorithm proposed in this paper is expected to be useful in various practical applications related to computer vision and image processing, such as gesture recognition, virtual reality implementation, 3D motion games, and sign recognition.

인간과 컴퓨터 사이의 보다 자연스러운 상호적인 인터페이스를 효과적으로 구현하기 위해서 사람의 제스처를 활용하려는 노력이 최근 들어 지속적으로 시도되고 있다. 본 논문에서는 연속적으로 입력되는 3차원의 깊이 영상을 받아들여서 손 모델을 정의하고, 정의된 손 모델을 기반으로 사람의 손 영역을 강인하게 추출하는 알고리즘을 제시한다. 본 논문에서 제시된 알고리즘에서는 먼저 21개의 관절을 사용하여 손 모델을 정의한다. 본 논문에서 정의한 손 모델은 6개의 손바닥 관절을 포함하는 손바닥 모델과 15개의 손가락 관절을 포함하는 손가락 모델로 구성된다. 그런 다음, 입력되는 3차원의 깊이 영상을 적응적으로 이진화함으로써, 배경과 같은 비관심 영역들은 제외하고, 관심 영역인 사람의 손 영역만을 정확하게 추출한다. 실험 결과에서는 제시된 알고리즘이 연속적으로 입력되는 깊이 영상으로부터 배경과 같은 영역들은 제외하고 사람의 손 영역만을 기존의 알고리즘에 비해 약 2.4% 보다 강인하게 검출한다는 것을 보여준다. 본 논문에서 제안된 손 영역 추출 알고리즘은 제스처 인식, 가상현실 구현, 3차원 운동 게임, 수화 인식 등과 같은 컴퓨터 비전 및 영상 처리와 관련된 여러 가지의 실제적인 분야에서 유용하게 활용될 것으로 기대된다.

Keywords

References

  1. T. G. Gonçalves, K. M. Oliveira, and C. Kolski, "Identifying HCI approaches to support CMMI-DEV for interactive system development," Computer Standards and Interfaces, Vol.58, pp.53-86, May 2018. DOI: https://doi.org/10.1016/j.csi.2017.12.003
  2. J. S. Kim and H. K. Bahn, "Analysis of Users' Gestures by Application in Smartphone Touch Interfaces" Journal of The Institute of Internet, Broadcasting and Communication, Vol.15, No.2, pp.9-14, 2015. DOI : http://dx.doi.org/10.7236/JIIBC.2015.15.2.9
  3. S. S. Kim, "Ranking Artificial Bee Colony for Design of Wireless Sensor Network", Journal of Society of Korea Industrial and Systems Engineering, Vol.42, No.1, pp.87-94, 2019. DOI : https://doi.org/10.11627/jkise.2019.42.1.087
  4. J. Y. Chang, S. M. Hong. D. M. Son, H. J. Yoo, and H. W. Ahn, "Development of Real-time Video Surveillance System Using the Intelligent Behavior Recognition Technique", Journal of The Institute of Internet, Broadcasting and Communication, Vol.19, No.2, pp.161-168, 2019. DOI : https://doi.org/10.7236/JIIBC.2019.19.2.161
  5. D. K. Shin, M. U. Ahmed, J. W. Kim, and P. K. Rhee, "Open set Object Detection combining Multi-branch Tree and ASSL", Journal of The Institute of Internet, Broadcasting and Communication, Vol.18, No.5, pp.171-177, 2018. DOI : https://doi.org/10.7236/JIIBC.2018.18.5.171
  6. T. Vuletic, A. Duffy, L. Hay, C. McTeague, and M. Grealy, "Systematic literature review of hand gestures used in human computer interaction interfaces," International Journal of Human-Computer Studies, Vol.129, pp.74-94, Sep. 2019. DOI: https://doi.org/10.1016/j.ijhcs.2019.03.011
  7. M. S. Jang and W. B. Lee, "Implementation of Hand-Gesture Interface to manipulate a 3D Object of Augmented Reality", Journal of The Institute of Internet, Broadcasting and Communication, Vol.16, No.4, pp.117-123, 2016. DOI : http://dx.doi.org/10.7236/JIIBC.2016.16.4.117
  8. J. S. Kim, J. Y. Jeong, and J. H. Lee, "Optimizing Work-In-Process Parameter using Genetic Algorithm", Journal of Society of Korea Industrial and Systems Engineering, Vol.40, No.1, pp.78-86, 2017. DOI : https://doi.org/10.11627/jkise.2017.40.1.079
  9. Q. Fan, X. Shen, Y. Hu, and C. Yu, "Simple very deep convolutional network for robust hand pose regression from a single depth image," Pattern Recognition Letters, Vol.119, pp.205-213, Mar. 2019. DOI: https://doi.org/10.1016/j.patrec.2017.10.019
  10. M. Imran and C. W. Kang, "A Synchronized Job Assignment Model for Manual Assembly Lines Using Multi-Objective Simulation Integrated Hybrid Genetic Algorithm (MO-SHGA)", Journal of Society of Korea Industrial and Systems Engineering, Vol.40, No.4, pp.211-11, 2017. DOI : https://doi.org/10.11627/jkise.2017.40.4.211
  11. D. J. Shin, J. H. Park, J. H. Kim, K. J. Kwak, J. M. Park, and J. J. Kim, "Big Data-based Sensor Data Processing and Analysis for IoT Environment", Journal of The Institute of Internet, Broadcasting and Communication, Vol.19, No.1, pp.117-126, 2019. DOI : https://doi.org/10.7236/JIIBC.2019.19.1.117
  12. A. T. S. Chan, H. V. Leong, and S. H. Kong, "Real-time Tracking of Hand Gestures for Interactive Game Design," Proceedings of the IEEE International Symposium on Industrial Electronics, Seoul, Korea, pp.98-103, Jul. 2009. DOI: 10.1109/ISIE.2009.5219910
  13. Q. D. Smedt, H. Wannous, and J.-P. Vandeborre, "Skeleton-based dynamic hand gesture recognition", Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops , Las Vegas, USA, Jun. 2016. DOI: 10.1109/CVPRW.2016.153
  14. S. Sridhar, F. Mueller, A. Oulasvirta, and C. Theobalt, "Fast and robust hand tracking using detection-guided optimization," Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Boston, USA, pp. 3213-3221, Jun. 2015. DOI: 10.1109/CVPR.2015.7298941
  15. X. Suau, M. Alcoverro, A. L. Mendez, J. R. Hidalgo, and J. R. Casas, "Real-time fingertip localization conditioned on hand gesture classification," Image and Vision Computing, Vol.32, No.8, pp.522-532, Aug. 2014. DOI: https://doi.org/10.1016/j.imavis.2014.04.015
  16. H. Tang, H. Liu, W. Xiao, and N. Sebe, "Fast and robust dynamic hand gesture recognition via key frames extraction and feature fusion," Neurocomputing, Vol.331, pp.424-433, Feb. 2019. DOI: https://doi.org/10.1016/j.neucom.2018.11.038
  17. A. Pujol-Miro, J. R. Casas, and J. Ruiz-Hidalgo, "Correspondence matching in unorganized 3D point clouds using convolutional neural networks," Image and Vision Computing, Vol.83, pp.51-60, Mar. 2019. DOI: https://doi.org/10.1016/j.imavis.2019.02.013
  18. Y. Ben-Shabat, T. Avraham, M. Lindenbaum, and A. Fischer, "Graph based over-segmentation methods for 3D point clouds," Computer Vision and Image Understanding, Vol.174, pp.12-23, Sep. 2018. DOI: https://doi.org/10.1016/j.cviu.2018.06.004