Hand Gesture Interface for Manipulating 3D Objects in Augmented Reality

증강현실에서 3D 객체 조작을 위한 손동작 인터페이스

  • 박건희 (전남대학교 전자컴퓨터공학과) ;
  • 이귀상 (전남대학교 전자컴퓨터공학과)
  • Received : 2009.12.30
  • Accepted : 2010.04.27
  • Published : 2010.05.28


In this paper, we propose a hand gesture interface for the manipulation of augmented objects in 3D space using a camera. Generally a marker is used for the detection of 3D movement in 2D images. However marker based system has obvious defects since markers are always to be included in the image or we need additional equipments for controling objects, which results in reduced immersion. To overcome this problem, we replace marker by planar hand shape by estimating the hand pose. Kalman filter is for robust tracking of the hand shape. The experimental result indicates the feasibility of the proposed algorithm for hand based AR interfaces.


Augmented Reality;Hand Gesture Interface;Human Computer Interaction


  1. P. Milgram and A. F. Kishino, “A Taxonomy of Mixed Reality Visual Displays,” IEICE Transactions on Information and Systems, Vol.E77-D, No.12, pp.1321-1329, 1994.
  2. R. Azuma, “A Survey of Augmented Reality Presence,” Teleoperators and Virtual Environments, Vol.6, No.4, pp.355-385, 1997.
  3. F. Zhou, H. B. Duh, and M. Billinghurst, "Trends in augmented reality tracking, interaction and display: A review of ten years of ISMAR," IEEE/ACM International Symposium on Mixed and Augmented Reality, pp.193-202, 2008.
  4. M. Fiala, "ARTag, a fiducial marker system using digital techniques," IEEE Computer Society Conference on Computer Vision and Pattern Recognition(CVPR'05), Vol.2, pp.590-596, 2005.
  5. H. Kato and M. Billinghurst, "Marker tracking and HMD Calibration for a Video-based Augmented Reality Conferencing System,"Proceedings International Workshop on Augmented Reality(IWAR'99), pp.85-94, 1999.
  6. H. Ishii and B. Ullmer, “Tangible bits: towards seamless interfaces between people, bits and atoms,” Conference on Human Factors in Computing System(CHI'97), Vol.22, No.27, pp.234-241, 1997.
  7. H. Kato, M. Billinghurst, I. Poupyrev, K. Imamoto, and K. Tachibana. "Virtual object manipulation on a table-top AR environment,"IEEE and ACM International Symposium on Augmented Reality(ISAR'00), pp.111-119, 2000.
  8. Z. Zhou, A. David Cheok, J. Pan, and Y. Li,"Magic Story Cube: an interactive tangible interface for storytelling," ACM SIGCHI International Conference on Advances in computer entertainment technology, Vol.74, pp.364-365, 2004.
  9. J. Lee, T. Ha, J. Ryu, and W. Woo,"Development of Pen-type Haptic User Interface and Haptic Effect Design for Digilog Book Authoring," Korea Human Computer Interaction Conference, pp.402-405, 2009.
  10. T. Lee and T. Holerer, "Handy AR:Markerless Inspection of Augmented Reality Objects Using Fingertip Tracking", IEEE International Symposium on Wearable Computers, pp.241-242, 2007.
  11. J. Joseph and LaViola, "A Survey of Hand Posture and Gesture Recognition Techniques and Technology," Technical Report CS-99-11, Brown University, 1999.
  12. Q. Chen, N. D. Georganas, and E. M. Petriu, "Real-time Vision based Hand Gesture Recognition Using Haar-like features," IEEE Instrumentation and Measurement Technology Conference, pp.1-6, 2007.
  13. 김기영, 박영민, 백운혁, 우운택, “미니어처 AR:증강 현실 기반 차세대 디지로그형 콘텐츠 체험전시 시스템”, 한국차세대컴퓨팅학회논문지, 제5권, 제2호, pp.35-41, 2009.
  14. A. J. Heap and D. C. Hogg, "Towards 3-D hand tracking using a deformable model,"International Face and Gesture Recognition Conference, pp.140-145, 1996.
  15. C. C. Wang and K. C. Wang, "Hand Posture recognition using Adaboost with SIFT for human robot interaction," Springer Lecture Notes in Control and Information Sciences, Vol.370, pp.317-329. 2008.
  16. J. Segen and S. Kumar, "Shadow Gestures: 3D Hand Pose Estimation Using a Single Camera,"IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'99), Vol.1, pp.1479-1485, 1999.
  17. R. Lockton and A. W. Fitzgibbon, "Real-time gesture recognition using deterministic boosting," British Machine Vision Conference, Vol.2, pp.817-826, 2002.
  18. A. A. Argyros and M. I. A. Lourakis, "Vision-Based Interpretation of Hand Gestures for Remote Control of a Computer Mouse,"Lecture Notes in Computer Science, Vol.3979, pp.40-51, 2006.
  19. N. Liu and B. C. Lovell, "Hand Gesture Extraction by Active Shape Models," Digital Image Computing: Techniques and Applications pp.10-15, 2005.
  20. 박건희, 이귀상, “피부색을 이용한 공간에서의 손끝 추적 연구”, 한국컴퓨터 정보학회 하계학술대회, 제17권, 제1호, pp.59-62, 2009.
  21. M. I. Shamos, “Computational Geometry,” Ph.D Thesis, Yale University, 1978.