DOI QR코드

DOI QR Code

A Study on the Gesture Based Virtual Object Manipulation Method in Multi-Mixed Reality

  • Received : 2021.02.01
  • Accepted : 2021.02.25
  • Published : 2021.02.26

Abstract

In this paper, We propose a study on the construction of an environment for collaboration in mixed reality and a method for working with wearable IoT devices. Mixed reality is a mixed form of virtual reality and augmented reality. We can view objects in the real and virtual world at the same time. And unlike VR, MR HMD does not occur the motion sickness. It is using a wireless and attracting attention as a technology to be applied in industrial fields. Myo wearable device is a device that enables arm rotation tracking and hand gesture recognition by using a triaxial sensor, an EMG sensor, and an acceleration sensor. Although various studies related to MR are being progressed, discussions on developing an environment in which multiple people can participate in mixed reality and manipulating virtual objects with their own hands are insufficient. In this paper, We propose a method of constructing an environment where collaboration is possible and an interaction method for smooth interaction in order to apply mixed reality in real industrial fields. As a result, two people could participate in the mixed reality environment at the same time to share a unified object for the object, and created an environment where each person could interact with the Myo wearable interface equipment.

본 논문에서 혼합현실에서 공동 작업을 위한 환경 구축에 대한 연구와 웨어러블 IOT 디바이스와 연동하기 위한 방법을 제안한다. 혼합현실은 가상현실과 증강현실의 혼합되어진 형태로서 실세계와 가상세계의 물체가 동시에 보이고, 가상현실과 달리 멀미 현상이 일어나지 않고 무선 방식이라 산업현장에서 적용될 기술로 주목 받고 있다. 마이오 웨어러블 디바이스는 삼축 센서, 근전도 센서, 가속도 센서를 이용하여 팔의 회전 트래킹과 손 제스처 인식을 가능하게 하는 디바이스이다. 혼합현실 관련 다양한 연구가 진행되고 있지만 여러 사람이 함께 혼합 현실 참여할 수 있는 환경 구축, 가상 객체를 직접 손으로 조작하는 것에 대한 논의는 미비한 상황이다. 본 논문에서는 실제 산업 현장에서 혼합 현실을 적용하기 위해 공동 작업이 가능한 환경 구축 방법에 대해 연구하였고, 상호작용이 원활하게 이루어지기 위한 인터랙션 방법에 대해 논의하였다. 그 결과 2인이 동시에 혼합현실 환경에 참여하여 객체에 대한 일원화된 객체를 공유할 수 있었고, 마이오 웨어러블 인터페이스 장비를 가지고 각자 인터랙션 할 수 있는 환경을 조성하였다.

Keywords

References

  1. Maximilian Spicher, Brian D.Hall, Michael Nebeling, "What is Mixed Reality?," CHI Conference, No.537, pp.1-15, May 2019, DOI:10.1145/3290605.3300767
  2. Chunfern Xu, Yange Wang, Wei Quan, He Yang, "Multi-person Collaborative Interaction Algorithm and Application Based on Hololens," Springer AISC, Vol. 1006, pp 303-315, Oct. 2019
  3. Rafael Radkowski, "Hololens Integration into a Multi-Kinnect Tracking Environment," IEEE, April, 2019, DOI:10.1109/ISMAR-Adunct.2018.00052
  4. P.Hubner, M.Weinmann, S.Wursthorn, "MARKER-BASED LOCALIZATION OF THE MICROSOFT HOLOELENS IN BUILDING MODELS," Vol.XLII-1, pp.195-202, DOI:10.5194/isprs-archives-XLII-1-195-2018
  5. M.Ostanin, R.Yagfarov, A.Klimchik, "Interactive Robots Control Using Mixed Reality," IFAC, pp.695-700, 2019, DOI:10.1016/j.ifacol.2019.11.307
  6. Muun Wu, Yanbin Xu, Chenguang Yang, Ying Feng, "Omnidirectinal Mobile Robot Control based on Mixed Reality and sEMG Signals," IEEE Xplore, January 2019, DOI:10.1109/CAC.2018.8623114
  7. Mulun Wu, Shi-Lu Dai, and Chenguang Yang, "Mixed Reality Enhanced User Interaction Path Planning for Omnidirectional Mobile Robot," Applied Sciences, Vol, DOI:10.3390/app10031135
  8. Tobias Mulling, Mithileysh Sathiyanarayanan, "Characteristics of Hand Gesture Navigation: a case study using a wearable device(MYO)," British HCI'15:Proceedings of the 2015 British HCI Conference, pp.283-284, July 2015, DOI:10.1145/2783446.2783612
  9. Lee Stearns, Victor DeSouza, Jessica Yin, Leah Findlater, Jon E. Froehlich, "Augmented Reality Magnification for Low Vision Users with the Microsoft Hololens and a Finger-Worn Camera," ACM SGACCESS Conference, pp.361-362, Oct. 2017, DOI:10.1145/3132525.3134812
  10. Will Guest, Fridolin Wild, Alla Vovk, Mikhail Fominykh, Bibeg Limbu, Roland Klemke, Puneet Sharma, jaakko Karjalainen, Carl Smith, Jazz Rasool, Soyeb Aswat, Kajh Helin, Daniele Di Mitri, Jan Schneider, "Affordances for Capturing and Re-enacting Expert Performance with Wearables," Springer LNCS, Vol. 10474, pp. 403-409, Sep. 2017.
  11. Kdhong, "An Efficient Dynamic Workload Balancing Strategy," Journal of The Korea Society of Computer and Information, Vol. 15, No. 1, pp. 1-10, Nov. 2010. https://doi.org/10.9708/jksci.2010.15.1.001
  12. Kei Sato, Syunta Sato, "Pedestrian Navigation System for Visually Impaired People Using HoloLens and RFID," IEEE Xplore, 2017, DOI:10.1109/TAAI.2017.9