DOI QR코드

DOI QR Code

AR Anchor System Using Mobile Based 3D GNN Detection

  • Jeong, Chi-Seo (Graduate School of Smart Convergence, Kwangwoon University) ;
  • Kim, Jun-Sik (Department of Electronic Engineering, Kwangwoon University) ;
  • Kim, Dong-Kyun (Department of Electronic Engineering, Kwangwoon University) ;
  • Kwon, Soon-Chul (Graduate School of Smart Convergence, Kwangwoon University) ;
  • Jung, Kye-Dong (Ingenium College of liberal arts, Kwangwoon University)
  • Received : 2020.11.15
  • Accepted : 2020.11.23
  • Published : 2021.02.28

Abstract

AR (Augmented Reality) is a technology that provides virtual content to the real world and provides additional information to objects in real-time through 3D content. In the past, a high-performance device was required to experience AR, but it was possible to implement AR more easily by improving mobile performance and mounting various sensors such as ToF (Time-of-Flight). Also, the importance of mobile augmented reality is growing with the commercialization of high-speed wireless Internet such as 5G. Thus, this paper proposes a system that can provide AR services via GNN (Graph Neural Network) using cameras and sensors on mobile devices. ToF of mobile devices is used to capture depth maps. A 3D point cloud was created using RGB images to distinguish specific colors of objects. Point clouds created with RGB images and Depth Map perform downsampling for smooth communication between mobile and server. Point clouds sent to the server are used for 3D object detection. The detection process determines the class of objects and uses one point in the 3D bounding box as an anchor point. AR contents are provided through app and web through class and anchor of the detected object.

Keywords

References

  1. HONG, Dong-Pyo, et al. A Study on the Mobile Augmented Reality system trends. Communications of the Korean Institute of Information Scientists and Engineers, 2008, 26.1: 88-97.
  2. JEON, Jong-hong; LEE, Seung-yoon. The trend of standardization of mobile augmented reality technology. Electronic Communication Trend Analysis, 2011, 26.2: 61-74.
  3. LI, Xingdong, et al. Generating colored point cloud under the calibration between TOF and RGB cameras. In: 2013 IEEE International Conference on Information and Automation (ICIA). IEEE, 2013. p. 483-488. DOI: doi.org/10.1109/ICInfA.2013.6720347
  4. Jung, Tae-Won; Jeong, Chi-Seo, et al. Object Detection with LiDAR Point Cloud and RGBD Synthesis Using GNN. International Journal of Advanced Smart Convergence, 2020, 9.3 p. 192-98. DOI: doi.org/10.7236/IJASC.2020.9.3.192
  5. COUCLELIS, Helen, et al. Exploring the anchor-point hypothesis of spatial cognition. Journal of environmental psychology, 1987, 7.2: 99-122. https://doi.org/10.1016/S0272-4944(87)80020-8
  6. LEE, Yongjae, et al. Unified Representation for XR Content and its Rendering Method. In: The 25th International Conference on 3D Web Technology. 2020. p. 1-10. DOI: doi.org/10.1145/3424616.3424695
  7. LEE, Jaehyun, et al. A Study on Virtual Studio Application using Microsoft HoloLens. International journal of advanced smart convergence, 2017, 6.4 p. 80-87. DOI: doi.org/10.7236/IJASC.2017.6.4.12
  8. MACLNTYRE, Blair; SMITH, Trevor F. Thoughts on the Future of WebXR and the Immersive Web. In: 2018 IEEE International Symposium on Mixed and Augmented Reality Adjunct (ISMAR-Adjunct). IEEE, 2018. p. 338-342. DOI: doi.org/10.1109/ISMAR-Adjunct.2018.00099
  9. SHI, Shaoshuai, et al. Pv-rcnn: Point-voxel feature set abstraction for 3d object detection. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2020. p. 10529-10538. DOI: doi.org/10.1109/CVPR42600.2020.01054
  10. SHI, Weijing; RAJKUMAR, Raj. Point-gnn: Graph neural network for 3d object detection in a point cloud. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2020. p. 1711-1719. DOI: doi.org/10.1109/CVPR42600.2020.00178
  11. Jeong, Chi-Seo, et al. Deep Learning-Based 3D Object Recognition System for Mobile AR. In: 8th International Symposium on Advanced & Applied Convergence, 2020. p. 40-42.