모바일 증강 현실 및 항공사진을 이용한 건물의 3차원 모델링

Towards 3D Modeling of Buildings using Mobile Augmented Reality and Aerial Photographs

  • 김세환 (산타바바라 캘리포니아대학 전산과) ;
  • ;
  • 장재식 (삼성전자 정보통신연구소) ;
  • 이태희 (로스앤젤레스 캘리포니아대학 전산과) ;
  • Kim, Se-Hwan (Computer Science Department, University of California, Santa Barbara) ;
  • Ventura, Jonathan (Computer Science Department, University of California) ;
  • Chang, Jae-Sik (Telecommunication R&D Center, Samsung Electronics) ;
  • Lee, Tae-Hee (Computer Science Department, University of California, Los Angeles) ;
  • Hollerer, Tobias (Computer Science Department, University of California, Santa Barbara)
  • 발행 : 2009.03.25

초록

본 논문에서는 모바일 증강 현실 시스템 및 항공사진을 이용하여 건물의 부분적 3D 모델을 생성하고, 이를 비디오 영상과 비교하여 사용자의 위치를 실시간으로 추적하는 방법을 제안한다. 제안된 시스템은 미리 생성된 모델을 사용하는 대신, 시스템 동작 중에 사용자 뷰와 항공 뷰를 결합하여 3D 모델을 생성한다. 우선 GPS의 위치에 따라 데이터베이스로부터 검색된 항공사진과, 피치를 추정하는 관성 센서를 이용하여 사용자의 초기 자세를 계산한다. 그리고 그래프 컷을 이용하여 건물의 상단의 에지를 검출하고, 제안된 비용 함수를 최소화함으로써 하단의 에지와 모퉁이 위치를 찾는다. 실시간으로 사용자의 자세를 추적하기 위해, 사용자가 관촬 중인 건물의 에지 및 벽면에서의 특이점을 이용하여 추적을 수행한다. 본 논문에서는 최소 자승 추정법과 언센티트 칼만 필터를 사용하여 카메라 자세 추정 방법을 구현하고 비교하였다. 또한 두 방법에 대하 속도와 정확도를 비교하고, Anywhere Augmentation 시나리오에 대한 중요한 기본 구성 요소들로서 실험결과의 유용성을 보였다.

This paper presents an online partial 3D modeling methodology that uses a mobile augmented reality system and aerial photographs, and a tracking methodology that compares the 3D model with a video image. Instead of relying on models which are created in advance, the system generates a 3D model for a real building on the fly by combining frontal and aerial views. A user's initial pose is estimated using an aerial photograph, which is retrieved from a database according to the user's GPS coordinates, and an inertial sensor which measures pitch. We detect edges of the rooftop based on Graph cut, and find edges and a corner of the bottom by minimizing the proposed cost function. To track the user's position and orientation in real-time, feature-based tracking is carried out based on salient points on the edges and the sides of a building the user is keeping in view. We implemented camera pose estimators using both a least squares estimator and an unscented Kalman filter (UKF). We evaluated the speed and accuracy of both approaches, and we demonstrated the usefulness of our computations as important building blocks for an Anywhere Augmentation scenario.

키워드

참고문헌

  1. T. Hollerer, J. Wither, and S. DiVerdi, Anywhere Augmentation: Towards Mobile Augmented Reality in Unprepared Environments, G. Gartner, M.P. Peterson, and W. Cartwright (Eds.), Location Based Services and TeleCartography, Springer Verlag, 2007
  2. G. Reitmayr, and T. Drummond, 'Going out: Robust model-based tracking for outdoor augmented reality,' Proc. of IEEE ISMAR'06, pp. 109-118, 2006
  3. G. Simon, A. Fitzgibbon, and A. Zisserman, 'Markerless tracking using planar structures in the scene,' Proc. of ISAR'00, pp. 120-128, 2000
  4. E. Rosten, and T. Drummond, T., 'Fusing points and lines for high performance tracking,' Proc. of ICCV'05, pp. 1508-1511, 2005
  5. A. Davison, W. Mayol, and D. Murray, 'Real-time localisation and mapping with wearable active vision,' Proc. of ISMAR'03, 2003
  6. G. King, W. Piekarski, and B. Thomas, 'Arvino - outdoor augmented reality visualisation of viticulture gis data,' Proc. of IEEE ISMAR'05, pp. 52-55, 2005
  7. J. Wither, S. DiVerdi, and T. Hollerer, 'Using aerial photographs for improved mobile ar annotation,' Proc. of IEEE ISMAR'06, pp. 159-162, 2006
  8. S. Kim, S. DiVerdi, J. S. Chang, T. Kang, R. Iltis, and T. Hollerer, 'Implicit 3D Modeling and Tracking for Anywhere Augmentation,' VRST'07, pp. 19-27, 2007
  9. C. Rother, V. Kolmogorov, and A. Blake, 'grabcut: interactive foreground extraction using iterated graph cuts,' ACM Trans. Graph, 23(3), pp. 309-314, 2004 https://doi.org/10.1145/1015706.1015720
  10. Y. Boykov and M.-P. Jolly, 'Interactive graph cuts for optimal boundary & region segmentation of objects in n-d images,' IEEE ICCV'01, pp. 105-112, 2001
  11. Intel, OpenCV Library, http://www.intel.com/technology/computing/opencv/, 2007
  12. J. Hershberger and J. Snoeyink, 'Speeding up the douglas-peucker line simplification algorithm,' Spatial Data Handling. IGU Commission on GIS, pp. 134-143, 1992
  13. J. Shi, and C. Tomasi, 'Good features to track,' Proc. of CVPR'94, pp. 593-600, 1994
  14. B. Lucas, and T. Kanade, 'An iterative image registration technique with an application to stereo vision,' Proc. of the International Joint Conference on AI, pp. 674-679, 1981
  15. S. Haykin, Kalman Filtering and Neural Networks, John Wiley & Sons, Inc., 2001
  16. S. Julier, and J. Uhlmann, 'Unscented filtering and nonlinear estimation,' Proc. of the IEEE, vol. 92, pp. 401-422, 2004 https://doi.org/10.1109/JPROC.2003.823141
  17. T. Lee and T. Hollerer, 'Hybrid Feature Tracking and User Interaction for Markerless Augmented Reality,' Proc. of IEEE VR'08, pp. 145-152, 2008
  18. D. G. Lowe, 'Distinctive Image Features from Scale-Invariant Keypoints,' IJCV, vol. 60, no. 2, pp.91–110, 2004 https://doi.org/10.1023/B:VISI.0000029664.99615.94
  19. Z. Zhang, 'A flexible new technique for camera calibration,' IEEE Transactions on PAMI, 22, 11, pp. 1330-1334, 2008