Journal of the Institute of Electronics Engineers of Korea SP (대한전자공학회논문지SP)

The Institute of Electronics and Information Engineers (대한전자공학회)
권호 표지

ROI-Based 3D Video Stabilization Using Warping

관심영역 기반 와핑을 이용한 3D 동영상 안정화 기법

  • Received : 2011.06.21
  • Accepted : 2011.11.30
  • Published : 2012.03.25
Download PDF
⟨ Previous Next ⟩

Abstract

As the portable camcorder becomes popular, various video stabilization algorithms for de-shaking of camera motion have been developed. In the past, most video stabilization algorithms were based on 2-dimensional camera motion, but recent algorithms show much better performance by considering 3-dimensional camera motion. Among the previous video stabilization algorithms, 3D video stabilization algorithm using content-preserving warps is known as the state-of-the art owing to its superior performance. But, the major demerit of the algorithm is its high computational complexity. So, we present a computationally light full-frame warping algorithm based on ROI (region-of-interest) while providing comparable visual quality to the state-of-the art in terms of ROI. First, a proper ROI with a target depth is chosen for each frame, and full-frame warping based on the selected ROI is applied.

가정용 캠코더의 보급으로 인해서 손떨림이나 카메라 흔들림을 보상하기 위해 다양한 동영상 안정화 기법들이 연구되고 있다. 동영상 안정화 기법은 초기에 2차원 움직임만을 고려하였지만, 최근에는 3차원 움직임까지 고려하여서 더 좋은 성능을 얻을 수 있게 되었다. 이러한 기법들 중 가장 좋은 성능을 보이는 것으로 알려진 기법이 바로 content preserving warping을 이용한 기법인데 이것은 뛰어난 성능을 보이지만 방대한 연산량이 단점이다. 그래서, 우리는 ROI 측면에서 종래 기술 대비 동등한 화질을 보이면서도 연산량이 적은 full frame warping을 제안한다. 먼저, 목표로 하는 깊이 정보를 바탕으로 관심 영역을 설정하고, 설정된 관심 영역을 기반으로 full frame warping을 수행한다.

Keywords

References

  1. K. Ratakonda, "Real-time digital video stabilization for multi-media applications," IEEE International Symposium on Circuits and Systems, pp. 69-72, 1998.
  2. Y. M. Yeh, H. C. Chiang, and S. J. Wang, "A digital camcorder image stabilizer based on gray coded bit-plane block matching," IPPR Conf. Computer Vision, Graphics and Image Processing, Taipei, Taiwan, pp.244-251, 2000.
  3. Y. Matsushita, E. Ofek, W. Ge, X. Tang, and H. Y. Shum, "Full-frame video stabilization with motion inpainting," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 28, no. 7, pp. 1150-1163, 2006. https://doi.org/10.1109/TPAMI.2006.141
  4. S. Battiato, G. Gallo, G. Puglisi, and S. Scellato, "SIFT features tracking for video stabilization", ICIAP, pp. 825-830, 2007.
  5. F. Liu, M. Gleicher, H. Jin, and A. Agarwala, "Content-preserving warps for 3D video stabilization," ACM SIGGRAPH, New York, NY, USA, pp. 1-9, 2009.
  6. J. M. Wang, H. P. Chou, S. W. Chen, and C. S. Fuh, "Video stabilization for a hand-held camera based on 3d motion model," IEEE International Conference on Image Processing, pp. 3477-3480, 2009.
  7. G. Zhang, W. Hua, X. Qin, Y. Shao, and H. Bao. "Video stabilization based on a 3D perspective camera model," The Visual Computer.
  8. M. Alexa, D. Cohen-Or, and D. Levin, "As-rigid-as-possible shape interpolation," ACM SIGGRAPH, New Orleans, USA, pp. 157-164, 2000.
  9. T. Igarashi, T. Moscovich, and J. F. Hughes, "As-rigid-as-possible shape manipulation," ACM Trans. Graph., pp.1134-1141, 2005.
  10. R. Gal, O. Sorkine, and D. Cohen-Or, "Feature-aware texturing," Eurographics Symposium on Rendering, pp. 297-303, 2006.
  11. S. Schaefer, T. McPhail, and J.Warren, "Image deformation using moving least squares," ACM Trans. Graph., vol. 25, no. 3, pp. 533-540, 2006. https://doi.org/10.1145/1141911.1141920
  12. S. Avidan et al., "Seam carving for content-aware image resizing," ACM Trans. Graphics, vol. 26, no. 3, 2007.
  13. J. Lee and S. Y. Shin, "General construction of time-domain filters for orientation data," IEEE Trans. Visualization and Computer Graphics, vol. 8, no. 2, pp. 119-128, 2002. https://doi.org/10.1109/2945.998665
  14. R. I. Hartley and A. Zisserman, "Multiple view geometry in computer vision," Cambridge University Press, 2000.
  15. P. S. Heckbert, "Fundamentals of texture mapping and image warping", Master thesis, Computer Science Division, UC Berkeley, 1989.
  16. J. More, "The Levenberg-Marquardt algorithm, implementation, and theory", Springer-Verlag, numerical analysis edition, 1977.
  17. B. Triggs, P. McLauchlan, R. I. Hartley, and A. Fitzgibbon, "Bundle adjustment-a modern synthesis." Vision Algorithms: Theory and Practice, pp. 298-373, 1999.
  18. T. Jebara, A. Azarbayejani, and A. Pentland, "3D structure from 2D motion", IEEE Signal Processing Magazine, vol. 16, no. 3, pp. 66-84, 1999. https://doi.org/10.1109/79.768574