Journal of the Korea Institute of Information and Communication Engineering (한국정보통신학회논문지)

The Korea Institute of Information and Commucation Engineering (한국정보통신학회)
권호 표지
DOI QR코드

DOI QR Code

Illumination Compensation Algorithm based on Segmentation with Depth Information for Multi-view Image

깊이 정보를 이용한 영역분할 기반의 다시점 영상 조명보상 기법

  • Received : 2013.02.12
  • Accepted : 2013.03.01
  • Published : 2013.04.30
Download PDF
⟨ Previous Next ⟩

Abstract

In this paper, a new illumination compensation algorithm by segmentation with depth information is proposed to improve the coding efficiency of multi-view images. In the proposed algorithm, a reference image is first segmented into several layers where each layer is composed of objects with a similar depth value. Then we separate objects from each other even in the same layer by labeling each separate region in the layered image. Then, the labeled reference depth image is converted to the position of the distortion image view by using 3D warping algorithm. Finally, we apply an illumination compensation algorithm to each of matched regions in the converted reference view and distorted view. The occlusion regions that occur by 3D warping are also compensated by a global compensation method. Through experimental results, we are able to confirm that the proposed algorithm has better performance to improve coding efficiency.

본 논문에서는 영상 분할을 이용한 다시점 영상의 조명보상 기법을 제안한다. 제안하는 기법에서는 깊이 정보를 이용하여 일정 거리에 따라 참조 영상의 깊이 영상을 레이어로 분리한다. 분리된 레이어에서 서로 다른 객체를 분리하기 위하여 각 레이어에 레이블링 과정을 수행한다. 레이블링 된 참조 영상의 깊이 영상은 3D 워핑 기법을 통하여 왜곡 영상의 시점으로 변환되고 레이블링 된 영역을 찾아 히스토그램을 이용한 조명 보상을 각 영역에서 독립적으로 수행한다. 3D 워핑으로 발생하는 가려짐 영역은 전역적인 방법을 이용하여 보상하게 된다. 다양한 실험을 통해 제안하는 기법으로 조명보상 전처리를 수행한 다시점 영상의 부호화 효율이 향상되는 것을 확인할 수 있었다.

Keywords

References

  1. ISO/IEC JTC1/SC29/WG11, "Introduction to 3D video," M9784, May. 2008.
  2. M. Tanimoto, M. P. Tehrani, T. Fujii, and T. Yendo, "Free-viewpoint TV," IEEE Signal Processing Magazine, vol.28, pp.67-76, Jan. 2011. https://doi.org/10.1109/MSP.2010.939077
  3. ISO/IEC JTC1/SC29/WG11, "Applications and Requirments on FTV," N9466, Oct. 2007.
  4. Y. Mori, N. Fukushima, T. Yendo, T. Fujii, and M. Tanimoto, "View generation with 3D warping using depth information for FTV," Signal Processing : Image Communication, vol.24, pp.65-72, Jan. 2009. https://doi.org/10.1016/j.image.2008.10.013
  5. D. Scharstein and R. Szeliski, "A Taxonomy and Evaluation of Dense Two-Frame Stereo Correspondence Algorithms," IEEE Workshop on Stereo and Multi-Baseline Vision, pp.131-140, 2001.
  6. ITU-T RECOMMENDATION "H.264 Advanced Video Coding for Generic Audiovisual Services," May, 2003.
  7. http://mpeg.chiariglione.org/standards.htm
  8. Joint Video Team (JVT) ISO/IEC MPEG &ITU& T VCEG, "Joint draft 3 Multi-view video coding," JVT-V209-M, Jan. 2007.
  9. Y. Su, A. Vetro, and A. Smolic, "Common test conditions for multiview video coding," Doc. JVT-U211, [Online]. Available: http://ftp3.itu.ch/ av-arch/jvt-site/, 2005.
  10. ISO/IEC JTC1/SC29/WG11, "Call for proposals on 3D video coding technology," n12036, March 2011.
  11. ISO/IEC JTC1/SC29/WG11, "Test model for AVC-based 3D video coding v2.0," n12743, July 2012.
  12. ISO/IEC JTC1/SC29/WG11, "Test model under consideration for HEVC based 3D video coding v3.0," n12744, April 2012.
  13. S. Yea and A. Vetro, "View synthesis prediction for multiview video coding," Signal Processing : Image Communication, vol.24, pp.89-100, Jan. 2009. https://doi.org/10.1016/j.image.2008.10.007
  14. C. Lee, B.H. Choi, and Y.S. Ho, "Efficient multiview depth video coding using depth synthesis prediction," Optical Engineering, vol.50, no.7, pp.1-14, July 2011.
  15. H. Schwarz, C. Bartnik, S. Bosse, H. Brust, T. Hinz, H. Lakshman, D. Marpe, P. Merkle, K. Muller, H. Rhee, G. Tech, M. Winken, and T. Wiegand, "3D video coding using advanced prediction Depth modeling, and encoder control methods," Picture Coding Symposium, pp.1-4, May 2012.
  16. M. Gilge, "Motion estimation by scene adaptive block matching and illumination correction," Image Process. Algorithms and Techni., R. J. Moorhead and K. S. Pennington, Eds., Canada, vol. 1244, pp. 355-366, June 1990. https://doi.org/10.1117/12.19525
  17. K. Kamikura, H. Watanabe, H. Jozawa, H. Kotera, and S. Ichinose, "Global brightness-variation compensation for video coding," IEEE Trans. Circuits Syst. Video Technol., vol.8, no.8, pp.988-1000, Dec. 1998. https://doi.org/10.1109/76.736731
  18. J. Lopez, J. Kim, A. Ortega, and G. Chen, "Block-based illumination compensation and search techniques for multiview video coding," in Proceeding of Picture Coding Symp. (PCS), Dec. 2004.
  19. J. H. Kim, et al., "New coding tools for illumination and focus mismatch compensation in multiview video coding," IEEE Trans. on Circuits and Systems for Video Technology, vol.17, no.11, pp.1519-1534, Nov. 2007. https://doi.org/10.1109/TCSVT.2007.909976
  20. P. L. Lai, A. Ortega, P. Pandit, P. Yin, and C. Gomila, "Adaptive Reference Filtering for MVC," JVT-W065.doc, pp.21-27, San Jose, USA, April 2007.
  21. U. Fecker, M. Barkowsky, and A. Kaup, "Histogram-based pre-filtering for luminance and chrominance compensation of multi-view video," IEEE Trans. Circuits Syst. Video Technol., vol.18, no.9, pp.1258-1267, Sep. 2008. https://doi.org/10.1109/TCSVT.2008.926997
  22. U. Fecker, M. Barkowsky, and A. Kaup, "Timeconstant histogram matching for colour compensation of multi-view video sequences," in Proceeding of Picture Coding Symp. (PCS), Nov. 2007.
  23. 박성희, 유지상, "다시점 비디오의 색상 성분 보정을 위한 특징점 기반의 전처리 방법," 한국정보통신학회 논문지, 제15권, 제12호, pp.2527-2533, 2011. https://doi.org/10.6109/jkiice.2011.15.12.2527
  24. Dong-Seok Lee, Min-Soo Ko, Young-Ho Seo, Dong-Wook Kim, Jisang Yoo, "Illumination compensation for multi-view video based on layered histogram matching with depth information," optics communication, Vol.286, pp.74-84, Jan. 2013. https://doi.org/10.1016/j.optcom.2012.08.022
  25. 황선규, "영상처리 프로그래밍 by Visual C++", 한빛미디어, 2007.
  26. Y. Mori, N. Fukushima, T. Yendoa, T. Fujii and M. Tanimotoa, "View generation with 3D warping using depth information for FTV," ELSEVIER Signal Processing : Image Communication, vol. 24, no. 1-2, pp. 65-72, Jan. 2009. https://doi.org/10.1016/j.image.2008.10.013