Multi-view Synthesis Algorithm for the Better Efficiency of Codec

부복호화기 효율을 고려한 다시점 영상 합성 기법

Choi, In-kyu;Cheong, Won-sik;Lee, Gwangsoon;Yoo, Jisang

  • Received : 2015.10.20
  • Accepted : 2015.12.01
  • Published : 2016.02.29


In this paper, when stereo image, satellite view and corresponding depth maps were used as the input data, we propose a new method that convert these data to data format suitable for compressing, and then by using these format, intermediate view is synthesized. In the transmitter depth maps are merged to a global depth map and satellite view are converted to residual image corresponding hole region as out of frame area and occlusion region. And these images subsampled to reduce a mount of data and stereo image of main view are encoded by HEVC codec and transmitted. In the receiver intermediate views between stereo image and between stereo image and bit-rate are synthesized using decoded global depth map, residual images and stereo image. Through experiments, we confirm good quality of intermediate views synthesized by proposed format subjectively and objectively in comparison to intermediate views synthesized by MVD format versus total bit-rate.


Format conversion;global depth map;residual view;view synthesis


  1. Z.-F. Wang and Z.-G. Zheng, "A region based stereo matching algorithm using cooperative optimization," in proceeding of the IEEE Conference on Computer Vision and Pattern Recognition, Anchorage, pp. 1-8, 2008.
  2. X. Mei, X. Sun, M. Zhou, S. Jiao, H. Wang and X. Zhang, "On building an accurate stereo matching system on graphics hardware," Computer Vision Workshops (ICCV Workshops), 2011 IEEE International Conference on, pp. 467-474, Nov. 2011.
  3. J. Liu, C. Li, F. Mei and Z. Wang, "3D entity-based stereo matching with ground control points and joint second-order smoothness prior," The Visual Computer, July 2014
  4. P.-J. Lee and Effendi, "Nongeometric Distortion Smoothing Approach for Depth Map Preprocessing," IEEE TRANSACTIONS ON MULTIMEDIA, vol. 13, no. 2, pp. 246-254, April 2011.
  5. C.-H. Wei, C.-K. Chiang and S.-H. Lai, "Iterative depth recovery for multi-view video synthesis from stereo videos," Asia-Pacific Signal and Information Processing Association, 2014 Annual Summit and Conference (APSIPA), pp. 1-8, Dec. 2014.
  6. I. Daribo, and H. Saito, "A Novel Inpainting-Based Layered Depth Video for 3DTV," IEEE TRANSACTIONS ON BROADCASTING, vol. 57, no. 2, pp. 533-541, June 2011.
  7. A. Criminisi, P. Perez and K. Toyama, "Region filling and object removal by exemplar-based image inpainting," IEEE TRANSACTIONS ON IMAGE PROCESSING, vol. 13, no. 9, pp. 1200-1212, Sept. 2004
  8. B. Bartczak, P. Vandewalle, O. Grau, G. Briand, J. Fournier, P. Kerbiriou, M. Murdoch, M. Muller, R. Goris, R. Koch and R. van der Vleuten, "Display-Independent 3D-TV Production and Delivery Using the Layered Depth Video Format," IEEE TRANSACTIONS ON BROADCASTING, vol. 57, no. 2, pp. 477-490, June 2011.
  9. T. Senoh, Y. Ichihashi, H. Sasaki and K. Yamamoto, "Simple Multi-View Coding with Depth Map", 3D Research, vol. 4, no.2, June 2013.
  10. R. C. Gonzalez and R. E. Woods, "Digital Image Processing," Pearson Education, Inc, 2010.
  11. M. Bertalmio, A. L. Bertozzi and G. Sapiro, "Navier-Stokes, Fluid Dynamics, and Image and Video Inpainting," Computer Vision and Pattern Recognition, 2001. CVPR 2001. Proceedings of the 2001 IEEE Computer Society Conference on, vol. 1, pp. 355-362, 2001.


Grant : Development of UHD Realistic Broadcasting, Digital Cinema, and Digital Signage Convergence Service Technology

Supported by : Institute for Information &communications Technology Promotion(IITP)