Journal of the Korea Institute of Information and Communication Engineering (한국정보통신학회논문지)

The Korea Institute of Information and Commucation Engineering (한국정보통신학회)
권호 표지
DOI QR코드

DOI QR Code

Multi-view Synthesis Algorithm for the Better Efficiency of Codec

부복호화기 효율을 고려한 다시점 영상 합성 기법

  • Choi, In-kyu (Department of Electronic Engineering, Kwangwoon University) ;
  • Cheong, Won-sik (Electronics and Telecommunications Research Institute) ;
  • Lee, Gwangsoon (Electronics and Telecommunications Research Institute) ;
  • Yoo, Jisang (Department of Electronic Engineering, Kwangwoon University)
  • Received : 2015.10.20
  • Accepted : 2015.12.01
  • Published : 2016.02.29
Download PDF
⟨ Previous Next ⟩

Abstract

In this paper, when stereo image, satellite view and corresponding depth maps were used as the input data, we propose a new method that convert these data to data format suitable for compressing, and then by using these format, intermediate view is synthesized. In the transmitter depth maps are merged to a global depth map and satellite view are converted to residual image corresponding hole region as out of frame area and occlusion region. And these images subsampled to reduce a mount of data and stereo image of main view are encoded by HEVC codec and transmitted. In the receiver intermediate views between stereo image and between stereo image and bit-rate are synthesized using decoded global depth map, residual images and stereo image. Through experiments, we confirm good quality of intermediate views synthesized by proposed format subjectively and objectively in comparison to intermediate views synthesized by MVD format versus total bit-rate.

본 논문에서는 스테레오 영상과 스테레오 영상 바깥 시점의 위성 영상 그리고 상응하는 깊이 영상이 입력으로 주어질 때 이 입력 영상들을 압축에 적합한 포맷으로 변환하고 활용하여 중간시점 영상을 합성하는 새로운 기법을 제안한다. 송출 단에서 깊이 영상들은 하나의 global 영상으로, 위성영상은 시점 이동 시에 발생하는 프레임 밖의 영역 및 가려짐 영역과 같은 홀 영역에만 해당하는 residual 영상으로 변환 후에 데이터 량을 줄이기 위하여 다운샘플링하고 주시점의 스테레오 영상과 더불어 HEVC 코덱으로 부호화 한 후 전송한다. 수신 단에서 복호화된 각 입력 영상들을 이용하여 스테레오 영상 간 그리고 스테레오 영상과 위성 영상간의 중간시점 영상을 합성한다. 실험을 통하여 제안한 포맷을 이용하여 합성한 중간 시점 영상을 MVD(Multiview Video plus Depth) 포맷을 이용하여 합성한 중간 시점 영상과 비교했을 때 전송하는 데이터의 비트율(bit-rate) 대비 객관적 그리고 주관적 화질이 뛰어남을 확인하였다.

Keywords

References

  1. Z.-F. Wang and Z.-G. Zheng, "A region based stereo matching algorithm using cooperative optimization," in proceeding of the IEEE Conference on Computer Vision and Pattern Recognition, Anchorage, pp. 1-8, 2008.
  2. X. Mei, X. Sun, M. Zhou, S. Jiao, H. Wang and X. Zhang, "On building an accurate stereo matching system on graphics hardware," Computer Vision Workshops (ICCV Workshops), 2011 IEEE International Conference on, pp. 467-474, Nov. 2011.
  3. J. Liu, C. Li, F. Mei and Z. Wang, "3D entity-based stereo matching with ground control points and joint second-order smoothness prior," The Visual Computer, July 2014
  4. P.-J. Lee and Effendi, "Nongeometric Distortion Smoothing Approach for Depth Map Preprocessing," IEEE TRANSACTIONS ON MULTIMEDIA, vol. 13, no. 2, pp. 246-254, April 2011. https://doi.org/10.1109/TMM.2010.2100372
  5. C.-H. Wei, C.-K. Chiang and S.-H. Lai, "Iterative depth recovery for multi-view video synthesis from stereo videos," Asia-Pacific Signal and Information Processing Association, 2014 Annual Summit and Conference (APSIPA), pp. 1-8, Dec. 2014.
  6. I. Daribo, and H. Saito, "A Novel Inpainting-Based Layered Depth Video for 3DTV," IEEE TRANSACTIONS ON BROADCASTING, vol. 57, no. 2, pp. 533-541, June 2011. https://doi.org/10.1109/TBC.2011.2125110
  7. A. Criminisi, P. Perez and K. Toyama, "Region filling and object removal by exemplar-based image inpainting," IEEE TRANSACTIONS ON IMAGE PROCESSING, vol. 13, no. 9, pp. 1200-1212, Sept. 2004 https://doi.org/10.1109/TIP.2004.833105
  8. B. Bartczak, P. Vandewalle, O. Grau, G. Briand, J. Fournier, P. Kerbiriou, M. Murdoch, M. Muller, R. Goris, R. Koch and R. van der Vleuten, "Display-Independent 3D-TV Production and Delivery Using the Layered Depth Video Format," IEEE TRANSACTIONS ON BROADCASTING, vol. 57, no. 2, pp. 477-490, June 2011. https://doi.org/10.1109/TBC.2011.2120790
  9. T. Senoh, Y. Ichihashi, H. Sasaki and K. Yamamoto, "Simple Multi-View Coding with Depth Map", 3D Research, vol. 4, no.2, June 2013.
  10. R. C. Gonzalez and R. E. Woods, "Digital Image Processing," Pearson Education, Inc, 2010.
  11. M. Bertalmio, A. L. Bertozzi and G. Sapiro, "Navier-Stokes, Fluid Dynamics, and Image and Video Inpainting," Computer Vision and Pattern Recognition, 2001. CVPR 2001. Proceedings of the 2001 IEEE Computer Society Conference on, vol. 1, pp. 355-362, 2001.