Advanced SearchSearch Tips
Hole-filling Algorithm Based on Extrapolating Spatial-Temporal Background Information for View Synthesis in Free Viewpoint Television
facebook(new window)  Pirnt(new window) E-mail(new window) Excel Download
  • Journal title : Journal of IKEEE
  • Volume 20, Issue 1,  2016, pp.31-44
  • Publisher : Institude of Korean Electrical and Electronics Engineers
  • DOI : 10.7471/ikeee.2016.20.1.031
 Title & Authors
Hole-filling Algorithm Based on Extrapolating Spatial-Temporal Background Information for View Synthesis in Free Viewpoint Television
Kim, Beomsu; Nguyen, Tien-Dat; Hong, Min-cheol;
  PDF(new window)
This paper presents a hole-filling algorithm based on extrapolating spatial-temporal background information used in view synthesis for free-viewpoint television. A new background codebook is constructed and updated in order to extract reliable temporal background information. In addition, an estimation of spatial local background values is conducted to discriminate an adaptive boundary between the background region and the foreground region as well as to update the information about the hole region. The holes then are filled by combining the spatial background information and the temporal background information. In addition, an exemplar-based inpainting technique is used to fill the rest of holes, in which a priority function using background-depth information is defined to determine the order in which the holes are filled. The experimental results demonstrated that the proposed algorithm outperformed the other comparative methods about average 0.3-0.6 dB, and that it synthesized satisfactory views regardless of video characteristics and type of hole region.
hole-filling;spatial-temporal background information;background codebook;priority function;DIBR;View Synthesis;
 Cited by
A. Smolic, P. Kauff, S. Knorr, A. Hornung, M. Kunter, M. Muller, and M. Lang, "Three-dimensional video postproduction and processing," Proc. of the IEEE, vol. 99, no. 4, pp. 607-625, Apr. 2011. crossref(new window)

N. A. Dodgson, "Autostereoscopic 3D displays", Computer, vol. 38, no. 8, pp.31-36, Aug. 2005. crossref(new window)

C. Fehn, "Depth-image-based rendering (DIBR), compression, and transmission for a new approach on 3D TV," SPIE Proc. Stereoscopic Image Proc. and Rendering, pp. 93-104, May 2004.

P.-J. Lee, and Effendi, "Nongeometric distortion smoothing approach for depth map preprocessing," IEEE Trans. Multimedia, vol. 13, no. 2, pp. 246-254, Apr. 2011. crossref(new window)

S. Zinger, D. Ruijters, L. Do and P. H. N. de With, "View interpolation for medical images on autostereoscopic displays," IEEE Trans. Circuits and Systems for Video Tech., vol. 21, no. 5-6, pp. 533-541, Jan. 2012.

S. Zinger, L. Do and P. H. N. de With, "Free-viewpoint depth image based rendering," J. of Visual Commu. and Image Representation, vol. 21, no. 5-6, pp. 533-541, Jul. 2012.

M. Solh and G. Alregib, "Hierarchical hole-filling for depth-based view synthesis in FTV and 3D video," IEEE J. of Selected Topics in Signal Processing, vol. 6, no. 5, pp. 495-504, Sep. 2012 crossref(new window)

B. Marcelo, S. Guillermo, C. Vincent and B. Coloma, "Image inpainting," The 27th Annual Conf. on Comput. Graphics and Interactive Techniques, pp. 417-424, 2000.

K. Oh, S. Yea and Y.-S. Ho, "Hole-filling method using depth based inpainting for view synthesis in free viewpoint television and 3D video," Picture Coding Symposium, pp. 1-4, June 2009.

I. Daribo, H. Saito, "A novel inpainting-based layered depth video for 3D TV," IEEE Trans. Broadcasting, vol. 57, no. 2, pp. 533-541, June 2011. crossref(new window)

A. Criminisi, P. Perez and K. Toyama, "Object removal by exemplar-based inpainting," IEEE Conf. on Computer Vision and Pattern Recognition, pp. 721-728, June 2003.

M. Koppel, X. Wang, D. Doshkov, T. Wiegand and P. Ndjiki-Nya, "Consitent spatio-temporal filling of disocclusions in the multiview video plus depth format," IEEE Int. Workshop on Multimedia Signal Process., pp. 25-30, Sept. 2012.

M. Xi, L.-H. Wang, Q.-Q. Yang, D.-X. Li and M. Zhang, "Depth image based rendering with spatial and temporal texture synthesis for 3DTV," EURASIP J. on Image and Video Process., vol. 2013, no. 1, DOI:10.1186/1687-5281-2013-9, Feb. 2013. crossref(new window)

C. Yao, T. Tillo, Y. Zhao, J. Xiao, H. Bai and C. Lin, "Depth map driven hole filling algorithm exploiting temporal correlation information," IEEE Trans. Broadcasting, vol. 60, no. 2, pp. 394-404, June 2014. crossref(new window)

K. Kim, T. H. Chalidabhongse, D. Harwood and L. Davis, "Background modeling and subtraction by codebook construction," IEEE Int. Conf. on Image Processing, vol. 5, pp. 3061-3064, Oct. 2004.

E. J. Fernandez-Sanchez, J.r Diaz and E. Ros, "Background subtraction based on color and depth using active sensors", Sensors, vol. 13, no. 7, pp. 8895-8915, July 2013. crossref(new window)

A. S. Glassner, Graphics Gems, Academic Press, Cambridge, MA, 1990.

Z. Wang, A. C. Bovik, H. R. Sheikh and E. P. Simoncelli, "Image quality assessment: From error visibility to structural similarity," IEEE Trans. Image Processing, vol. 13, no. 4, pp. 600-612, Apr. 2004. crossref(new window)