Publisher : The Korean Institute of Electrical Engineers
DOI : 10.5370/JEET.2016.11.4.1003
Title & Authors
Hole Filling Algorithm for a Virtual-viewpoint Image by Using a Modified Exemplar Based In-painting Ko, Min Soo; Yoo, Jisang;
In this paper, a new algorithm by using 3D warping technique to effectively fill holes that are produced when creating a virtual-viewpoint image is proposed. A hole is defined as the region that cannot be seen in the reference view when a virtual view is created. In the proposed algorithm, to reduce the blurring effect that occurs on the hole region filled by conventional algorithms and to enhance the texture quality of the generated virtual view, Exemplar Based In-painting algorithm is used. The boundary noise which occurs in the initial virtual view obtained by 3D warping is also removed. After 3D warping, we estimate the relative location of the background to the holes and then pixels adjacent to the background are filled in priority to get better result by not using only adjacent object's information. Also, the temporal inconsistency between frames can be reduced by expanding the search region up to the previous frame when searching for most similar patch. The superiority of the proposed algorithm compared to the existing algorithms can be shown through the experimental results.
Y. Mori, N. Fukushima, T. Yendoa, T. Fujii and M. Tanimotoa, “View generation with 3D warping using depth information for FTV”, ELSEVIER, Signal Processing: Image Communication, vol. 24, issue. 1-2, pp. 65-72, Jan. 2009.
Z. Tauber, Ze-Nian Li, and M.S. Drew, “Review and preview : disocclusion by inpainting for image-based rendering,” IEEE Transactions on Systems, Man, and Cybernetics, Part C: Applications and Reviews, vol. 37, no. 4, pp. 527-540, Jul. 2007.
S.-H. Park, H. Song, E.-Y. Jang, N. Hur, J. Kim, J.- S. Kim, S.-H. Lee, and J. Yoo, “Compensation method for occluded-region of arbitrary-view image synthesized from multi-view video”, The Journal of Korea Information and Communications Society, vol. 33, no. 12, pp. 1029-1038, Dec. 2008.
M. Bertalmio, G. Sapiro, V. Caselles and C. Ballester, “Image inpainting”, Proc. of 27th Conference Computer Graphics and Interactive Algorithms (ACM SIGGRAPH 2000), pp. 417-424, July 2000.
ISO/IEC JTC1/SC29/WG11 "Introduction to 3D video”, M9784, May 2008.
A. Telea, “An Image In-painting Algorithm Based on the Fast Marching Method”, Journal of Graphics Tools, vol. 9, no. 1, pp. 25-36, 2004.
A. Criminisi, P. Perez and K. Toyama, “Region filling and Object Removal by Exemplar-Based Image Inpainting”, IEEE Trans. on Image Processing, vol. 13, no. 9, pp. 1200-1212, Sept. 2004.
Y.-J. Kim, S. H. Lee and J.-I. Park, "A High-quality Occlusion Filling Method Using Image Inpainting", Journal of Broadcast Engineering, vol. 15, no. 1, pp. 3-13, Jan. 2010.
I. Daribo and H. Saito, “A Novel Inpainting-Based Layered Depth Video for 3DTV”, IEEE Transactions on Broadcasting, vol. 57, no. 2, pp. 533-541, June 2011.
M. S. Ko and J. Yoo, “Virtual View Generation by a New Hole Filling Algorithm”, The Journal of Electrical Engineering & Technology, vol. 9, no. 3, pp. 1023-1033, March 2014.
ISO/IEC JTC1/SC29/WG11 “Boundary noise removal and hole filling for VSRS 3.5 alpha”, M19992, March 2011.
ISO / IEC JTC1 / SC29 / WG11 "Implementation of Hole Filling Methods for VSRS 3.5. alpha", M20005, March 2011.