Journal of the Korea Institute of Information and Communication Engineering (한국정보통신학회논문지)

The Korea Institute of Information and Commucation Engineering (한국정보통신학회)
권호 표지
DOI QR코드

DOI QR Code

Super-resolution Reconstruction Method for Plenoptic Images based on Reliability of Disparity

시차의 신뢰도를 이용한 플렌옵틱 영상의 초고해상도 복원 방법

  • Jeong, Min-Chang (Graduate School of Information and Communication Engineering, Chungbuk National University) ;
  • Kim, Song-Ran (Graduate School of Information and Communication Engineering, Chungbuk National University) ;
  • Kang, Hyun-Soo (Graduate School of Information and Communication Engineering, Chungbuk National University)
  • Received : 2017.11.02
  • Accepted : 2018.01.11
  • Published : 2018.03.28
Download PDF
⟨ Previous Next ⟩

Abstract

In this paper, we propose a super-resolution reconstruction algorithm for plenoptic images based on the reliability of disparity. The subperture image generated by the Flanoptic camera image is used for disparity estimation and reconstruction of super-resolution image based on TV_L1 algorithm. In particular, the proposed image reconstruction method is effective in the boundary region where disparity may be relatively inaccurate. The determination of reliability of disparity vector is based on the upper, lower, left and right positional relationship of the sub-aperture image. In our method, the unreliable vectors are excluded in reconstruction. The performance of the proposed method was evaluated by comparing to a bicubic interpolation method, a conventional disparity based method and dictionary based method. The experimental results show that the proposed method provides the best performance in terms of PSNR(Peak Signal to noise ratio), SSIM(Structural Similarity).

본 논문에서는 시차의 신뢰도를 기반으로 플렌옵틱 영상의 초고해상도 복원 알고리즘을 제안한다. 그리고 플렌옵틱 카메라 영상으로부터 생성한 서브어퍼처(sub-aperture) 이미지는 TV_L1알고리즘에 기반한 시차 추정과 초고해상도 영상 복원에 활용된다. 특히 제안된 알고리즘은 시차가 부정확하게 나타날 수 있는 경계 역역에서 향상된 성능을 보인다. 시차 벡터의 신뢰도는 서브어퍼처 이미지의 상하좌우 각 위치별 영역에 따른 분산을 고려하여 판단한다. 신뢰도가 낮은 시차벡터는 초고해상도 영상 복원시 제외된다. 제안된 방법은 바이큐빅 보간 방법과 기존의 시차기반방법 그리고 사전기반 방법과 비교하여 평가되었다. 성능 평가에서 초고해상도 영상복원의 결과는 PSNR, SSIM 관점에서 성능을 비교하여 최상의 성능을 보여준다.

Keywords

References

  1. I. H. Choe, H. J. Lee, and B. C. Song. "Super Resolution Image Reconstruction Technique for Digital video devices," The Magazine of the IEIE, vol. 39, no. 10, pp. 36-44, Oct. 2012.
  2. J. Sanchez, E. M-Llopis, and G. Facciolo, "TV-L1 optical flow estimation," IPOL(Image Processing On Line), vol. 3, pp. 137-150, July. 2013. https://doi.org/10.5201/ipol.2013.26
  3. L. I. Rudin, S. Osher, and E. Fatemi. "Nonlinear total variation based noise removal algorithms," Physica D, vol. 60, pp. 259-268, Nov. 1992. https://doi.org/10.1016/0167-2789(92)90242-F
  4. A. Chambolle, "An Algorithm for Total Variation Minimization and Applications," Journal of Mathematical Imaging and Vision, vol. 20, no. 1-2, pp. 89-97, Jan. 2004. https://doi.org/10.1023/B:JMIV.0000011321.19549.88
  5. Sven Wanner and Bastian Goldluecke, "Variational Light Field Analysis for Disparity Estimation and Super-Resolution," IEEE Transactions of Pattern Analysis and Machine Intelligence, vol. 36, no. 3, pp. 606-619, March. 2014. https://doi.org/10.1109/TPAMI.2013.147
  6. D. Dansereau, O. Pizarro, and S. Williams. "Decoding, calibration and rectification for lenselet-based plenoptic cameras," in Proceeding of IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1027-1034, June 2013.
  7. S. R. Kim, M. C. Jeong, H. S. Kang, "Calibration Method of Plenoptic Camera using CCD Camera Model", in Proceedings of The Korea Institute of Information and Communication Engineering, Journal of the Korea Institute of Information and Communication Engineering, vol. 22, no. 2, pp. 261-269, Feb. 2018.
  8. D. H. Kim, "SVD-based Image Enhancement Method using Weighted Average of Histogram Stretching and Equalization," in Proceeding of Asia-pacific Journal of Multimedia Services Convergent with Art, Humanities, and Sociology, vol.5, no.5, pp. 77-85, Oct. 2015
  9. J. Yang, J. Wright, T. Huang, and Y. Ma, "Image superresolution via sparse representation," IEEE Trans. Image Processing, vol. 19, no. 11, pp. 2861-2873, Nov. 2010. https://doi.org/10.1109/TIP.2010.2050625
  10. R. Zeyde, M. Elad, and M. Protter, "On single image scale-up using sparse-representations," Lecture Note in Computer Science, vol. 6920, pp. 711-730, 2012.
  11. R. Timofte, V. De, and L. Van Gool, "Anchored neighborhood regression for fast example-based superresolution," in Proceeding of IEEE International Conference Computer Vision, pp. 1920-1927, Dec. 2013.
  12. J. Jiang, X. Ma, C. Chen, L. Tao, Z. Wang, and J. Ma, "Single Image Super-Resolution via Locally Regularized Anchored Neighborhood Regression and Nonlocal Means," IEEE Trans. on Multimedia, vol. 19, no. 1, pp. 15-26, Aug. 2016. https://doi.org/10.1109/TMM.2016.2599145