Current Optics and Photonics

Optical Society of Korea (한국광학회)
권호 표지
DOI QR코드

DOI QR Code

Identification and Correction of Microlens-array Error in an Integral-imaging-microscopy System

  • Imtiaz, Shariar Md (School of Information and Communication Engineering, Chungbuk National University) ;
  • Kwon, Ki-Chul (School of Information and Communication Engineering, Chungbuk National University) ;
  • Alam, Md. Shahinur (School of Information and Communication Engineering, Chungbuk National University) ;
  • Hossain, Md. Biddut (School of Information and Communication Engineering, Chungbuk National University) ;
  • Changsup, Nam (Department of Mechanical ICT Engineering, College of Future Convergence, Hoseo University) ;
  • Kim, Nam (School of Information and Communication Engineering, Chungbuk National University)
  • Received : 2021.07.06
  • Accepted : 2021.09.06
  • Published : 2021.10.25
Download PDF
⟨ Previous Next ⟩

Abstract

In an integral-imaging microscopy (IIM) system, a microlens array (MLA) is the primary optical element; however, surface errors impede the resolution of a raw image's details. Calibration is a major concern with regard to incorrect projection of the light rays. A ray-tracing-based calibration method for an IIM camera is proposed, to address four errors: MLA decentering, rotational, translational, and subimage-scaling errors. All of these parameters are evaluated using the reference image obtained from the ray-traced white image. The areas and center points of the microlens are estimated using an "8-connected" and a "center-of-gravity" method respectively. The proposed approach significantly improves the rectified-image quality and nonlinear image brightness for an IIM system. Numerical and optical experiments on multiple real objects demonstrate the robustness and effectiveness of our proposed method, which achieves on average a 35% improvement in brightness for an IIM raw image.

Keywords

Acknowledgement

This work was supported by the National Research Foundation of Korea (NRF) (NRF-2018R1D1A3B07044041, NRF-2020R1A2C1101258), and was supported under the Grand Information Technology Research Center support program (IITP-2020-0-01462) supervised by the IITP (Institute for Information & communications Technology Planning & Evaluation), grant funded by the Korean government.

References

  1. M. Levoy, "Light fields and computational imaging," Computer 39, 46-55 (2006). https://doi.org/10.1109/MC.2006.270
  2. K.-C. Kwon, K. H. Kwon, M.-U. Erdenebat, Y.-L. Piao, Y.-T. Lim, M. Y. Kim, and N. Kim, "Resolution-enhancement for an integral imaging microscopy using deep learning," IEEE Photonics J. 11, 6900512 (2019).
  3. M. S. Alam, K.-C. Kwon, M.-U. Erdenebat, M. Y. Abbass, A. Alam, and N. Kim, "Super-resolution enhancement method based on generative adversarial network for integral imaging microscopy," Sensors 21, 2164 (2021). https://doi.org/10.3390/s21062164
  4. M. Y. Abbass, K.-C. Kwon, M. S. Alam, Y.-L. Piao, K.-Y. Lee, and N. Kim, "Image super resolution based on residual dense CNN and guided filters," Multimed. Tools Appl. 80, 5403-5421 (2021). https://doi.org/10.1007/s11042-020-09824-3
  5. K.-C. Kwon, M.-U. Erdenebat, S. Alam, Y.-T. Lim, K. G. Kim, and N. Kim, "Integral imaging microscopy with enhanced depth-of-field using a spatial multiplexing," Opt. Express 24, 2072-2083 (2016). https://doi.org/10.1364/OE.24.002072
  6. C. Shin, H.-G. Jeon, Y. Yoon, I. S. Kweon, and S. J. Kim, "Epinet: A fully-convolutional neural network using epipolar geometry for depth from light field images," in Proc. IEEE Conference on Computer Vision and Pattern Recognition (Salt Lake City, UT, USA. Jun. 2018), pp. 4748-4757.
  7. K.-C. Kwon, K. H. Kwon, M.-U. Erdenebat, Y.-L. Piao, Y.-T. Lim, Y. Zhao, and M. Y. Kim, and N. Kim, "Advanced three-dimensional visualization system for an integral imaging microscope using a fully convolutional depth estimation network," IEEE Photonics J. 12, 3900714 (2020).
  8. T. Georgiev and C. Intwala, "Light field camera design for integral view photography," Tech. Rep. (Adobe Systems Incorporated, 2006).
  9. R. Ng, M. Levoy, M. Bredif, G. Duval, M. Horowitz, and P. Hanrahan, "Light field photography with a hand-held plenoptic camera," Tech. Rep. CTSR 2005-02 (Department of Computer Science, Stanford University, 2005).
  10. K.-C Kwon, J.-S. Jeong, M.-U. Erdenebat, Y.-T. Lim, K.-H. Yoo, and N. Kim, "Real-time interactive display for integral imaging microscopy," Appl. Opt. 53, 4450-4459 (2014). https://doi.org/10.1364/AO.53.004450
  11. S. Li, Y. Yuan, Z. Gao, and H. Tan, "High-accuracy correction of a microlens array for plenoptic imaging sensors," Sensors 19, 3922 (2019). https://doi.org/10.3390/s19183922
  12. T.-J. Li, S.-N. Li, S. Li, Y. Yuan, and H.-P. Tan, "Correction model for microlens array assembly error in light field camera," Opt. Express 24, 24524-24543 (2016). https://doi.org/10.1364/OE.24.024524
  13. S.-N. Li, Y. Yuan, B. Liu, F.-Q. Wang, and H.-P. Tan, "Influence of microlens array manufacturing errors on light-field imaging," Opt. Commun. 410, 40-52 (2018). https://doi.org/10.1016/j.optcom.2017.09.055
  14. S. Shi, J. Wang, J, Ding, Z. Zhao, and T. H. New, "Parametric study on light field volumetric particle image velocimetry," Flow Meas. Instrum. 49, 70-88 (2016). https://doi.org/10.1016/j.flowmeasinst.2016.05.006
  15. J. Zhao, Z. Liu, and B. Guo, "Three-dimensional digital image correlation method based on a light field camera," Opt. Lasers Eng. 116, 19-25 (2019). https://doi.org/10.1016/j.optlaseng.2018.12.008
  16. L. Su, Q. Yan, J. Cao, and Y. Yuan, "Calibrating the orientation between a microlens array and a sensor based on projective geometry," Opt. Lasers Eng. 82, 22-27 (2016). https://doi.org/10.1016/j.optlaseng.2016.01.018
  17. P. Suliga and T. Wrona, "Microlens array calibration method for a light field camera," in Proc. 19th International Carpathian Control Conference-ICCC (Szilvasvarad, Hungary, May. 2018), pp. 19-22.
  18. J. Jin, Y. Cao, W. Cai, W. Zheng, and P. Zhou, "An effective rectification method for lenselet-based plenoptic cameras," Proc. SPIE 10020, 100200F (2016).
  19. D. Cho, M. Lee, S. Kim, and Y.-W. Tai, "Modeling the calibration pipeline of the lytro camera for high quality light-field image reconstruction," in Proc. IEEE International Conference on Computer Vision (Sydney, Australia, Apr. 2013), pp. 3280-3287.
  20. Z. Zhao, M. Hui, M. Liu, L. Dong, X. Liu, and Y. Zhao, "Centroid shift analysis of microlens array detector in interference imaging system," Opt. Commun. 354, 132-139 (2015). https://doi.org/10.1016/j.optcom.2015.05.049
  21. X. Liu, X. Zhang, F. Fang, Z. Zeng, H. Gao, and X. Hu, "Influence of machining errors on form errors of microlens arrays in ultra-precision turning," Int. J. Mach. Tools Manuf. 96, 80-93 (2015). https://doi.org/10.1016/j.ijmachtools.2015.05.008
  22. V. Dembele, I. Choi, S. Kheiryzadehkhanghah, S. Choi, J. Kim, C. S. Kim, and D. Kim, "Interferometric snapshot spectro-ellipsometry: calibration and systematic error analysis," Curr. Opt. Photon. 4, 345-352 (2020). https://doi.org/10.3807/COPP.2020.4.4.345
  23. K. Wu, E. Otoo, and A. Shoshani, "Optimizing connected component labeling algorithms," Proc. SPIE 5747, 1965-1976 (2005).
  24. N. Otsu, "A threshold selection method from gray-level histograms," IEEE Trans. Syst. Man Cybern. SMC-9, 62-66 (1979). https://doi.org/10.1109/TSMC.1979.4310076
  25. S. Li, Y. Zhu, C. Zhang, Y. Yuan, and H. Tan, "Rectification of images distorted by microlens array errors in plenoptic cameras," Sensors 18, 2019 (2019). https://doi.org/10.3390/s18072019
  26. Y.-T. Lim, J.-H. Park, K.-C. Kwon, and N. Kim, "Resolution-enhanced integral imaging microscopy that uses lens array shifting," Opt. Express 17, 19253-19263 (2009). https://doi.org/10.1364/OE.17.019253
  27. S. Alam, K.-C. Kwon, M.-U. Erdenebat, Y.-T. Lim, S. Imtiaz, A. Sufian, S.-H. Jeon, and N. Kim, "Resolution Enhancement of an Integral Imaging Microscopy Using Generative Adversarial Network," in Proc. Conference on Lasers and Electro-Optics Pacific Rim-CLEO-PR (Sydney, Australia, Aug. 2020) paper C3G_4.