DOI QR코드

DOI QR Code

Implementation of Real-Time Post-Processing for High-Quality Stereo Vision

  • Choi, Seungmin (IT Convergence Technology Research Laboratory, ETRI) ;
  • Jeong, Jae-Chan (IT Convergence Technology Research Laboratory, ETRI) ;
  • Chang, Jiho (IT Convergence Technology Research Laboratory, ETRI) ;
  • Shin, Hochul (IT Convergence Technology Research Laboratory, ETRI) ;
  • Lim, Eul-Gyoon (IT Convergence Technology Research Laboratory, ETRI) ;
  • Cho, Jae Il (IT Convergence Technology Research Laboratory, ETRI) ;
  • Hwang, Daehwan (IT Convergence Technology Research Laboratory, ETRI)
  • Received : 2014.12.11
  • Accepted : 2015.06.08
  • Published : 2015.08.01

Abstract

We propose a novel post-processing algorithm and its very-large-scale integration architecture that simultaneously uses the passive and active stereo vision information to improve the reliability of the three-dimensional disparity in a hybrid stereo vision system. The proposed architecture consists of four steps - left-right consistency checking, semi-2D hole filling, a tiny adaptive variance checking, and a 2D weighted median filter. The experimental results show that the error rate of the proposed algorithm (5.77%) is less than that of a raw disparity (10.12%) for a real-world camera image having a $1,280{\times}720$ resolution and maximum disparity of 256. Moreover, for the famous Middlebury stereo image sets, the proposed algorithm's error rate (8.30%) is also less than that of the raw disparity (13.7%). The proposed architecture is implemented on a single commercial field-programmable gate array using only 13.01% of slice resources, which achieves a rate of 60 fps for $1,280{\times}720$ stereo images with a disparity range of 256.

Keywords

References

  1. J.-C. Jeong et al., "High-Quality Stereo Depth Map Generation Using Infrared Pattern Projection," ETRI J., vol. 35, no. 6, Dec. 2013, pp. 1011-1020. https://doi.org/10.4218/etrij.13.2013.0052
  2. J.H. Chang, J.C. Jeong, and D.-H. Hwang, "High-Quality Stereo Depth Map Generation Using Infrared Pattern Projection," Proc. British Mach. Vis. Conf., Nottingham, UK, Sept. 2014.
  3. C. Tomasi and R. Manduchi, "Bilateral Filtering for Gray and Color Images," Proc. IEEE Int. Conf. Comput. Vis., Bombay, India, Jan. 4-7, 1998, pp. 839-846.
  4. D. Scharstein and R. Szeliski, "A Taxonomy and Evaluation of Dense Two-Frame Stereo Correspondence Algorithms," Int. J. Comput. Vis., vol. 47, no. 1, Apr. 2002, pp. 7-42. https://doi.org/10.1023/A:1014573219977
  5. J. Heikkila and O. Silven, "A Four-Step Camera Calibration Procedure with Implicit Image Correction," IEEE Comput. Soc. Conf. Comput. Vis. Pattern Recogn., San Juan, Puerto Rico, June 17-19, 1997, pp. 1106-1112.
  6. D.I. Han et al., "The Design of HD Image Rectification Architecture Using Floating Point IP," IERI Procedia, Saipan, USA, vol. 6, 2014, pp. 39-44. https://doi.org/10.1016/j.ieri.2014.03.007
  7. R. Zabih and J. Woodfill, "Non-parametric Local Transforms for Computing Visual Correspondence," Proc. ECCV, Stockholm, Sweden, May 2-6, 1994, pp. 151-158.
  8. A. Hosni, M. Gelautz, and M. Bleyer, "Accuracy-Efficiency Evaluation of Adaptive Support Weight Techniques for Local Stereo Matching," DAGM OAGM Symp., Graz, Australia, vol. 7476, Aug. 28-31, 2012, pp. 337-346.
  9. F. Tombari et al., "Classification and Evaluation of Cost Aggregation Methods for Stereo Correspondence," Proc. IEEE Comput. Vis. Pattern Recogn., Anchorage, AK, USA, June 23-28, 2008, pp. 1-8.
  10. C. Cigla and A.A. Alatan, "Efficient Edge-Preserving Stereo Matching," IEEE Int. Conf. Comput. Vis. Workshops, Barcelona, Spain, Nov. 6-13, 2011, pp. 696-699.
  11. C.C. Pham and J.W. Jeon, "Domain Transformation-Based Efficient Cost Aggregation for Local Stereo Matching," IEEE Trans. Circuits Syst. Video Technol., vol. 23, no. 7, Oct. 2012, pp. 1119-1130. https://doi.org/10.1109/TCSVT.2012.2223794
  12. S.M. Choi et al., "Post-Processing Algorithms for Real-Time Active Stereo Vision," IEEE Int. Symp. Consum. Electron., Jeju, Rep. of Korea, June 22-25, 2014, pp. 1-2.
  13. S.M. Choi et al., "A FPGA Based Real-Time Post-Processing Architecture for Active Stereo Vision," IEEE Int. Symp. Consum. Electron., Jeju, Rep. of Korea, June 22-25, 2014, pp. 1-2.
  14. S.B. Kang, R. Szeliski, and J. Chai, "Handling Occlusions in Dense Multi-view Stereo," Proc. IEEE Comput. Soc. Conf. Comput. Vis. Pattern Recogn., Kauai, HI, USA, vol. 1, Dec. 8-14, 2001, pp. 103-110.
  15. L. Wang et al., "Stereoscopic Inpainting: Joint Color and Depth Completion from Stereo Images," IEEE Comput. Vis. Pattern Recogn., Anchorage, AK, USA, June 23-28, 2008, pp. 1-8.
  16. H. Jiang, R. Gao, and X. Liu, "Research of Stereo Matching Based on Improved Median Filter," Int. Conf. Electr. Electron., Nanchang, China, vol. 4, June 20-22, 2011, pp. 479-486.
  17. L. Yin et al., "Weighted Median Filters: A Tutorial," IEEE Trans. Circuits Syst. II: Analog Digital Signal Process., vol. 43, no. 3, Mar. 1996, pp. 157-192. https://doi.org/10.1109/82.486465
  18. S.A. Fahmy, P.Y.K. Cheung, and W. Luk, "Novel FPGA-Based Implementation of Median and Weighted Median Filters for Image Processing," Int. Conf. Field Programmable Logic Appl., Tampere, Finland, Aug. 24-26, 2005, pp. 142-147.
  19. D. Scharstein and R. Szeliski, "High-Accuracy Stereo Depth Maps Using Structured Light," Proc. IEEE Comput. Soc. Conf. Comput. Vis. Pattern Recogn., Madison, WI, USA, no. 1, June 18-20, 2003, pp. 195-202.
  20. S. Jin et al., "FPGA Design and Implementation of a Real-Time Stereo Vision System," IEEE Trans. Circuits Syst. Video Technol., vol. 20, no. 1, Jan. 2010, pp. 15-26. https://doi.org/10.1109/TCSVT.2009.2026831
  21. J.I. Woodfill, G. Gordon, and R. Buck, "Tyzx DeepSea High Speed Stereo Vision System," Conf. Comput. Vis. Pattern Recogn., Washington, DC, USA, 2004, pp. 41-46.
  22. Y. Jia et al., "A Miniature Stereo Vision Machine (MSVM-III) for Dense Disparity Mapping," Proc. Int. Conf. Pattern Recogn., Cambridge, UK, Aug. 23-26, 2004, pp. 728-731.
  23. C. Banz et al., "Real-Time Stereo Vision System Using Semiglobal Matching Disparity Estimation: Architecture and FPGAImplementation," Int. Conf. Embedded Comput. Syst., Samos, Greece, July 19-22, 2010, pp. 93-101.