Smart Structures and Systems

Techno-Press (테크노프레스)
권호 표지
DOI QR코드

DOI QR Code

One-step deep learning-based method for pixel-level detection of fine cracks in steel girder images

  • Li, Zhihang (Department of Civil Engineering, Monash University) ;
  • Huang, Mengqi (Department of Civil Engineering, Monash University) ;
  • Ji, Pengxuan (Department of Civil Engineering, Monash University) ;
  • Zhu, Huamei (Department of Civil Engineering, Monash University) ;
  • Zhang, Qianbing (Department of Civil Engineering, Monash University)
  • Received : 2021.04.15
  • Accepted : 2021.08.07
  • Published : 2022.01.25
⟨ Previous Next ⟩

Abstract

Identifying fine cracks in steel bridge facilities is a challenging task of structural health monitoring (SHM). This study proposed an end-to-end crack image segmentation framework based on a one-step Convolutional Neural Network (CNN) for pixel-level object recognition with high accuracy. To particularly address the challenges arising from small object detection in complex background, efforts were made in loss function selection aiming at sample imbalance and module modification in order to improve the generalization ability on complicated images. Specifically, loss functions were compared among alternatives including the Binary Cross Entropy (BCE), Focal, Tversky and Dice loss, with the last three specialized for biased sample distribution. Structural modifications with dilated convolution, Spatial Pyramid Pooling (SPP) and Feature Pyramid Network (FPN) were also performed to form a new backbone termed CrackDet. Models of various loss functions and feature extraction modules were trained on crack images and tested on full-scale images collected on steel box girders. The CNN model incorporated the classic U-Net as its backbone, and Dice loss as its loss function achieved the highest mean Intersection-over-Union (mIoU) of 0.7571 on full-scale pictures. In contrast, the best performance on cropped crack images was achieved by integrating CrackDet with Dice loss at a mIoU of 0.7670.

Keywords

Acknowledgement

This study was supported by Monash University for the scholarships and the high-performance computation platform. The authors appreciate the organizations of the IPC-SHM 2020 ANCRiSST, Harbin Institute of Technology (China), and University of Illinois at Urbana-Champaign (USA) for their generously providing invaluable data from actual structures. The authors also would like to thank the chairs of IPC-SHM 2020 Prof. Hui Li, and Prof. Billie F. Spencer Jr for their leadership on the competition.

References

  1. Abdel-Qader, I., Abudayyeh, O. and Kelly, M.E. (2003), "Analysis of edge-detection techniques for crack identification in bridges", J. Comput. Civil Eng., 17(4), 255-263. https://doi.org/10.1061/(ASCE)0887-3801(2003)17:4(255)
  2. Bao, Y., Chen, Z., Wei, S., Xu, Y., Tang, Z. and Li, H. (2019), "The state of the art of data science and engineering in structural health monitoring", Engineering, 5(2), 234-242. https://doi.org/10.1016/j.eng.2018.11.027
  3. Bao, Y., Li, J., Nagayama, T., Xu, Y., Spencer Jr, B.F. and Li, H. (2021), "The 1st International Project Competition for Structural Health Monitoring (IPC-SHM, 2020): A summary and benchmark problem", Struct. Health Monitor., 20(4), 2229-2239. https://doi.org/10.1177/14759217211006485
  4. Cha, Y.J., Choi, W. and Buyukozturk, O. (2017), "Deep learning- based crack damage detection using convolutional neural networks", Comput.-Aided Civil Infrastr. Eng., 32(5), 361-378. https://doi.org/10.1111/mice.12263
  5. Cha, Y.J., Choi, W., Suh, G., Mahmoudkhani, S. and Buyukozturk, O. (2018), "Autonomous structural visual inspection using region-based deep learning for detecting multiple damage types", Comput.-Aided Civil Infrastr. Eng., 33(9), 731-747. https://doi.org/10.1111/mice.12334
  6. Dung, C.V. (2019), "Autonomous concrete crack detection using deep fully convolutional neural network", Automat. Constr., 99, 52-58. https://doi.org/10.1016/j.autcon.2018.11.028
  7. Goodfellow, I., Bengio, Y. and Courville, A. (2016), Deep learning, MIT press Cambridge.
  8. He, K., Zhang, X., Ren, S. and Sun, J. (2014), "Spatial pyramid pooling in deep convolutional networks for visual recognition", IEEE Transact. Pattern Anal. Mach. Intell., 37(9), 1904-1916. https://doi.org/10.1007/978-3-319-10578-9_23
  9. Huang, M.Q., Ninic, J. and Zhang, Q.B. (2021), "BIM, machine learning and computer vision techniques in underground construction: Current status and future perspectives", Tunnell. Undergr. Space Technol., 108, 103677. https://doi.org/10.1016/j.tust.2020.103677
  10. Hutchinson, T.C. and Chen, Z. (2006), "Improved image analysis for evaluating concrete damage", J. Comput. Civil Eng., 20(3), 210-216. https://doi.org/10.1061/(ASCE)0887-3801(2006)20:3(210)
  11. Kang, D., Benipal, S.S., Gopal, D.L. and Cha, Y.J. (2020), "Hybrid pixel-level concrete crack segmentation and quantification across complex backgrounds using deep learning", Automat. Constr., 118, 103291. https://doi.org/10.1016/j.autcon.2020.103291
  12. Koch, C., Georgieva, K., Kasireddy, V., Akinci, B. and Fieguth, P. (2015), "A review on computer vision based defect detection and condition assessment of concrete and asphalt civil infrastructure", Adv. Eng. Inform., 29(2), 196-210. https://doi.org/10.1016/j.aei.2015.01.008
  13. Lattanzi, D. and Miller, G.R. (2014), "Robust automated concrete damage detection algorithms for field applications", J. Comput. Civil Eng., 28(2), 253-262. https://doi.org/10.1061/(asce)cp.1943-5487.0000257
  14. Lei, B., Ren, Y., Wang, N., Huo, L. and Song, G. (2020), "Design of a new low-cost unmanned aerial vehicle and vision-based concrete crack inspection method", Struct. Health Monitor., 19(6), 1871-1883. https://doi.org/10.1177/1475921719898862
  15. Li, L., Wang, Q., Zhang, G., Shi, L., Dong, J. and Jia, P. (2018), "A method of detecting the cracks of concrete undergo high-temperature", Constr. Build. Mater., 162, 345-358. https://doi.org/10.1016/j.conbuildmat.2017.12.010
  16. Lin, T.Y., Dollar, P., Girshick, R., He, K., Hariharan, B. and Belongie, S. (2017), "Feature pyramid networks for object detection", Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 2117-2125. https://doi.org/10.1109/CVPR.2017.106
  17. Lin, T.Y., Goyal, P., Girshick, R., He, K. and Dollar, P. (2018), "Focal loss for dense object detection", Proceedings of 2017 IEEE International Conference on Computer Vision (ICCV), pp. 2980-2988. https://doi.org/10.1109/ICCV.2017.324
  18. Long, J., Shelhamer, E. and Darrell, T. (2015), "Fully convolutional networks for semantic segmentation", IEEE Transact. Pattern Anal. Mach. Intell., 39(4), 640-651. https://doi.org/10.1109/CVPR.2015.7298965
  19. Milletari, F., Navab, N. and Ahmadi, S.A. (2016), "Fully convolutional neural networks for volumetric medical image segmentation", Proceedings of 2016 Fourth International Conference on 3D Vision (3DV), pp. 565-571. https://doi.org/10.1109/3DV.2016.79
  20. Nash, W., Drummond, T. and Birbilis, N. (2018), "A review of deep learning in the study of materials degradation", npj Materials Degradation, 2(1), 1-12. https://doi.org/10.1038/s41529-018-0058-x
  21. Nayyeri, F., Hou, L., Zhou, J. and Guan, H. (2018), "Foreground-background separation technique for crack detection", Comput.- Aided Civil Infrastr. Eng., 34(6), 457-470. https://doi.org/10.1111/mice.12428
  22. Prasanna, P., Dana, K., Gucunski, N. and Basily, B. (2012), "Computer-vision based crack detection and analysis", Proceedings of SPIE - The International Society for Optical Engineering, Vol. 8345, pp. 834542. https://doi.org/10.1117/12.915384
  23. Ren, Y., Huang, J., Hong, Z., Lu, W., Yin, J., Zou, L. and Shen, X. (2020), "Image-based concrete crack detection in tunnels using deep fully convolutional networks", Constr. Build. Mater., 234, 117367. https://doi.org/10.1016/j.conbuildmat.2019.117367
  24. Ronneberger, O., Fischer, P. and Brox, T. (2015), "U-net: Convolutional networks for biomedical image segmentation", Proceedings of International Conference on Medical Image Computing and Computer-Assisted Intervention, pp. 234-241. https://doi.org/10.1007/978-3-662-54345-0_3
  25. Russell, B.C., Torralba, A., Murphy, K.P. and Freeman, W.T. (2008), "LabelMe: a database and web-based tool for image annotation", Int. J. Comput. Vision, 77(1-3), 157-173. https://doi.org/10.1007/s11263-007-0090-8
  26. Salehi, S.S.M., Erdogmus, D. and Gholipour, A. (2017), "Tversky loss function for image segmentation using 3D fully convolutional deep networks", International Workshop on Machine Learning in Medical Imaging, pp. 379-387. https://doi.org/10.1007/978-3-319-67389-9_44
  27. Spencer Jr., B.F., Hoskere, V. and Narazaki, Y. (2019), "Advances in computer vision-based civil infrastructure inspection and monitoring", Engineering, 5(2), 199-222. https://doi.org/10.1016/j.eng.2018.11.030
  28. Sudre, C.H., Li, W., Vercauteren, T., Ourselin, S. and Cardoso, M.J. (2017), "Generalised dice overlap as a deep learning loss function for highly unbalanced segmentations", In: Deep learning in medical image analysis and multimodal learning for clinical decision support, pp. 240-248. https://doi.org/10.1007/978-3-319-67558-9_28
  29. Tversky, A. (1988), "Features of similarity", Readings in Cognitive Science, 84(4), 290-302. https://doi.org/10.1037/0033-295X.84.4.327
  30. Xu, Y., Li, S., Zhang, D., Jin, Y., Zhang, F., Li, N. and Li, H. (2018), "Identification framework for cracks on a steel structure surface by a restricted Boltzmann machines algorithm based on consumer-grade camera images", Struct. Control Health Monitor., 25(2), e2075. https://doi.org/10.1002/stc.2075
  31. Yamaguchi, T. and Hashimoto, S. (2009), "Fast crack detection method for large-size concrete surface images using percolation-based image processing", Mach. Vision Applicat., 21(5), 797-809. https://doi.org/10.1007/s00138-009-0189-8
  32. Yang, X., Li, H., Yu, Y., Luo, X., Huang, T. and Yang, X. (2018), "Automatic pixel-level crack detection and measurement using fully convolutional network", Comput.-Aided Civil Infrastr. Eng., 33(12), 1090-1109. https://doi.org/10.1111/mice.12412
  33. Yeum, C.M. and Dyke, S.J. (2015), "Vision-based automated crack detection for bridge inspection", Comput.-Aided Civil Infrastr. Eng., 30(10), 759-770. https://doi.org/10.1111/mice.12141
  34. Yu, F. and Koltun, V. (2016), "Multi-scale context aggregation by dilated convolutions", arXiv preprint arXiv:1511.07122.
  35. Zhang, L., Yang, F., Zhang, Y.D. and Zhu, Y.J. (2016), "Road crack detection using deep convolutional neural network", Proceedings of IEEE International Conference on Image Processing (ICIP), Phoenix, AZ, USA, pp. 3708-3712. https://doi.org/doi/10.1109/ICIP.2016.7533052
  36. Zhao, H., Qin, G. and Wang, X. (2010), "Improvement of canny algorithm based on pavement edge detection", Proceedings of 2010 3rd International Congress on Image and Signal Processing, Yantai, China. https://doi.org/10.1109/CISP.2010.5646923