Journal of the Korea institute for structural maintenance and inspection (한국구조물진단유지관리공학회 논문집)

Korea Institute for Structural Maintenance Inspection (한국구조물진단유지관리공학회)
권호 표지
DOI QR코드

DOI QR Code

Semantic Segmentation for Multiple Concrete Damage Based on Hierarchical Learning

계층적 학습 기반 다중 콘크리트 손상에 대한 의미론적 분할

  • Shim, Seungbo (Department of Geotechnical Engineering, Korea Institute of Civil Engineering and Building Technology) ;
  • Min, Jiyoung
  • 심승보 (한국건설기술연구원 지반연구본부) ;
  • 민지영 (한국건설기술연구원 구조연구본부)
  • Received : 2022.10.24
  • Accepted : 2022.11.13
  • Published : 2022.12.31
Download PDF
⟨ Previous Next ⟩

Abstract

The condition of infrastructure deteriorates as the service life increases. Since most infrastructure in South Korea were intensively built during the period of economic growth, the proportion of outdated infrastructure is rapidly increasing now. Aging of such infrastructure can lead to safety accidents and even human casualties. To prevent these issues in advance, periodic and accurate inspection is essential. For this reason, the need for research to detect various types of damage using computer vision and deep learning is increasingly required in the field of remotely controlled or autonomous inspection. To this end, this study proposed a neural network structure that can detect concrete damage by classifying it into three types. In particular, the proposed neural network can detect them more accurately through a hierarchical learning technique. This neural network was trained with 2,026 damage images and tested with 508 damage images. As a result, we completed an algorithm with average mean intersection over union of 67.04% and F1 score of 52.65%. It is expected that the proposed damage detection algorithm could apply to accurate facility condition diagnosis in the near future.

구조물의 공용연수가 증가함에 따라 각종 성능 저하가 발생한다. 특히 국내 인프라 구조물은 대부분 경제가 성장하는 시기에 집중적으로 건설되었기 때문에 노후 인프라 비율 급증이 최근 주요 이슈가 되고 있다. 인프라의 노후화는 자칫 안전사고로 이어질 수 있으며 인명 피해까지 유발할 수 있다. 이러한 문제를 사전에 예방하기 위하여 주기적이고 정확한 점검 및 유지관리가 필수적이다. 이 같은 이유로 최근 컴퓨터 비전과 딥러닝을 활용하여 다양한 손상을 탐지하는 연구에 대한 수요가 원격점검 혹은 점검자동화 분야에서 증가하고 있다. 따라서 본 논문에서는 콘크리트 손상의 종류를 세 가지로 구분하여 이를 탐지할 수 있는 신경망 구조를 제안했다. 특히 계층적 학습 기법을 통해 보다 정확하게 다양한 손상을 탐지할 수 있는 신경망을 개발하였다. 이 신경망은 2,026장의 손상 영상으로 학습되었고, 508장의 손상 영상으로 실험하였다. 그 결과 67.04%의 평균 중첩 정확도와 52.65%의 F1 점수를 갖는 알고리즘을 완성하였다. 이 같은 손상 탐지 알고리즘은 향후 구조물의 정확한 상태 진단에 활용될 수 있으리라 기대한다.

Keywords

Acknowledgement

본 논문은 2021년 해양수산부 재원으로 해양수산과학기술진흥원(과제번호 20210659)의 지원을 받아 수행되었으며, 이에 감사드립니다.

References

  1. Ai, D., Jiang, G., Kei, L. S., and Li, C. (2018), Automatic pixel-level pavement crack detection using information of multi-scale neighborhoods, IEEE Access, IEEE., 6, 24452-24463. https://doi.org/10.1109/access.2018.2829347
  2. Feng, C., Zhang, H., Wang, H., Wang, S., and Li, Y. (2020), Automatic pixel-level crack detection on dam surface using deep convolutional network, Sensors, MDPI, 20(7), 2069. https://doi.org/10.3390/s20072069
  3. Gao, R. (2021), Rethink dilated convolution for real-time semantic segmentation, arXiv:2111.09957. [Online]. Available: https://arxiv.org/abs/2111.09957
  4. Hong, Y., Pan, H., Sun, W., and Jia, Y. (2021), Deep dual-resolution networks for real-time and accurate semantic segmentation of road scenes, arXiv:2101.06085. [Online]. Available: https://arxiv.org/abs/2101.06085
  5. Jenkins, M. D., Carr, T. A., Iglesias, M. I., Buggy, T., and Morison, G. (2018), A deep convolutional neural network for semantic pixel-wise segmentation of road and pavement surface cracks, Proceeding European signal processing conference, IEEE., Rome, Italy, 2120-2124.
  6. Jung, H. J., An, H. J., Park, K. T., Jung, K. S., Kim, Y. H., and Lee, J. H. (2021), Correlation Analysis between Damage of Expansion Joints and Response of Deck in RC Slab Bridges, Journal of the Korea Institute for Structural Maintenance and Inspection, 25(6), 245-253. https://doi.org/10.11112/JKSMI.2021.25.6.245
  7. Kawahara, S., Shirato, M., Kajifusa, N., and Kutsukake, T. (2014), Investigation of the tunnel ceiling collapse in the central expressway in Japan, Proceeding Transportation Research Board 93rd Annual Meeting, Washington, D.C., USA, 14, 2559.
  8. Kim, B., and Cho, S. (2019), Image-based concrete crack assessment using mask and region-based convolutional neural network, Structural Control and Health Monitoring, Wiley, 26(8), e2381. https://doi.org/10.1002/stc.2381
  9. Li, S., Zhao, X., and Zhou, G. (2019), Automatic pixel-level multiple damage detection of concrete structure using fully convolutional network, Computer-Aided Civil and Infrastructure Engineering, Wiley, 34(7), 616-634. https://doi.org/10.1111/mice.12433
  10. Liu, Y., Yao, J., Lu, X., Xie, R., and Li, L. (2019), DeepCrack: A deep hierarchical feature learning architecture for crack segmentation, Neurocomputing, Elsevier, 338, 139-153. https://doi.org/10.1016/j.neucom.2019.01.036
  11. Long, J., Shelhamer, E., and Darrell, T. (2015), Fully convolutional networks for semantic segmentation, Proceedings of the IEEE conference on computer vision and pattern recognition, Boston, MA, USA, pp. 3431-3440.
  12. Ministry of Land, Infrastructure, Transportation, and Tourism. (2013), White paper on land, infrastructure, transportation, and tourism in Japan, 2013.
  13. Ronneberger, O., Fischer, P., and Brox, T. (2015), U-net: Convolutional networks for biomedical image segmentation, Proceeding International Conference on Medical image computing and computer-assisted intervention, Munich, Germany, 234-241.
  14. Shim, S., and Jeong, J.-J. (2021), Detection Algorithm of Road Damage and Obstacle Based on Joint Deep Learning for Driving Safety, The Journal of The Korea Institute of Intelligent Transport Systems, 20(2), 95-111.
  15. Shim, S., Kim, J., Cho, G. C., and Lee, S. W. (2020), Multiscale and adversarial learning-based semi-supervised semantic segmentation approach for crack detection in concrete structures, IEEE Access, IEEE., 8, 170939-170950. https://doi.org/10.1109/access.2020.3022786
  16. Shim, S., Kim, J., Cho, G. C., and Lee, S. W. (2022), Stereo-vision-based 3D concrete crack detection using adversarial learning with balanced ensemble discriminator networks, Structural Health Monitoring, SAGE Publication, p. 14759217221097868.
  17. Shim, S., Kim, J., Lee, S. W., and Cho, G. C. (2021), Road surface damage detection based on hierarchical architecture using lightweight auto-encoder network, Automation in Construction, 130, 103833. https://doi.org/10.1016/j.autcon.2021.103833
  18. Spencer Jr, B. F., Hoskere, V., and Narazaki, Y. (2019), Advances in computer vision-based civil infrastructure inspection and monitoring, Engineering, Elsevier, 5(2), 199-222. https://doi.org/10.1016/j.eng.2018.11.030
  19. Witcher, T. R. (2017), From disaster to prevention: The silver bridge, Civil Engineering Magazine Archive, ASCE., 87(11), 44-47. https://doi.org/10.1061/ciegag.0001250
  20. Xu, J., Xiong, Z., and Bhattacharyya, S. P. (2022), PIDNet: A Real-time Semantic Segmentation Network Inspired from PID Controller, arXiv:2206.02066. [Online]. Available: https://arxiv.org/abs/2206.02066
  21. Zhang, C., Chang, C. C., and Jamshidi, M. (2020), Concrete bridge surface damage detection using a single-stage detector, Computer-Aided Civil and Infrastructure Engineering, Wiley, 35(4), 389-409. https://doi.org/10.1111/mice.12500
  22. Zhang, L., Shen, J., and Zhu, B. (2021), A research on an improved Unet-based concrete crack detection algorithm, Structural Health Monitoring, SAGE Publication, 20(4), 1864-1879. https://doi.org/10.1177/1475921720940068