Journal of the Korea institute for structural maintenance and inspection (한국구조물진단유지관리공학회 논문집)

Korea Institute for Structural Maintenance Inspection (한국구조물진단유지관리공학회)
권호 표지
DOI QR코드

DOI QR Code

Mean Teacher Learning Structure Optimization for Semantic Segmentation of Crack Detection

균열 탐지의 의미론적 분할을 위한 Mean Teacher 학습 구조 최적화

  • Seungbo Shim (Department of Geotechnical Engineering Research, Korea Institute of Civil Engineering and Building Technology)
  • 심승보 (한국건설기술연구원 지반연구본부 )
  • Received : 2023.09.13
  • Accepted : 2023.10.10
  • Published : 2023.10.31
Download PDF
⟨ Previous Next ⟩

Abstract

Most infrastructure structures were completed during periods of economic growth. The number of infrastructure structures reaching their lifespan is increasing, and the proportion of old structures is gradually increasing. The functions and performance of these structures at the time of design may deteriorate and may even lead to safety accidents. To prevent this repercussion, accurate inspection and appropriate repair are requisite. To this end, demand is increasing for computer vision and deep learning technology to accurately detect even minute cracks. However, deep learning algorithms require a large number of training data. In particular, label images indicating the location of cracks in the image are required. To secure a large number of those label images, a lot of labor and time are consumed. To reduce these costs as well as increase detection accuracy, this study proposed a learning structure based on mean teacher method. This learning structure was trained on a dataset of 900 labeled image dataset and 3000 unlabeled image dataset. The crack detection network model was evaluated on over 300 labeled image dataset, and the detection accuracy recorded a mean intersection over union of 89.23% and an F1 score of 89.12%. Through this experiment, it was confirmed that detection performance was improved compared to supervised learning. It is expected that this proposed method will be used in the future to reduce the cost required to secure label images.

인프라 구조물은 대부분 경제 성장기에 완공되었다. 이러한 인프라 구조물은 최근 들어 공용연수가 점차 증가하고 있어 노후 구조물의 비중이 점차 증가하고 있다. 이러한 노후 구조물은 설계 당시의 기능과 성능이 저하될 수 있고 안전사고로까지 이어질 수 있다. 이를 예방하기 위해서는 정확한 점검과 적절한 보수가 필수적이다. 이를 위해서는 우선 미세한 균열까지 정확히 탐지할 수 있도록 컴퓨터 비전과 딥러닝 기술에 수요가 증가하고 있다. 하지만 딥러닝 알고리즘은 다수의 학습 데이터가 있어야 한다. 특히 영상 내 균열의 위치를 표시한 라벨 영상은 필수적이다. 이러한 라벨 영상을 다수 확보하기 위해서는 많은 노동력과 시간이 필요한 실정이다. 이러한 비용을 절감하고 탐지 정확도를 높이기 위해서 본 연구에서는 mean teacher 방식의 학습 구조를 제안하였다. 이 학습 구조는 900장의 라벨 영상 데이터 세트와 3000장의 비라벨 영상 데이터 세트로 훈련되었다. 학습된 균열 탐지 신경망 모델은 300여장의 실험용 데이터 세트를 통해 평가되었고 탐지 정확도는 89.23%의 mean intersection over union과 89.12%의 F1 score를 기록하였다. 이 설험을 통해 지도학습과 비교하여 탐지 성능이 향상된 것을 확인하였다. 향후에 이러한 방법은 라벨 영상을 확보하는데 필요한 비용을 절감하는데 활용될 것으로 기대한다.

Keywords

Acknowledgement

이 성과는 정부(과학기술정보통신부)의 재원으로 한국연구재단의 지원을 받아 수행된 연구입니다(No. 2022R1F1A1074663). 지원에 감사합니다.

References

  1. Jeong, Y., Kim, W., Lee, I., and Lee, J. (2018), Bridge inspection practices and bridge management programs in China, Japan, Korea, and US, Journal of Structural Integrity and Maintenance, 3(2), 126-135.  https://doi.org/10.1080/24705314.2018.1461548
  2. Kim, H., and Kim, C. (2020), Deep-learning-based classification of point clouds for bridge inspection, Remote Sensing, 12(22), 3757. 
  3. Adhikari, R. S., Moselhi, O., and Bagchi, A. (2014), Image-based retrieval of concrete crack properties for bridge inspection, Automation in construction, 39, 180-194.  https://doi.org/10.1016/j.autcon.2013.06.011
  4. Ye, X. W., Jin, T., Yun, C. B. (2019), A review on deep learning-based structural health monitoring of civil infrastructures, Smart Structures and Systems, 24(5), 567-585.  https://doi.org/10.12989/SSS.2019.24.5.567
  5. Abdel-Qader, I., Abudayyeh, O., and Kelly, M. E. (2003), Analysis of edge-detection techniques for crack identification in bridges, Journal of Computing in Civil Engineering, 17(4), 255-263.  https://doi.org/10.1061/(ASCE)0887-3801(2003)17:4(255)
  6. Long, J., Shelhamer, E., and Darrell, T. (2015), Fully convolutional networks for semantic segmentation, Proceedings of the IEEE conference on computer vision and pattern recognition, Boston, MA, USA, 3431-3440. 
  7. Ali, R., Chuah, J. H., Talip, M. S. A., Mokhtar, N., and Shoaib, M. A. (2022), Structural crack detection using deep convolutional neural networks, Automation in Construction, 133, 103989. 
  8. Li, G., Wan, J., He, S., Liu, Q., and Ma, B. (2020), Semi-supervised semantic segmentation using adversarial learning for pavement crack detection, IEEE Access, 8, 51446-51459.  https://doi.org/10.1109/ACCESS.2020.2980086
  9. Shim, S., Kim, J., Lee, S. W., and Cho, G. C. (2022), Road damage detection using super-resolution and semi-supervised learning with generative adversarial network, Automation in Construction, 135, 104139. 
  10. Shim, S., Kim, J., Cho, G. C., and Lee, S. W. (2023), Stereovision-based 3D concrete crack detection using adversarial learning with balanced ensemble discriminator networks, Structural Health Monitoring, 22(2), 1353-1375.  https://doi.org/10.1177/14759217221097868
  11. Maeda, H., Kashiyama, T., Sekimoto, Y., Seto, T., and Omata, H. (2021), Generative adversarial network for road damage detection, Computer-Aided Civil and Infrastructure Engineering, 36(1), 47-60.  https://doi.org/10.1111/mice.12561
  12. Zhang, K., Zhang, Y., and Cheng, H. D. (2020), Self-supervised structure learning for crack detection based on cycle-consistent generative adversarial networks, Journal of Computing in Civil Engineering, 34(3), 04020004. 
  13. Shim, S., Kim, J., Cho, G. C., and Lee, S. W. (2020), Multiscale and adversarial learning-based semi-supervised semantic segmentation approach for crack detection in concrete structures, IEEE Access, 8, 170939-170950.  https://doi.org/10.1109/ACCESS.2020.3022786
  14. Tarvainen, A., and Valpola, H. (2017), Mean teachers are better role models: Weight-averaged consistency targets improve semi-supervised deep learning results. Advances in neural information processing systems, 30. 
  15. Zheng, M., You, S., Huang, L., Wang, F., Qian, C., and Xu, C. (2022), Simmatch: Semi-supervised learning with similarity matching, Proceedings of the IEEE conference on computer vision and pattern recognition, New Orleans, LA, USA, 14471-14481. 
  16. Romera, E., Alvarez, J. M., Bergasa, L. M., and Arroyo, R. (2017), Erfnet: Efficient residual factorized convnet for real-time semantic segmentation, IEEE Transactions on Intelligent Transportation Systems, 19(1), 263-272.  https://doi.org/10.1109/TITS.2017.2750080
  17. Bang, S., Park, S., Kim, H., and Kim, H. (2019), Encoder-decoder network for pixel-level road crack detection in black-box images. Computer-Aided Civil and Infrastructure Engineering, 34(8), 713-727. https://doi.org/10.1111/mice.12440