IEMEK Journal of Embedded Systems and Applications (대한임베디드공학회논문지)

Institute of Embedded Engineering of Korea (대한임베디드공학회)
권호 표지
DOI QR코드

DOI QR Code

Development of a Steel Plate Surface Defect Detection System Based on Small Data Deep Learning

소량 데이터 딥러닝 기반 강판 표면 결함 검출 시스템 개발

  • Received : 2022.03.30
  • Accepted : 2022.04.19
  • Published : 2022.06.30
Download PDF
⟨ Previous Next ⟩

Abstract

Collecting and labeling sufficient training data, which is essential to deep learning-based visual inspection, is difficult for manufacturers to perform because it is very expensive. This paper presents a steel plate surface defect detection system with industrial-grade detection performance by training a small amount of steel plate surface images consisting of labeled and non-labeled data. To overcome the problem of lack of training data, we propose two data augmentation techniques: program-based augmentation, which generates defect images in a geometric way, and generative model-based augmentation, which learns the distribution of labeled data. We also propose a 4-step semi-supervised learning using pseudo labels and consistency training with fixed-size augmentation in order to utilize unlabeled data for training. The proposed technique obtained about 99% defect detection performance for four defect types by using 100 real images including labeled and unlabeled data.

Keywords

Acknowledgement

This paper was supported by Research Fund, Kumoh National Institute of Technology (2019-104-059).

References

  1. C. Shorten, T. M. Khoshgoftaar, "A Survey on Image Data Augmentation for Deep Learning," J Big Data, Vol. 6, No. 1, pp. 1-48, 2019. https://doi.org/10.1186/s40537-018-0162-3
  2. Y. Wang, Q. Yao, J. T. Kwok, L. M. Ni, "Generalizing from a Few Examples: A Survey on Few-shot Learning," ACM Comput. Surv. Vol. 53, No. 3, pp. 1-3, 2021.
  3. K. Weiss, T. M. Khoshgoftaar, D. D. Wang, "A Survey of Transfer Learning," J Big Data Vol. 3, No. 1, pp. 1-40, 2016. https://doi.org/10.1186/s40537-015-0036-x
  4. S. Bernhard, P. John, H. Thomas, "Analysis of Representations for Domain Adaptation," Advances in Neural Information Processing Systems 19: Proceedings of the 2006 Conference , MIT Press, pp. 137-144, 2007.
  5. M. Khanam, T. Mahboob, W. Imtiaz, H. A. Ghafoor, R. Sehar, "A Survey on Unsupervised Machine Learning Algorithms for Automation, Classification and Maintenance," International Journal of Computer Applications. Vol. 119, No.13, pp. 34-39, 2015.
  6. A. Jaiswal, A. R. Babu, M. Z. Zadeh, D. Banerjee, F. Makedon, "A Survey on Contrastive Self- Supervised Learning," Technologies, Vol. 9, No.1 pp. 2, 2021. https://doi.org/10.3390/technologies9010002
  7. J. E. Van Engelen, H. H. Hoos, "A Survey on Semi-supervised Learning," Mach Learn Vol. 109, No. 2, pp. 373-440, 2020. https://doi.org/10.1007/s10994-019-05855-6
  8. T. DeVries, G. W. Taylor, "Improved Regularization of Convolutional Neural Networks with Cutout," arXiv:1708.04552, 2017.
  9. H. Zhang, M. Cisse, Y. N. Dauphin, David Lopez-Paz, "mixup: Beyond Empirical Risk Minimization," arXiv:1710.09412, 2017.
  10. A. Bochkovskiy, C. Y. Wang, H. Y. M. Liao, "YOLOv4: Optimal Speed and Accuracy of Object Detection", arXiv:2004.10934, 2020.
  11. G. Ghiasi, Y. Cui, A. Srinivas, R. Qian, Tsung-Yi Lin, Ekin D. Cubuk, Quoc V. Le, Barret Zoph, "Simple Copy-Paste is a Strong Data Augmentation Method for Instance Segmentation," Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 2918-2928, 2021.
  12. B. Christopher, C. Liang, G. P. B. Ricardo, G. Roger, H. Alexander, A. D. David, V. H. Maria, W. Joanna, R. Daniel, "GAN Augmentation: Augmenting Training data Using Generative Adversarial Networks," arXiv preprint. 2018.
  13. A. Antoniou, A. Storkey, H. Edwards. "Data Augmentation Generative Adversarial Networks," In Proceedings of the International Conference on Learning Representations Workshops (ICLR Workshops), 2018.
  14. M. Arjovsky, S. Chintala, L. Bottou, "Wasserstein GAN," arXiv:1701.07875, 2017.
  15. Y. Xian, T. Lorenz, B. Schiele, Z. Akata. "Feature Generating Networks for Zero-shot Learning," In CVPR, 2018.
  16. Y. Wang, Q. Yao, J. Kwok, L. M. Ni, "Generalizing from a Few Examples: A Survey on Few-Shot Learning," ACM Computing Surveys, Vol. 53, No. 3, pp. 1-34, 2020.
  17. D. Kim, D. Cho, D. Yoo, I. S. Kweon, "Learning Image Representations by Completing Damaged Jigsaw Puzzles," 2018 IEEE Winter Conference on Applications of Computer Vision (WACV), pp. 793-802, 2018.
  18. S. Gidaris, P. Singh, N. Komodakis, "Unsupervised Representation Learning by Predicting Image Rotations," 6th International Conference on Learning Representations (ICLR), Vancouver, Canada, 2018.
  19. L. Songtao, Z. Li, J. Sun. "Self-EMD: Self-Supervised Object Detection Wthout ImageNet." ArXiv abs/2011.13677, 2020.
  20. D. H. Lee, "Pseudo-Label : The Simple and Efficient Semi-Supervised Learning Method for Deep Neural Networks," ICML 2013 Workshop: Challenges in Representation Learning (WREPL), Vol. 3, No. 2, pp. 896, 2013
  21. S. Laine, T. Aila, "Temporal Ensembling for Semi-Supervised Learning," arXiv:1610.02242, 2016.
  22. J. Jeong, S. Lee, J. Kim, N. Kwak, "Consistency-based Semi-supervised Learning for Object Detection," NIPS'19: Proceedings of the 33rd International Conference on Neural Information Processing Systems, pp. 10759-10768, 2019.
  23. A. Tarvainen, H. Valpola, "Mean Teachers are Better Role Models: Weight-averaged Consistency Targets Improve Semi-supervised Deep Learning Results," NIPS'17: Proceedings of the 31st International Conference on Neural Information Processing Systems, pp. 1195-1204, 2017.
  24. Q. Xie, Z. Dai, E. Hovy, T. Luong, Q. Le, "Unsupervised Data Augmentation for Consistency Training," Advances in Neural Information Processing Systems 33 (NeurIPS 2020), pp. 6256-6268, 2020.
  25. D. Berthelot, N. Carlini, I. Goodfellow, N. Papernot, A. Oliver, C. Raffel, "MixMatch: a Holistic Approach to Semi-supervised Learning," NIPS'19: Proceedings of the 33rd International Conference on Neural Information Processing Systems, pp. 5049-5059, 2019.
  26. Q. Guo, Y. Mu, J. Chen, T. Wang, Y. Yu, P. Luo, "Scale-Equivalent Distillation for Semi-Supervised Object Detection," Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 14522-14531, 2022.
  27. F. Guo, J. Zhao, P. Jiang, "Surface Defects Detection of Steel Plate Based on Visual Attention Mechanism," 2017 Chinese Automation Congress (CAC), pp. 3174-3177, 2017.
  28. K. He, X. Zhang, S. Ren, J. Sun, "Deep Residual Learning for Image Recognition," Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770-778, 2016.
  29. S. Ren, K. He, R. Girshick, J. Sun, "Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks," Advances in Neural Information Processing Systems Vol. 28 (NIPS 2015), 2015.
  30. S. Wang, X. Xia, L. Ye, B. Yang, "Automatic Detection and Classification of Steel Surface Defect Using Deep Convolutional Neural Networks." Metals, Vol. 11, No. 3, pp. 388, 2021. https://doi.org/10.3390/met11030388
  31. D. Amin, S. Akhter, "Deep Learning-Based Defect Detection System in Steel Sheet Surfaces," 2020 IEEE Region 10 Symposium (TENSYMP), pp. 444-448, 2020,
  32. P. Damacharla, A. R. M. V., J. Ringenberg, A. Y. Javaid, "TLU-Net: A Deep Learning Approach for Automatic Steel Surface Defect Detection," 2021 International Conference on Applied Artificial Intelligence (ICAPAI), pp. 1-6, 2021.
  33. A. M. Deshpande, A. A. Minai, M. Kumar, "One-Shot Recognition of Manufacturing Defects in Steel Surfaces," Procedia Manufacturing, Vol. 48, pp. 1064-1071, 2020. https://doi.org/10.1016/j.promfg.2020.05.146
  34. J. Bozic, D. Tabernik, D. Skocaj, "Mixed Supervision for Surface-defect Detection: from Weakly to Fully Supervised Learning," Computers in Industry, Vol. 129, 103459, 2021. https://doi.org/10.1016/j.compind.2021.103459
  35. Wikipedia, "Jensen-Shannon Divergence", see https://en.wikipedia.org/wiki/Jensen-Shannon_divergence.
  36. L. C. Chen, Y. Zhu, G. Papandreou, F. Schroff, H. Adam, "Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation," Proceedings of the European Conference on Computer Vision (ECCV), pp. 801-818, 2018.
  37. J. Hu, L. Shen, S. Albanie, G. Sun, E. Wu, "Squeeze-and-Excitation Networks," Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2018.
  38. Ultralytics, "YOLOv5 v5.0 YOLOv5-P6 models," see https://github.com/ultralytics/yolov5/releases, 2021.
  39. T. Y. Lin, P.a Goyal, R. Girshick, K. He, P. Dollar, "Focal Loss for Dense Object Detection," Proceedings of the IEEE International Conference on Computer Vision, pp. 2980-2988, 2017.