Smart Media Journal (스마트미디어저널)

THE KOREAN INSTITUTE OF SMART MEDIA (한국스마트미디어학회)
권호 표지
DOI QR코드

DOI QR Code

Improved Method of License Plate Detection and Recognition Facilitated by Fast Super-Resolution GAN

Fast Super-Resolution GAN 기반 자동차 번호판 검출 및 인식 성능 고도화 기법

  • 민동욱 (한국교통대학교 소프트웨어학과) ;
  • 임현석 (한국교통대학교 소프트웨어학과 대학원) ;
  • 곽정환 (한국교통대학교 소프트웨어학과)
  • Received : 2020.09.04
  • Accepted : 2020.11.11
  • Published : 2020.12.31
Download PDF
⟨ Previous Next ⟩

Abstract

Vehicle License Plate Recognition is one of the approaches for transportation and traffic safety networks, such as traffic control, speed limit enforcement and runaway vehicle tracking. Although it has been studied for decades, it is attracting more and more attention due to the recent development of deep learning and improved performance. Also, it is largely divided into license plate detection and recognition. In this study, experiments were conducted to improve license plate detection performance by utilizing various object detection methods and WPOD-Net(Warped Planar Object Detection Network) model. The accuracy was improved by selecting the method of detecting the vehicle(s) and then detecting the license plate(s) instead of the conventional method of detecting the license plate using the object detection model. In particular, the final performance was improved through the process of removing noise existing in the image by using the Fast-SRGAN model, one of the Super-Resolution methods. As a result, this experiment showed the performance has improved an average of 4.34% from 92.38% to 96.72% compared to previous studies.

자동차 번호판 인식 기술은 도로의 교통상황 통제, 과속차량 단속, 도주 차량의 추적 등 현대 교통 시설 및 교통 안전망을 책임지고 있는 핵심 기술 중 하나이다. 이 기법은 과거에도 연구되었던 분야였으나 최근 딥러닝 기술의 발전으로 다양한 기법들을 적용하여 향상된 성능을 보이는 분야이며, 크게 자동차 번호판 검출과 번호판 인식으로 나뉜다. 본 연구에서는 다양한 객체 검출 모델과 WPOD-Net(Warped Planar Object Detection Network) 모델을 활용하여 자동차 번호판 검출 성능을 향상시키기 위한 실험을 진행하였으며, 객체 검출 모델을 활용하여 번호판을 검출하는 기존 방식들 대신 차량을 검출한 다음 번호판을 검출하는 방식을 택하여 정확도를 높였다. 특히 Super-Resolution 기법 중 하나인 Fast-SRGAN 모델을 활용하여 이미지 내에 존재하는 노이즈를 제거하는 처리를 통해 최종 성능을 향상시켰다. 결과적으로 92.38%에서 96.72%로 선행 연구 대비 평균 4.34% 향상된 성능이 실험을 통해 확인되었다.

Keywords

References

  1. H. Li, P. Wang, C. Shen, "Towards End-to-End Car License Plates Detection and Recognition with Deep Neural Networks," IEEE TRANSACTIONS ON INTELLIGENT TRANSPORTATION SYSTEMS, vol. 20, pp. 1126-1136, Mar. 2019. https://doi.org/10.1109/tits.2018.2847291
  2. 민동욱, 임현석, 이광, 김봉근, 곽정환, "Fast-Super-Resolution GAN 기반 자동차 번호판 검출 및 인식 성능 향상 방법에 대한 연구," 스마트미디어학회 춘계 학술대회, 광주, 한국, 2020년 5월
  3. 박승현, 조성원, "Haar-like Feature 및 CLNF 알고리즘을 이용한 차량 번호판 인식," 스마트미디어저널, 제5권, 제1호, 15-23쪽, 2016년 3월
  4. 이광옥, 배상현, "차량 번호판 밝기 제어를 이용한 인식률 개선 방안," 스마트미디어저널, 제6권, 제3호, 57-63쪽, 2017년 9월
  5. 김운기, Fatemeh Dehghan, 조성원, "SSD-Mobilenet과 ResNet을 이용한 모바일 기기용 자동차 번호판 인식시스템," 스마트미디어저널, 제9권 제2호, 92-98쪽, 2020년 06월 https://doi.org/10.30693/SMJ.2020.9.2.92
  6. S. Lee, J. Gwak and M. Jeon, "Vehicle model recognition in video," International Journal of Signal Processing, Image Processing and Pattern Recognition, vol. 6, no. 2, 2013.
  7. O. Bulan, V. Kozitsky, P. Ramesh and M. Shreve, "Segmentation and Annotation Free License Plate Recognition With Deep Localization and Failure Identification," IEEE Transactions on Intelligent Transportation Systems, vol. 18, no. 9, pp. 2351-2363, Sept. 2017. https://doi.org/10.1109/TITS.2016.2639020
  8. J. Redmon, S. Divvala, R. Girshick, A. Farhadi, "You Only Look Once: Unified, Real-Time Object Detection," IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 779-788. Nevada, USA, Jun. 2016.
  9. J. Redmon, A. Farhadi, "YOLO9000: Better, Faster, Stronger," IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 6517-6525, Hawaii, USA, Jun. 2017.
  10. S.M. Silva, C.R. Jung, "Real-time brazilian license plate detection and recognition using deep convolutional neural networks," SIBGRAPI Conference on Graphics, Patterns and Images (SIBGRAPI), pp. 55-62, Rio de Janeiro, Brazil, Oct. 2017.
  11. G.S. Hsu, A. Ambikapathi, S.L. Chung, C.P. Su, "Robust license plate de-tection in the wild," IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS), pp. 1-6, Lecce Italy, Aug. 2017.
  12. L. Xie, T. Ahmad, L. Jin, Y. Liu, S. Zhang, "A New CNN-Based Method for Multi-Directional Car License Plate Detection," IEEE Transactions on Intelligent Transportation Systems, Volume 19, pp. 507-517, Feb. 2018. https://doi.org/10.1109/tits.2017.2784093
  13. R. Laroca, E. Severo, L.A. Zanlorensi, L.S. Oliveira, G.R. Goncalves, W.R. Schwartz, D. Menotti, "A robust real-time automatic license plate recognition based on the YOLO detector," International Joint Conference on Neural Networks (IJCNN), Rio de Janeiro, Brazil, Jul. 2018.
  14. D.K. Francisco, R. Minetto, B.T. Nassu, "Convolutional neural networks for license plate detection in images," IEEE International Conference on Image Processing (ICIP), pp. 3395-3399, Athens, Greece, Sep. 2017.
  15. Z. Selmi, H.M. Ben, A.M. Alimi, "Deep Learning System for Automatic License Plate Detection and Recognition," IAPR International Conference on Document Analysis and Recognition (ICDAR), pp. 1132-1138, Kyoto, Japan, Nov. 2017.
  16. W. Liu, D. Anguelov, D. Erhan, C. Szegedy, S. Reed, C.Y. Fu, A.C. Berg, "SSD: Single Shot MultiBox Detector," European Conference on Computer Vision (ECCV), pp. 21-37, Amsterdam, Netherlands, Oct. 2016.
  17. S. Ren, K. He, R. Girshick, J. Sun, "Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks," IEEE Transactions on Pattern Analysis and Machine Intelligence, Volume 39, pp. 1137-1149, Jun. 2017. https://doi.org/10.1109/TPAMI.2016.2577031
  18. M. Jaderberg, K. Simonyan, A. Vedaldi, A. Zisserman, "Synthetic Data and Artificial Neural Networks for Natural Scene Text Recognition," Conference on Neural Information Processing Systems, pp. 1-10, Montreal, Canada, Dec. 2014.
  19. A. Gupta, A. Vedaldi, A. Zisserman, "Synthetic Data for Text Localisation in Natural Images," IEEE Conference on Computer Vision and Pattern Recognition, Nevada, USA, 2016.
  20. F. Wang, L. Zhao, X. Li, X. Wang, D. Tao, "Geometry-Aware Scene Text Detection with Instance Transformation Network," IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1381-1389, Utah, USA, Jun. 2018.
  21. H. Li, P. Wang, C. Shen, "Towards end-to-end car license plates detection and recognition with deep neural networks," http://arxiv.org/abs/1709.08828, (accessed Feb., 8, 2020).
  22. S. M. Silva and C. R. Jung, "License plate detection and recognition in unconstrained scenarios," European Conference on Computer Vision, pp. 593-609, Munich, Germany, Sept. 2018.
  23. T.Y. Lin, "Microsoft COCO: Common Objects in Context," European Conference on Computer Vision(ECCV), Lecture Notes in Computer Science, vol 8693, Zurich, Switzland, Sept. 2014, (accessed on Feb, 24, 2020).
  24. J. Spanhel, J. Sochor, R. Juranek, A. Herout, L. Marsik and P. Zemcik, "Holistic recognition of low quality license plates by CNN using track annotated data," IEEE International Conference on Advanced Video and Signal Based Surveillance(AVSS), pp. 1-6, Lecce, Italy, Aug. 2017.
  25. OpenALPR Benchmarks, https://github.com/openalpr/benchmarks (accessed Feb., 20, 2020).
  26. J. Redmon, A. Farhadi, "YOLOv3: An Incremental Improvement," https://arxiv.org/abs/1709.08828, (accessed Feb., 8, 2020).
  27. M. Tan, R. Pang, Q. V. Le, "EfficientDet: Scalable and Efficient Object Detection," The IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 10781-10790, Washington, USA, Jun. 2020.
  28. K. Simonyan, A. Zisserman. "Very deep convolutional networks for large-scale image recognition," The International Conference on Learning Representations(ICLR), California, USA, May. 2015.
  29. M. Tan, Q. Le, "EfficientNet: Rethinking model scaling for convolutional neural networks," https://arxiv.org/abs/1905.11946, (accessed Feb., 12, 2020).
  30. C. Ledig, L. Theis, F. Huszar, J. Caballero, A. Cunningham, A. Acosta, A. Aitken, A. Tejani, J. Totz, Z. Wang, W. S. Twitter, "Photo-realistic single image Super-Resolution using a generative adversarial network," https://arxiv.org/abs/1609.04802, (accessed Feb., 15, 2020).
  31. Fast-SRGAN. https://github.com/HasnainRaz/Fast-SRGAN (accessed Feb., 12, 2020).
  32. O. Prakash, J. Gwak, M. Khare, A. Khare and M. Jeon, "Human detection in complex real scenes based on combination of biorthogonal wavelet transform and Zernike moments," Optik - International Journal for Light and Electron Optics, vol. 157, pp. 1267-1281, 2018. https://doi.org/10.1016/j.ijleo.2017.12.061