Journal of the Korea Institute of Information and Communication Engineering (한국정보통신학회논문지)

The Korea Institute of Information and Commucation Engineering (한국정보통신학회)
권호 표지
DOI QR코드

DOI QR Code

Image Super-Resolution for Improving Object Recognition Accuracy

객체 인식 정확도 개선을 위한 이미지 초해상도 기술

  • Lee, Sung-Jin (Department of AI Convergence, Chonnam National University) ;
  • Kim, Tae-Jun (Department of SW Engineering, Chonnam National University) ;
  • Lee, Chung-Heon (Department of SW Engineering, Chonnam National University) ;
  • Yoo, Seok Bong (Department of AI Convergence, Chonnam National University)
  • Received : 2021.04.14
  • Accepted : 2021.04.23
  • Published : 2021.06.30
Download PDF
⟨ Previous Next ⟩

Abstract

The object detection and recognition process is a very important task in the field of computer vision, and related research is actively being conducted. However, in the actual object recognition process, the recognition accuracy is often degraded due to the resolution mismatch between the training image data and the test image data. To solve this problem, in this paper, we designed and developed an integrated object recognition and super-resolution framework by proposing an image super-resolution technique to improve object recognition accuracy. In detail, 11,231 license plate training images were built by ourselves through web-crawling and artificial-data-generation, and the image super-resolution artificial neural network was trained by defining an objective function to be robust to the image flip. To verify the performance of the proposed algorithm, we experimented with the trained image super-resolution and recognition on 1,999 test images, and it was confirmed that the proposed super-resolution technique has the effect of improving the accuracy of character recognition.

객체 검출 및 인식 과정은 컴퓨터비전 분야에서 매우 중요한 과업으로써, 관련 연구가 활발하게 진행되고 있다. 그러나 실제 객체 인식 과정에서는 학습된 이미지 데이터와 테스트 이미지 데이터간 해상도 차이로 인하여 인식기의 정확도 성능이 저하되는 문제가 종종 발생한다. 이를 해결하기 위해 본 논문에서는 객체 인식 정확도 향상을 위한 이미지 초해상도 기법을 제안하여 객체 인식 및 초해상도 통합 프레임워크를 설계하고 개발하였다. 세부적으로는 11,231장의 차량 번호판 훈련용 이미지를 웹 크롤링, 인조데이터 생성 등을 통해 자체적으로 구축하고, 이를 활용하여 이미지 좌우 반전에 강인하도록 목적함수를 정의하여 이미지 초해상도 인공 신경망을 훈련시켰다. 제안 방법의 성능을 검증하기 위해 훈련된 이미지 초해상도 및 번호 인식기 1,999장의 테스트 이미지에 실험하였고, 이를 통해 제안한 초해상도 기법이 문자 인식 정확도 개선 효과가 있음을 확인하였다.

Keywords

Acknowledgement

This work was supported by Institute of Information & Communications Technology Planning & Evaluation(IITP) grant funded by the Korea government(MSIT) (No.2020-0-00004, Development of Previsional Intelligence based on Long-term Visual Memory Network)

References

  1. C. Dong, C. C. Loy, K. He, and X. Tang, "Image super-resolution using deep convolutional networks," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 38, no. 2, pp. 295-307, Feb. 2016. https://doi.org/10.1109/TPAMI.2015.2439281
  2. W. T. Freeman, T. R. Jones, and E. C. Pasztor, "Example-bas ed super-resolution," IEEE Computer Graphics and Applications, vol. 22, no. 2, pp. 56-65, Mar-Apr. 2002. https://doi.org/10.1109/38.988747
  3. J. Yang, J. Wright, T. Huang, and Y. Ma, "Image super-resolution as sparse representation of raw image patches," in Proceeding of the IEEE Conference on Computer Vision and Pattern Recognition, Anchorage, AK, USA, pp. 1-8, 2008.
  4. C. Dong, C. C. Loy, and X. Tang, "Accelerating the super-resolution convolutional neural network," in Proceeding of the European Conference on Computer Vision, Springer, Cham. pp. 391-407, Aug. 2016.
  5. W. Shi, J. Caballero, F. Huszar, J. Totz, A. P. Aitken, R. Bishop, D. Rueckert, and Z. Wang, "Real-time single image and video super-resolution using an efficient sub-pixel convolutional neural network," in Proceeding of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1874-1883, Sep. 2016.
  6. K. He, X. Zhang, S. Ren, and J. Sun, "Deep residual learning for image recognition," in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770-778, Jun-Jul. 2016.
  7. B. Lim, S. Son, H. Kim, S. Nah, and K. M. Lee, "Enhanced Deep Residual Networks for Single Image Super-Resolution," in Proceeding of the 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops, Honolulu, HI, USA, pp. 1132-1140, 2017.
  8. K. Simonyan and A. Zisserman. "Very deep convolutional networks for large-scale image recognition," arXiv preprint arXiv:1409.1556, 2014.
  9. J. W. Kim, J. K. Lee, and K. M. Lee, "Accurate image super-resolution using very deep convolutional networks," in Proceeding of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1646-1654, 2016.
  10. G. Huang, Z. Liu, L. Maaten, and K. Q. Weinberger, "Densely Connected Convolutional Networks," in Proceeding of the 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops, Honolulu, HI, USA, pp. 2261-2269, 2017.
  11. M. Haris, G. Shakhnarovich, and N. Ukita, "Deep back-projection networks for super-resolution," in Proceeding of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1664-1673, Mar. 2018.
  12. Y. Zhang, Y. Tian, Y. Kong, B. Zhong, and Y. Fu, "Residual dense network for image super-resolution," in Proceeding of the IEEE conference on Computer Vision and Pattern Recognition, pp. 2472-2481, Mar. 2018.
  13. Y. Guo, J. Chen, J. Wang, Q. Chen, J. Cao, Z. Deng, Y. Xu, and M. Tan, "Closed-loop Matters: Dual Regression Networks for Single Image Super-Resolution," in Proceeding of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 5407-5416, May. 2020.
  14. K. Zhang, L. V. Gool, and R. Timofte, "Deep unfolding network for image super-resolution," in Proceeding of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3217-3226, Mar. 2020.
  15. J. W. Soh, S. Cho, and N. I. Cho, "Meta-Transfer Learning for Zero-Shot Super-Resolution," in Proceeding of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3516-3525, Feb. 2020.
  16. J. H. Lee, S. M. Cho, S. J. Lee, and C. H. Kim, "License Plate Recognition System Using Synthetic Data," Journal of the Institute of Electronics and Information Engineers, vol. 57, no. 1, pp. 107-115, Jan. 2020. https://doi.org/10.5573/ieie.2020.57.1.107
  17. Y. J. Lee, S. J. Kim, K. M. Park, and K. M. Park, "Comparison of number plate recognition performance of Synthetic number plate generator using 2D and 3D rotation," The Korean Institute of Broadcast and Media Engineers Summer Conference, pp. 141-144, Jul. 2020.
  18. Z. Sergey and G. Alexey, "LPRNet: License Plate Recognition via Deep Neural Networks," arXiv preprint arXiv:1806.10447, Jun. 2018.
  19. W. Liu, D. Anguelov, D. Derhan, C. Szegedy, S. Reed, C. Y. Fu, and A. C. Berg, "SSD: Single Shot MultiBox Detector," in Proceeding of the European Conference on Computer Vision, pp. 21-37, Dec. 2016.
  20. Z. Xu, W. Yang, A. Meng, N. Lu, H. Huang, C. Ying, and L. Huang, "Towards end-to-end license plate detection and recognition: A large dataset and baseline," in Proceeding of the European Conference on Computer Vision, pp. 261-277, Sep. 2018.
  21. H. C. Lee, "Design and Implementation of Efficient Place Number Region Detecting System in Vehicle Number Plate Image," Korea Society Of Computer Information Journal, vol. 10, no. 5, pp. 87-93, Nov. 2005.
  22. Translate darknet to tensorflow [Internet]. Available: https://github.com/thtrieu/darkflow.
  23. S. B. Yoo and M. Han, "Temporal matching prior network for vehicle license plate detection and recognition in videos," ETRI Journal, vol. 42, no. 3, pp. 411-419, Feb. 2020. https://doi.org/10.4218/etrij.2019-0245
  24. J. H. Baek and N. H. Kim, "Noise Removal Method using Entropy in High-Density Noise Environments," Journal of the Korea Institute of Information and Communication Engineering, vol. 24, no. 10, pp. 1255-1261, Oct. 2020. https://doi.org/10.6109/JKIICE.2020.24.10.1255
  25. D. J. Kim and P. L. Manjusha, "Building Detection in High Resolution Remotely Sensed Images based on Automatic Histogram-Based Fuzzy C-Means Algorithm," Asia-pacific Journal of Convergent Research Interchange, vol. 3, no. 1, pp. 57-62, Mar. 2017. https://doi.org/10.21742/apjcri.2017.12.11