DOI QR코드

DOI QR Code

A Study of CNN-based Super-Resolution Method for Remote Sensing Image

원격 탐사 영상을 활용한 CNN 기반의 초해상화 기법 연구

  • Choi, Yeonju (Artificial Intelligence Research Section, Korea Aerospace Research Institute) ;
  • Kim, Minsik (Naraspace Technology) ;
  • Kim, Yongwoo (Department of System Semiconductor Engineering, Sangmyung University) ;
  • Han, Sanghyuck (Artificial Intelligence Research Section, Korea Aerospace Research Institute)
  • 최연주 (한국항공우주연구원 인공지능연구실) ;
  • 김민식 (나라스페이스 테크놀로지) ;
  • 김용우 (상명대학교 시스템반도체공학과) ;
  • 한상혁 (한국항공우주연구원 인공지능연구실)
  • Received : 2020.06.04
  • Accepted : 2020.06.16
  • Published : 2020.06.30

Abstract

Super-resolution is a technique used to reconstruct an image with low-resolution into that of high-resolution. Recently, deep-learning based super resolution has become the mainstream, and applications of these methods are widely used in the remote sensing field. In this paper, we propose a super-resolution method based on the deep back-projection network model to improve the satellite image resolution by the factor of four. In the process, we customized the loss function with the edge loss to result in a more detailed feature of the boundary of each object and to improve the stability of the model training using generative adversarial network based on Wasserstein distance loss. Also, we have applied the detail preserving image down-scaling method to enhance the naturalness of the training output. Finally, by including the modified-residual learning with a panchromatic feature in the final step of the training process. Our proposed method is able to reconstruct fine features and high frequency information. Comparing the results of our method with that of the others, we propose that the super-resolution method improves the sharpness and the clarity of WorldView-3 and KOMPSAT-2 images.

References

  1. Arjovsky, M., S. Chintala, and L. Bottou, 2017. Wasserstein GAN, arXiv preprint arXiv:1701.07875.
  2. Bosch, M., C. M. Gifford, and P. A. Rodriguez, 2018. Super-Resolution for Overhead Imagery Using DenseNets and Adversarial Learning, Proc. of 2018 IEEE Winter Conference on Applications of Computer Vision (WACV), Lake Tahoe, NV, Mar. 12-15, pp. 1414-1422.
  3. Canny, J., 1986. A computational approach to edge detection, IEEE Transactions on Pattern Analysis and Machine Intelligence, 8(6): 679-698.
  4. Dong, C., C.C. Loy, K. He, and X. Tang, 2014. Learning a deep convolutional network for image superresolution, Proc. of the European Conference on Computer Vision, Zurich, Sep. 6-12, vol. 8692, pp. 184-199.
  5. Feldman, J. A., G. M. Feldman, G. Falk, G. Grape, J. Pearlman, I. Sobel, and J. M. Tenebaum, 1969. The stanford hand-eye project, Proc. of International Joint Conferences on Artificial Intelligence Organization, Washington, D.C., May 7-9, pp. 521-526.
  6. Fernandez-Beltran, R., P. Latorre-Carmona, and F. Pla, 2017. Single-frame super-resolution in remote sensing: a practical overview, International Journal of Remote Sensing, 38(1): 314-354. https://doi.org/10.1080/01431161.2016.1264027
  7. Goodfellow, I., J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. Courville, and Y. Bengio, 2014. Generative adversarial nets, Proc. of Neural Information Processing Systems Conference, Montreal, Dec. 8-13, pp. 2672-2680.
  8. Haris, M., G. Shakhnarovich, and N. Ukita, 2018. Deep back-projection networks for super-resolution, Proc. of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, Jun. 18-22, pp. 1664-1673.
  9. He, K., X. Zhang, S. Ren, and J. Sun, 2015. Delving deep into rectifiers: Surpassing human-level performance on ImageNet classification, Proc. of the IEEE International Conference on Computer Vision, Las Condes, Dec. 11-18, pp. 1026-1034.
  10. Johnson, J., A. Alahi, and L. Fei-Fei, 2016. Perceptual losses for real-time style transfer and superresolution, Proc. of the European Conference on Computer Vision, Amsterdam, Oct. 11-14, pp. 694-711.
  11. Keys, R., 1981. Cubic convolution interpolation for digital image processing, IEEE Transactions on Acoustics, Speech, and Signal Processing, 29(6): 1153-1160. https://doi.org/10.1109/TASSP.1981.1163711
  12. Kim, J., J. K. Lee, and K. M. Lee, 2016. Accurate image super-resolution using very deep convolutional networks, Proc. of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, Jun. 27-30, pp. 1646-1654.
  13. Kingma, D. and J. Ba, 2014. Adam: A method for stochastic optimization, arXiv preprint arXiv:1412.6980.
  14. Ledig, C., L. Theis, F. Huszar, J. Caballero, A. Cunningham, A. Acosta, A. Aitken, A. Tejani, J. Totz, Z. Wang, and W. Shi, 2017. Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network, Proc. of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, Jul. 21-26, pp. 105-114.
  15. Liebel, L. and M. Korner, 2016. Single-Image Super Resolution for Multispectral Remote Sensing Data Using Convolutional Neural Networks, Proc. of 2016 XXIII ISPRS Congress, Prague, Jul. 12-19, vol. XLI-B3, pp. 883-890.
  16. Park, S. C., M. K. Park, and M. G. Kang, 2003. Superresolution image reconstruction: a technical overview, IEEE Signal Processing Magazine, 20(3): 21-36. https://doi.org/10.1109/MSP.2003.1203207
  17. Simonyan, K. and A. Zisserman, 2014. Very deep convolutional networks for large scale image recognition, arXiv preprint arXiv:1409.1556.
  18. SpaceNet, 2020. SpaceNet on AWS, http://explore.digital globe.com/spacenet, Accessed on Jun. 16, 2020.
  19. Wang, X., K. Yu, S. Wu, J. Gu, Y. Liu, C. Dong, C. C. Loy, Y. Qiao, and X. Tang, 2018. ESRGAN: Enhanced super-resolution generative adversarial networks, Proc. of the European Conference on Computer Vision, Munich, Sep. 8-14, pp. 1-16.
  20. Wang, Z., A. C. Bovik, H. R. Sheikh, and E. P. Simoncell, 2004. Image quality assessment: from error visibility to structural similarity, IEEE Transactions on Image Processing, 13(4): 600-612. https://doi.org/10.1109/TIP.2003.819861
  21. Weber, N., M. Waechter, S.C. Amend, S. Guthe, and M. Goesele, 2016. Rapid, Detail-Preserving Image Downscaling, Association for Computing Machinery Transactions on Graphics, 35(6): 205.
  22. Yan, Q., Y. Xu, X. Yang, and T. Q. Nguyen, 2015. Single image super resolution based on gradient profile sharpness, IEEE Transactions on Image Processing, 24(10): 3187-3202. https://doi.org/10.1109/TIP.2015.2414877