Smart Media Journal (스마트미디어저널)

THE KOREAN INSTITUTE OF SMART MEDIA (한국스마트미디어학회)
권호 표지
DOI QR코드

DOI QR Code

Image-to-Image Translation with GAN for Synthetic Data Augmentation in Plant Disease Datasets

  • Nazki, Haseeb (Department of Electronics Engineering, Chonbuk National University) ;
  • Lee, Jaehwan (Department of Electronics Engineering, Chonbuk National University) ;
  • Yoon, Sook (Department of Computer Engineering, Mokpo National University) ;
  • Park, Dong Sun (IT Convergence Research Center, Department of Electronics Engineering, Chonbuk National University)
  • Received : 2019.03.08
  • Accepted : 2019.05.05
  • Published : 2019.06.30
Download PDF
⟨ Previous Next ⟩

Abstract

In recent research, deep learning-based methods have achieved state-of-the-art performance in various computer vision tasks. However, these methods are commonly supervised, and require huge amounts of annotated data to train. Acquisition of data demands an additional costly effort, particularly for the tasks where it becomes challenging to obtain large amounts of data considering the time constraints and the requirement of professional human diligence. In this paper, we present a data level synthetic sampling solution to learn from small and imbalanced data sets using Generative Adversarial Networks (GANs). The reason for using GANs are the challenges posed in various fields to manage with the small datasets and fluctuating amounts of samples per class. As a result, we present an approach that can improve learning with respect to data distributions, reducing the partiality introduced by class imbalance and hence shifting the classification decision boundary towards more accurate results. Our novel method is demonstrated on a small dataset of 2789 tomato plant disease images, highly corrupted with class imbalance in 9 disease categories. Moreover, we evaluate our results in terms of different metrics and compare the quality of these results for distinct classes.

Keywords

References

  1. Y. LeCun, Y. Bengio, and G. Hinton, "Deep learning," Nature, vol. 521, pp. 436-444, May 2015 https://doi.org/10.1038/nature14539
  2. K. He, X. Zhang, S. Ren, and J. Sun, " Delving deep into rectifiers: Surpassing human-level performance on ImageNet classification, " 2015 IEEE International Conference on Computer Vision (ICCV). IEEE Computer Society, 2015, pp. 1026-1034
  3. I. Goodfellow, Y. Bengio, and A. Courville, "Deep Learning" MIT Press, 2016, http://www.deeplearningbook.org. ( accessed May 2019)
  4. O. Ronneberger, P. Fischer, and T. Brox, " U-Net: Convolutional networks for biomedical image segmentation, " in Medical Image Computing and Computer-Assisted Intervention - MICCAI 2015. Springer, Cham, pp. 234-241, 2015
  5. Hughes, D.P.; Salathe, M. "An open access repository of images on plant health to enable the development of mobile disease diagnostics" arXiv:1511.08060v2, 2016
  6. Ciresan, D.C., Meier, U., Masci, J., Gambardella, L.M., Schmidhuber, J. "High performance neural networks for visual object classification" arXiv preprint arXiv:1102.0183, (2011)
  7. Krizhevsky, A., Sutskever, I., Hinton, G.E. "Imagenet classification with deep convolutional neural networks" NIPS. pp. 1097-1105, 2012
  8. A. Rozantsev, V. Lepetit, and P. Fua, "On rendering synthetic images for training an object detector, " Computer Vision and Image Understanding (CVIU), vol. 137, pp. 24-37, 2015. https://doi.org/10.1016/j.cviu.2014.12.006
  9. G. Riegler, M. Urschler, M. Ruther, H. Bischof, and D. Stern, " Anatomical landmark detection in medical applications driven by synthetic data, " 2015 IEEE International Conference on Computer Vision Workshop (ICCVW), pp. 85-89. Dec 2015
  10. I. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. Courville, and Y. Bengio, " Generative Adversarial Nets, " Advances in Neural Information Processing Systems 27 (NIPS). Curran Associates, Inc., pp. 2672-2680, 2014
  11. T.Karras, T. Aila, S. Laine, and J. Lehtinen, " Progressive growing of GANs for improved quality, stability, and variation, " Proceedings of the Sixth International Conference on Learning Representations (ICLR), 2018.
  12. A. Shrivastava, T. Pfister, O. Tuzel, J. Susskind, W. Wang, and R. Webb, "Learning from simulated and unsupervised images through adversarial training, " 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 2242-2251, 2017
  13. T. Neff, C. Payer, D. Stern, and M. Urschler, "Generative Adversarial Network based synthesis for supervised medical image segmentation, " Proceedings of the OAGM&ARW Joint Workshop, pp. 140-145, 2017
  14. Tim Salimans, Ian J. Goodfellow, Wojciech Zaremba, Vicki Cheung, Alec Radford, Xi Chen, "Improved Techniques for Training GANs" CoRR abs/1606.03498, 2016
  15. Japkowicz, N. "The class imbalance problem: Significance and strategies" Proceedings of the 2000 International Conference on Artificial Intelligence (ICAI' 2000). Citeseer, 2000
  16. P. Costa et al., " Towards adversarial retinal image synthesis," arXiv preprint arXiv:1701.08974, 2017.
  17. D. Nie et al., Medical "Image Synthesis with ContextAware Generative Adversarial Networks", Springer International Publishing, Cham, pp.417-425, 2017.
  18. Z. Yi, H. Zhang, P. Tan, and M. Gong, "DualGAN: Unsupervised Dual Learning for Image-to-Image Translation," ArXiv:1704.02510, Apr. 2017.
  19. M.-Y. Liu, T. Breuel, and J. Kautz, " Unsupervised Image-to-Image Translation Networks, " ArXiv:1703.00848, 2017.
  20. J.-Y. Zhu, T. Park, P. Isola, and A. A. Efros, " Unpaired Image-to-Image Translation using CycleConsistent Adversarial Networks," ArXiv:1703.10593, 2017.
  21. Y. Choi, M. Choi, M. Kim, J.-W. Ha, S. Kim, and J. Choo, " StarGAN: Unified Generative Adversarial Networks for Multi-Domain Image-to-Image Translation, " ArXiv:1711.09020, 2017.
  22. A. Radford, L. Metz, and S. Chintala, " Unsupervised representation learning with deep convolutional generative adversarial networks, " arXiv preprint arXiv:1511.06434, 2015.
  23. V. Dumoulin and F. Visin, "A guide to convolution arithmetic for deep learning, " arXiv preprint arXiv:1603.07285, 2016
  24. Tung, H.-Y. F., Harley, A. W., Seto, W., and Fragkiadaki, K. "Adversarial inverse graphics networks: Learning 2d-to3d lifting and image-to-image translation from unpaired supervision" The IEEE International Conference on Computer Vision (ICCV), vol. 2, pp.4364-4372, 2017.
  25. Wolf, L., Taigman, Y., and Polyak, A. "Unsupervised creation of parameterized avatars" arXiv preprint arXiv:1704.05693, 2017.
  26. Hoffman, J., Tzeng, E., Park, T., Zhu, J.-Y., Isola, P., Saenko, K., Efros, A. A., and Darrell, T. Cycada:"Cycleconsistent adversarial domain adaptation" arXiv preprint arXiv:1711.03213, 2017.
  27. Li, C., Liu, H., Chen, C., Pu, Y., Chen, L., Henao, R., and Carin, L. Alice "Towards understanding adversarial learning for joint distribution matching" Advances in Neural Information Processing Systems, pp.5501-5509, 2017.
  28. Ciresan, D.C., Meier, U., Masci, J., Gambardella, L.M., Schmidhuber, J. "High performance neural networks for visual object classification" arXiv preprint arXiv:1102.0183, 2011
  29. Simard, P.Y., Steinkraus, D., Platt, J.C., et al. "Best practices for convolutional neural networks applied to visual document analysis" ICDAR. vol.3, pp.958-962, 2003
  30. I. Gulrajani, F. Ahmed, M. Arjovsky, V. Dumoulin, and A. Courville. "Improved training of wasserstein gans" arXiv preprint arXiv:1704.00028, 2017.
  31. O. Ronneberger, P. Fischer, and T. Brox. "U-net: Convolutional networks for biomedical image segmentation" International Conference on Medical Image Computing and Computer-Assisted Intervention, pp. 234-241, 2015.
  32. M. Heusel, H. Ramsauer, T. Unterthiner, B. Nessler, and S. Hochreiter. "Gans trained by a two time-scale update rule converge to a local nash equilibrium" Advances in Neural Information Processing Systems, pp.6629-6640, 2017.
  33. Talebi, H., Milanfar, P.: "NIMA: neural image assessment," IEEE Trans. Image Process, Vol.27, pp.3998-4011, 2018 https://doi.org/10.1109/TIP.2018.2831899
  34. ChangSeong Kim, DongSuk Lee, Dong Sun Park, "Automatic Pedestrian Removal Algorithm Using Multiple Frames", Smart media journal, vol.4, no.2, pp.26-33, June. 2015
  35. Seo Jeong Kim, Jae Su Lee, Hyong Suk Kim, "Deep learning-based Automatic Weed Detection on Onion Field", Smart media journal, vol.7, no.3, pp.16-21, Sept. 2018 https://doi.org/10.30693/SMJ.2018.7.3.16
  36. Nazki, H., Lee, J., Yoon, S. and Park, D.S., "Synthetic Data Augmentation for Plant Disease Image Generation using GAN," Proceedings of the Korea Contents Association Conference, pp.459-460.