Journal of the Korea Society of Computer and Information (한국컴퓨터정보학회논문지)

Korean Society of Computer Information (한국컴퓨터정보학회)
권호 표지
DOI QR코드

DOI QR Code

Multi Cycle Consistent Adversarial Networks for Multi Attribute Image to Image Translation

  • Jo, Seok Hee (Dept. of Computer Science, Inha Technical College) ;
  • Cho, Kyu Cheol (Dept. of Computer Science, Inha Technical College)
  • Received : 2020.06.23
  • Accepted : 2020.08.25
  • Published : 2020.09.29
Download PDF
⟨ Previous Next ⟩

Abstract

Image-image translation is a technology that creates a target image through input images, and has recently shown high performance in creating a more realistic image by utilizing GAN, which is a non-map learning structure. Therefore, there are various studies on image-to-image translation using GAN. At this point, most image-to-image translations basically target one attribute translation. But the data used and obtainable in real life consist of a variety of features that are hard to explain with one feature. Therefore, if you aim to change multiple attributes that can divide the image creation process by attributes to take advantage of the various attributes, you will be able to play a better role in image-to-image translation. In this paper, we propose Multi CycleGAN, a dual attribute transformation structure, by utilizing CycleGAN, which showed high performance among image-image translation structures using GAN. This structure implements a dual transformation structure in which three domains conduct two-way learning to learn about the two properties of an input domain. Experiments have shown that images through the new structure maintain the properties of the input area and show high performance with the target properties applied. Using this structure, it is possible to create more diverse images in the future, so we can expect to utilize image generation in more diverse areas.

이미지-이미지 변환은 입력 이미지를 통해서 목적 이미지를 만들어내는 기술로 최근 비지도 학습 구조인 GAN을 활용하여 더 실제와 같은 이미지를 만들어내는 높은 성과를 보였다. 이에 따라 GAN을 활용한 이미지-이미지 변환 연구는 다양하게 진행되고 있다. 이때 일반적으로 이미지-이미지 변환은 하나의 속성 변환을 목표한다. 그러나 실제 생활에서 사용되고 얻을 수 있는 자료들은 한 가지 특징으로 설명하기 힘든 다양한 특징으로 이루어진다. 그래서 다양한 속성을 활용하기 위하여 속성별로 이미지 생성 과정을 나누어 학습할 수 있도록 하는 다중 속성 변화를 목표로 한다면 더 이미지-이미지 변환의 역할을 잘 수행할 수 있을 것이다. 본 논문에서는 GAN을 활용한 이미지-이미지 변환 구조 중 높은 성과를 보인 CycleGAN을 활용해 이중 속성 변환 구조인 Multi CycleGAN을 제안한다. 이 구조는 입력 도메인을 두 가지의 속성에 대하여 학습하기 위하여 3개의 도메인이 양방향 학습을 진행하는 이중 변환 구조를 구현하였다. 새로운 구조를 통해 생성된 이미지와 기존 이미지-이미지 변환 구조들을 통해 생성된 이미지를 비교할 수 있도록 실험을 진행하였다. 실험 결과 새로운 구조를 통한 이미지는 입력 도메인의 속성을 유지하며 목표한 속성이 적용되는 높은 성능을 보였다. 이 구조를 활용한다면 앞으로 더 다양한 이미지를 생성하는 일이 가능지기 때문에 더 다양한 분야에서의 이미지 생성의 활용을 기대할 수 있다.

Keywords

References

  1. Choi, Y., Choi, M., Kim, M., Ha, J. W., Kim, S., and Choo, J., "Stargan: Unified generative adversarial networks for multi-doma in image to image translation", In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 8789-8797, 2018
  2. Gatys, L. A. and Ecker, A. S. and Bethge, M., "Image style transfer using convolutional neural network"s, In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 2414-2423, 2016
  3. Girshick, R., "Fast r-cnn", In Proceedings of the IEEE international conference on computer vision, pp. 1440-1448, 2015
  4. Goodfellow, I., Pouget-Abadie, J., Mirza, M., Xu, B., Warde- Farley, D., Ozair, S., ... and Bengio, Y., "Generative adversarial nets", In Advances in neural information processing systems, pp. 2672-2680, 2014
  5. He, K., Zhang, X., Ren, S., and Sun, J., "Deep residual learning for image recognition", In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 770-778, 2016
  6. Isola, P., Zhu, J. Y., Zhou, T., and Efros, A. A., "image to image translation with conditional adversarial networks", In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 1125-1134, 2017
  7. Kingma, D. P., and Ba, J., "Adam: A method for stochastic optimization", arXiv preprint arXiv:1412.6980, 2014
  8. Ledig, C., Theis, L., Huszár, F., Caballero, J., Cunningham, A., Acosta, A., ... and Shi, W., "Photo-realistic single image super-resolution using a generative adversarial network", In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 4681-4690. 2017
  9. Li, C., and Wand, M., "Precomputed real-time texture synthesis with markovian generative adversarial networks", In European conference on computer vision, pp. 702-716. Springer, Cham. 2016, October
  10. Lin, J., Xia, Y., Wang, Y., Qin, T., and Chen, Z., "image to image translation with multi-path consistency regularization", arXiv preprint arXiv:1905.12498, 2019
  11. Liu, Z., Luo, P., Wang, X., and Tang, X., "Deep learning face attributes in the wild", In Proceedings of the IEEE international conference on computer vision, pp. 3730-3738. 2015
  12. Mao, X., Li, Q., Xie, H., Lau, R. Y., Wang, Z., and Paul Smolley, S., "Least squares generative adversarial networks", In Proceedings of the IEEE International Conference on Computer Vision, pp. 2794-2802, 2017
  13. Mao, X., Shen, C., and Yang, Y. B., "Image restoration using very deep convolutional encoder-decoder networks with symmetric skip connections", In Advances in neural information processing systems, pp. 2802-2810, 2016
  14. Radford, A., Metz, L., and Chintala, S., "Unsupervised representation learning with deep convolutional generative adversarial networks", arXiv preprint arXiv:1511.06434, 2015
  15. Zhou, T., Krahenbuhl, P., Aubry, M., Huang, Q., and Efros, A. A., "Learning dense correspondence via 3d-guided cycle consistency", In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 117-126, 2016
  16. Zhu, J. Y., Park, T., Isola, P., and Efros, A. A., "Unpaired image to image translation using cycle-consistent adversarial networks", In Proceedings of the IEEE international conference on computer vision, pp. 2223-2232, 2017