Journal of the Korea Society of Computer and Information (한국컴퓨터정보학회논문지)

Korean Society of Computer Information (한국컴퓨터정보학회)
권호 표지
DOI QR코드

DOI QR Code

Text Augmentation Using Hierarchy-based Word Replacement

  • Kim, Museong (Graduate School of Business IT, Kookmin University) ;
  • Kim, Namgyu (Graduate School of Business IT, Kookmin University)
  • Received : 2020.11.30
  • Accepted : 2021.01.12
  • Published : 2021.01.29
Download PDF
⟨ Previous Next ⟩

Abstract

Recently, multi-modal deep learning techniques that combine heterogeneous data for deep learning analysis have been utilized a lot. In particular, studies on the synthesis of Text to Image that automatically generate images from text are being actively conducted. Deep learning for image synthesis requires a vast amount of data consisting of pairs of images and text describing the image. Therefore, various data augmentation techniques have been devised to generate a large amount of data from small data. A number of text augmentation techniques based on synonym replacement have been proposed so far. However, these techniques have a common limitation in that there is a possibility of generating a incorrect text from the content of an image when replacing the synonym for a noun word. In this study, we propose a text augmentation method to replace words using word hierarchy information for noun words. Additionally, we performed experiments using MSCOCO data in order to evaluate the performance of the proposed methodology.

최근 딥 러닝(Deep Learning) 분석에 이질적인 데이터를 함께 사용하는 멀티모달(Multi-modal) 딥러닝 기술이 많이 활용되고 있으며, 특히 텍스트로부터 자동으로 이미지를 생성해내는 Text to Image 합성에 관한 연구가 활발하게 수행되고 있다. 이미지 합성을 위한 딥러닝 학습은 방대한 양의 이미지와 이미지를 설명하는 텍스트의 쌍으로 구성된 데이터를 필요로 하므로, 소량의 데이터로부터 다량의 데이터를 생성하기 위한 데이터 증강 기법이 고안되어 왔다. 텍스트 데이터 증강의 경우 유의어 대체에 기반을 둔 기법들이 다수 사용되고 있지만, 이들 기법은 명사 단어의 유의어 대체 시 이미지의 내용과 상이한 텍스트를 생성할 가능성이 있다는 한계를 갖는다. 따라서 본 연구에서는 단어가 갖는 품사별 특징을 활용하는 텍스트 데이터 증강 방안, 즉 일부 품사에 대해 단어 계층 정보를 활용하여 단어를 대체하는 방안을 제시하였다. 또한 제안 방법론의 성능을 평가하기 위해 MSCOCO 데이터를 사용하여 실험을 수행하여 결과를 제시하였다.

Keywords

References

  1. I. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. Courville, and Y. Bengio, "Generative Adversarial Nets," Advances in Neural Information Processing Systems 27, 2014.
  2. S. Surya, A. Setlur, A. Biswas, and S. Negi, "ReStGAN: A Step towards Visually Guided Shopper Experience via Text to Image Synthesis," 2020 IEEE Winter Conference on Applications of Computer Vision (WACV), Mar, 2020.
  3. C. Shorten and T. M. Khoshgoftaar, "A Survey on Image Data Augmentation for Deep Learning," Journal of Big Data, No. 60, Feb, 2019.
  4. X. Zhang, J. Zhao, and Y. LeCun, "Character-level Convolutional Networks for Text Classification," Advances in Neural Information Processing Systems 28, 2015.
  5. W. Y. Wang and D. Yang, "That's So Annoying!!!: A Lexical and Frame Semantic Embedding-based Data Augmentation Approach to Automatic Categorization of Annoying Behaviors Using #petpeeve Tweets," Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pp. 2557-2563, Sep, 2015.
  6. T. Salimans, I. Goodfellow, W. Zaremba, V. Cheung, A. Radford, and X. Chen, "Improved Techniques for Training GANs," Advances in Neural Information Processing Systems 29, 2016.
  7. Y. Lecun, L. Bottou, Y. Bengio, and P. Haffner, "Gradient-based Learning Applied to Document Recognition," Proceedings of the IEEE, Vol. 86, No. 11, pp. 2278-2324, 1998. https://doi.org/10.1109/5.726791
  8. T. Mikolov, S. Kombrink, L. Burget, J. Cernocky, and S. Khudanpur, "Extensions of Recurrent Neural Network Language Model," Proceedings of 2011 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 5528-5531, 2011.
  9. S. Reed, Z. Akata, X. Yan, L. Logeswaran, B. Schiele, and H. Lee, "Generative Adversarial Text to Image Synthesis," arXiv:1605.05396, May, 2016.
  10. H. Zhang, T. Xu, H. Li, S. Zhang, X. Wang, X. Huang, and D. N. Metaxas, "StackGAN: Text to Photo Realistic Image Synthesis with Stacked Generative Adversarial Networks," Proceedings of the IEEE International Conference on Computer Vision (ICCV), pp. 5907-5915, 2017.
  11. T. Xu, P. Zhang, Q. Huang, H. Zhang, Z. Gan, X. Huang, and X. He, "AttnGAN: Fine Grained Text to Image Generation with Attentional Generative Adversarial Networks," Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1316-1324, 2018.
  12. T. DeVries and G. W. Taylor, "Dataset Augmentation in Feature Space," arXiv:1702.05538, Feb, 2017.
  13. Y. Li, N. Wang, J. Liu, and X. Hou, "Demystifying Neural Style Transfer," arXiv:1701.01036, Jul, 2017.
  14. C. Bowles, L. Chen, R. Guerrero, P. Bentley, R. Gunn, A. Hammers, D. A. Dickie, M. V. Hernandez, J. Wardlaw, and D. Rueckert, "GAN Augmentation: Augmenting Training Data Using Generative Adversarial Networks," arXiv:1810.10863, Oct, 2018.
  15. T. Mikolov, C. Kai, G. Corrado, and J. Dean, "Efficient Estimation of Word Representations in Vector Space," arXiv:1301.3781, Jan, 2013.
  16. P. Bojanowski, E. Grave, A. Joulin, and T. Mikolov, "Enriching Word Vectors with Subword Information," arXiv:1607.04606, Jul, 2016.
  17. J. Pennington, R. Socher, and C. D. Manning, "Glove: Global Vectors for Word Representation," Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing, pp. 1532-1543, 2014.
  18. J. Devlin, M. W. Chang, K. Lee, and K. Toutanova, "BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding," arXiv:1810.04805, May, 2019.
  19. A. Radford, J. Wu, R. Child, D. Luan, D. Amodei, and I. Sutskever, "Language Models are Unsupervised Multitask Learners," https://cdn.openai.com/better-language-models/language_models_are_unsupervised_multitask_learners.pdf, Feb, 2019.
  20. Q. Xie, Z. Dai, E. Hovy, M. T. Luong, and Q. V. Le, "Unsupervised Data Augmentation for Consistency Training," arXiv:1904.12848, Jun, 2020.
  21. A. Anaby-Tavor, B. Carmeli, E. Goldbraich, A. Kantor, G. Kour, S. Shlomov, N. Tepper, and N. Zwerdling, "Not Enough Data? Deep Learning to the Rescue!," arXiv:1911.03118, Nov, 2019.
  22. V. Kumar, A. Choudhary, and E. Cho, "Data Augmentation Using Pre-trained Transformer Models," arXiv:2003.02245, Mar, 2020.
  23. E. Loper and S. Bird, "NLTK: The Natural Language Toolkit," arXiv:cs/0205028, May, 2002.
  24. Y. Tian and D. Lo, "A comparative study on the effectiveness of part-of-speech tagging techniques on bug reports," 2015 IEEE 22nd International Conference on Software Analysis, Evolution, and Reengineering (SANER), Mar, 2015.
  25. T. Lin, M. Maire, S. Belongie, J. Hays, P. Perona, D. Ramanan, P. Dollar and C. L. Zitnick, "Microsoft COCO: Common Objects in Context," European Conference on Computer Vision, pp. 740-755, 2014.