The Journal of Society for e-Business Studies (한국전자거래학회지)

Society for e-Business Studies (한국전자거래학회)
권호 표지
DOI QR코드

DOI QR Code

Multimodal Sentiment Analysis Using Review Data and Product Information

리뷰 데이터와 제품 정보를 이용한 멀티모달 감성분석

  • Hwang, Hohyun (Seoul National University of Science and Technology) ;
  • Lee, Kyeongchan (Seoul National University of Science and Technology) ;
  • Yu, Jinyi (Seoul National University of Science and Technology) ;
  • Lee, Younghoon (Department of Industrial & Systems Engineering, Seoul National University of Science and Technology)
  • Received : 2021.09.07
  • Accepted : 2021.12.21
  • Published : 2022.02.28
Download PDF
⟨ Previous Next ⟩

Abstract

Due to recent expansion of online market such as clothing, utilizing customer review has become a major marketing measure. User review has been used as a tool of analyzing sentiment of customers. Sentiment analysis can be largely classified with machine learning-based and lexicon-based method. Machine learning-based method is a learning classification model referring review and labels. As research of sentiment analysis has been developed, multi-modal models learned by images and video data in reviews has been studied. Characteristics of words in reviews are differentiated depending on products' and customers' categories. In this paper, sentiment is analyzed via considering review data and metadata of products and users. Gated Recurrent Unit (GRU), Long Short-Term Memory (LSTM), Self Attention-based Multi-head Attention models and Bidirectional Encoder Representation from Transformer (BERT) are used in this study. Same Multi-Layer Perceptron (MLP) model is used upon every products information. This paper suggests a multi-modal sentiment analysis model that simultaneously considers user reviews and product meta-information.

최근 의류 등의 특정 쇼핑몰의 온라인 시장이 크게 확대되면서, 사용자의 리뷰를 활용하는 것이 주요한 마케팅 방안이 되었다. 이를 이용한 감성분석에 대한 연구들도 많이 진행되고 있다. 감성분석은 사용자의 리뷰를 긍정과 부정 그리고 필요에 따라서 중립으로 분류하는 방법이다. 이 방법은 크게 머신러닝 기반의 감성분석과 사전기반의 감성분석으로 나눌 수 있다. 머신러닝 기반의 감성분석은 사용자의 리뷰 데이터와 그에 대응하는 감성 라벨을 이용해서 분류 모델을 학습하는 방법이다. 감성분석 분야의 연구가 발전하면서 리뷰와 함께 제공되는 이미지나 영상 데이터 등을 함께 고려하여 학습하는 멀티모달 방식의 모델들이 연구되고 있다. 리뷰 데이터에서 제품의 카테고리와 사용자별로 사용되는 단어 등의 특징이 다르다. 따라서 본 논문에서는 리뷰데이터와 제품 정보를 동시에 고려하여 감성분석을 진행한다. 리뷰를 분류하는 모델로는 기본 순환신경망 구조에서 Gate 방식을 도입한 Gated Recurrent Unit(GRU), Long Short-Term Memory(LSTM) 그리고 Self Attention 기반의 Multi-head Attention 모델, Bidirectional Encoder Representation from Transformer(BERT)를 사용해서 각각 성능을 비교하였다. 제품 정보는 모두 동일한 Multi-Layer Perceptron(MLP) 모델을 이용하였다. 본 논문에서는 사용자 리뷰를 활용한 Baseline Classifier의 정보와 제품 정보를 활용한 MLP모델의 결과를 결합하는 방법을 제안하며 실제 데이터를 통해 성능의 우수함을 보인다.

Keywords

References

  1. Agarwal, A., Yadav, A., and Vishwakarma, D. K., "Multimodal sentiment analysis via RNN variants," 2019 IEEE International Conference on Big Data, Cloud Computing, Data Science & Engineering (BCD), IEEE, 2019.
  2. Ambartsoumian, A. and Popowich, F., "Self-attention: A better building block for sentiment analysis neural network classifiers," arXiv preprint arXiv:1812.07860, 2018.
  3. Bahdanau, D., Cho, K., and Bengio, Y., "Neural machine translation by jointly learning to align and translate," arXiv preprint arXiv:1409.0473, 2014.
  4. Baltrusaitis, T., Ahuja, C., and Morency, L., "Multimodal machine learning: A survey and taxonomy," IEEE transactions on pattern analysis and machine intelligence, Vol. 41, No. 2, pp. 423-443, 2018. https://doi.org/10.1109/tpami.2018.2798607
  5. Cho, K., Van Merrienboer, B., Gulcehre, C., Bahdanau, D., Bougares, F., Schwenk, H., and Bengio, Y., "Learning phrase representations using RNN encoder-decoder for statistical machine translation," arXiv preprint arXiv:1406.1078, 2014.
  6. Devlin, J., Chang, M., Lee, K., and Toutanova, K., "Bert: Pre-training of deep bidirectional transformers for language understanding," arXiv preprint arXiv:1810.04805, 2018.
  7. Gruber, N. and Jockisch, A., "Are GRU cells more specific and LSTM cells more sensitive in motive classification of text?," Frontiers in Artificial Intelligence, Vol. 3, 2020.
  8. Hochreiter, S. and Schmidhuber, J., "Long short-term memory," Neural Computation, Vol. 9. No. 8, pp. 1735-1780, 1997. https://doi.org/10.1162/neco.1997.9.8.1735
  9. Huang, Q., Chen, R., Zheng, X., and Dong, Z., "Deep sentiment representation based on CNN and LSTM", 2017 International Conference on Green Informatics (ICGI), IEEE, 2017.
  10. Hwang, S. and Kim, D., "BERT-based Classification Model for Korean Documents," The Journal of Society for e-Business Studies, Vol. 25, No. 1, 2020.
  11. Jeon, W., Lee, Y., and Geum, Y., "Airline Service Quality Evaluation Based on Customer Review Using Machine Learning Approach and Sentiment Analysis," The Journal of Society for e-Business Studies, Vol. 26, No. 4, pp. 15-36, 2021.
  12. Jin, Z., Cao, J., Guo, H., Zhang, Y., and Luo, J., "Multimodal fusion with recurrent neural networks for rumor detection on microblogs," Proceedings of the 25th ACM international conference on Multimedia, 2017.
  13. Kingma, D. P. and Ba, J., "Adam: A method for stochastic optimization," arXiv preprint arXiv:1412.6980, 2014.
  14. Mohan, S., Mullapudi, S., Sammeta, S., Vijayvergia, P., and Anastasiu, D. C., "Stock price prediction using news sentiment analysis," 2019 IEEE Fifth International Conference on Big Data Computing Service and Applications (BigDataService), IEEE, 2019.
  15. Oh, P. and Hwang, B., "Real-time Spatial Recommendation System based on Sentiment Analysis of Twitter," The Journal of Society for e-Business Studies, Vol. 21, No. 3, pp. 15-28, 2016.
  16. Pannala, N. U., Nawarathna, C. P., Jayakody, J. T. K., Rupasinghe, L., and Krishnadeva, K., "Supervised learning based approach to aspect based sentiment analysis," 2016 IEEE International Conference on Computer and Information Technology (CIT), IEEE, 2016.
  17. Pires, T., Schlinger, E., and Garrette, D., "How Multilingual is Multilingual BERT?," Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pp. 4996-5001, 2019.
  18. Poria, S., Cambria, E., and Gelbukh, A., "Deep convolutional neural network textual features and multiple kernel learning for utterance-level multimodal sentiment analysis," Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, 2015.
  19. Severyn, A. and Moschitti, A., "Twitter sentiment analysis with deep convolutional neural networks," Proceedings of the 38th International ACM SIGIR Conference on Research and Development in Information Retrieval, 2015.
  20. Smeureanu, I. and Zurini, M., "Spam Filtering for Optimization in Internet Promotions using Bayesian Analysis," Journal of Applied Quantitative Methods, Vol. 5, No. 2, pp. 198-211, 2010.
  21. Sun, C., Huang, L., and Qiu, X., "Utilizing BERT for aspect-based sentiment analysis via constructing auxiliary sentence," arXiv preprint arXiv:1903.09588, 2019.
  22. Sutskever, I., Vinyals, O., and Le, Q. V., "Sequence to sequence learning with neural networks," Advances in Neural Information Processing Systems, 2014.
  23. Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., and Polosukhin, I., "Attention is all you need," In Advances in Neural Information Processing Systems, pp. 6000-6010, 2017.
  24. Wang, H., Can, D., Kazemzadeh, A., Bar, F., and Narayanan, S., "A system for real- time twitter sentiment analysis of 2012 us presidential election cycle," Proceedings of the ACL 2012 System Demonstrations, 2012.
  25. Xu, N. and Mao, W., "Multisentinet: A deep semantic network for multimodal sentiment analysis," Proceedings of the 2017 ACM on Conference on Information and Knowledge Management, 2017.
  26. Yang, L., Li, Y., Wang, J., and Sherratt, R. S., "Sentiment analysis for E-commerce product reviews in Chinese based on sentiment lexicon and deep learning," IEEE Access, Vol. 8, pp. 23522-23530, 2020. https://doi.org/10.1109/access.2020.2969854