Journal of Information Technology Services (한국IT서비스학회지)

Korea Society of IT Services (한국IT서비스학회)
권호 표지
DOI QR코드

DOI QR Code

Development of Deep Learning Models for Multi-class Sentiment Analysis

딥러닝 기반의 다범주 감성분석 모델 개발

  • Received : 2017.10.31
  • Accepted : 2017.12.07
  • Published : 2017.12.31
Download PDF
⟨ Previous Next ⟩

Abstract

Sentiment analysis is the process of determining whether a piece of document, text or conversation is positive, negative, neural or other emotion. Sentiment analysis has been applied for several real-world applications, such as chatbot. In the last five years, the practical use of the chatbot has been prevailing in many field of industry. In the chatbot applications, to recognize the user emotion, sentiment analysis must be performed in advance in order to understand the intent of speakers. The specific emotion is more than describing positive or negative sentences. In light of this context, we propose deep learning models for conducting multi-class sentiment analysis for identifying speaker's emotion which is categorized to be joy, fear, guilt, sad, shame, disgust, and anger. Thus, we develop convolutional neural network (CNN), long short term memory (LSTM), and multi-layer neural network models, as deep neural networks models, for detecting emotion in a sentence. In addition, word embedding process was also applied in our research. In our experiments, we have found that long short term memory (LSTM) model performs best compared to convolutional neural networks and multi-layer neural networks. Moreover, we also show the practical applicability of the deep learning models to the sentiment analysis for chatbot.

Keywords

References

  1. Bakliwal, A., J. Foster, J. Van der puil, R. O'Brien, L. Tounsi, and M. Hughes, "Sentiment Analysis of Political Tweets : Towards An Accurate Classifier", Proceedings of the NAACL Workshop on Language Analysis in Social Media. Association for Computational Linguistics, Atlanta, GA, 2013, 50-58.
  2. Bollen, J., H. Mao, and X. Zeng, "Twitter Mood Predicts The Stock Market", Journal of Computational Science, Vol.2, No.1, 2010, 1-8. https://doi.org/10.1016/j.jocs.2010.12.007
  3. Ding, X., B. Liu, and P.S. Yu, "A Holistic Lexiconbased Approach to Opinion Mining", Proceedings of the 2008 International Conference on Web Search and Data Mining, New York, USA, 2008, 231-240.
  4. Dos Santos, C.N. and M. Gatti, "Deep Convolutional Neural Networks for Sentiment Analysis of Short Texts", Proceedings of COLING, the 25th International Conference on Computational Linguistics, Dublin, Ireland, 2014, 23-29.
  5. Feldman, R., "Techniques and Applications for Sentiment Analysis", Communications of the ACM, Vol.56, No.4, 2013, 82-89. https://doi.org/10.1145/2436256.2436274
  6. Hochreiter, S. and J. Schmidhuber, "Long Shortterm Memory", Neural Computation, Vol.9, No.8, 1997, 1735-1780. https://doi.org/10.1162/neco.1997.9.8.1735
  7. Kim, Y., "Convolutional Neural Networks for Sentence Classification", arXiv Preprint arXiv : 1408.5882, 2014.
  8. Krizhevsky, A., I. Sutskever, and G.E. Hinton, "Imagenet Classification with Deep Convolutional Neural Networks", Proceedings of the 25th International Conference on Neural Information Processing Systems, Vol.1, Nevada, USA, 2012, 1097-1105.
  9. Liu, B., Sentiment Analysis and Opinion Mining, Morgan & Claypool Publishers, USA, 2012.
  10. Liu, Y., J.W. Bi, and Z.P. Fan, "Multi-class Sentiment Classification : The Experimental Comparisons of Feature Selection and Machine Learning Algorithms", Expert Systems with Applications, Vol.80, No.1, 2017, 323-339. https://doi.org/10.1016/j.eswa.2017.03.042
  11. Medhat, W., A. Hassan, and H. Korashy, "Sentiment Analysis Algorithms and Applications : A Survey", Ain Shams Engineering Journal, Vol.5, No.4, 2014, 1093-1113. https://doi.org/10.1016/j.asej.2014.04.011
  12. Mikolov, T., K. Chen, G. Corrado, and J. Dean, "Efficient Estimation of Word Representations in Vector Space", arXiv Preprint arXiv : 1301.3781, 2013.
  13. Mullen, T. and N. Collier, "Sentiment Analysis Using Support Vector Machines with Diverse Information Sources", Processing of the Conference on Empirical Methods in Natural Language Processing, Barcelona, Spain, 2004, 412-418.
  14. Pang, B., L. Lee, and S. Vaithyanathan, "Thumbs up? : Sentiment Classification Using Machine Learning Techniques", Proceedings of the Conference on Empirical Methods in Natural Language Processing, Stroudsburg, USA, 2002, 79-86.
  15. Pang, B. and L. Lee, "Opinion Mining and Sentiment Analysis", Foundations and Trends in Information Retrieval, Vol.2, No.1-2, 2008, 1-135. https://doi.org/10.1561/1500000011
  16. Poria, S., E. Cambria, and A. Gelbukh, "Deep Convolutional Neural Network Textual Features and Multiple Kernel Learning for Utterance-level Multimodal Sentiment Analysis", Proceedings of the Conference on Empirical Methods in Natural Language Processing, Lisbon, Portugal, 2015, 2539-2544.
  17. Scherer, K.R. and H.G. Wallbott, "Evidence for Universality and Cultural Variation of Differential Emotion Response Patterning", Journal of Personality and Social Psychology, Vol.66, No.2, 1994, 310-338. https://doi.org/10.1037/0022-3514.66.2.310
  18. Taboada, M., J. Brooke, M. Tofiloski, K. Voll, and M. Stede, "Lexicon-based Methods for Sentiment Analysis", Computational Linguistics, Vol.37, No.2, 2011, 267-307. https://doi.org/10.1162/COLI_a_00049
  19. Tai, K.S., R. Socher, and C.D. Manning, "Improved Semantic Representations from Treestructured Long Short-term Memory Networks", Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing, Beijing, China, 2015, 1556-1566.
  20. Tan, S., X. Cheng, Y. Wang, and H. Xu, "Adapting Naive Bayes to Domain Adaptation for Sentiment Analysis", Advances in Information Retrieval, Lecture Notes in Computer Science, Vol.5478, 2009, 337-349.
  21. Tang, D., B. Qin, X. Feng, and T. Liu, "Effective LSTMs for Target-Dependent Sentiment Classification", arXiv Peprint arXiv : 1512.01100, 2016.
  22. Tarasov, D., "Deep Recurrent Neural Networks for Multiple Language Aspect-based Sentiment Analysis of User Reviews", Proceedings of the 21st International Conference on Computational Linguistics, Kazan, Russia, 2015.
  23. Zeiler, M.D. and R. Fergus, "Visualizing and Understanding Convolutional Networks", arXiv Peprint arXiv : 1311.2901, 2013.