Journal of Information Technology Services (한국IT서비스학회지)

Korea Society of IT Services (한국IT서비스학회)
권호 표지
DOI QR코드

DOI QR Code

Predicting Missing Ratings of Each Evaluation Criteria for Hotel by Analyzing User Reviews

사용자 리뷰 분석을 통한 호텔 평가 항목별 누락 평점 예측 방법론

  • 이동훈 (국민대학교 비즈니스IT전문대학원) ;
  • 부현경 (국민대학교 비즈니스IT전문대학원) ;
  • 김남규 (국민대학교 경영정보학부)
  • Received : 2017.10.18
  • Accepted : 2017.11.26
  • Published : 2017.12.31
Download PDF
⟨ Previous Next ⟩

Abstract

Recently, most of the users can easily get access to a variety of information sources about companies, products, and services through online channels. Therefore, the online user evaluations are becoming the most powerful tool to generate word of mouth. The user's evaluation is provided in two forms, quantitative rating and review text. The rating is then divided into an overall rating and a detailed rating according to various evaluation criteria. However, since it is a burden for the reviewer to complete all required ratings for each evaluation criteria, so most of the sites requested only mandatory inputs for overall rating and optional inputs for other evaluation criteria. In fact, many users input only the ratings for some of the evaluation criteria and the percentage of missed ratings for each criteria is about 40%. As these missed ratings are the missing values in each criteria, the simple average calculation by ignoring the average 40% of the missed ratings can sufficiently distort the actual phenomenon. Therefore, in this study, we propose a methodology to predict the rating for the missed values of each criteria by analyzing user's evaluation information included the overall rating and text review for each criteria. The experiments were conducted on 207,968 evaluations collected from the actual hotel evaluation site. As a result, it was confirmed that the prediction accuracy of the detailed criteria ratings by the proposed methodology was much higher than the existing average-based method.

Keywords

References

  1. Augustine, A. and M. Pathak, "User Rating Prediction for Movies", University of Texas at Austin, 2007.
  2. Aurelio G.M. and R. Minazzi, "Web Reviews Influence on Expectations and Purchasing Intentions of Hotel Potential Customers", International Journal of Hospitality Management, Vol.34, 2013, 99-107. doi : http://dx.doi.org/10.1016/j.ijhm.2013.02.012 (Downloaded August 30, 2017).
  3. Bao, Z. and H. Xia, "Movie Rating Estimation and Recommendation", CS229 2012 Project, Stanford University, 2012.
  4. Blei, D.M., "Probabilistic Topic Models", Communications of the ACM, Vol.55, No.4, 2012, 77-84. doi : 10.1145/2133806.2133826 (Downloaded August 30, 2017).
  5. Blei, D.M. and J.D. Lafferty, "Dynamic Topic Models", ICML '06 Proceedings of the 23rd International Conference on Machine Learning, 2006, 113-120.
  6. Blei, D.M., A.Y. Ng, and M.I. Jordan, "Latent Dirichlet Allocation", Journal of Machine Learning Research, Vol.3, 2003, 993-1022.
  7. Casalo, L.V., C. Flavian, M. Guinaliu, and Y. Ekinci, "Do Online Hotel Rating Schemes Influence Booking Behaviors?", International Journal of Hospitality Management, Vol. 49, 2015, 28-36. https://doi.org/10.1016/j.ijhm.2015.05.005
  8. Cui, G., H.K. Lui, and X.N. Guo, "The Effect of Online Consumer Reviews on New Product Sales", International Journal of Electronic Commerce, Vol.17, No.1, 2012, 39-58. https://doi.org/10.2753/JEC1086-4415170102
  9. Davis, A. and D. Khazanchi, "The Influence of Online Word of Mouth on Product Sales in Retail E-commerce : An Empirical Investigation", Proceedings of the 13th Annual Americas Conference on Information Systems, Vol.13, No.3, 2007, 2149-2157.
  10. Duan, W.J., G. Gu, and A.B. Whinston, "Do Online Reviews Matter?-An Empirical Investigation of Panel Data", Decision Support Systems, Vol.45, No.4, 2008, 1007-1016. https://doi.org/10.1016/j.dss.2008.04.001
  11. Gauri, D.K., A. Bhatnagar, and R. Rao, "Role of Word of Mouth in Online Store Loyalty", Magazine Communications of the ACM, Vol. 51, No.3, 2008, 89-91.
  12. Gayatree, G., N. Elhadad, and A. Marian, "Beyond the Stars : Improving Rating Predictions Using Review Text Content", Twelfth International Workshop on the Web and Databases, 2009.
  13. Ghose, A. and G.I. Panagiotis, "Designing Ranking Systems for Consumer Reviews : The Impact of Review Subjectivity on Product Sales and Review Quality", Proceedings of the International Conference on Decision Support Systems, 2006.
  14. Ghose, A. and G.I. Panagiotis, "Designing Novel Review Ranking Systems : Predicting the Usefulness and Impact of Reviews", Proceedings of the Ninth International Conference on Electronic Commerce, 2007, 303-310.
  15. Gupta, N., G.D. Fabbrizio, and P. Haffner, "Capturing the Stars : Predicting Ratings for Service and Product Reviews", Proceedings of the NAACL HLT 2010 Workshop on Semantic Search, Los Angeles, California, 2010, 36-43.
  16. Ha, H.S., H.J. Kim, S.W. Choi, and S.R. Yoon, "Comparing and Predicting Movie Rating Using Social Network Service", Proceedings of Korean Institute of Information Scientists and Engineers, 2014, 276-278. (하헌석, 김한주, 최성운, 윤성로, "소셜 네트워크 서비스를 활용한 영화 평점 비교 및 예측", 한국정보과학회 학술발표논문집, 2014, 276-278.)
  17. Hofmann, T., "Probabilistic Latent Semantic Indexing", Proceedings of the 22nd Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, Berkeley, California, 1999, 50-57. doi : 10.1145/312624.312649 (Downloaded August 30, 2017).
  18. Hu, N., N.S. Koh, and S.K. Reddy, "Ratings Lead You to the Product Reviews Help You Clinch It? The Mediating Role of Online Review Sentiments on Product Sales", Decision Support Systems, Vol.57, 2014, 42-53. https://doi.org/10.1016/j.dss.2013.07.009
  19. Kim, G.M., M.H. Ahn, and Y.H. Lee, "Prediction of Movies based on Naver DB Information and Filtering Malicious Rating Markers", Proceedings of Korean Institute of Industrial Engineers, 2013, 1651-1662. (김경민, 안무혁, 이윤호, "네이버 DB정보 기반 영화 평점 예측 및 악의적 평점 기록자 Filtering", 대한산업공학회 춘계공동학술대회논문집, 2013, 1651-1662.)
  20. Landauer, T.K., P.W. Foltz, and D. Laham, "An Introduction to Latent Semantic Analysis", Discourse Processes, Vol.25, No.2-3, 1998, 259-284. doi : http://dx.doi.org/10.1080/01638539809545028 (Downloaded August 30, 2017).
  21. Lee, J., D.H. Park, and I. Han, "The Effect of Online Negative Online Consumer Reviews on Product Attitude : An Information Processing View", Electronic Commerce Research, and Applications, Vol.7, No.3, 2008, 341-352. https://doi.org/10.1016/j.elerap.2007.05.004
  22. Lee, S., T.R. Jeon, G.D. Baek, and S.S. Kim, "A Movie Rating Prediction System of User Propensity Analysis based on Collaborative Filtering, and Fuzzy System", Journal of Korean Institute of Intelligent Systems, Vol.19, No.2, 2009, 242-247. (이수진, 전태룡, 백경동, 김성신, "협업적 필터링 및 퍼지시스템 기반 사용자 성향분석에 의한 영화 평가 예측 시스템", 한국지능시스템학회논문지, 제19권, 제2호, 2009, 242-247.) https://doi.org/10.5391/JKIIS.2009.19.2.242
  23. Li, F., N. Liu, H. Jin, K. Zhao, Q. Yang, and X. Zhu, "Incorporating Reviewer, and Product Information for Review Rating Prediction", 22nd International Joint Conferences on Artificial Intelligence, Vol.3, 2011, 1820-1825.
  24. Liu, C. and N. Kim "Methodology for Improving the Reliability of the Rating System for Leisure Activity Information Sites : Focusing on a Movie Information Site", Journal of Tourism & Leisure Research, Vol.27, No.7, 2015, 187-200. (류 신, 김남규, "여가활동 정보 사이트의 평점 시스템 신뢰성 향상 방안", 관광레저연구, 제27권, 제7호, 2015, 187-200.)
  25. Liu, C.L., W.H. Hsaio, C.H. Lee, G.C. Lu, and E. Jou, "Movie Rating and Review Summarization in Mobile Environment", IEEE Transactions on Systems, Man, and Cybernetics-Part C : Applications and Reviews, Vol.42 No.3, 2012, 397-407. https://doi.org/10.1109/TSMCC.2011.2136334
  26. Marovic, M., M. Mihokovic, M. Miksa, S. Pribil, and A. Tus, "Automatic Movie Ratings Prediction Using Machine Learning", Proceedings of the 34th International Convention, 2011, 1640-1645.
  27. Moghaddam, S. and E. Martin, "ILDA : Interdependent LDA Model for Learning Latent Aspects and Their Ratings from Online Product Reviews", Proceedings of the 34th International ACM SIGIR Conference on Research and Development in Information Retrieval, 2011, 665-674.
  28. Mudambi, S.M. and D. Schuff, "What Makes a Helpful Online Review? A Study of Customer Reviews on Amazon.com", Journal MIS Quarterly Archive, Vol.34, No.1, 2010, 185-200. https://doi.org/10.2307/20721420
  29. Oh, B.H., J.H. Yang, and H.J. Lee, "A Hybrid Recommender System based on Collaborative Filtering with Selective Utilization of Content-based Predicted Ratings", Journal of KISS : Software and Applications, Vol.41, No.4, 2014, 289-294. (오병화, 양지훈, 이현진, "콘텐츠 기반 예측 평점을 선택적으로 활용하는 협력적 필터링 기반 복합추천 시스템", 정보과학회논문지 : 소프트웨어 및 응용, 제41권, 제4호, 2014, 289-294.)
  30. Park, D.H., J. Lee, and I. Han, "The Effect of Online Consumer Reviews on Consumer Purchasing Intention : The Moderating Role of Involvement", International Journal of Electronic Commerce, Vol.11, No.4, 2007, 125-148. https://doi.org/10.2753/JEC1086-4415110405
  31. Patrali, C., "Online Reviews: Do Consumers Use Them?", Advances in Consumer Research, Vol.28, No.1, 2001, 129-133.
  32. Persson, K., "Predicting Movie Ratings : a Comparative Study on Random Forests and Support Vector Machines", Dissertation, 2015.
  33. Seo, S.Y., J. Huangy, H. Yangy, and Y. Liu, "Representation Learning of Users, and Items for Review Rating Prediction Using Attention- based Convolutional Neural Network", 3rd International Workshop on Machine Learning Methods for Recommender Systems, Houston, Texas, 2017.
  34. Sharma, R.D., S. Tripathi, S.K. Sahu, S. Mittal, and A. Anand, "Predicting Online Doctor Ratings from User Reviews Using Convolutional Neural Networks", International Journal of Machine Learning, and Computing, Vol.6, No.2, 2016, 149-154. doi : 10.18178/ijmlc.2016.6.2.590 (Downloaded August 30, 2017).
  35. Son, S.B. and J.H. Chun, "Product Feature Extraction and Rating Distribution Using User Reviews", Journal of Society for e-Business Studies, Vol.22, No.1, 2017, 65-87. (손수빈, 전종훈, "사용자 리뷰를 이용한 상품 특징 추출 및 평점 분배", 한국전자거래학회지, 제22권, 제1호, 2017, 65-87.) https://doi.org/10.7838/jsebs.2017.22.1.065
  36. Song, C.H., B.C. Kim, and K.D. Park, "Impact of Consumer Rating on Mobile Application Sales : A Comparison between Korean, and American Consumers", Korean Management Review, Vol.43, No.5, 2014, 1493-1518. (송치훈, 김병조, 박경도, "소비자 평점이 모바일 애플리케이션 구매에 미치는 영향-한국과 미국 소비자와의 비교를 중심으로", 경영학연구, 제43권, 제5호, 2014, 1493-1518.)
  37. Tang, D., B. Qiny, T. Liuy, and Y. Yang, "User Modeling with Neural Network for Review Rating Prediction", Proceedings of the Twenty-Fourth International Joint Conference on Artificial Intelligence, 2015.
  38. Wang, B., Y. Huang, and X. Li, "Combining Review Text Content, and Reviewer-Item Rating Matrix to Predict Review Rating", Computational Intelligence, and Neuroscience, 2016. doi : 10.1155/2016/5968705 (Downloaded August 29, 2017).
  39. Wang, H., Y. Lu, and C.X. Zhai, "Latent Aspect Rating Analysis without Aspect Keyword Supervision", Proceedings of the 17th ACM SIGKDD International Conference on Knowledge Discovery, and Data Mining (KDD'11), 2011, 618-626.
  40. Ye, Q., R. Law, and B. Gu, "The Impact of Online User Reviews on Hotel Room Sales", International Journal of Hospitality Management, Vol.28, No.1, 2009, 180-182. doi : http://dx.doi.org/10.1016/j.ijhm.2015.05.005 (Downloaded August 30, 2017).
  41. Yew, J.L. and W.T. Yee, "Variational Bayesian Approach to Movie Rating Prediction", Association for Computing Machinery, 2007.
  42. Yoon, D.M. and K.J. Kim, "Prediction of Rating Score from Short Comments on Movies Using Word-Rating Correlation Analysis", Proceedings of the HCI KOREA, 2011, 484-486. (윤두밈, 김경중, "단어-평점 상관관계 분석을 통한 단문 영화평 평점 예측", 한국HCI학회 학술대회, 2011, 484-486.)