경영정보학연구 (Information Systems Review)

  • 제27권3호
  • /
  • Pages.105-126
  • /
  • 2025
  • /
  • 2982-6551(pISSN)
  • /
  • 2982-6837(eISSN)
한국경영정보학회 (The Korea Society of Management Information Systems)
권호 표지
DOI QR코드

DOI QR Code

AI vs. 인간 생성리뷰 요약에서의 리뷰 양면성의 역할: 정보처리 노력에 대한 정교화 가능성 모델(ELM) 관점에서

The Role of Review Sidedness in AI vs. Human-Generated Summaries: An ELM Perspective on Information Processing Effort

  • 정한나 (경희대학교 스마트관광원) ;
  • 구철모 (경희대학교 스마트관광원 ) ;
  • 김기헌 (영산대학교 관광컨벤션학과)
  • Hanna Jeong (Smart Tourism Education Platform (STEP), Kyung Hee University) ;
  • Chulmo Koo (Smart Tourism Education Platform (STEP), Kyung Hee University) ;
  • Keehun Kim (College of Hotel and Tourism Management, Youngsan University )
  • 투고 : 2025.03.25
  • 심사 : 2025.06.25
  • 발행 : 2025.08.31
PDF 다운로드
⟨ 이전 논문 다음 논문 ⟩

초록

본 연구는 온라인 호텔 예약 맥락에서 리뷰 요약문의 출처(AI vs. 인간)와 리뷰의 양면성(단면 vs. 양면)이 소비자의 정보처리 노력(IPE)에 미치는 영향에 관하여 연구하였다. 본 연구에서는 정교화 가능성 모델(ELM)을 이론적 틀로 활용하여 두 가지 지표(이해도 점수, 읽는 시간)의 IPE를 측정하였으며 이를 위해 두 차례의 실험연구를 진행하였다. 스터디 1에서는 리뷰의 출처보다 리뷰 양면성이 인지적 노력에 더 큰 영향을 미치며, 특히 단면적인 AI 생성 리뷰가 가장 높은 정보처리 노력을 요구함을 확인할 수 있었다. 흥미롭게도, 양면 리뷰의 경우 인간이 작성한 요약이 AI가 생성한 요약보다 더 높은 IPE를 유발하였다. 스터디 2에서는 참가자들이 양면 리뷰보다 단면 리뷰를 더 오래 읽음을 확인할 수 있었다. 이러한 결과는 리뷰 출처와 IPE 간의 관계에서 리뷰 양면성이 조절 변수로 작용될 수 있음을 시사한다. 또한, 리뷰 내용의 불충분함과 모호성이 더 깊은 인지적 처리를 유발하며, 확인 편향과 인간 우월성 인식 또한 AI와 인간 생성 콘텐츠에 대한 정보처리 노력에 영향을 미침을 보여준다. 본 연구는 AI-인간 생성 콘텐츠 비교 연구에서 리뷰 양면성을 조절 변수로 도입함으로써 이론적 공헌을 하며, AI 기반 플랫폼에서 신뢰성과 설득력을 갖춘 리뷰 시스템 설계를 위한 실무적 시사점을 제공한다.

This study investigates how the source (AI vs. Human) and sidedness (One-sided vs. Two-sided) of review summaries affect consumer information processing effort (IPE) in the context of online hotel booking. Drawing upon the Elaboration Likelihood Model (ELM), two experimental studies were conducted to measure IPE using two indicators: comprehension score and reading time. Study 1 reveals that review sidedness, rather than review source, significantly influences cognitive effort, with one-sided AI-generated reviews requiring the highest processing effort. Interestingly, for two-sided reviews, human-generated summaries demanded higher IPE than AI-generated ones. Study 2 finds that participants spent more time reading one-sided reviews than two-sided ones. These findings support the moderating role of review-sidedness in the relationship between review source and IPE. Additionally, the results highlight that perceived insufficiency and ambiguity in review content trigger deeper cognitive processing. Notably, confirmation bias and perceived human superiority also contribute to shaping consumers' cognitive effort with AI versus human-generated content. This study contributes to existing literature by introducing review-sidedness as a moderating variable in AI-human content comparisons and offers practical implications for designing persuasive and trustworthy review systems in AI-assisted platforms.

키워드

과제정보

This work was supported by the Ministry of Education of the Republic of Korea and the National Research Foundation of Korea (NRF-2023S1A5C2A03095253).

참고문헌

  1. Alashoor, T., M. Keil, H. J. Smith, and A. R. McConnell, "Too tired and in too good of a mood to worry about privacy: Explaining the privacy paradox through the lens of effort level in information processing", Information Systems Research, Vol.34, No.4, 2023, pp. 1415-1436. https://doi.org/10.1287/isre.2022.1182.
  2. Allen, M., "Meta-analysis comparing the persuasiveness of one-sided and two-sided messages", Western Journal of Speech Communication, Vol.55, No.4, 1991, pp. 390-404, https://doi.org/10.1080/10570319109374395.
  3. Amazon, "How Amazon continues to improve the customer reviews experience with generative AI", US About Amazon, August 14, 2023, Available at https://www.aboutamazon.com/news/amazon-ai/amazon-improves-customer-reviews-with-generative-ai.
  4. Amos, C. and L. Zhang, "Consumer reactions to perceived undisclosed ChatGPT usage in an online review context", Telematics and Informatics, Vol.93, 2024, p. 102163, https://doi.org/10.1016/j.tele.2024.102163.
  5. Baek, T. H., J. Kim, and J. H. Kim, "Effect of disclosing AI-generated content on prosocial advertising evaluation", International Journal of Advertising, 2024, pp. 1-22, https://doi.org/10.1080/02650487.2024.2320161.
  6. Cao, Y., S. Li, Y. Liu, Z. Yan, Y. Dai, P. S. Yu, and L. Sun, "A comprehensive survey of AI-generated content (AIGC): A history of generative AI from GAN to ChatGPT", arXiv, 2023, https://doi.org/10.48550/arXiv.2303.04226.
  7. Carvalho, I. and S. Ivanov, "ChatGPT for tourism: Applications, benefits and risks", Tourism Review, Vol.79, No.2, 2024, pp. 290-303, https://doi.org/10.1108/TR-02-2023-0088.
  8. Cheng, R., A. Smith-Renner, K. Zhang, J. R. Tetreault, and A. Jaimes, Mapping the Design Space of Human-AI Interaction in Text Summarization (Version 1), arXiv, 2022, https://doi.org/10.48550/ARXIV.2206.14863.
  9. Cheung, M. Y., C. L. Sia, and K. K. Y. Kuan, "Is this review believable? A study of factors affecting the credibility of online consumer reviews from an ELM perspective", Journal of the Association for Information Systems, Vol.13, No.8, 2012, Available at https://aisel.aisnet.org/jais/vol13/iss8/2.
  10. Choi, M., Y. Choi, E. Bangura, and D. Kim, "ChatGPT or online review: Which is a better determinant of customers' trust in Airbnb listings and stay intention?", International Journal of Hospitality Management, Vol.122, 2024, p. 103852, https://doi.org/10.1016/j.ijhm.2024.103852.
  11. ColorWhistle, "How OTAs can utilize generative AI on travel industry", March 20, 2024, Available at https://colorwhistle.com/generative-ai-in-travel-ota/.
  12. Flanagin, A. J., S. Winter, and M. J. Metzger, "Making sense of credibility in complex information environments: The role of message sidedness, information source, and thinking styles in credibility evaluation online", Information, Communication & Society, Vol.23, No.7, 2020, pp. 1038-1056, https://doi.org/10.1080/1369118X.2018.1547411.
  13. Gang, V. and H. Reja, "Revolutionizing travel reviews: Tripadvisor's AI-driven summaries", LinkedIn, January 22, 2024, Available at https://www.linkedin.com/pulse/revolutionizing-travel-reviews-tripadvisors-ai-driven-summaries-2xpqe/.
  14. Giroux, M., J. Kim, J. C. Lee, and J. Park, "Artificial intelligence and declined guilt: Retailing morality comparison between human and AI", Journal of Business Ethics, Vol.178, No.4, 2022, pp. 1027-1041, https://doi.org/10.1007/s10551-021-04951-9.
  15. Grant, M. J. and A. Booth, "A typology of reviews: An analysis of 14 review types and associated methodologies", Health Information & Libraries Journal, Vol.26, No.2, 2009, pp. 91-108, https://doi.org/10.1111/j.1471-1842.2009.00848.x.
  16. Hendriks, F., I. Janssen, and R. Jucks, "Balance as credibility? How presenting one- vs. two-sided messages affects ratings of scientists' and politicians' trustworthiness", Health Communication, Vol.38, No.12, 2023, pp. 2757-2764, https://doi.org/10.1080/10410236.2022.2111638.
  17. Jia, S., O. H. Chi, and C. G. Chi, "Unpacking the impact of AI vs. human-generated review summary on hotel booking intentions", International Journal of Hospitality Management, Vol.126, 2025, p. 104030, https://doi.org/10.1016/j.ijhm.2024.104030.
  18. Jin, M., Z. Yang, T. L. Freling, and N. Janakiraman, "The human superiority effect in advice taking: A multimethod exploration and implications for policy makers and governmental organizations", Journal of Public Policy & Marketing, Vol.44, No.3, 2025, pp. 350-369, https://doi.org/10.1177/07439156251320314.
  19. Johnson, E. J. and J. W. Payne, "Effort and Accuracy in Choice", Management Science, Vol.31, No.4, 1985, pp. 395-414, https://doi.org/10.1287/mnsc.31.4.395.
  20. Kamins, M. A., M. J. Brand, S. A. Hoeke, and J. C. Moe, "Two-sided versus one-sided celebrity endorsements: The impact on advertising effectiveness and credibility", Journal of Advertising, Vol.18, No.2, 1989, pp. 4-10, https://doi.org/10.1080/00913367.1989.10673146.
  21. Kao, D. T., "Message sidedness in advertising: The moderating roles of need for cognition and time pressure in persuasion", Scandinavian Journal of Psychology, Vol.52, No.4, 2011, pp. 329-340, https://doi.org/10.1111/j.1467-9450.2011.00882.x.
  22. Kiyoumarsi, F., "Evaluation of automatic text summarizations based on human summaries", Procedia - Social and Behavioral Sciences, Vol.192, 2015, pp. 83-91, https://doi.org/10.1016/j.sbspro.2015.06.013.
  23. Koo, W. W., J. Pu, and N. Gao, "AI review summary, customer ratings, and performance entrenchment[Preprint]", SSRN, January 10, 2025, https://doi.org/10.2139/ssrn.5107429.
  24. Korzynski, P., J. Paniagua, and M. A. Rodríguez-Molina, "The role of artificial intelligence in online customer experience: Review and research agenda", Journal of Business Research, Vol.156, 2023, pp. 113446, https://doi.org/10.1016/j.jbusres.2022.113446.
  25. Kwon, B., J. Lee, J. Min, C. Kwak, and H. S. Choi, "Beyond the stars: The impact of rating-text inconsistency on perceived review usefulness", Asia Pacific Journal of Information Systems, Vol.35, No.1, 2025, pp. 49-72, https://doi.org/10.14329/apjis.2025.35.1.49.
  26. Lim, S. and R. Schmälzle, "The effect of source disclosure on evaluation of AI-generated mes-sages", Computers in Human Behavior: Artificial Humans, Vol.2, No.1, 2024, p. 100058, https://doi.org/10.1016/j.chbah.2024.100058.
  27. Liu, S. Q., M. Ozanne, and A. S. Mattila, "Does expressing subjectivity in online reviews enhance persuasion?", Journal of Consumer Marketing, Vol.35, No.4, 2018, pp. 403-413, https://doi.org/10.1108/JCM-02-2017-2109.
  28. Mudambi, S. M. and D. Schuff, "What makes a helpful online review? A study of customer reviews on Amazon.com", MIS Quarterly, Vol.34, No.1, 2010, pp. 185-200, https://doi.org/10.2307/20721420.
  29. Ou, M., S. S. Ho, and S. A. Wijaya, "Harnessing AI to address misinformation on cultivated meat: The impact of chatbot expertise and correction sidedness", Science Communication, Advance online publication, 2025, https://doi.org/10.1177/10755470251315097.
  30. Pentina, I., A. A. Bailey, and L. Zhang, "Exploring effects of source similarity, message valence, and receiver regulatory focus on Yelp review persuasiveness and purchase intentions", Journal of Marketing Communications, Vol.24, No.2, 2018, pp. 125-145, https://doi.org/10.1080/13527266.2015.1005115.
  31. Petty, R. E. and J. T. Cacioppo, "The elaboration likelihood model of persuasion", Advances in Experimental Social Psychology, Vol.19, 1986, pp. 123-205, https://doi.org/10.1016/S0065-2601(08)60214-2.
  32. Purnawirawan, N., P. De Pelsmacker, and N. Dens, "Balance and sequence in online reviews: How perceived usefulness affects attitudes and intentions", Journal of Interactive Marketing, Vol.26, No.4, 2012, pp. 244-255. https://doi.org/10.1016/j.intmar.2012.04.002
  33. Schlosser, A. E., "Can including pros and cons increase the helpfulness and persuasiveness of online reviews? The interactive effects of ratings and arguments", Journal of Consumer Psychology, Vol.21, No.3, 2011, pp. 226-239, https://doi.org/10.1016/j.jcps.2011.04.002.
  34. Sundar, S. S. and C. Nass, "Conceptualizing sources in online news", Journalism & Mass Communication Quarterly, Vol.78, No.1, 2001, pp. 87-104, https://doi.org/10.1177/107769900107800107.
  35. Susarla, A., R. Gopal, J. B. Thatcher, and S. Sarker, "The janus effect of generative AI: Charting the path for responsible conduct of scholarly activities in information systems", Information Systems Research, Vol.34, No.2, 2023, pp. 399-408, https://doi.org/10.1287/isre.2023.ed.v34.n2.
  36. Tsai, C.-F., K. Chen, Y.-H. Hu, and W.-K. Chen, "Improving text summarization of online hotel reviews with review helpfulness and sentiment", Tourism Management, Vol.80, 2020, p. 104122, https://doi.org/10.1016/j.tourman.2020.104122.
  37. Tversky, A. and D. Kahneman, "Judgment under uncertainty: Heuristics and biases", Science, Vol.185, No.4157, 1974, pp. 1124-1131, https://doi.org/10.1126/science.185.4157.1124.
  38. Varga, S. and P. Albuquerque, "The impact of negative reviews on online search and purchase decisions", Journal of Marketing Research, Vol.61, No.5, 2024, pp. 803-820, https://doi.org/10.1177/00222437231190874.
  39. Varsha, "How OTAs can utilize generative AI on travel industry", ColorWhistle, March 20, 2024, Available at https://colorwhistle.com/generative-ai-in-travel-ota/.
  40. Wang, Y., K. Zhong, and Q. Liu, "Let criticism take precedence: Effect of side order on consumer attitudes toward a two-sided online review", Journal of Business Research, Vol.140, 2022, pp. 403-419, https://doi.org/10.1016/j.jbusres.2021.11.010.
  41. Westerwick, A., B. K. Johnson, and S. KnoblochWesterwick, " Confirmation biases in selective exposure to political online information: Source bias vs. content bias", Communication Monographs, Vol.84, No.3, 2017, pp. 343-364, https://doi.org/10.1080/03637751.2016.1272761
  42. Whitmore, G., "Will ChatGPT replace travel agents?", Forbes, February 7, 2023, Available at https://www.forbes.com/sites/geoffwhitmore/2023/02/07/will-chatgpt-replace-travel-agents/.
  43. Xu, X., "Examining an asymmetric effect between online customer reviews emphasis and overall satisfaction determinants", Journal of Business Research, Vol.106, 2020, pp. 196-210, https://doi.org/10.1016/j.jbusres.2018.07.022.
  44. Yazdani, E., S. Gopinath, and S. J. Carson, "The role of reviewer badges in the dynamics of online reviews", International Journal of Research in Marketing, Vol.41, No.3, 2024, pp. 547-566, https://doi.org/10.1016/j.ijresmar.2024.01.003.
  45. Zhu, T., I. Weissburg, K. Zhang, and W. Y. Wang, "Human bias in the face of AI: The role of human judgement in AI-generated text evaluation", arXiv, 2024, https://doi.org/10.48550/arXiv.2410.03723.
  46. Złotowski, H., H. Sumioka, F. Eyssel, S. Nishio, C. Bartneck, and H. Ishiguro, "Model of dual anthropomorphism: The relationship between the media equation effect and implicit anthropomorphism", International Journal of Social Robotics, Vol.10, No.5, 2018, pp. 701-714, https://doi.org/10.1007/s12369-018-0476-5.