The Journal of the Convergence on Culture Technology (문화기술의 융합)

The International Promotion Agency of Culture Technology (국제문화기술진흥원)
권호 표지
DOI QR코드

DOI QR Code

A Study on User Experience Factors of Display-Type Artificial Intelligence Speakers through Semantic Network Analysis : Focusing on Online Review Analysis of the Amazon Echo

의미연결망 분석을 통한 디스플레이형 인공지능 스피커의 사용자 경험 요인 연구 : 아마존 에코의 온라인 리뷰 분석을 중심으로

  • 이정명 (연세대학교 정보대학원 UX 트랙) ;
  • 김혜선 (연세대학교 정보대학원 UX 트랙) ;
  • 최준호 (연세대학교 정보대학원 UX 트랙)
  • Received : 2019.05.09
  • Accepted : 2019.07.08
  • Published : 2019.08.31
Download PDF
⟨ Previous Next ⟩

Abstract

The artificial intelligence speaker market is in a new age of mounting displays. This study aimed to analyze the difference of experience using artificial intelligent speakers in terms of usage context, according to the presence or absence of displays. This was achieved by using semantic network analysis to determine how the online review texts of Amazon Echo Show and Echo Plus consisted of different UX issues with structural differences. Based on the physical context and the social context of the user experience, the ego network was constructed to draw out major issues. Results of the analysis show that users' expectation gap is generated according to the display presence, which can lead to negative experiences. Also, it was confirmed that the Multimodal interface is more utilized in the kitchen than in the bedroom, and can contribute to the activation of communication among family members. Based on these findings, we propose a user experience strategy to be considered in display type speakers to be launched in Korea in the future.

인공지능 스피커 시장은 디스플레이 탑재라는 새로운 흐름 속에 놓여 있다. 이 연구는 디스플레이 유무에 따른 인공지능 스피커 사용 경험의 차이를 사용 맥락에 따라 분석하고자 한다. 이를 위해 아마존 에코 쇼(Echo Show)와 에코 플러스(Echo Plus)의 온라인 리뷰 텍스트가 어떠한 구조적 차이를 보이며 차별화된 UX 이슈들로 구성되어 있는지 의미연결망 분석을 통해 살펴보고자 한다. 사용자 경험의 물리적 맥락과 사회적 맥락에 따른 에고 네트워크 분석을 실시하여 주요 이슈를 도출하였다. 분석 결과 디스플레이 탑재에 따라 사용자의 기대격차가 발생하고 이로 인해 부정적 경험이 유도되는 것으로 나타났다. 또한, 멀티모달 인터페이스는 침실보다 부엌에서 활용도가 높으며, 가족 구성원 간의 커뮤니케이션 활성화에 기여할 수 있음을 확인하였다. 이러한 발견을 바탕으로 향후 국내에서도 출시될 디스플레이형 스피커가 고려해야 할 사용자 경험 전략을 제안한다.

Keywords

References

  1. J.H. Lee, S.W. Jeon, and J.T. Lee, "A study of Times Pressure Affecting on the User's Acceptance of Convergence Technologies : Focusing on the Diffusion of A.I Speaker", in Proc. Korea Technology Innovation Society, pp.1355-1368, Nov, 2017
  2. Yonhap news agency, https://www.yna.co.kr/view/AKR20180202163300980
  3. Seoul Economy. https://www.sedaily.com/NewsView/1VJ87YRT6W
  4. D.J. Kim, G.R. Lee, and H.W. Kim, "Analysis of speech recognition technology and personal assistant service using AI," in Proc. Korea Institute Of Communication Sciences, pp. 213-214, Jun, 2017.
  5. B. Smith, Ok, Google: Designing information architecture for smart speakers, Master's Thesis. University of Aalborg, Aalborg, Denmark., 2017.
  6. G.E. Jo, and S.I. Kim, "A study on User Experience of Artificial Intelligence speaker," Journal of the Korea Convergence Society, Vol. 9, No. 8, pp. 127-133, August 2018. https://doi.org/10.15207/JKCS.2018.9.8.127
  7. M.S. Suh, S.H. Hong, and J.M. Lee, "The Effect of AI Agent's Multi Modal Interaction on the Driver Experience in the Semi-autonomous Driving Context : With a Focus on the Existence of Visual Character," Journal of the Korea Contents Association, Vol. 18, No. 8, pp. 92-101, August 2018. https://doi.org/10.5392/JKCA.2018.18.08.092
  8. The Korea Times, http://www.koreatimes.co.kr/www/tech/2019/04/133_267403.html
  9. Maekyung, https://www.mk.co.kr/news/it/view/2019/04/241076/
  10. Maeil Business News Korea. https://www.mk.co.kr/news/business/view/2019/04/274590/
  11. NewsTomato. http://www.newstomato.com/ReadNews.aspx?no=826208
  12. H.J. Lee, C.H. Cho, S.Y. Lee, and Y.H. Keel, "A Study on Consumers Perception of and Use Motivation of Artificial Intelligence(AI) Speaker," Journal of the Korea Contents Association, Vol. 19, No. 3, pp.138-154, March 2019. https://doi.org/10.5392/JKCA.2019.19.03.138
  13. A. Purington, J.G. Taft, S. Sannon, N.N. Bazarova, and S.H Taylor, "Alexa is my new BFF: social roles, user satisfaction, and personification of the amazon echo," in Proc. 2017 CHI Conference Extended Abstracts on Human Factors in Computing Systems, ACM, pp.2853-2859, 2017.
  14. S. Kujala, V. Roto, K. Vaananen-Vainio-Mattila, E. Karapanos, and A. Sinnela, "UX Curve: A method for evaluating long-term user experience," Interacting with computers, Vol. 23, No. 5, pp. 473-483, 2011. https://doi.org/10.1016/j.intcom.2011.06.005
  15. J.H. Park, S.H. Han, H.K. Kim, H.K. Moon, and J.H. Park, "Developing and verifying a questionnaire for evaluating user value of a mobile device," Human Factors and Ergonomics in Manufacturing & Service Industries, Vol. 25, No. 6, pp. 724-739, 2015. https://doi.org/10.1002/hfm.20588
  16. L. Nigay, and J. Cautaz, "A design space for multimodal systems: concurrent processing and data fusion," in Proc. the INTERACT'93 and CHI'93 conference on Human factors in computing systems, ACM, pp. 172-178, 1993.
  17. B.M. Jang, S.D. Park, H.K, Jang, C.B. Jang, and E.I. Choi, "U -healthcare Context-Aw areness Security Framework Using Multi-modal Interaction Agent," Journal of Security Engineering, Vol. 6, No. 5, pp. 138-154, October 2009.
  18. B. Dumas, D. Lalanne, and S. Oviatt, "Multimodal interfaces: A survey of principles, models and frameworks," Human machine interaction, Vol. 5440, pp. 3-26, 2009. https://doi.org/10.1007/978-3-642-00437-7_1
  19. P. Biswas, C. Duarte, P. Langdon, L. Almeida, and C. Jung, A multimodal end-2-end approach to accessible computing. Springer, 2013.
  20. S. Oviatt, and P. Cohen, "Perceptual user interfaces: multimodal interfaces that process what comes naturally," Communications of the ACM, Vol.43, No.3, pp.45-53. 2000. doi:10.1145/330534.330538
  21. H.J. Lee, A study on the light interaction of the AI device for improving emotional transmission and increasing the use sustainability, Master's Thesis. Ewha Womans University, Seoul, Republic of Korea., 2019.
  22. E.L.C. Law, V. Roto, M. Hassenzah, A.P. Vermeeren, and J. Kort, "Understanding, scoping and defining user experience: A survey approach," in Proc. the SIGCHI Conference on Human Factors in Computing Systems, 2009.
  23. ISO 9241-210. Ergonomics of human system interaction-Part 210: Human-centered design for interactive systems (formerly known as 13407). Geneva, Switzerland: International Organization for Standardization (ISO). 2009.
  24. J.W. Kim, Human Computer Interaction, Ahn Graphics Publishers, 2012.
  25. X. Li, H. Lim, and H. Hwang, "Daily life with Clova Smart Speaker: Topic Modeling of Clova Review Data," In Proceedings of the XX International Conference on Human Computer Interaction, pp. 57. ACM., June 2019. doi:10.1145/3335595.3335603
  26. Y.L. Rhie, J.H. Lim, and M.H. Yun, "Evaluating representativeness of qualitative text data in identifying UX issues," International Journal of Human-Computer Interaction, Vol.33, No. 11, pp. 868-881, 2017. doi:10.1080/10447318.2017.1294337
  27. J.H. Chung, K. Nah, and S.B. Kim, "Study on Design Research using Semantic Network Analysis," Journal of the Ergonomics Society of Korea, Vol.34, No. 6, pp. p563-581, 2015. https://doi.org/10.5143/JESK.2015.34.6.563
  28. S.M. Mudambi, and D. Schuff, "What makes a helpful review? A study of customer reviews on Amazon. com," MIS quarterly, Vol. 34, No. 1, pp. 185-200. 2010. doi:10.2307/20721420
  29. K.H. Kim, and S.R. Oh, "Methodology for Applying Text Mining Techniques to Analyzing Online Customer Reviews for Market Segmentation," Journal of the Korea Contents Association, Vol. 9, No. 8, pp. 272-284, August 2009. https://doi.org/10.5392/JKCA.2009.9.8.272
  30. B. Yang, Y. Liu, Y. Liang, and M. Tang, "Exploiting user experience from online customer reviews for product design," International Journal of Information Management, Vol. 46, pp. 173-186. June 2019. doi:10.1016/j.ijinfomgt.2018.12.006
  31. H.J. Hwang, H.R. Shim, and J. Choi, "Exploration of user experience research method with big data analysis: Focusing on the online review analysis of echo," The Journal of the Korea Contents Association, Vol. 16, No. 8, pp. 517-528, 2016. doi:10.5392/jkca.2016.16.08.517
  32. S.H. Chae, J.I. Lim, and J.Y. Kang, "A Comparative Analysis of Social Commerce and Open Market Using User Reviews in Korean Mobile Commerce," Journal of Intelligence and Information Systems, Vol. 21, No. 4, pp. 23-23, December 2015.
  33. The Korea Economic, https://www.hankyung.com/it/article/2019051375941
  34. C. Yu, H. Shane, R. W. Schlosser, A. O'Brien, A. Allen, J. Abramson, and S. Flynn, "An Exploratory Study of Speech-Language Pathologists Using the Echo Show to Deliver Visual Supports," Advances in Neurodevelopmental Disorders, Vol. 2, No. 3, pp. 286-292. September 2018. https://doi.org/10.1007/s41252-018-0075-3
  35. W.R. Neuman, L. Guggenheim, S. M. Jang, and S. Y. Bae, "The dynamics of public attention: Agenda-setting theory meets big data," Journal of Communication, Vol. 64, No. 2, pp. 193-214. doi:10.1111/jcom.12088, 2014.
  36. R. Zamith, and S. C. Lewis, Content analysis and the algorithmic coder: What computational social science means for traditional modes of media analysis. The ANNALS of the American Academy of Political and Social Science, 659, 307?318. doi:10.1177/0002716215570576, 2015.
  37. H.B. Kim, S.M. Jang, S.H. Kim, and A. Wan, "Evaluating sampling methods for content analysis of Twitter data," Social Media+ Society, Vol. 4, No. 2, 2056305118772836, 2018. https://doi.org/10.1177/2056305118772836
  38. M.E. Roberts, B.M. Stewart, D. Tingley, C. Lucas, J. Leder-Luis, S. K. Gadarian, B. Albertson and D.G. Rand "Structural topic models for open-ended survey responses," American Journal of Political Science, Vol. 58, No. 4, pp. 1064-1082, 2014. https://doi.org/10.1111/ajps.12103
  39. J.H. Choi, K.H. Lee, and J.H. Choi, "Determinants of User Satisfaction with Mobile VR Headsets: The Human Factors Approach by the User Reviews Analysis and Product Lab Testing," International Journal of Contents, Vol. 15, No. 1. 2019.
  40. E. Cowley, and A.A. Mitchell, "The moderating effect of product knowledge on the learning and organization of product information," Journal of Consumer Research, Vol. 30, No. 3, pp. 443-454, 2003. https://doi.org/10.1086/378620
  41. C. Grebitus, and M. Bruhn, "Analyzing semantic networks of pork quality by means of concept mapping," Food Quality and Preference, Vol. 19, No. 1, pp.86-96, 2008. doi:10.1016/j.foodqual.2007.07.007
  42. W. Zhang, Q. Zhang, B. Yu, and L. Zhao, "Knowledge map of creativity research based on keywords network and co-word analysis," Quality & Quantity, Vol. 49, No. 3, pp. 1023-1038, 2015. https://doi.org/10.1007/s11135-014-0032-9
  43. S.S. Lee, "A Content Analysis of Journal Articles Using the Language Network Analysis Methods," Journal of the Korean Society for information Management, Vol. 31, No. 4, pp. 49-68, 2014. doi:10.3743/kosim.2014.31.4.049
  44. S. Wasserman, and K. Faust, "Social network analysis: Methods and applications," Cambridge university press, Vol. 8, 1994.
  45. B.J. Seo and S.C. Lee, "Network Analysis on Communication of Welfare Policy Using Twitter Data," The International Journal of Advanced Culture Technology, Vol. 6, No. 2, pp. 58-64, 2018. https://doi.org/10.17703/IJACT.2018.6.2.58
  46. H.W. Park, and L. Leydesdorff "Understanding the KrKwic: A computer program for the analysis of Korean text," Journal of The Korean Data Analysis Society, Vol. 6, No. 5, pp. 1377-1387, October 2004.
  47. Y.Y. Jung, "Semantic Network Analysis for the President Directions Item : Focusing on Patterns(2001-2009)". The Journal of the Convergence on Culture Technology (JCCT), Vol. 4, No. 1, pp. 129-137, 2018. https://doi.org/10.17703/JCCT.2018.4.1.129
  48. H.K. Cha, and K.Y. Hwa, "Analysis on Semantic Structure Shared among the Older Adults in the Online Community" Social Welfare Policy, Vol. 41, No. 4, pp. 325-349, December 2014. doi:10.15855/swp.2014.41.4.325
  49. G.W. Kim, J. Lim, and M. Yun, "Analysis of Consumer Value Using Semantic Network: The Comparison of Hierarchical and Nonhierarchical Value Structures," Human Factors and Ergonomics in Manufacturing and Service Industries, Vol. 26, No. 3, pp. 393-407, 2016. doi:10.1002/hfm.20665
  50. S.J. Lee, and J.Y. Choeh, "The Determinants of Helpfulness of Online Reviews," Behavior and Information Technology, Vol. 35, No. 10, pp. 853-863, 2016. doi:10.1080/0144929x.2016.1173099
  51. V. Motti, and K. Caine, "Understanding the wearability of head-mounted devices from a human-centered perspective," in Proc. International Semantic Web Conference (ISWC) Seattle, 2014.
  52. S. Mudambi, and D. Schuff, "What Makes a Helpful Review?: A Study of Customer Reviews on Amazon.com," MIS Quarterly, Vol. 34, No. 1, pp. 185-200, 2010. doi:10.2307/20721420
  53. J.H. Chung, K. Nah, and S.B. Kim, "Study on Design Research using Semantic Network Analysis," Journal of the Ergonomics Society of Korea, Vol. 34, No. 6, pp. 563-581, 2015. doi:10.5143/jesk.2015.34.6.563
  54. Y.H. Kim, Social Network Analysis, Pakyongsa, 2016
  55. P. Bonacich, "Power and centrality: A family of measures," American journal of sociology, Vol. 92, No. 5, pp. 1170-1182, 1987. https://doi.org/10.1086/228631
  56. J. Scott, Social network analysis, communication books, 2012.
  57. J.C. Mitchell, "The concept and use of social networks," Social networks in urban situations, 1969.
  58. Korea, N. NodeXL Catching up on node Excel(노드엑셀따라잡기), paradigm book, 2014.
  59. D. Hansen, B. Shneiderman, and M.A. Smith, Analyzing Social Media Networks with Nodexl Insights from a Connected World, Morgan Kaufmann Publ Inc, 2010.
  60. E. Luger, and A. Sellen, "Like having a really bad PA: the gulf between user expectation and experience of conversational agents," in Proc. the 2016 CHI Conference on Human Factors in Computing Systems pp. 5286-5297. ACM. 2016. doi:10.1145/2858036.2858288
  61. H.A. Park, M.Y. Tae, Y.J. Huh, and J.H. Lee, "Expectation and Expectation Gap towards intelligent properties of AI-based Conversational Agent," Journal of the HCI Society of Korea, Vol. 14, No. 1, pp. 15-22. February 2019. doi:10.17210/jhsk.2019.02.14.1.15
  62. S.P.A. Ringland, and F.J. Scahill, "Multimodality-the future of the wireless user interface," BT technology journal, Vol. 21, No. 3, pp. 181-191, 2003. https://doi.org/10.1023/A:1025127603820
  63. S. Lemmela, A. Vetek, K. Makela, and D. Trendafilov, "Designing and evaluating multimodal interaction for mobile contexts," in Proc. the 10th international conference on Multimodal interfaces, pp. 265-272, ACM. 2008. doi:10.1145/1452392.1452447
  64. H.M. Koo, S.C. Kim, and C.G. Nam, "Speaker Wars begins: Which applications will be the killer content for smart speaker?," in Proc. 14th International Telecommunications Society (ITS) Asia-Pacific Regional Conference, Jun. 24-17, 2017.
  65. T.L. Saaty, and J.S. Shang, "An innovative orders-of-magnitude approach to AHP-based mutli-criteria decision making: Prioritizing divergent intangible humane acts," European Journal of Operational Research, Vol. 214, No. 3, pp. 703-715, 2011. doi:10.1016/j.ejor.2011.05.019
  66. L. Kim, "The Lessons of Google Glass: Aligning Key Benefits and Sociability," In International Conference on Human Interface and the Management of Information pp. 371-380, Springer, Cham. 2018. doi : 10.1007/978-3-319-92043-6_31
  67. D.H. Byun, "Evaluating User Experience of Smart Television Using Emotional Representation Language" Journal of the Korea Contents Association, Vol. 15, No. 5, pp. 132-141, May 2015. doi:10.5392/jkca.2015.15.05.132