Journal of the HCI Society of Korea (한국HCI학회논문지)

The HCI Society of Korea (한국HCI학회)
권호 표지

The Effects of Interface Modality on Cognitive Load and Task Performance in Media Multitasking Environment

미디어 멀티태스킹 환경에서 인터페이스의 감각양식 차이가 인지부하와 과업수행에 미치는 영향에 관한 연구 다중 자원 이론과 스레드 인지 모델을 기반으로

  • 이다나 (연세대학교 일반대학원 인지과학협동과정) ;
  • 한광희 (연세대학교 심리학과, 인지과학협동과정)
  • Received : 2019.01.14
  • Accepted : 2019.03.15
  • Published : 2019.05.31
Download PDF
⟨ Previous Next ⟩

Abstract

This research examined the changes that fast-growing voice-based devices would bring in the media multitasking environment. Based on the theoretical background that information processing efficiency improves when performing multiple tasks requiring different resource structures at the same time, we conducted an experiment where participants searched for information with voice-based or screen-based devices while performing an additional visual task. Results showed that both task performance environment and interface modality had significant main effects on cognitive load. The overall cognitive load level was higher in the voice interface group, but the difference in cognitive load between the two groups decreased in a multitasking environment where the additional visual resources was required. The visual task performance was significantly higher when using the voice interface than the screen interface. Our findings suggest that voice interfaces offered advantages in the cognitive load and task performance by distributing two tasks to the auditory and visual channels. The results of this study imply that voice-based devices have the potential to facilitate efficient information processing in the screen-centric environment where visual resources collide. We provided theoretical evidence of resource distribution using multiple resource theory and tried to identify the advantages of the voice interface more specifically based on the threaded cognition model.

본 연구는 빠르게 발전하는 음성 기반의 디바이스가 스크린 중심의 미디어 멀티태스킹 환경에 어떤 변화를 가져올 수 있을지 확인하고자 했다. 서로 다른 자원 구조를 가진 과업을 동시에 수행할 때 정보 처리 효율이 높아진다는 이론적 근거를 토대로, 시각 주의가 필요한 과제와 음성 또는 스크린 기반의 디바이스를 활용해 정보를 검색하는 과업을 동시에 수행하는 실험이 진행되었다. 실험 결과, 과업수행 환경과 인터페이스 감각양식은 모두 인지부하에 유의미한 영향을 미쳤다. 음성 인터페이스 그룹에서 전반적으로 인지부하 수준이 높게 나타났는데, 단독으로 사용된 단일 과업 조건보다 시각 과제를 동시에 수행한 다중 과업 조건에서 시각 인터페이스 그룹과의 차이가 줄어들었다. 과업 수행도의 경우 음성 인터페이스 그룹에서 시각 과제에 대한 수행능력이 시각 인터페이스 그룹보다 더 높게 측정되었다. 이러한 결과는 멀티태스킹 환경에서 음성 인터페이스를 사용했을 때 동시적 과업을 청각 경로와 시각 경로로 나누어 처리함으로써 인지부하와 과업수행에 이점이 나타났음을 의미한다. 이는 시각 자원의 충돌이 발생하기 쉬운 스크린 중심의 미디어 멀티태스킹 환경에서 음성 기반의 디바이스가 효율적 정보 처리를 촉진시키는 잠재적 역할을 할 수 있다는 함의점을 제공한다. 본 연구는 다중 자원 이론을 통해 자원의 분산처리에 대한 이론적 증거를 제시하고, 스레드 인지 모델을 기반으로 음성 인터페이스를 활용했을 때의 이점을 더욱 구체적으로 규명하고자 했다.

Keywords

References

  1. Van Cauwenberge, A., Schaap, G. and Van Roy, R. "TV no longer commands our full attention": Effects of second-screen viewing and task relevance on cognitive load and learning from news. Computers in Human Behavior. 38. Elsevier. pp. 100-109. 2014. https://doi.org/10.1016/j.chb.2014.05.021
  2. NewsMediaWorks. What we learned fromthe 2018 Deloitte Media Consumer Survey. https://newsmediaworks.com.au/what-we-learnedfrom-the-2018-deloitte-media-consumer-survey January 14. 2019.
  3. Facebook business. Mobile and TV: Between the Screens. https://www.facebook.com/business/news/insights/mobile-and-tv-between-the-screens#Multiscreening-has-gone-mainstream January 14. 2019.
  4. Wang, Z., Irwin, M., Cooper, C. and Srivastava, J. Multidimensions of media multitasking and adaptive media selection. Human Communication Research. 41(1). International Communication Association. pp. 102-127. 2015. https://doi.org/10.1111/hcre.12042
  5. Canalys. Smart speaker installed base to hit 100 million by end of 2018. https://www.canalys.com/newsroom/smart-speaker-installed-base-to-hit-100-million-by-end-of-2018 January 14. 2019.
  6. Lang, A. The limited capacity model of mediated message processing. Journal of communication. 50(1). International Communication Association. pp. 46-70. 2000. https://doi.org/10.1111/j.1460-2466.2000.tb02833.x
  7. Mayer, R. E. and Moreno, R. Nine ways to reduce cognitive load in multimedia learning. Educational psychologist. 38(1). Taylor & Francis. pp. 43-52. 2003. https://doi.org/10.1207/S15326985EP3801_6
  8. Wickens, C. D. Multiple resources and performance prediction. Theoretical issues in Ergonomics Science. 3(2). Taylor & Francis. pp. 159-177. 2002. https://doi.org/10.1080/14639220210123806
  9. Salvucci, D. D. and Taatgen, N. A. Threaded cognition: An integrated theory of concurrent multitasking. Psychological review. 115(1). American Psychological Association. pp. 101-130. 2008. https://doi.org/10.1037/0033-295X.115.1.101
  10. Salvucci, D. D., Taatgen, N. A. and Borst, J. P. Toward a unified theory of the multitasking continuum: From concurrent performance to task switching, interruption, and resumption. In Proceedings of the SIGCHI conference on human factors in computing systems. ACM. pp. 1819-1828. 2009.
  11. Armstrong, G. B. and Chung, L. Background television and reading memory in context: Assessing TV interference and facilitative context effects on encoding versus retrieval processes. Communication Research. 27(3). SAGE journals. pp. 327-352. 2000.
  12. Bolls, P. D. and Muehling, D. D. The effects of dual-task processing on consumers' responses to high-and low-imagery radio advertisements. Journal of Advertising. 36(4). Taylor & Francis. pp. 35-47. 2007. https://doi.org/10.2753/JOA0091-3367360403
  13. Bowman, L. L., Levine, L. E., Waite, B. M. and Gendron, M. Can students really multitask? An experimental study of instant messaging while reading. Computers & Education. 54(4). Elsevier. pp. 927-931. 2010. https://doi.org/10.1016/j.compedu.2009.09.024
  14. Pool, M. M., Van Der Voort, T. H., Beentjes, J. W. and Koolstra, C. M. Background television as an inhibitor of performance on easy and difficult homework assignments. Communication Research. 27(3). SAGE journals. pp. 293-326. 2000.
  15. Jeong, S. H. and Hwang, Y. Multitasking and persuasion: The role of structural interference. Media Psychology. 18(4). Taylor & Francis. pp. 451-474. 2015. https://doi.org/10.1080/15213269.2014.933114
  16. Hwang, Y. and Jeong, S. H. Multitasking and task performance: Roles of task hierarchy, sensory interference, and behavioral response. Computers in Human Behavior. 81. Elsevier. pp. 161-167. 2018. https://doi.org/10.1016/j.chb.2017.12.008
  17. Lauricella, A. R., Blackwell, C. K. and Wartella, E. The "New" technology environment: The role of content and context on learning and development from mobile media. In Media exposure during infancy and early childhood. Springer. pp. 1-23. 2017.
  18. Wang, Z., David, P., Srivastava, J., Powers, S., Brady, C., D'Angelo, J. and Moreland, J. Behavioral performance and visual attention in communication multitasking: A comparison between instant messaging and online voice chat. Computers in Human Behavior. 28(3). Elsevier. pp. 968-975. 2012. https://doi.org/10.1016/j.chb.2011.12.018
  19. Myers, C., Furqan, A., Nebolsky, J., Caro, K. and Zhu, J. Patterns for How Users Overcome Obstacles in Voice User Interfaces. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems. ACM. pp. 6. 2018.
  20. D Hart, S. G. and Staveland, L. E. Development of NASA-TLX (Task Load Index): Results of empirical and theoretical research. In Advances in psychology. 52. Elsevier. pp. 139-183. 1988. https://doi.org/10.1016/S0166-4115(08)62386-9
  21. NAVER Corp. 네이버 - NAVER. https://itunes.apple.com/us/app/id393499958?mt=8 2019.1.14.
  22. Maulsby, D., Greenberg, S. and Mander, R. Prototyping an intelligent agent through Wizard of Oz. In Proceedings of the INTERACT '93 and CHI'93 conference on Human factors in computing systems. ACM. pp. 277-284. 1993.
  23. Paas, F., Tuovinen, J. E., Tabbers, H. and VanGerven, P. W. Cognitive load measurement as a means to advance cognitive load theory. Educational psychologist. 38(1). Taylor & Francis. pp. 63-71. 2003. https://doi.org/10.1207/S15326985EP3801_8
  24. Yeh, Y. Y. and Wickens, C. D. Dissociation of performance and subjective measures of workload. Human Factors. 30(1). SAGE journals. pp. 111-120. 1988.
  25. Sweller, J. Cognitive load theory, learning difficulty, and instructional design. Learning and instruction. 4(4). Elsevier. pp. 295-312. 1994. https://doi.org/10.1016/0959-4752(94)90003-5
  26. Limerick, H., Moore, J. W. and Coyle, D. Empirical evidence for a diminished sense of agency in speech interfaces. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems. ACM. pp. 3967-3970. 2015.
  27. Corbett, E. and Weber, A. What can I say?: addressing user experience challenges of a mobile voice user interface for accessibility. In Proceedings of the 18th International Conference on Human-Computer Interaction with Mobile Devices and Services. ACM. pp. 72-82. 2016.
  28. Getty, D., Biondi, F., Morgan, S. D., Cooper, J. M. and Strayer, D. L. The Effects of Voice System Design Components on Driver Workload. Transportation Research Record. 0361198118777382. SAGE journals. pp. 1-7. 2018. https://doi.org/10.1177/0361198118777382