Journal of the Korea Academia-Industrial cooperation Society (한국산학기술학회논문지)

The Korea Academia-Industrial cooperation Society (한국산학기술학회)
권호 표지
DOI QR코드

DOI QR Code

Web Usability Testing by Using Scanpath Similarity Analysis

탐색경로 일치도 분석을 이용한 웹사이트 사용성 평가

  • Received : 2012.11.21
  • Accepted : 2013.02.06
  • Published : 2013.02.28
Download PDF
⟨ Previous Next ⟩

Abstract

This study was performed to determine the usefulness of scanpath similarity analysis as one of new web usability testing. The 5 websites of public institutions were used and 15 students participated. First of all, eye movements were tracked and visual appeal ratings were measured as participants freely viewed each website for 3 seconds. Subsequently in continuously tracking the eye movements we asked the participants to perform 17 missions. Finally, in interview the participants rated on satisfaction, awareness, and mission difficulty. Results of this study showed that scanpath similarity had a significant relationship with both the visual appeal ratings(first impression) and the satisfaction. In other words, higher the visual appeal ratings were related to higher scanpath similarity. This result showed that measurement such as scanpath similarity of eye movements could become an objective index for usability testing instead of subjective evaluation such as the satisfaction. We discussed possibility that the usability testing by using the scanpath similarity with both fixation and duration on eye movements will find more appropriately inference on observers' experiences in websites.

본 연구는 새로운 웹사용성 평가 방법의 하나로 탐색경로 일치도 분석의 유용성을 알아보기 위해 수행되었다. 사용한 웹사이트는 5개 공공기관이었으며, 15명의 피험자가 참여하였다. 우선, 참가자들에게 각 웹사이트를 3초간 보여주고 자유롭게 탐색하게 하면서 안구운동을 추적하였고 호감도를 측정하였다. 그 후 안구운동 추적 상태에서 참가자들에게 17개의 임무과제를 수행하도록 하였으며. 마지막 인터뷰에서는 웹사이트에 대한 만족도, 인지도, 과제 난이도 등에 응답하게 하였다. 여러 측정치들 간의 관련성을 분석하였으며, 분석 결과 본 연구에서 관심을 가졌던 탐색경로 일치도가 호감도 및 만족도와 의미 있는 관련성을 보였다. 즉 웹사이트에 대한 호감도가 높을수록 탐색경로 일치도가 높은 결과를 얻었다. 이 결과는 웹사이트에 대한 주관적 만족도 평가 방식을 대신해, 안구운동의 탐색경로 일치도와 같은 측정치가 객관적인 사용성 평가 지수가 될 수 있음을 보여주는 것이다. 안구운동 고정 및 응시시간 등과 함께 사용함으로써 웹사이트에서 보이는 관찰자의 행동을 보다 더 적절하게 추론할 수 있는 가능성을 논의하였다.

Keywords

References

  1. Y. Puzis. "Accessible web automation interface: A user study". Proc. of the 14th International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS). 2012. DOI: http://dx.doi.org/10.1145/2384916.2384999
  2. D. A. Norman, The Design of Everyday Things, New York: Doubleday. 1998.
  3. J. Nielsen, Usability Engineering, Boston, MA: Academic Press. 1993.
  4. B. Keevil, Measuring the usability index of your Web site, Proc. of the 16th annual international conference on Computer documentation(ACM), pp. 271-277. 1998.
  5. A. Fernandez, S. Abrahao, and E. Insfran, "Empirical validation of a usability inspection method for model-driven Web development", Journal of Systems and Software, 86, 1, pp. 161-186, 2013. DOI: http://dx.doi.org/10.1016/j.jss.2012.07.043
  6. A. P. Afonso, J. R. Lima, and M. P. Cota. "A heuristic evaluation of usability of Web interfaces". Proc. of Iberian Conference on Information Systems and Technologies(CISTI), 2012.
  7. J. M. Spool, Web Site Usability : A Designer's Guide. San Francisco, EUA: Morgan Kaufmann. 1999.
  8. T. E. Moon and H. N. Moon, "A study on the evaluation and improvement methods of web accessibility and usability of Korea government department websites.", Daehan Journal of Business, 22, 3, pp. 1511-1535, 2009.
  9. J. G. Min and K. S. Lee, "Quality Evaluation for the Usability of Multimedia Web Sites", Journal of the Korea Society of Computer and Information, 11, 5, pp. 139-148, 2006.
  10. M. S. Lee and K. S. Lee, "Quality Evaluation for Usability of Internet Shopping Mall", Journal of the Korea Society of Computer and Information, 12, 6, pp. 59-68, 2007.
  11. H. W. Kwak, J. U. Kwahk, S. J. Kim, and J. M. Lee, "Usability testing of the domestic web site design: questionnaire and heuristic evaluation", Koean Journal of Cognitive Science, 11, 1, pp. 33-45, 2000.
  12. B. J. Kim and K. P. Lee, "Development of Integrated Analysis Model and Tool for Web Usability Test-with Emphasis on Eye-Tracking, Mouse-Tracking, and Retrospective Think Aloud.", Journal of Korean Society of Design Science, 20, 5, pp. 39-50, 2007.
  13. K. Choi, J. Shu, and Y. Kim, "An Investigation of Banner Ad Effects Using Eye Tracking: An Explorative Observation", The Korean Journal of Experimental Psychology, 16, 4, pp. 421-434, 2004.
  14. D. Rosaci and G. M. L. Sarne, "Recommending multimedia web services in a multi-device environment", Information Systems, 38, 2, pp. 198-212, 2013. DOI: http://dx.doi.org/10.1016/j.is.2012.08.002
  15. A. L. Yarbus, Eye movements and vision, New York: Plenum Press. 1967.
  16. S. Josephson and M. E. Holmes, "Attention to repeated images on the World-Wide Web: Another look at scanpath theory", Behavior Research Methods Instruments & Computers, 34, 4, pp. 539-548, 2002. DOI: http://dx.doi.org/10.3758/BF03195483
  17. C. M. Privitera and L. W. Stark, "Algorithms for defining visual regions-of-interest: Comparison with eye fixations", Ieee Transactions on Pattern Analysis and Machine Intelligence, 22, 9, pp. 970-982, 2000. DOI: http://dx.doi.org/10.1109/34.877520
  18. J. M. Findlay and V. Brown, "Eye scanning of multi-element displays: I. Scanpath planning", Vision Research, 46, 1-2, pp. 179-195, 2006. https://doi.org/10.1016/j.visres.2005.06.010
  19. R. Pieters, E. Rosbergen, and M. Wedel, "Visual attention to repeated print advertising: A test of scanpath theory", Journal of Marketing Research, 36, 4, pp. 424-438, 1999. DOI: http://dx.doi.org/10.2307/3151998
  20. D. Noton and L. Stark, "Scanpaths in saccadic eye movements while viewing and recognizing patterns", Vision Research, 11, 9, pp. 929-932, 1971. DOI: http://dx.doi.org/10.1016/0042-6989(71)90213-6
  21. J. H. Goldberg and J. I. Helfman. "Visual scanpath representation". Proc. of Eye Tracking Research and Applications Symposium (ETRA), 2010.
  22. C. Holland, O. Komogortsev, and D. Tamir. "Identifying usability issues via algorithmic detection of excessive visual search". Proc. of Conference on Human Factors in Computing Systems.2012.
  23. L. W. Stark and Y. S. Choi, Experimental metaphysics: The scanpath as an epistemological mechanism. pp. 3-69. 1996.
  24. Y. S. Choi, A. D. Mosley, and L. W. Stark, "STRING EDITING ANALYSIS OF HUMAN VISUAL-SEARCH", Optometry and Vision Science, 72, 7, pp. 439-451, 1995. DOI: http://dx.doi.org/10.1097/00006324-199507000-00003
  25. A. T. Duchowski, J. Driver, S. Jolaoso, W. Tan, B. N. Ramey, and A. Robbins, Scanpath comparison revisited, Proc. of the 2010 Symposium on Eye-Tracking Research & Applications(ACM), pp. 219-226. 2010.
  26. G. Lindgaard and C. Dudek, User Satisfaction, Aesthetics and Usability: Beyond Reductionism, Proc. of the IFIP 17th World Computer Congress-TC13 Stream on Usability: Gaining a Competitive Edge, pp. 231-246. 2002.
  27. G. Lindgaard, G. Fernandes, C. Dudek, and J. Brown, "Attention web designers: you have 50 milliseconds to make a good first impression!", Behaviour & Information Technology, 25, 2, pp. 115-126, 2006. DOI: http://dx.doi.org/10.1080/01449290500330448
  28. A. Campbell and S. Pisterman, "A Fitting Approach to Interactive Service Design: THE IMPORTANCE OF EMOTIONAL NEEDS", Design Management Journal (Former Series), 7, 4, pp. 10-14, 1996.
  29. W. Schneider, A. Eschman, and A. Zuccolotto, E-Prime User's Guide, Pittsburgh: Psychology Software Tools, Inc. 2002.
  30. A. VoBkuhler. OGAMA Description (for Version 2.5). Available from: http://www.ogama.net/sites/default/files/pdf/OGAMA-De scriptionV25.pdf,(accessed Dec, 27, 2012).
  31. A. VoBkuhler, V. Nordmeier, L. Kuchinke, and A. Jacobs, "OGAMA (Open Gaze and Mouse Analyzer): Open-source software designed to analyze eye and mouse movements in slideshow study designs", Behavior Research Methods, 40, 4, pp. 1150-1162, 2008. DOI: http://dx.doi.org/10.3758/BRM.40.4.1150
  32. J. H. An, "A Study on the User's Evaluation Factors of a Broadcasting Company Web Site", Korean journal of journalism & communication studies, 47, 1, pp. 191-215, 2003.
  33. J. Nielsen. Thinking Aloud: The #1 Usability Tool. Alertbox: Current Issues in Web Usability 2012, Available from: http://www.useit.com/alertbox/thinking-aloud-tests.html, (accessed Nov, 12, 2012).
  34. J. E. Kwahk and H. W. Kwak, "Usability of Internet Search Interfaces: Establishing Design Guidelines based on Expert Review and User Testing Methods", The Korean journal of experimental and cognitive and biological psychology, 14, 4, pp. 463-483, 2002.
  35. J. Brooke, SUS: A quick and dirty usability scale, in Usability evaluation in industry, Edited by P. W. Jordan, B. Weerdmeester, A. Thomas, and I. L. McLelland, Taylor and Francis. 1996.