Journal of Korean Society of Archives and Records Management (한국기록관리학회지)

Korean Society of Archives and Records Management (한국기록관리학회)
권호 표지
DOI QR코드

DOI QR Code

A Study on Improvement Plans for National Archives of Korea Website's Search Service through Its Usability Evaluation

국가기록원 웹사이트 검색서비스의 사용성 평가를 통한 개선방안 연구

  • 이효진 (이화여자대학교 정책과학대학원 기록관리학전공) ;
  • 김지현 (이화여자대학교 사회과학대학 문헌정보학과)
  • Received : 2021.07.20
  • Accepted : 2021.08.13
  • Published : 2021.08.31
Download PDF
⟨ Previous Next ⟩

Abstract

Archives should provide a web-based archival information service with good usability based on the understanding of users' needs. Thus, this study analyzed the usability of the National Archives of Korea (NAK) website's search service through expert and usability evaluations for various users and suggested improvement plans. A literature review, heuristic evaluation, one of the usability expert evaluation methods, and usability evaluation for users were performed. As a result of expert evaluation, the severity rating for the usability problem was highest regarding a match between the system and real world and the lowest in error prevention. In the usability evaluation, it was assumed that the user's background and whether the website's help was provided is associated with usability. Usability evaluation was conducted for 22 participants, including those majoring in records management, web/application developers, and people without such backgrounds. Furthermore, a presurvey, a five-second test, and search tasks were conducted in a row. The analysis based on usability components in ISO 9241-11: 2018, i.e., effectiveness, efficiency, and satisfaction, showed that web/application developers had the highest search effectiveness, and those majoring in records management showed high effectiveness only in the search for archival content services. Background factors did not affect efficiency, and the more familiar with the finding aids, the better the efficiency. Moreover, providing the website's help was found to be positively associated with effectiveness and satisfaction. This study suggests that the NAK website will offer a user-friendly interface and search function, expand help support, and display a consistent web interface.

기록관에서는 이용자의 요구사항에 대한 이해를 바탕으로 설계된 사용성이 높은 웹 기반 기록정보서비스를 제공할 필요가 있다. 이에 본 연구는 국가기록원 웹사이트 검색서비스의 사용성을 전문가 평가와 다양한 그룹의 이용자 대상 사용성 평가를 통해 분석하였고 이를 기반으로 개선방안을 제시하였다. 연구방법으로 문헌연구, 인터페이스 사용성 전문가 평가 중 하나인 휴리스틱 평가 및 이용자 대상 사용성 평가를 수행하였다. 전문가 평가 결과 정확성 영역에서 심각도가 가장 높았으며 에러대처 영역의 심각도가 가장 낮았다. 이용자 대상 사용성 평가에서는 이용자의 배경요인과 도움말 제공여부가 사용성에 영향을 준다고 가정하였다. 따라서 기록관리학 전공자, 웹/앱 개발자 및 그러한 배경요인이 없는 일반인 등 총 22명을 대상으로 사용성 평가를 실시하였다. 사용성 평가는 사전 설문조사, 5초테스트, 검색과업 수행 순으로 진행하였다. ISO 9241-11:2018의 사용성 구성요소인 효과성, 효율성, 만족도를 기준으로 분석을 진행한 결과 웹/앱 개발자의 검색 효과성이 가장 높았고 기록관리학 전공자의 경우 기록정보콘텐츠 검색에서만 효과성이 높은 것으로 나타났다. 효율성에 있어 배경요인은 영향을 주지 않는 것으로 나타났으며 검색도구에 익숙해질수록 효율성이 향상되었다. 도움말 제공은 효과성과 만족도에 긍정적인 영향을 주는 것으로 나타났다. 개선방안으로 이용자에게 익숙한 인터페이스와 검색기능의 제공, 도움말 지원의 확대, 일관성 있는 웹 인터페이스 제공을 제안하였다.

Keywords

References

  1. Baek, Ji-yeon, & Oh, Hyo-Jung (2019). User Information Needs Analysis based on Query Log Big Data of the National Archives of Korea. Journal of the Korean Society for information Management, 30(4), 183-205. https://doi.org/10.3743/KOSIM.2019.36.4.183
  2. Choi, Chan-jin (2019). A Study on the Improvement of Medical Devices User Interface using Heuristic Evaluation. Unpublished doctoral dissertation, Graduate School of Dongguk University.
  3. Choi, Young-Sun (2014). Comparative study on usability evaluation of text vs. graphic-based drugs retrieval system. Unpublished doctoral dissertation, Graduate School of Sungkyunkwan University.
  4. Chun, Junghyun (2018). A Study on the Development of User Experience Indicators for The Knowledge and Information Resource System. Unpublished doctoral dissertation, Graduate School of Yonsei University.
  5. Gang, Ju-Yeon, Nam, Yeon-Hwa, & Oh, Hyo-Jung (2017). An Evaluation of Web-Based Research Records Archival Information Services and Recommendations for Their Improvement: NTIS vs. NKIS. Journal of Korean Society of Archives and Records Management, 17(3), 139-160. https://doi.org/10.14404/JKSARM.2017.17.3.139
  6. Jang, Hee-Jung (2012). A Study on Evaluation of National Archives Websites. Journal of Korean Society of Archives and Records Management, 12(2), 51-70. https://doi.org/10.14404/JKSARM.2012.12.2.051
  7. Jin, Ju Yeong, & Rieh, Hae-young (2018). Analysis of Users' Inflow Route and Search Terms of the Korea National Archives' Web Site. Journal of the Korean Society for information Management, 35(1), 183-203. https://doi.org/10.3743/KOSIM.2018.35.1.183
  8. Kang, Yerim (2020). A Study on the Improvement of Archival Reference Service in the National Archives of Korea. Unpublished Master's thesis, Graduate School of Ewha Womans University Department of Library & Information Science.
  9. Kim, Hyun-Jee, & Rieh, Hae-Young (2009). Evaluation Model Development for Web Archival Information Services and the Service Evaluation: Focused on University Archives. Journal of Korean Society of Archives and Records Management, 9(2), 59-87. https://doi.org/10.14404/JKSARM.2009.9.2.059
  10. Kim, Jihyun (2012). A Study on Users' Perception of Reference Services in National Archives of Korea. Journal of Korean Society of Archives and Records Management, 12(1), 167-187. https://doi.org/10.14404/JKSARM.2012.12.1.167
  11. Kim, Yeon-Hee (2010). An usability assessment of presidential archives websites. Unpublished Master's thesis, Graduate School of Hannam University.
  12. KISA, Ministry of Science and ICT (2018). Internet Use Survey on 2018. KISA & Ministry of Science and ICT.
  13. Lee Eun-Kyung (2008). A Study on Evaluation Method of Website Usability based on Web UI(User Interface) Design - Centering on Internet Shopping Malls. Unpublished Master's thesis, Graduate School of Information Chung-Ang University.
  14. Lee, Eung-Bong, Ryu, Bum-Jong, Kim, Woo-Sung, & Kim, Suk-Dong (2002). A Study on Usability Evaluation for Improving Quality of University Digital Library Website in Korea. Journal of the Korean Society for Library and Information Science, 36(1). https://doi.org/10.4275/KSLIS.2002.36.1.041
  15. Lee, Yoon-Ju (2007). A Study on the Evaluating Standards On-Line Service for Archives. Unpublished Master's thesis, Graduate School of Hankuk University of Foreign Studies
  16. Lee, Yoon-Ryeong, & Rieh, Hae-Young (2014). A Study on the Improvement Direction for Online Finding Aids- Based on the Assessment of National Archives. Journal of Korean Society of Archives and Records Management, 14(1), 75-100. https://doi.org/10.14404/JKSARM.2014.14.1.075
  17. Namkoong, Keun (2018). Research methods for public administration. Seoul: bum moon sa.
  18. National Archives of Korea (2020). Plan for major projects in 2020 (Draft). National Archives of Korea, Ministry of the Interior and Safety.
  19. NIA, Ministry of the Interior (2016). Guidelines for applying e-Government User Experience (UX). Smart Service Division, Ministry of the Interior & e-Government Planning Team, NIA.
  20. Seol, Moon-Won (2015). A Study of Online Research Guides for User Support to Find and Understand Archival Records in National Archives. Journal of Korean Society of Archives and Records Management, 15(1), 53-87. https://doi.org/10.14404/JKSARM.2015.15.1.053
  21. Shin, Ju-hee (2018). User Satisfactions on the Digital Content of Archives & Records - Focused on the Records of Republic of Korea of National Archives of Korea. Unpublished Master's thesis, Graduate School of Hansung University.
  22. Sinn, Dong-Hee (2003). The Concept of Finding Aids. Journal of Korean Society of Archives and Records Management, 3(1), 93-109. https://doi.org/10.14404/JKSARM.2003.3.1.093
  23. Yu, Eun-Yeong, & Kang, Soon-Ae (2019). A Study on the Usability Evaluation of the Archival Contents in the National Archives of Korea- Focus on Decisions Relating to the Independence Movement Collection. Journal of Korean Society of Archives and Records Management, 19(1), 137-166. https://doi.org/10.14404/JKSARM.2019.19.1.137
  24. Aitta, M., Kaleva, S., & Kortelainen, T. (2008). Heuristic evaluation applied to library web services. New Library World, 109(1/2), 25-45. https://doi.org/10.1108/03074800810845985
  25. Alex B. (2019). How to Use Heat Maps to Improve Your Marketing. CXL Coperation. https://cxl.com/blog/heat-maps/
  26. Bevan, N. (2001). International standards for HCI and usability. International Journal of Human-Computer Studies, 55(4), 533-552. https://doi.org/10.1006/ijhc.2001.0483
  27. Bevan, N., Carter, J., Earthy, J., Geis, T., & Harker, S. (2016). New ISO standards for usability, usability reports and usability measures. In Kurosu M. (eds.) Human-Computer Interaction. Theory, Design, Development and Practice. HCI 2016. Lecture Notes in Computer Science, vol 9731. Springer, Cham. https://doi.org/10.1007/978-3-319-39510-4_25
  28. Chen, M., Anderson, J. R., & Sohn, M. (2001). What can a mouse cursor tell us more? Correlation of eye/mouse movements on Web browsing. In Proceedings of CHI '2001 Conference on Human Factors in Computing Systems, New York: ACM Press. https://doi.org/10.1145/634067.634234
  29. Chow, A. S., Bridges, M., & Commander, P. (2014). The website design and usability of US academic and public libraries. Reference & User Services Quarterly, 53(3), 253-265. https://www.jstor.org/stable/refuseserq.53.3.253 https://doi.org/10.5860/rusq.53n3.253
  30. DeRidder, J., Presnell, A., & Walker, K. (2012). Leveraging encoded archival description for access to digital content: a cost and usability analysis. The American Archivist, 75(1), 143-170. https://doi.org/10.17723/aarc.75.1.5641v61p422u0u90
  31. Doncaster, P. (2014). The UX five-second rules. Nederland: Elsevier/Morgan Kaufmann.
  32. Elaine, W. (1993). Making computers people-literate. California: Jossey-Bass Publishers.
  33. Ergonomics of human-system interaction - Part 11: Usability: Definitions and concepts. ISO 9241-11: 2018.
  34. Jeffries, R., Miller, J.R., Wharton, C., & Uyeda, K. M. (1991). User interface evaluation in the real world: a comparison of four techniques, Proceedings of ACM CHI'91 Conference on human factors in computing systems, New York: ACM press, l19-124. https://doi.org/10.1145/108844.108862
  35. Joo, S. & Lee, J. Y. (2011). Measuring the usability of academic digital libraries: Instrument development and validation. The Electronic Library, 29(4), 523-537. https://doi.org/10.1108/02640471111156777
  36. Kous, K., Pusnik, M., Hericko, M., & Polancic, G. (2020). Usability evaluation of a library website with different end user groups. Journal of Librarianship and Information Science, 52(1), 75-90. https://doi.org/10.1177/0961000618773133
  37. Marina, P. (2019). Two types of cost-effective and easy usability tests. UX collective. https://uxdesign.cc/two-types-of-cost-effective-and-easy-usability-tests-cef41e4c13f
  38. Nielsen, J. (1993). Iterative user-interface design, IEEE Computer, 26(11), 32-41. https://doi.org/10.1109/2.241424
  39. Nielsen, J. (1994). Heuristic evaluation. In Nielsen, J. & Mack, R. L. (Eds.), Usability Inspection Methods. New York: John Wiley & Sons.
  40. Nielsen, J. (2000). Desiginig Web Usability: The Practice of Simplicity, Indianapolis, Indiana USA: New Riders Publishing.
  41. Powell, J. E. (1991). A Framework for evaluating CD-ROM retrieval software. Laserdisk Professional, 3(2), 41-46.
  42. Pugh, M. J. (2005). Providing Archival Reference services for Archives and Manuscripts. 2nd ed. SAA. Chicago: Archival Fundamental Series II.
  43. Ritthiron, S. & Jiamsanguanwong, A. (2017). Usability Evaluation of the University Library Network's Website Using an Eye-Tracking Device. Proceedings of the International Conference on Advances in Image Processing, 184-188. https://doi.org/10.1145/3133264.3133294
  44. Seffah, A., Donyaee, M., Kline, R. B., & Padda, H. K. (2006). Usability measurement and metrics: A consolidated model. Software quality journal, 14(2), 159-178. https://doi.org/10.1007/s11219-006-7600-8
  45. Yeh, S. T. & Fontenelle, C. (2012). Usability study of a mobile website: The health sciences library, University of Colorado Anschutz medical campus, experience. Journal of the Medical Library Association: JMLA, 100(1), 64-68. https://doi.org/10.3163/1536-5050.100.1.012