Development of Evaluation Indicators for the Usability Evaluation of Smart Home App Design

스마트 홈(Smart Home) 앱 디자인의 사용성 평가를 위한 평가지표 개발

  • 이만 (충남대학교 디자인창의학과) ;
  • 김맹호 (충남대학교 디자인창의학과)
  • Received : 2019.03.12
  • Accepted : 2019.05.10
  • Published : 2019.05.28


Currently, people are paying more attention to smart home businesses. Therefore, the research on smart home system is becoming important. Smart Home App, which is an important component of smart home system, needs to consider the development of evaluation indicators for usability. In this regard, this paper takes smart home App as the object to develop the usability evaluation indicators of smart home App design. For this purpose, this study performed three rounds of survey by using the Delphi method. In the first round, 8 suitable areas were selected by collecting opinions from experts and analyzing the Content Validity Ratio, which was derived from literature review. Based on the results of the first survey, a total of 38 evaluation items were collected through prior research and user interview surveys. For the 38 collected evaluation items, 27 evaluation items have been selected by analyzing Average Value, Standard Deviation, Content Validity Ratio, Consultation and Convergence in the second and third rounds. The result of this study can be used as a guideline when evaluating the usability of smart home App design.


Smart Home App;Usability;Evaluation Indications;Delphi Technique

표 1. 다양한 사용성 평가지표

CCTHCV_2019_v19n5_249_t0001.png 이미지

표 2. 앱의 평가 기준에 관한 문헌 고찰

CCTHCV_2019_v19n5_249_t0002.png 이미지

표 3. 스마트 홈 앱 사용성 평가 특성 영역

CCTHCV_2019_v19n5_249_t0003.png 이미지

표 4. 15개 사용성 평가 영역

CCTHCV_2019_v19n5_249_t0004.png 이미지

표 5. 델파이 1차 설문지

CCTHCV_2019_v19n5_249_t0005.png 이미지

표 6. The corresponding reference values of CVR

CCTHCV_2019_v19n5_249_t0006.png 이미지

표 7. 1차 델파이 조사 CVR 결과

CCTHCV_2019_v19n5_249_t0007.png 이미지

표 8. 33개 평가 항목

CCTHCV_2019_v19n5_249_t0008.png 이미지

표 9. 2차 델파이 조사 결과

CCTHCV_2019_v19n5_249_t0009.png 이미지

표 10. 3차 델파이 조사 결과

CCTHCV_2019_v19n5_249_t0010.png 이미지

표 11. 최종 평가지표

CCTHCV_2019_v19n5_249_t0011.png 이미지


  1. L. Atzori and A. Iera, "Morabito G. The internet of things: 2018 Global Smart Home Market Forecast," Computer networks, Vol.54, No.15, pp.2787-2805, 2010.
  2., 2018.
  3. 편정민, 모바일 폰 GUI 디자인 사용성 평가방법의 최적화, 단국대학교 대학원, 박사학위논문, p.4, 2005.
  4. 한국 스마트홈 산업협회, 스마트홈(홈IOT)생태계 6대 구성요소, p.2, 2014.
  5. KISTIMIRIAN report, 사물인터넷, 실천과 상상력, 미래의 창, 2015.
  6. 김민주, 모바일 스마트 홈 어플리케이션 GUI디자인유형에 따른 사용편의성이 이용의도에 미치는 영향에 관한연구, 홍익대학교, 석사학위논문, p.11, 2016.
  7. J. Nielsen, Usability engineering, SanFrancisco, Calif. Morgan Kaufman Publishers, 1994.
  8. B. Shackel, "Usability, context, framework, definiti on, designand evaluation," In B. Shackel and S. Richardson(Eds.), Human Factors for Informatics Usability, Cambridge University Press, Cambridge, pp.21-38, 1991.
  9. J. Nielsen, Designing Web Usability : The Practice of Simplicity, Indianapolis, Indiana USA : New Riders Publishing, 2000.
  10. J. Nielson, Usability Engineering, Acadenic Press, Inc., 1993.
  11. ISO 9241-11, Ergonomic Requirements for Of Tice Work with Visual Display Terminals (VDT)s-Part II Guidanceon Usability, 1998.
  12. J. Preece and D. Benyon, A guide to usability:Human factors in computing, Addison-Wesley, 1993.
  13. B. Shackel, Human factors for informatics usability, Cambridge University Press, 1991.
  14. P. Morville, Ambient Findability, O'Reilly Media, Inc., 2005.
  15. Whitney Quesenbery, Design and Research in a Connected World, Elsevier Australia, 2001.
  16. R. Baharuddin, D. Singh, and R. Razali, "Usability dimensions for mobile applicationsareview," Research Journal of Applied Sciences, Engineering and Technology, Vol.5, No.6, pp.2225-2231, 2013.
  17. A. Kekalainen, A. Kaikkonen, A. Kankainen, M. Cankar, and T. Kallio, "Usability testing of mobile applications: a comparison between laboratory and field testing," Journal of Usability Studies, Vol.1, No.1, pp.4-17, 2005.
  18. J. S. Lai and D. Zhang, "Extended thumb: a target acquisition approach for one-handed interaction with touch-screen mobile phones," IEEE Trans. Human Machine Systems, Vol.45, No.3, pp.362-370, 2015.
  19. Jung-Gil Cho and Won-Whoi Huh, "Usability Evaluation Frameworks of Mobile Apps on Smart Phone," Asia-pacific Journal of Multimedia Services Convergent with Art, Humanities, and Sociology, Vol.7, No.12, pp.841-848, 2017.
  20. J. Heo, D. H. Ham, S. Park, C. Song, and W. C. Yoon, "A framework for evaluating the usability of mobile phones based on multi-level, hierarchical model of usability factors," interacting with Computers, Vol.21, No.4, pp.263-275, 2009.
  21. Xuemei Yang, Li Xin, and Lining Shen, "Construction of App Evaluation Index System from the Perspective of User Experience," Digital Library Forum, pp.59-66, 2017.
  22. Yuhang Jiang, Application Research of Intelligent Home base on Internet of Things Technology, Beijing University of Posts and Telecommunications, 2011.
  23. Li Jiang, Da-you Liu, and Bo Yang, "Smart Home tow Research," Machine Learning to and Cybernetics, Proceedings of 2014 International Conference, Vol.652, pp.659- 663, 2014.
  24. 안진성, 델파이 기법(Delphi)과 계층적 의사결정방법(AHP)의 적용을 통한 전통정원의 보존상태 평가지표 개발, 성균관대학교, 박사학위논문, p.38, 2010.
  25. Z. S. Lee, Research Methods 21: Delphi Method, Seoul, Education Science Society, 2001.
  26. 권도연, 김보연, "주문형 운송서비스의 모바일 애플리케이션 사용성 평가 연구: 카카오택시와 우버를 중심으로," 커뮤니케이션 디자인학연구, 제53권, pp.60-70, 2015.
  27. C. H. Lawshe, "A Quantitative approach to content validity," Personnel Psychology, Vol.28, No.4, pp.563-575, 1975.
  28. J. S. Lee, Research Method 21: Delphi Method, Seoul: Kyoyook-book, 2001.
  29. E. A. Im, K. C. Son, and J. K. Kam, "Development of elements of horticultural therapy evaluation indices(HTEI) through Delphi method," Korean Journal of Horticultural Science & Technology, Vol.30, No.3, pp.308-324, 2012.
  30. H. Xu, M. J. Seok, W. D. Yu, and T. H. Kang, "A study on indicators selection of management effectiveness evaluation for world cultural heritage used by Delphi technique," Journal of Korean Institute of Traditional Landscape Architecture, Vol.31, No.2, pp.136-144, 2013.
  31. 이자경, 모바일 UI의 사용성 평가요인에 관한 연구, 이화여자대학교, 석사학위논문, 2005.
  32. 정회준, 김광명, 조선, 고영준, "서비스디자인 플랫폼을 위한 사용성 평가지표 연구," 한국과학예술포럼, 제20권, pp.409-419, 2015.
  33. 최지혜, 모바일 웹 UI/UX 의 사용성 향상을위한 감리 프레임워크 연구, 건국대학교 대학원, 석사학위논문, 2014.
  34. 김민주, 모바일 스마트홈 어플리케이션 GUI디자인유형에 따른 사용편의성이 이용의도에 미치는 영향에 관한 연구, 홍익대학교 대학원, 석사학위논문, 2016.
  35. Jiang Yue, Research on App Availability Evaluation of Social Mobile Phone Based on User Perspective, Nankai University, 2016.
  36. Peisong Wang, Research on Interface Interaction Design of Smart Home Control Terminal Based on General Design Concept, Xi'an University of Engineering, 2016.
  37. Qianqian Hu, Research on Intelligent Product Design of Empty Nest Elderly Home Care, Zhengzhou Institute of Light Industry, 2018.