Journal of Information Technology Services (한국IT서비스학회지)

Korea Society of IT Services (한국IT서비스학회)
권호 표지
DOI QR코드

DOI QR Code

Software Test Automation Using Data-Driven Approach : A Case Study on the Payment System for Online Shopping

데이터 주도 접근법을 활용한 소프트웨어 테스트 자동화 : 온라인 쇼핑몰 결제시스템 사례

  • 김성용 (고려대학교 일반대학원 디지털경영학과, (주)넥슨) ;
  • 민대환 (고려대학교 경영정보/디지털경영학과) ;
  • 임성택 (고려대학교 경영정보/디지털경영학과)
  • Received : 2017.11.05
  • Accepted : 2017.12.26
  • Published : 2018.03.31
Download PDF
⟨ Previous Next ⟩

Abstract

This study examines a data-driven approach for software test automation at an online shopping site. Online shopping sites typically change prices dynamically, offer various discounts or coupons, and provide diverse delivery and payment options such as electronic fund transfer, credit cards, mobile payments (KakaoPay, NaverPay, SyrupPay, ApplePay, SamsungPay, etc.) and so on. As a result, they have to test numerous combinations of possible customer choices continuously and repetitively. The total number of test cases is almost 584 billion. This requires somehow automation of tests in settling payments. However, the record playback approach has difficulties in maintaining automation scripts due to frequent changes and complicated component identification. In contrast, the data-driven approach minimizes changes in scripts and component identification. This study shows that the data-driven approach to test automation is more effective than the traditional record playback method. In 2014 before the test automation, the monthly average defects were 5.6 during the test and 12.5 during operation. In 2015 after the test automation, the monthly average defects were 9.4 during the test and 2.8 during operation. The comparison of live defects and detected errors during the test shows statistically significant differences before and after introducing the test automation using the data-driven approach.

Keywords

References

  1. Bolt, W., D. Humphrey, and R. Uittenbogaard, "Transaction Pricing and the Adoption of Electronic Payments : A Cross-Country Comparison", International Journal of Central Banking, Vol.4, No.1, 2008, 89-123.
  2. Chae, H.S., "Model-based Test-Concepts and Issues", Communications of the Korean Institute of Information Scientists and Engineers, Vol.32, No.4, 2014, 59-71. (채흥석, "모델 기반 테스트-개념과 이슈", 정보과학회지, 제32권, 제4호, 2014, 59-71.)
  3. Damm, L.O. and L. Lundberg, "Results from Introducing Component-level Test Automation and Test-Driven Development", Journal of Systems and Software, Vol.79, No.7, 2006, 1001-1014. https://doi.org/10.1016/j.jss.2005.10.015
  4. DMC Media, "2016 The Market Trends of Mobile Easy Payment Services", DMC, 2016. (DMC Media, "2016 모바일 간편결제 서비스 시장 현황과 전망", DMC, 2016.)
  5. Do, H. and G. Rothermel, "On the Use of Mutation Faults in Empirical Assessments of Test Case Prioritization Techniques", IEEE Transactions On Software Engineering, Vol.32, No.9, 2006, 733-752. https://doi.org/10.1109/TSE.2006.92
  6. Elbaum, S., A.G. Malishevsky, and G. Rothermel, "Prioritizing Test Cases for Regression Testing", Proceedings of the International Symposium of Software Testing and Analysis, 2000, 102-112.
  7. Gerrard, P., "Testing GUI Applications", The Fifth EuroSTAR(Software Testing and Review) Conference, 1997, 24-28.
  8. Hwang, Y. and J. Jeong, "Electronic Commerce and Online Consumer Behavior Research : A Literature Review", Information Development, Vol.32, No.3, 2016, 377-388. https://doi.org/10.1177/0266666914551071
  9. Jeong, K.C., S.H. Lee, J.H. Jo, Y.M. Sin, and Y.O. Park, "An Efficient Regression Testing Using Testing Automation Tool for Industrial Automation Software", Proceedings of Control Automation System Symposium, 2008, 245- 249. (정광철, 이상훈, 조주현, 신영민, 박용운, "자동화 테 스트 도구를 활용한 산업용 자동화 소프트웨어 의 효율적인 회귀 테스트", 제어로봇시스템학 회 국내학술대회논문집, 2008, 245-249.)
  10. Jin, J.S., H.M. Kim, and J.S. Park, "A Study on Behavior in Using Fin-Tech Based on Life Style Types", Journal of Information Technology Services, Vol.16, No.1, 2017, 119-138. (진정숙, 김현모, 박주석, "라이프스타일 유형에 따 른 모바일 간편결제 서비스의 이용행태 연구", 한국IT서비스학회지, 제16권, 제1호, 2017, 119-138.)
  11. Jo, H. and J.M. Lee, "A Study on Antecedents of WOM in the Context of Internet ECommerce", Journal of Information Technology Services, Vol.12, No.2, 2013, 231- 242. (조 현, 이정민, "인터넷 전자상거래 환경에서의 구 전효과의 선행 요인에 관한 연구", 한국IT서비스학회지, 제12권, 제2호, 2013, 231-242.)
  12. Jung, W.J., S.R. Rah, and Y.L. Choi, "A Study on the Selection of Test Scope and the Prioritization of Test Case Based on Modification Method for Regression Testing", Journal of Information Technology Services, Vol.14, No.2, 2015, 129-142. (정우진, 나상린, 최용락, "변경 메서드 기반의 회귀 테스트 검증 범위 선택 및 검증 항목 우선순위 선정에 관한 연구", 한국IT서비스학회지, 제14권, 제2호, 2015, 129-142.)
  13. Kim, S.D., P. Park, and S.B. Yang, "Influencing Factors on Users' Resistance to the Mobile Easy Payment Services : Focusing on the Case of KakaoPay Users", Journal of Information Technology Services, Vol.16, No.2, 2017, 139-156. (김소담, Philip Park, 양성병, "모바일 간편결제 서비스에 대한 사용자 수용저항 요인 : 카카오페 이 사용자를 중심으로", 한국IT서비스학회지, 제16권, 제2호, 2017, 139-156.)
  14. Kwon, W.I., H.J. Lee, S.H. Choi, S.H. Lee, and E.Y. Park, Practical Software Testings for the Developers, Third Edition, STA, 2010. (권원일, 이현주, 최승희, 이승호, 박은영, 개발자도 알아야 할 소프트웨어 테스팅 실무, 3판, STA, 2010.)
  15. Lee, J.G., H.S. Kim, S.H. Kuk, and D.W, Cho, "Record-Playback based Automatic Test Case Generation for GUI Test", Proc. of the KIISE Korea Computer Congress, 2007, Vol.34, No.1(B), 2007, 96-100. (이정규, 김현수, 국승학, 조대환, "Record-Playback 기술 기반의 GUI 테스트 케이스 자동 생성", 한국컴퓨터종합학술대회논문집, 제34권, 제1 호(B), 2007, 96-100.
  16. Memon, A., A. Nagarajan, and Q. Xie, "Automating Regression Testing for Evolving GUI Software", Journal of Software Maintenance and Evolution : Research and Practice, Vol.17, No.1, 2005, 27-64. https://doi.org/10.1002/smr.305
  17. Meszaros, G., R. Bohnet, and J. Andrea, "Agile Regression Testing Using Record and Playback" In F. Maurer & D. Wells(Eds.), Extreme Programming and Agile Methods- XP/Agile Universe, Lecture Notes in Computer Science, Springer, Berlin, Heidelberg, Vol.2753, 2003, 111-119.
  18. Morris, S., "Automated GUI Testing", Tessella Support Services PLC, 1999.
  19. Patton, R., Software Testing, Second Edition, Information publishing Group, 2006. (Ron Patton, 소프트웨어 테스팅, 제2판, 김도균 옮김, 정보문화사, 2006.)
  20. Polo, M., S. Tendero, and M. Piattini, "Integrating Techniques and Tools for Testing Automation", Software Testing Verification and Reliability, Vol.17, No.1, 2007, 3-39. https://doi.org/10.1002/stvr.348
  21. Prototech Solutions, "9 Tips for selecting test cases for Regression Testing", available at http://www.prototechsolutions.com/regres sion-testing/(Accessed on May 18, 2017).
  22. Rothermel, G., S. Elbaum, A. Malishevsky, P. Kallakuri, and X. Qiu, "On Test Suite Composition and Cost-Effective Regression Testing", ACM Transactions on Software Engineering and Methodology, Vol.13, No.3, 2004, 277-331. https://doi.org/10.1145/1027092.1027093
  23. Schwartz, A. and H. Do, "Cost-effective Regression Testing through Adaptive Test Prioritization Strategies", Journal of Systems and Software, Vol.115, 2016, 61-81. https://doi.org/10.1016/j.jss.2016.01.018
  24. w3ii.com, "Data Driven Testing", available at http://www.w3ii.com/software_testing_dictionary/data_driven_testing.html (Accessed on May 15, 2017).
  25. Zech, P., P. Kalb, M. Felderer, C. Atkinson, and R. Breu, "Model-based Regression Testing by OCL", International Journal on Software Tools for Technology Transfer, Vol. 19, No.1, 2017, 115-131. https://doi.org/10.1007/s10009-015-0408-8