Machine Learning Frameworks for Automated Software Testing Tools : A Study

  • Kim, Jungho (Analysis & Consulting Team, Big Data Business Division, ONYCOM Inc.) ;
  • Ryu, Joung Woo (Analysis & Consulting Team, Big Data Business Division, ONYCOM Inc.) ;
  • Shin, Hyun-Jeong (School of IT Convergence Engineering/Computer Science & Engineering Shinhan University) ;
  • Song, Jin-Hee (School of IT Convergence Engineering/Computer Science & Engineering Shinhan University)
  • Received : 2017.01.31
  • Accepted : 2017.02.13
  • Published : 2017.03.28


Increased use of software and complexity of software functions, as well as shortened software quality evaluation periods, have increased the importance and necessity for automation of software testing. Automating software testing by using machine learning not only minimizes errors in manual testing, but also allows a speedier evaluation. Research on machine learning in automated software testing has so far focused on solving special problems with algorithms, leading to difficulties for the software developers and testers, in applying machine learning to software testing automation. This paper, proposes a new machine learning framework for software testing automation through related studies. To maximize the performance of software testing, we analyzed and categorized the machine learning algorithms applicable to each software test phase, including the diverse data that can be used in the algorithms. We believe that our framework allows software developers or testers to choose a machine learning algorithm suitable for their purpose.


Supported by : Korea Evaluation Institute of Industrial Technology


  2. Boris Beizer, Software testing techniques (2nd ed.), Van Nostrand Reinhold Co., 1990.
  7. Moataz A. Ahmed and Irman Hermadi, "GA-based multiple paths test data generator," Computer & Operations Research, vol. 35, issue 10, 2008, pp. 3107-3124.
  8. Joachim Wegener, Andre Baresel, and Harmen Sthamer, "Evolutionary test environment for automatic structural testing," Information and Software Technology, vol. 43, no. 14, 2001, pp. 841-854.
  9. A. von Mayrhauser, C. Anderson, and R. Mraz, "Using a neural network to predict test case effectiveness," In Aerospace Applications Conference Proceedings, vol. 2, no. 0, 1995, pp. 77-91.
  10. Lionel C. Briand, Yvan Labiche, and Xuetao Liu, "Using machine learning to support debugging with tarantula," In Proceedings of the 18th IEEE International Symposium on Software Reliability, Washington DC, USA, 2007, pp. 137-146.
  11. B. Uma Maheswari and S. Vali, "Survey on Graphical User Interface and Machine Learning Based Testing Techniques," Journal of Artificial Intelligence, vol. 7, no. 3, 2014, pp. 1994-5750.
  12. Pooja Paramshetti and D.A. Phalke, "Survey on Software Defect Prediction Using Machine Learning Techniques," International Journal of Science and Research, 2012, ISSN (Online) pp. 2319-7064.
  13. M. Noorian, E. Bagheri, and W. Du, "Machine learningbased software testing : Towards a classification framework," Proceedings of the International Conference on Software Engineering and Knowledge Engineering, Boston, USA, 2011, pp. 225-229.
  14. Paul Ammann and Jeff Offutt, Introduction to software testing, Cambridge University Press, 2008.
  15. Glenford J. Myers and Corey Sandler, The Art of Software Testing, John Wiley & Sons, 2004.
  16. Tom M. Mitchell, Machine learning, McGraw Hill series in computer science, McGraw-Hill, 1997.
  17. Thomas J. Cheatham, Jungsoon P. Yoo, and Nancy J. Wahl, "Software testing: a machine learning experiment," Proceedings of 23rd annual conference on Computer Science, Tennessee, USA, 1995, pp. 135-141.
  18. Daniel G. e Silva, Mario Jino, and Bruno T. de Abreu, "Machine Learning Methods and Asymmetric Cost Function to Estimate Execution Effort of Software," Proceeding of 2010 Third International Conference on Software Testing, Verification and Validation, Paris, France, 2010, pp. 257-284.
  19. Lionel C. Briand, Yvan Labiche, and Zaheer Bawar, "Using machine learning to refine black-box test specifications and test suites," In Proceedings of the 2008 Eighth International Conference on Quality Software Washington DC, USA, 2008, pp. 135-144.
  20. W. Eric Wong and Yu Qi, "BP neural network-based effective fault localization," International Journal of Software Engineering and Knowledge Engineering, vol. 19, issue 04, 2009, pp. 573-597.
  21. Susan A. Sherer, "Software fault prediction," Journal of Systems and Software, vol. 29, no. 2, 1995, pp. 97-105.
  22. Xinli Yang, David Lo, Xin Xia, Yun Zhang, and Jianling Sun, "Deep Learning for Just-In-Time Defect Prediction," In Proceedings of IEEE International Conference on Software Quality, Reliability and Security, Vancouver, Canada, 2015, pp. 17-26.