KIPS Transactions on Software and Data Engineering (정보처리학회논문지:소프트웨어 및 데이터공학)

Korea Information Processing Society (한국정보처리학회)
권호 표지
DOI QR코드

DOI QR Code

Design of Programming Failure Feedback System Based on Control Flow of Test Case to Support Programming Training

프로그래밍 훈련 지원을 위한 테스트케이스의 제어흐름에 기반한 프로그래밍 실패 피드백 시스템 설계

  • 이성희 (경북대학교 컴퓨터학부) ;
  • 김덕엽 (경북대학교 컴퓨터학부) ;
  • 서강복 (경북대학교 컴퓨터학부) ;
  • 이우진 (경북대학교 컴퓨터학부/소프트웨어기술연구소)
  • Received : 2019.03.26
  • Accepted : 2019.04.20
  • Published : 2019.08.31
Download PDF
⟨ Previous Next ⟩

Abstract

Programming judge systems for programming training support are typically built on the Web, where the examiners uploads a programming problem, which the student reads and submits an answer to the problem. The judge system executes the submitted answer of source code to provide feedback such as pass, failure, and error messages. Students who receive the feedback except for the pass continues debugging the source code until they are judged to pass. We developed an online judge system to support programming training and analyzed answers submitted by the students and found that many of the students who were not judged to pass that test did not know exactly where they were wrong but continued to solve the problem. The current judge system generally feeds runtime error messages back to students. However, with only runtime error message, it is difficult for student who train to find the wrong part of the answer. Therefore, in this paper, we propose a system that provides the feedback of programming failure by analyzing the control flow of the test cases used in the source code submitted by the student. The proposed system helps students find the wrong parts more quickly by feeding back the paths where faults in the control flow may exist. In addition, we show that this system is applicable to the answer source code that the actual student submitted.

프로그래밍 훈련 지원을 위한 프로그래밍 채점시스템은 일반적으로 웹 상에 구축되어 출제자는 프로그래밍 문제를 업로드하고 이를 학습자가 열람하고 문제에 대한 답안을 제출한다. 프로그래밍 채점시스템은 제출된 답안 소스코드를 실행하여 통과, 실패, 에러 메시지 등의 피드백을 제공한다. 통과를 제외한 피드백을 받은 학습자는 통과 판정받을 때까지 계속해서 소스코드를 디버깅한다. 본교에서 프로그래밍 훈련 지원을 위해 온라인 채점시스템을 자체 개발하였고 학생들이 제출한 답안을 분석한 결과 통과 판정을 받지 못한 학생들 중 다수가 자신이 틀린 부분을 정확하게 모른 채 계속해서 문제를 푼다는 사실을 확인하였다. 현재 프로그래밍 채점시스템은 주로 런타임 에러 메시지 위주로 피드백을 수행한다. 하지만 이러한 정보만으로는 프로그래밍 훈련을 하는 학습자들은 제출한 답안의 틀린 부분을 찾아내기가 어렵다. 따라서 본 논문에서는 학습자가 제출한 소스코드에 사용된 테스트케이스의 제어흐름을 분석한 결과를 프로그래밍 실패에 대한 피드백으로 제공하는 시스템을 제안한다. 제안한 시스템은 제어흐름 상의 결함이 존재할 수 있는 경로를 피드백하여 학습자들이 틀린 부분을 보다 빠르게 찾을 수 있게 도와준다. 또한 실제 학습자가 제출한 답안 소스코드를 예로 들어 본 시스템이 적용 가능함을 보인다.

Keywords

JBCRJM_2019_v8n8_317_f0001.png 이미지

Fig. 1. The General Process of Using Judge System

JBCRJM_2019_v8n8_317_f0002.png 이미지

Fig. 2. The Control Flow Graphs of the Answer Source Codes

JBCRJM_2019_v8n8_317_f0003.png 이미지

Fig. 3. The Possible Control Flow Graphs of the ‘If Statement’ Problem of the Table 3

JBCRJM_2019_v8n8_317_f0004.png 이미지

Fig. 4. The Answer Source Code of the ‘If Statement’ Problem of the Table 3 with Faults

JBCRJM_2019_v8n8_317_f0005.png 이미지

Fig. 5. The Feedback of Programming Failure Based on the Control Flow Graph

JBCRJM_2019_v8n8_317_f0006.png 이미지

Fig. 6. The Execution Paths of the 1st Submitted Answer

JBCRJM_2019_v8n8_317_f0007.png 이미지

Fig. 7. The Source Code Feedback of the 1st Submitted Answer

JBCRJM_2019_v8n8_317_f0008.png 이미지

Fig. 8. The Execution Paths of the 2nd Submitted Answer

JBCRJM_2019_v8n8_317_f0009.png 이미지

Fig. 9. The Source Code Feedback of the 2nd Submitted Answer

JBCRJM_2019_v8n8_317_f0010.png 이미지

Fig. 10. The Execution Paths of the Final Submitted Answer

JBCRJM_2019_v8n8_317_f0011.png 이미지

Fig. 11. The Source Code Feedback of the Final Submitted Answer

Table 1. Analysis Result of Students’ Submission History

JBCRJM_2019_v8n8_317_t0001.png 이미지

Table 2. Feedback Contents of Online Judge Systems

JBCRJM_2019_v8n8_317_t0002.png 이미지

Table 3. The Problem of ‘If Statement’ for Training

JBCRJM_2019_v8n8_317_t0003.png 이미지

References

  1. U. Y. Jung, J. M. Han, and Y. J. Lee, "Analysis on Domestic Research Trends related to Bebras Challenge," in Proceedings of the Korean Society of Computer and Information Winter Conference, Gumi, 2019. pp.207-210.
  2. J. H. Jeon, U. Y. Jung, and Y. J. Lee, "A Problem Solving Learning Model using Online Judge System," in Proceedings of the Korean Society of Computer and Information Summer Conference, Jeju, 2018. pp.417-418.
  3. H. J. Park, C. S. Im, S. H. Park, J. Y. Hong, and M. H. Park, "Multi-facetted Automatic Coding Evaluation and Feedback System," in Proceedings of the KIISE Korea Computer Congress, Busan, 2017. pp.1051-1053.
  4. J. H. Park, Y. H. Shin, and S. G. Lee, "Online Judge System for SQL Practice," in Proceedings of the KIISE Korea Computer Congress, Jeju, 2016. pp.1054-1056.
  5. S. J. Yi, G. H. An, S. Y. Yun, and S. H. Lee, "A Web-based Algorithm Development Practice Tutoring System," in Proceedings of the Proc. of the Korean Institute of Communications and Information Sciences Fall Conference, Seoul, 2016. pp.150-151.
  6. G. P. Wang, S. Y. Chen, X. Yang, and R. Feng, "OJPOT: online judge & practice oriented teaching idea in programming courses," the European Journal of Engineering Education, Vol.41, No.3, pp.304-319, 2013. https://doi.org/10.1080/03043797.2015.1056105
  7. ACM UVA [Internet], https://www.uva.onlinejudge.org
  8. Baekjoon Online Judge [Internet], https://www.acmicpc.net
  9. Y. M. Woo, J. W. Bang, J. M. Song, J. Y. Yoo, and S. J. Lee, "Design and Implementation of the Web-based Learning System for C Programming Language," The KIISE Transactions on Computing Practices, Vol.20, No.12, pp.640-645, 2014. https://doi.org/10.5626/KTCP.2014.20.12.640
  10. KNU Online Judge [Internet], http://selab.knu.ac.kr/training
  11. S. Y. Park, "Design and Implementation of a Web-based Programming Class Support System," The Journal of the Korea Institute of Maritime Information & Communication Sciences, Vol.14, No.12, pp.2275-2782, 2010.
  12. Codeup [Internet], http://www.codeup.kr
  13. T. Kakeshita, K. Ohta, and R. Yanagita, "A Programming Education Support Tool pgtracer utilizing Fill-in-the-Blank Questions," International Conference on Education Reform and Modern Management, Hong Kong, 2015, pp.164-167.
  14. DOMjudge [Internet], https://www.domjudge.org
  15. Algospot Online Judge [Internet], https://www.algospot.com
  16. Programmers [Internet], https://www.programmers.co.kr
  17. Sphere Online Judge [Internet], http://www.spoj.com
  18. KOI [Internet], http://www.koistudy.net