DOI QR코드

DOI QR Code

A New Statistical Index for Detecting Cheaters on Multiple Choice Tests

다중선택 시험에서 부정행위자 발견을 위한 새로운 통계적 측도

  • Han, Eun Su (Division of Planning and Research, Korea National Institute of Health) ;
  • Lim, Johan (Department of Statistics, Seoul National University) ;
  • Lee, Kyeong Eun (Department of Statistics, Kyungpook National University)
  • Received : 2012.11.07
  • Accepted : 2012.12.28
  • Published : 2013.02.28

Abstract

It is important to construct a firm basis for accusing potential violators of academic integrity in order to avoid spurious accusations and false convictions. Educational researchers have developed many statistical methods that can either uncover or confirm cases of cheating on tests. However, most of them rely on simple correlation-based measures, and often fail to account for patterns in responses or answers. In this paper, we propose a new statistical index denoted by a Standardized Signed Entropy Similarity Score to resolve this difficulty. In addition, we apply the proposed method to analyze a real data set and compare the results to other existing methods.

학문적 진실성(academic integrity)을 위반하는 잠재적 부적행위를 판단할 때, 잘못된 결정을 피하기 위해서는 확고한 근거를 마련하는 것이 중요하다. 교육학 연구자들은 부정행위를 발견 혹은 확신 할 수 있는 많은 통계적인 방법들을 발전시켰다. 그러나, 대부분의 방법들은 단순히 상관계수를 기초로한 방법들이어서 종종 응답자들의 패턴을 설명하기가 어렵다. 이 논문에서는, 이런 어려움을 해결하기 해결하기 위하여 표준화된 부호 엔트로피 유사성 점수(Standardized Signed Entropy Similarity Score)라는 새로운 통계적인 측도를 제안한다. 또한, 이 제안한 방법을 실제 시험 자료를 이용 부정행위자를 발견하는데 적용하였고, 다른 기존의 방법들과 비교하였다.

Keywords

References

  1. Angoff, W. H. (1974). The development of statistical indices for detecting cheaters, Journal of the American Statistical Association, 69, 44-49. https://doi.org/10.1080/01621459.1974.10480126
  2. Bellezza, F. S. and Bellezza, S. F. (1989). Detection of cheating on multiple-choice tests by using error similarity analysis, Teaching of Psychology, 16, 151-155. https://doi.org/10.1207/s15328023top1603_15
  3. Buss, W. G. and Novick, M. R. (1980). The detection of cheating on standardized tests: Statistical and legal analysis, Journal of Law and Education, 9, 1-64.
  4. Crawford, C. C. (1930). Dishonesty in objective tests, School Review, 38, 776-781. https://doi.org/10.1086/439105
  5. Haney, W. M. (1993). Cheating and escheating on standardized tests, Paper presented at the annual meeting of the American Educational Research Association, Atlanta, GA.
  6. Holland, P. W. (1996). Assessing unusual agreement between the incorrect answers of two examinees using the K-Index: Statistical theory and empirical support, ETS Research Report No.96-7. Princeton, NJ: Educational Testing Service.
  7. Storey, J. D. (2003). The positive False Discovery Rate: A Bayesian interpretation and the q-value, The Annals of Statistics, 6, 2013-2035.