Journal of the Korea Society of Computer and Information (한국컴퓨터정보학회논문지)

Korean Society of Computer Information (한국컴퓨터정보학회)
권호 표지
DOI QR코드

DOI QR Code

Assessing Classification Accuracy using Cohen's kappa in Data Mining

데이터 마이닝에서 Cohen의 kappa를 이용한 분류정확도 측정

  • Um, Yonghwan (Division of Industrial and Management Engineering, Sungkyul University)
  • 엄용환 (성결대학교 산업경영공학부)
  • Received : 2012.11.19
  • Accepted : 2012.12.18
  • Published : 2013.01.31
Download PDF
⟨ Previous Next ⟩

Abstract

In this paper, Cohen's kappa and weighted kappa are applied to measuring classification accuracy when performing classification in data minig. Cohen's kappa compensates for classifications that may be due to chance and is used for the data with nominal or ordinal scales. Especially, for the ordinal data, weighted kappa which measures the classification accuracy by quantifying the classification errors as weights is used. We used two weights (linear weight, quadratic weight) for calculations of weighted kappa. Also for the calculation and comparison of kappa and weighted kappa we used a real data set, fat-liver data.

본 논문에서는 데이터 마이닝에서 분류 작업을 실시할 때 그 분류정확도을 측정하기 위해 Cohen의 kappa 계수와 weighted kappa 계수를 제안하였다. kappa 계수는 우연에 의해 생기는 분류를 보정하여 분류정확도을 측정하며 명목척도와 순서척도의 데이터에 대해 사용된다. 특히 순서척도의 데이터에서는 오분류의 크기를 가중치에 의해 정량화하여 분류정확도을 측정하는 weighted kappa 계수가 더 유용하게 사용된다. weighted kappa 계수 계산을 위해서는 2가지 가중치(일차형 가중치, 이차형 가중치)를 사용하였다.. 또한 실제 데이터인 지방간 데이터에 대해 kappa 계수와 weighted kappa 계수를 계산하여 비교하였다.

Keywords

References

  1. J. Hollmen, M. Skubacz, and M. Taniguchi, "Input Dependent Misclassification costs for cost-sensitive classifiers". In N. Ebechen & C.Brebbia(Eds.), Data Mining II - Proceedings of the Second International Conference on Data Mining (pp. 495-503) Cambridge. MA: MIT Press, 2000.
  2. D. D. Margineantu, and T. G. Ditterich, "Boostrap Methods for the Cost-sensitive Evaluation of Classifiers", In: Proceedings of the Seventh International Conference on Machine Learning, Morgan Kauffmann (pp. 582-590), 2000
  3. J. Cohen, "A Coefficient of Agreement for Nominal Scales", Educational and Psychological Measurement, 20, 37-46, 1960. https://doi.org/10.1177/001316446002000104
  4. J. Cohen, "Weighted Kappa: Nominal Scale Agreement with Provision for Scaled Disagreement or Partial Credit", Psychological Bulletin, 70, 213-220, 1968. https://doi.org/10.1037/h0026256
  5. J. L. Fleiss, "Statistical Methods for Rates and Proportions (2nd ed.)", New York: Wiley, 1981.
  6. J. J. Barko, and W. T. Carpenter, "On the Methods and Theory of Reliability", The Journal of Nervous and Mental Disease, 163, 307-317, 1976. https://doi.org/10.1097/00005053-197611000-00003
  7. J. R. Landis, and G. G. Koch, "Measurement of Observer Agreement for Categorical Data", Biometrics, 33, 159-174, 1977. https://doi.org/10.2307/2529310
  8. H. S. Bae, et. al. "SAS Enterprise Miner, Data Mining", Kyowoo Co. 2004.
  9. B. S. Everitt, "The Analysis of Contingency Tables", London: Chapman and Hall, 1977.