Journal of the Korean Data and Information Science Society

The Korean Data and Information Science Society (한국데이터정보과학회)
권호 표지

Classification accuracy measures with minimum error rate for normal mixture

정규혼합분포에서 최소오류의 분류정확도 측도

  • Hong, C.S. (Department of Statistics, Sungkyunkwan University) ;
  • Lin, Meihua (Research Institute of Applied Statistics, Sungkyunkwan University) ;
  • Hong, S.W. (Research Institute of Applied Statistics, Sungkyunkwan University) ;
  • Kim, G.C. (Research Institute of Applied Statistics, Sungkyunkwan University)
  • 홍종선 (성균관대학교 경제학부 통계학과) ;
  • ;
  • 홍선우 (성균관대학교 응용통계연구소, 통계학과) ;
  • 김강천 (성균관대학교 응용통계연구소, 통계학과)
  • Received : 2011.05.21
  • Accepted : 2011.06.20
  • Published : 2011.08.01
Download PDF
⟨ Previous Next ⟩

Abstract

In order to estimate an appropriate threshold and evaluate its performance for the data mixed with two different distributions, nine kinds of well-known classification accuracy measures such as MVD, Youden's index, the closest-to- (0,1) criterion, the amended closest-to- (0,1) criterion, SSS, symmetry point, accuracy area, TA, TR are clustered into five categories on the basis of their characters. In credit evaluation study, it is assumed that the score random variable follows normal mixture distributions of the default and non-default states. For various normal mixtures, optimal cut-off points for classification measures belong to each category are obtained and type I and II error rates corresponding to these cut-off points are calculated. Then we explore the cases when these error rates are minimized. If normal mixtures might be estimated for these kinds of real data, we could make use of results of this study to select the best classification accuracy measure which has the minimum error rate.

본 연구에서는 두 분포함수의 혼합된 자료에서 적절한 분류점을 추정하고 평가하기 위하여 많이 사용하는 아홉 종류의 분류정확도 측도인 MVD, Youden지수, (0,1)까지최단기준, 수정된 (0,1)까지 최단기준, SSS, 대칭점, 정확도면적, TA, TR을 다섯 개의 조건범주로 군집시킨다. 신용평가분석에서 정상과 부도상태의 스코어 확률변수가 정규분포를 따르며 전체부도율로 혼합되었다고 가정한다. 다양한 정규혼합분포의 상황에서 군집된 측도들의 최적분류점을 발견하고, 그 분류점에 대응하는 제I종 오류율과 제II종 오류율 그리고 두 종류의 오류율 합을 구하여 각각의 오류율이 최소인 경우를 탐색적으로 살펴본다. 현실자료에 적합한 정규혼합분포를 추정하여 본 연구 결과를 적용하면 최소 오류율이 보장되는 분류정확도를 선택할 수 있으며, 이를 사용하여 모형의 판별력을 향상시킬 수 있다.

Keywords

References

  1. 홍종선, 이원용 (2011). 정규혼합분포를 이용한 ROC 분석. <응용통계연구>, 24, 269-278.
  2. 홍종선, 주재선, 최진수 (2010). 혼합분포에서 최적분류점. <응용통계연구>, 23, 13-28.
  3. 홍종선, 권태완 (2010). 수익률분포의 적합과 리스크값 추정. <한국데이터정보과학회지>, 21, 219-229.
  4. Brasil, P. (2010). Diagnostic test accuracy evaluation for medical professionals, Package 'DiagnosisMed' in R.
  5. Cantor, S. B., Sun, C. C., Tortolero-Luna, G., Richards-Kortum, R. and Follen, M. (1999). A comparison of C/B ratios from studies using receiver operating characteristic curve analysis. Journal of Clinical Epidemiology, 52, 885-892. https://doi.org/10.1016/S0895-4356(99)00075-X
  6. Connell, F. A. and Koepsell, T. D. (1985). Measures of gain in certainty from a diagnostic test. American Journal of Epidemiology, 121, 744-753. https://doi.org/10.1093/aje/121.5.744
  7. Freeman, E. A. and Moisen, G. G. (2008). A comparison of theperformance of threshold criteria for binary classification in terms of predicted prevalence and kappa. Ecological Modelling, 217, 48-58. https://doi.org/10.1016/j.ecolmodel.2008.05.015
  8. Greiner, M. M. and Gardner, I. A. (2000). Epidemiologic issues in the validation of veterinary diagnostic tests. Preventive Veterinary Medicine, 45, 3-22. https://doi.org/10.1016/S0167-5877(00)00114-8
  9. Krzanowski, W. J. and Hand, D. J. (2009). ROC curves for continuous data, Champman & Hall/CRC, Boca Raton, FL.
  10. Liu, C., White, M. and Newell1, G. (2009). Measuring the accuracy of species distribution models: A review. 18th World IMACS/MODSIM Congress. http://mssanz.org.au/modsim09.
  11. Lambert, J. and Lipkovich, I. (2008). A macro for getting more out of your ROC curve. SAS Global Forum, 231.
  12. Moses, L. E., Shapiro, D. and Littenberg, B. (1993). Combining independent studies of a diagnostic test into a summary ROC curve: Data-analytic approaches and some additional considerations. Statistics in Medicine, 12, 1293-1316. https://doi.org/10.1002/sim.4780121403
  13. Perkins, N. J. and Schisterman, E. F. (2006). The inconsistency of "optimal" cutpoints obtained using two criteria based on the receiver operating characteristic curve. American Journal of Epidemiology, 163, 670-675. https://doi.org/10.1093/aje/kwj063
  14. Pepe, M. S. (2003). The statistical evaluation of medical tests for classification and prediction, University Press, Oxford. .
  15. Velez, D. R., White, B. C., Motsinger, A. A., Bush, W. S., Ritchie, M. D., Williams, S. M. and Moore, J. H. (2007). A balanced accuracy function for epistasis modeling in imbalanced datasets using multifactor dimensionality reduction. Genetic Epidemiology, 31, 306-315. https://doi.org/10.1002/gepi.20211
  16. Youden, W. J. (1950). Index for rating diagnostic test. Cancer, 3, 32-35. https://doi.org/10.1002/1097-0142(1950)3:1<32::AID-CNCR2820030106>3.0.CO;2-3