Communications for Statistical Applications and Methods

  • 제17권6호
  • /
  • Pages.755-765
  • /
  • 2010
  • /
  • 2287-7843(pISSN)
  • /
  • 2383-4757(eISSN)
한국통계학회 (The Korean Statistical Society)
권호 표지
DOI QR코드

DOI QR Code

비용곡선과 ROC곡선에서의 비용비율

Cost Ratios for Cost and ROC Curves

  • 투고 : 20100700
  • 심사 : 20101000
  • 발행 : 2010.11.30
PDF 다운로드
⟨ 이전 논문 다음 논문 ⟩

초록

혼합분포의 분류문제에서 비용함수를 고려한 분류점은 최소 기대비용이라는 측면에서 최적이다. 비용에 관한 어떠한 정보가 주어지지 않은 경우에 ROC곡선을 이용하여 분류정확도 측도인 전체정확도와 진실율이 최대일 때의 분류점에 대응하는 기대비용에서의 비용비율을제안하고, 최소 기대비용의 비용비율과의 관계를 설명한다. 그리고 비용곡선을 이용하여 분류정확도 측도들에 기반하는 최소 기대비용에서의 비용비율을 제안하였고 이 비용비율은 대표적인 두 종류의 분류정확도가 최대일 때의 기대비용에 대한 비용비율들 사이에 존재하며, 최소 기대비용에서의 비용비율에 수렴하는 것을 발견하였다. 본 연구는 기대비용과 정규화된 기대비용을 최소화할 때의 비용비율과 분류정확도가 최대일 때의 비용비율들의 관계를 토론한다.

For classification problems on mixture distribution, a threshold based on cost functions is optimal from the viewpoint of a minimum expected cost. Assuming that there is no cost information, we propose cost ratios in the expected cost corresponding to thresholds where the total accuracy and the true rate are maximized to explain the relation of these cost ratios minimizing the expected cost. Other cost ratios are also proposed by comparing the normalized expected costs when classification accuracy is maximized. The values of these cost ratios are located between two cost ratios for the expected costs based on classification accuracies, and converge to that of the minimum expected cost. This work suggests two cost ratios: one is minimized by the expected cost and the normalized expected cost, and the other in the expected cost and the normalized expected cost functions that are maximized classification accuracies. We discuss their compatibility based on the relation of these cost ratios.

키워드

참고문헌

  1. 김지현 (2004). ROC and cost graphs for general cost matrix where correct classifications incur non-zero costs, <한국통계학회논문집>, 11, 21–30. https://doi.org/10.5351/CKSS.2004.11.1.021
  2. 홍종선, 주재선, 최진수 (2010). 혼합분포에서의 최적분류점, <응용통계연구>, 23, 13-28. https://doi.org/10.5351/KJAS.2010.23.5.943
  3. 홍종선, 최진수 (2009). ROC와 CAP 곡선에서의 최적분류점, <응용통계연구>, 22, 911-921. https://doi.org/10.5351/KJAS.2009.22.5.911
  4. Adams, N. M. and Hand, D. J. (1999). Comparing classifiers when the misallocation costs are uncertain, Pattern Recognition, 30, 1139–1147.
  5. Antonie, M. L., Zaiane, O. R. and Holte, R. C. (2006). Learning to use a learned model: A two-stage approach to classication, In Proceedings of the 6th IEEE International Conference on Data Mining(ICDM’06), 33–42. https://doi.org/10.1109/ICDM.2006.97
  6. Briggs, W. M. and Zaretzki, R. (2007). The skill plot: a graphical technique for the evaluating the predictive usefulness of continuous diagnostic tests, Biometrics, 63, 250–261. https://doi.org/10.1111/j.1541-0420.2007.00781_1.x
  7. Cantor, S. B., Sun, C. C., Tortolero-Luna, G., Richards-Kortum, R. and Follen, M. (1999). A comparison of C/B ratios from studies using receiver operating characteristic curve analysis, Journal of Clinical Epidemiology, 52, 885–892. https://doi.org/10.1016/S0895-4356(99)00075-X
  8. Davis, J. and Goadrich, M. (2006). The relationship between precision-recall and ROC curves, In Proceedings of the 23rd International Conference on Machine Learning(ICML’06), 233–240. https://doi.org/10.1145/1143844.1143874
  9. Drummond, C. and Holte, R. C. (2006). Cost curves: an improved method for visualizing classifier performance, Machine Learning, 65, 95–130. https://doi.org/10.1007/s10994-006-8199-5
  10. Fawcett, T. (2006). ROC graphs with instance-varying costs, Pattern Recognition Letters archive, 27, 882–891. https://doi.org/10.1016/j.patrec.2005.10.012
  11. Fielding, A. H. and Bell, J. F. (1997). A review of methods for the measurement of prediction errors in conservation presence/absence models, Environmental Conservation, 24, 38–49.
  12. Hand, D. J. (2009). Mismatched models, wrong results, and dreadful decisions: on choosing appropriate data mining tools, In Proceedings of the 15th ACM SIGKDD international conference on Knowledge discovery and data mining. https://doi.org/10.1145/1557019.1557021
  13. Hand, D. J. and Zhou, F. (2009). Evaluating models for classifying customers in retail banking collections, Journal of the Operational Society, doi:10.1057/jors.2009.129.
  14. Hilden, J. and Glasziou, P. (1996). Regret graphs, diagnostic uncertainty and Youden’s index, Statistics in Medicine, 15, 969–986.
  15. Holte, R. C. and Drummond, C. (2008). Cost-sensitive classifier evaluation using cost curves, Advances in Knowledge Discovery and Data Mining, 5012, 26–29. https://doi.org/10.1007/978-3-540-68125-0_4
  16. Hoshino, R., Coughtrey, D., Sivaraja, S., Volnyansky, I., Auer, S. and Trishtchenko, A. (2009). Applications and extensions of cost curves to marine container inspection, Annals of Operations Research, doi: 10.1007/s10479-009-0669-2.
  17. Jund, J., Rabilloud, M., Wallon, M. and Ecochard, R. (2005). Methods to estimate the optimal threshold for normally or log-narmally distributed biological tests, Medical Decision Making, 25, 406–415.
  18. Kaivanto, K. (2008). Maximization of the sum of sensitivity and specificity as a diagnostic cutpoint criterion, Journal of Clinical Epidemiology, 61, 516–518.
  19. Krzanowski, W. J. and Hand, D. J. (2009). ROC Curves for Continuous Data, Champman & Hall/CRC, Boca Raton, Florida.
  20. Liu, Y. (2002). The evaluation of classification models for credit scoring, Arbeitsbericht, 2, 1–65.
  21. Liu, Y. and Shriberg, E. (2007). Comparing evaluation metrics for sentence boundary detection, In Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing(ICASSP’07), 4, 185–188.
  22. Metz, C. E. (1978). Basic principles of ROC analysis, Seminars in Nuclear Medicine, 8, 283–298. https://doi.org/10.1016/S0001-2998(78)80014-2
  23. Pepe, M. S. (2003). The Statistical Evaluation of Medical Tests for Classification and Prediction, University Press, Oxford.
  24. Provost, F. and Fawcett, T. (1997). Analysis and visualization of classifier performance: comparison under imprecise class and cost distributions, In Proceedings of the Third International Conference on Knowledge Discovery and Data Mining, 43–48.
  25. Servigny, A. D. and Renault, O. (2004). Measuring and Managing Credit Risk, McGraw-Hill, New York.
  26. Tasche, D. (2006). Validation of Internal Rating Systems and PD Estimates, arXiv.org, eprint arXiv:physics/0606071.
  27. Turney, P. D. (1995). Cost-sensitive classification: Empirical evaluation of a hybrid genetic decision tree induction algorithm, Journal of Artificial Intelligence Research, 2, 369–409. https://doi.org/10.1613/jair.120
  28. Velez, D. R., White, B. C., Motsinger, A. A., Bush,W. S., Ritchie, M. D.,Williams, S. M. and Moore, J. H. (2007). A balanced accuracy function for epistasis modeling in imbalanced datasets using multifactor dimensionality reduction, Genetic Epidemiology, 31, 306–315.
  29. Vuk, M. and Curk, T. (2006). ROC curve, lift chart and calibration plot, Metodoloski Zvezki, 3, 89–108.
  30. Zhou, X. H., Obuchowski, N. A. and McClish, D. K. (2002). Statistical Methods in Diagnostic Medicine, Wiley, New York.
  31. Zweig, M. H. and Campbell, G. (1993). Receiver-operating characteristic(ROC) plots: A fundamental evaluation tool in clinical medicine, Clinical Chemistry, 39, 561–577.

피인용 문헌

  1. Alternative Optimal Threshold Criteria: MFR vol.27, pp.5, 2014, https://doi.org/10.5351/KJAS.2014.27.5.773
  2. Alternative accuracy for multiple ROC analysis vol.25, pp.6, 2014, https://doi.org/10.7465/jkdi.2014.25.6.1521