Journal of the Korean Data and Information Science Society

The Korean Data and Information Science Society (한국데이터정보과학회)
권호 표지
DOI QR코드

DOI QR Code

Standard criterion of hypervolume under the ROC manifold

ROC 다면체 아래 체적의 판단기준

  • Hong, C.S. (Department of Statistics, Sungkyunkwan University) ;
  • Jung, D.G. (Department of Statistics, Sungkyunkwan University)
  • Received : 2014.03.25
  • Accepted : 2014.04.28
  • Published : 2014.05.31
Download PDF
⟨ Previous Next ⟩

Abstract

Even though the ROC manifold for more than three dimensional space which is an extension of the ROC curve and surface has difficulty to represent graphically, the hypervolume under the ROC manifold (HUM) statistic can be defined and obtained based on AUC and VUS measures for the ROC curve and the ROC surface. Hence the definition and characteristics of the HUM for four dimensional space are studied in this work. By extension of the standard criterion of AUC for probabilities of default based on Basel II, the 13 classes of standard criterion of HUM are proposed in order to discriminate four classification models and some application methods are discussed. In order to explore the standard criterion of HUM whose values are obtained from various distributions, ternary plot is used and explained.

ROC 곡선과 ROC 곡면을 확장한 4차원 이상의 공간에서의 ROC 다면체는 시각적인 표현이 어렵기 때문에 활용하기 어려우나, ROC 다면체 아래 공간을 측정하는 HUM 통계량에 대하여는 AUC와 VUS 통계량을 기반으로 정의가 가능하고 값을 구할 수 있으므로 본 연구는 네 가지 범주의 분류모형의 판별력을 측정하는 확률을 정의하고 연구한다. 그리고 Basel II를 기반한 부도확률에 대한 AUC의 판별력 판단기준을 제안한 연구를 확장하여, 네 범주 분류모형의 판별력을 측정하는 HUM 통계량에 관한 판단기준을 13단계로 구분하여 제안하고 활용하는 방법을 설명한다. 다양한 분포함수에 대하여 얻은 HUM 값을 바탕으로 제안한 판단기준을 탐색하기 위하여 삼원구획그림을 활용하여 판단기준을 설명한다.

Keywords

References

  1. Dreiseitl, S., Ohno-Machado, L. and Binder, M. (2000). Comparing three-class diagnostic tests by three-way ROC analysis. Medical Decision Making, 20, 323-331. https://doi.org/10.1177/0272989X0002000309
  2. Fawcett, T. (2003). ROC graphs: Notes and practical considerations for data mining researchers, HP Labs Tech Report HPL-2003-4, HP Laboratories, http://www.hpl.hp.com/techreports/2003/.
  3. Heckerling, P. S. (2001). Parametric three-way receiver operating characteristic surface analysis using mathematica. Medical Decision Making, 21, 409-417. https://doi.org/10.1177/02729890122062703
  4. Hong, C. S., Jung, E. S. and Jung, D. G. (2013). Standardized criterion of VUS for ROC surface. The Korean Journal of Applied Statistics, 26, 977-985. https://doi.org/10.5351/KJAS.2013.26.6.977
  5. Hosmer, D. W. and Lemeshow, S. (2000). Applied logistic regression, John Wiley & Sons, New York.
  6. Li, J. and Fine, J. P. (2008). ROC analysis with multiple classes and multiple tests: Methodology and its application in microarray studies. Biostatistics, 9, 566-576. https://doi.org/10.1093/biostatistics/kxm050
  7. Joseph, M. P. (2005). A PD validation framework for Basel II internal ratings-based systems. Quantitative Analyst Basel II Project, Commonwealth Bank of Australia.
  8. Mossman, D. (1999). Three-way ROCs. Medical Decision Making, 19, 78-89. https://doi.org/10.1177/0272989X9901900110
  9. Nakas, C. T. and Yiannoutsos, C. T. (2004). Ordered multiple-class ROC analysis with continuous measurements. Statistics in Medicine, 23, 3437-3449. https://doi.org/10.1002/sim.1917
  10. Nakas, C. T., Alonzo, T. A. and Yiannoutsos, C. T. (2010). Accuracy and cut off point selection in three class classification problems using a generalization of the Youden index. Statistics in Medicine, 29, 2946-2955. https://doi.org/10.1002/sim.4044
  11. Patel, A. C. and Markey, M. K. (2005). Comparison of three-class classification performance metrics: A case study in breast cancer CAD. International Society for Optical Engineering, 5749, 581-589.
  12. Scurfield, B. K. (1996). Multiple-event forced-choice tasks in the theory of signal detectability. Journal of Mathematical Psychology, 40, 253-269. https://doi.org/10.1006/jmps.1996.0024
  13. Wandishin, M. S. and Mullen, S. J. (2009). Multiclass ROC analysis. Weather and Forecasting, 24, 530-547. https://doi.org/10.1175/2008WAF2222119.1
  14. Wilkie, A. D. (2004). Measures for comparing scoring systems. In Readings in Credit Scoring, edited by L.C. Thomas, D.B. Edelman, and J.N. Crook, Oxford University Press, Oxford.
  15. Zachos, C. K. (2012). Ternary plots for neutrino mixing visualization. ANL-HEP-PR-12-31, arXiv:1205.4772v3 [hep-ph].
  16. Zou, K. H., O'Malley, A. J. and Mauri, L. (2007). Receiver-operating characteristic analysis for evaluating diagnostic tests and predictive models. Circulation, 115, 654-657. https://doi.org/10.1161/CIRCULATIONAHA.105.594929

Cited by

  1. Proposition of polytomous discrimination index and test statistics vol.27, pp.2, 2016, https://doi.org/10.7465/jkdi.2016.27.2.337
  2. VUS and HUM Represented with Mann-Whitney Statistic vol.22, pp.3, 2015, https://doi.org/10.5351/CSAM.2015.22.3.223
  3. Parameter estimation for the imbalanced credit scoring data using AUC maximization vol.29, pp.2, 2016, https://doi.org/10.5351/KJAS.2016.29.2.309
  4. ROC curve and AUC for linear growth models vol.26, pp.6, 2015, https://doi.org/10.7465/jkdi.2015.26.6.1367
  5. Parameter estimation of linear function using VUS and HUM maximization vol.26, pp.6, 2015, https://doi.org/10.7465/jkdi.2015.26.6.1305
  6. Test Statistics for Volume under the ROC Surface and Hypervolume under the ROC Manifold vol.22, pp.4, 2015, https://doi.org/10.5351/CSAM.2015.22.4.377
  7. Discriminant analysis using empirical distribution function vol.28, pp.5, 2017, https://doi.org/10.7465/jkdi.2017.28.5.1179