Communications for Statistical Applications and Methods

The Korean Statistical Society (한국통계학회)
권호 표지
DOI QR코드

DOI QR Code

VUS and HUM Represented with Mann-Whitney Statistic

  • Hong, Chong Sun (Department of Statistics, Sungkyunkwan University) ;
  • Cho, Min Ho (Department of Statistics, Sungkyunkwan University)
  • Received : 2014.12.16
  • Accepted : 2015.02.05
  • Published : 2015.05.31
Download PDF
⟨ Previous Next ⟩

Abstract

The area under the ROC curve (AUC), the volume under the ROC surface (VUS) and the hypervolume under the ROC manifold (HUM) are defined and interpreted with probability that measures the discriminant power of classification models. AUC, VUS and HUM are expressed with the summation and integration notations for discrete and continuous random variables, respectively. AUC for discrete two random samples is represented as the nonparametric Mann-Whitney statistic. In this work, we define conditional Mann-Whitney statistics to compare more than two discrete random samples as well as propose that VUS and HUM are represented as functions of the conditional Mann-Whitney statistics. Three and four discrete random samples with some tie values are generated. Values of VUS and HUM are obtained using the proposed statistic. The values of VUS and HUM are identical with those obtained by definition; therefore, both VUS and HUM could be represented with conditional Mann-Whitney statistics proposed in this paper.

Keywords

References

  1. Bradley, A. P. (1997). The use of the area under the ROC curve in the evaluation of machine learning algorithms, Pattern Recognition, 30, 1145-1159. https://doi.org/10.1016/S0031-3203(96)00142-2
  2. Dreiseitl, S., Ohno-Machado, L. and Binder, M. (2000). Comparing three-class diagnostic tests by three-way ROC analysis, Medical Decision Making, 20, 323-331. https://doi.org/10.1177/0272989X0002000309
  3. Egan, J. P. (1975). Signal Detection Theory and ROC Analysis, Academic Press, New York.
  4. Engelmann, B., Hayden, E. and Tasche, D. (2003). Testing rating accuracy, Risk, 82-86.
  5. Fawcett, T. (2003). ROC graphs: notes and practical considerations for data mining researchers, Available from: http://www.hpl.hp.com/techreports/2003/HPL-2003-4.pdf.
  6. Hanley, J. A. and McNeil, B. J. (1982). The meaning and use of the area under a receiver operating characteristic (ROC) curve, Radiology, 143, 29-36. https://doi.org/10.1148/radiology.143.1.7063747
  7. Heckerling, P. S. (2001). Parametric three-way receiver operating characteristic surface analysis using mathematica, Medical Decision Making, 21, 409-417. https://doi.org/10.1177/02729890122062703
  8. Hong, C. S. (2009). Optimal threshold from ROC and CAP curves, Communications in Statistics - Stimulation and Computation, 38, 2060-2072. https://doi.org/10.1080/03610910903243703
  9. Hong, C. S. and Choi, J.S. (2009). Optimal thre shold from ROC and CAP curves, The Korean Journal of Applied Statistics, 22, 911-921. https://doi.org/10.5351/KJAS.2009.22.5.911
  10. Hong, C. S., Joo, J. S. and Choi, J. S. (2010). Optimal thresholds from mixture distributions, The Korean Journal of Applied Statistics, 23, 13-28. https://doi.org/10.5351/KJAS.2010.23.1.013
  11. Hong, C. S. and Jung, D. G. (2014). Standard criterion of hypervolume under the ROC manifold, Journal of the Korean Data & Information Science Society, 25, 473-483. https://doi.org/10.7465/jkdi.2014.25.3.473
  12. Hong, C. S., Jung, E. S. and Jung, D. G. (2013). Standard criterion of VUS for ROC surface, The Korean Journal of Applied Statistics, 26, 977-985. https://doi.org/10.5351/KJAS.2013.26.6.977
  13. Li, J. and Fine, J. P. (2008). ROC analysis with multiple classes and multiple tests: Methodology and its application in microarray studies, Biostatistics, 9, 566-576. https://doi.org/10.1093/biostatistics/kxm050
  14. Mossman, D. (1999). Three-way ROCs, Medical Decision Making, 19, 78-89. https://doi.org/10.1177/0272989X9901900110
  15. Nakas, C. T., Alonzo, T. A. and Yiannoutsos, C. T. (2010). Accuracy and cut-off point selection in three class classification problems using a generalization of the Youden index, Statistics in Medicine, 29, 2946-2955. https://doi.org/10.1002/sim.4044
  16. Nakas, C. T. and Yiannoutsos, C. T. (2004). Ordered multiple-class ROC analysis with continuous measurements, Statistics in Medicine, 23, 3437-3449. https://doi.org/10.1002/sim.1917
  17. Patel, A. C. and Markey, M. K. (2005). Comparison of three-class classification performance metrics:A case study in breast cancer CAD, Proceedings of SPIE, 5749, 581-589.
  18. Provost, F. and Fawcett, T. (2001). Robust classification for imprecise environments, Machine Learning, 42, 203-231. https://doi.org/10.1023/A:1007601015854
  19. Rosset, S. (2004). Model Selection via the AUC, In Proceedings of the 21st International Conference on Machine Learning (ICML'04), Banff, Canada, 89.
  20. Scureld, B. K. (1996). Multiple-event forced-choice tasks in the theory of signal detectability, Journal of Mathematical Psychology, 40, 253-269. https://doi.org/10.1006/jmps.1996.0024
  21. Sobehart, J. and Keenan, S. (2001). Measuring default accurately, Risk - Credit Risk Special Report, 14, 31-33.
  22. Swets, J. A. (1988). Measuring the accuracy of diagnostic systems, Science, 240, 1285-1293. https://doi.org/10.1126/science.3287615
  23. Swets, J. A., Dawes, R. M. and Monahan, J. (2000). Better decisions through science, Scientific American, 283, 82-87.
  24. Wandishin, M. S. and Mullen, S. J. (2009). Multiclass ROC analysis, Weather and Forecasting, 24, 530-547. https://doi.org/10.1175/2008WAF2222119.1
  25. Zou, K. H., O'Malley, A. J. and Mauri, L. (2007). Receiver-operating characteristic analysis for evaluating diagnostic tests and predictive models, Circulation, 115, 654-657. https://doi.org/10.1161/CIRCULATIONAHA.105.594929

Cited by

  1. Proposition of polytomous discrimination index and test statistics vol.27, pp.2, 2016, https://doi.org/10.7465/jkdi.2016.27.2.337
  2. Parameter estimation for the imbalanced credit scoring data using AUC maximization vol.29, pp.2, 2016, https://doi.org/10.5351/KJAS.2016.29.2.309
  3. Standardized polytomous discrimination index using concordance vol.27, pp.1, 2016, https://doi.org/10.7465/jkdi.2016.27.1.33
  4. Test Statistics for Volume under the ROC Surface and Hypervolume under the ROC Manifold vol.22, pp.4, 2015, https://doi.org/10.5351/CSAM.2015.22.4.377