DOI QR코드

DOI QR Code

Partial AUC using the sensitivity and specificity lines

민감도와 특이도 직선을 이용한 부분 AUC

  • Received : 2020.05.20
  • Accepted : 2020.06.14
  • Published : 2020.10.31

Abstract

The receiver operating characteristic (ROC) curve is expressed as both sensitivity and specificity; in addition, some optimal thresholds using the ROC curve are also represented with both sensitivity and specificity. In addition to the sensitivity and specificity, the expected usefulness function is considered as disease prevalence and usefulness. In particular, partial the area under the ROC curve (AUC) on a certain range should be compared when the AUCs of the crossing ROC curves have similar values. In this study, partial AUCs representing high sensitivity and specificity are proposed by using sensitivity and specificity lines, respectively. Assume various distribution functions with ROC curves that are crossing and AUCs that have the same value. We propose a method to improve the discriminant power of the classification models while comparing the partial AUCs obtained using sensitivity and specificity lines.

Receiver operating characteristic (ROC) 곡선은 민감도와 특이도로 표현되며, ROC 곡선을 이용하는 최적분류점도 민감도와 특이도만을 반영하지만, 본 연구에서는 질병률과 효용을 추가하여 고려하는 기대효용함수를 연구한다. 특히 교차하는 ROC 곡선들의 area under the ROC curve (AUC) 값들이 유사한 경우에 특정한 부분의 부분 AUC를 비교해야 한다. 본 연구에서는 정의된 민감도 직선과 특이도 직선을 바탕으로 각각 높은 민감도와 특이도를 나타내는 부분 AUC를 제안한다. ROC 곡선들이 교차하고 동일한 AUC 값을 갖는 다양한 분포함수를 설정하여, 민감도 직선과 특이도 직선을 이용하여 구한 부분 AUC를 비교하면서 모형의 판별력을 향상시키는 방법을 제안한다.

Keywords

References

  1. Bradley, A. P. (1997). The use of the area under the ROC curve in the evaluation of machine learning algorithms, Pattern Recognitions, 30, 1145-1159. https://doi.org/10.1016/S0031-3203(96)00142-2
  2. Brasil, P. (2010). Diagnostic test accuracy evaluation for medical professionals, Package DiagnosisMed in R.
  3. Centor, R. M. (1991). Signal detectability: the use of ROC analysis, Med Decision Making, 11, 102-106. https://doi.org/10.1177/0272989X9101100205
  4. Dodd, L. E. and Pepe, M. S. (2003). Partial AUC estimation and regression, Med Decision Making, 59, 614-623.
  5. Fawcett, T. (2003). ROC graphs: notes and practical considerations for data mining researchers, Laboratories, Palo Alto, HPL-2003-4.
  6. Greiner, M., Pfeiffer, D., and Smith, R. D. (2000). Principles and practical application of the receiver operating characteristic analysis for diagnostic tests, Veterinary Medicine, 45, 23-41.
  7. Irwin, J. R. and Irwin, C. T. (2012). Appraising credit ratings: does the CAP fit better than the ROC?, Monetary Fund Working paper, WP. 12/122.
  8. Jiang, Y., Metz, C., and Nishikawa, R. (1996). A receiver operating characteristic partial area index for highly sensitive diagnostic tests, Radiology, 201, 745-750. https://doi.org/10.1148/radiology.201.3.8939225
  9. Koepsell, T. D. and Connell, F. A. (1985). Measures of gain in certainty from a diagnostic test, American Journal of Epidemiology, 121, 744-753. https://doi.org/10.1093/aje/121.5.744
  10. Krzanowski, W. J. and Hand, D. J. (2009). ROC Curves for Continuous Data, Chapman & Hall/CRC, Boca Raton.
  11. McClish, D. K. (1989). Analyzing a portion of the ROC curve, Med Decision Making, 9, 190-195. https://doi.org/10.1177/0272989X8900900307
  12. Metz, C. E. (1978). Basic principles of ROC analysis, Seminars in Nuclear Medicine, 8, 283-298. https://doi.org/10.1016/S0001-2998(78)80014-2
  13. Perkins, N. J. and Schisterman, E. F. (2006). The inconsistency of "optimal" cutpoints obtained using two criteria based on the receiver operating characteristic curve, American Journal of Epidemiology, 163, 670-675. https://doi.org/10.1093/aje/kwj063
  14. Samawi, H. M., Yin, J., Rochani, H., and Panchal, V. (2017). Notes on the overlap measure as an alternative to the Youden index: How are they related?, Statistics in Medicine, 36, 4230-4240. https://doi.org/10.1002/sim.7435
  15. Sox, H. C., Blatt, M. A., Higgins, M. C., and Marton, K. I. (1988). Medical decision making, Butterworths, Stoneham.
  16. Subtil, F. and Rabilloud, M. (2015). An enhancement of ROC curves made them clinically relevant for diagnostic-test comparison and optimal-threshold determination, Journal of Clinical Epidemiology, 68, 752-759. https://doi.org/10.1016/j.jclinepi.2015.01.003
  17. Swets, J. A., Dawes, R. M., and Monahan, J. (2000). Better decisions through science, Scientific American, 82-87.
  18. Tasche, D. (2006). Validation of internal rating systems and PD estimates, The Analytics of Risk Model Validation, 169-196.
  19. Vickers, A. J. E. and Elkin, E. B. (2006). Decision curve analysis: a novel method for evaluating prediction models, Med Decis Making, 26, 565-574. https://doi.org/10.1177/0272989X06295361
  20. Youden, W. J. (1950). Index for rating diagnostic test, Cancer, 3, 32-35. https://doi.org/10.1002/1097-0142(1950)3:1<32::AID-CNCR2820030106>3.0.CO;2-3
  21. Zweig, M. and Campbell, G. (1993). Receiver-operating characteristics (ROC) plots: A fundamental evaluation tool in clinical medicine, Clinical Chemistry, 39, 561-577. https://doi.org/10.1093/clinchem/39.4.561