Journal of the Korean Data and Information Science Society

The Korean Data and Information Science Society (한국데이터정보과학회)
권호 표지
DOI QR코드

DOI QR Code

Parameter estimation of linear function using VUS and HUM maximization

VUS와 HUM 최적화를 이용한 선형함수의 모수추정

  • Received : 2015.08.03
  • Accepted : 2015.09.14
  • Published : 2015.11.30
Download PDF
⟨ Previous Next ⟩

Abstract

Consider the risk score which is a function of a linear score for the classification models. The AUC optimization method can be applied to estimate the coefficients of linear score. These estimates obtained by this AUC approach method are shown to be better than the maximum likelihood estimators using logistic models under the general situation which does not fit the logistic assumptions. In this work, the VUS and HUM approach methods are suggested by extending AUC approach method for more realistic discrimination and prediction worlds. Some simulation results are obtained with both various distributions of thresholds and three kinds of link functions such as logit, complementary log-log and modified logit functions. It is found that coefficient prediction results by using the VUS and HUM approach methods for multiple categorical classification are equivalent to or better than those by using logistic models with some link functions.

ROC 곡선을 구성하는 한 개의 스코어 변수로 이루어진 분류모형을 확장하여 선형 스코어의 함수인 리스크 스코어를 고려하고, 선형 스코어의 계수를 추정하기 위한 방법으로 AUC를 최대화하는 방법을 사용한다. 이런 AUC 접근방법으로 구한 스코어의 계수 추정량은 로지스틱모형을 이용한 선형 스코어의 모수의 최대가능도 추정량보다 자료가 로지스틱 가정이 맞지 않는 일반적인 상황에서도 좋은 추정 결과를 보인다. 본 연구에서는 다항범주로 분류되어 현실적인 판별 및 예측 상황을 고려하여 AUC 접근방법을 확장한 VUS와 HUM 접근방법을 제안한다. 연결함수로는 로짓, complementary log-log와 로짓을 변형한 함수의 세 종류와 그리고 다양한 분류점의 분포인 경우에 대하여도 모의실험을 실시하였다. 본 논문에서는 다항범주 판별결과에 대하여 VUS와 HUM 접근방법도 AUC 접근방법과 유사하게 다양한 연결함수에 대하여 로지스틱모형 추정방법보다 동등하거나 더 나은 모수추정 결과를 보이는 것을 확인하였다.

Keywords

References

  1. Bamber, D. C. (1975). The area above the ordinal dominance graph and the area below the receiver operating characteristic graph. Journal of Mathematical Psychology, 12, 387-415. https://doi.org/10.1016/0022-2496(75)90001-2
  2. Cavanagh, C. and Sherman, R. P. (1998). Rank estimators for monotonic index models. Journal of Econometrics, 84, 351-381. https://doi.org/10.1016/S0304-4076(97)00090-0
  3. Dreiseitl, S., Ohno-Machado, L. and Binder, M. (2000). Comparing three-class diagnostic tests by three-way ROC analysis. Medical Decision Making, 20, 323-331. https://doi.org/10.1177/0272989X0002000309
  4. Egan, J. P. (1975). Signal detection theory and ROC analysis, Academic Press, New York.
  5. Engelmann, B., Hayden, E. and Tasche, D. (2003). Measuring the discriminative power of rating systems. Risk, 82-86.
  6. Fawcett, T. (2003). ROC graphs: Notes and practical considerations for data mining researchers, HP Labs Technical Report HPL-2003-4, CA, USA.
  7. Han, A. K. (1987). Non-parametric analysis of a generalized regression model, The maximum rank correlation estimator. Journal of Economics, 35, 303-316. https://doi.org/10.1016/0304-4076(87)90030-3
  8. Heckerling, P. S. (2001). Parametric three-way receiver operating characteristic surface analysis using mathematica. Medical Decision Making, 21, 409-417. https://doi.org/10.1177/02729890122062703
  9. Hong, C. S. and Cho, M. H. (2015). Two optimal threshold criteria for ROC analysis. Journal of the Korean Data & Information Science Society, 26, 255-260. https://doi.org/10.7465/jkdi.2015.26.1.255
  10. Hong, C. S. and Choi, J. S. (2009). Optimal threshold from ROC and CAP curves. The Korean Journal of Applied Statistics, 22, 911-921. https://doi.org/10.5351/KJAS.2009.22.5.911
  11. Hong, C. S., Joo, J. S. and Choi, J. S. (2010). Optimal thresholds from mixture distributions. The Korean Journal of Applied Statistics, 23, 13-28. https://doi.org/10.5351/KJAS.2010.23.1.013
  12. Hong, C. S., Jung, E. S. and Jung, D. G. (2013). Standard criterion of VUS for ROC surface. The Korean Journal of Applied Statistics, 26, 1-8. https://doi.org/10.5351/KJAS.2013.26.1.001
  13. Hong, C. S. and Jung, D. G. (2014). Standard criterion of hypervolume under the ROC manifold. Journal of the Korean Data & Information Science Society, 25, 473-483. https://doi.org/10.7465/jkdi.2014.25.3.473
  14. Hosmer, D. W. (2000). Applied logistic regression, 2nd ed., Wiley, New York.
  15. Joseph, M. P. (2005). A PD validation framework for Basel II internal ratings-based systems. Quantitative Analyst Basel II Project, Commonwealth Bank of Australia.
  16. Kraus, A. (2014). Recent methods from statistics and machine learning for credit scoring, Dissertation an der Fakultat fur Mathematik, Informatik und Statistik, der Ludwig-Maximilians-Universitat Munchen, Munchen; http://edoc.ub.uni-muenchen.de/17143/1/Kraus_Anne.pdf.
  17. Li, J. and Fine, J. P. (2008). ROC analysis with multiple classes and multiple tests: Methodology and its application in microarray studies. Biostatistics, 9, 566-576. https://doi.org/10.1093/biostatistics/kxm050
  18. Mossman, D. (1999). Three-way ROCs. Medical Decision Making, 19, 78-89. https://doi.org/10.1177/0272989X9901900110
  19. Nakas, C. T., Alonzo, T. A. and Yiannoutsos, C. T. (2010). Accuracy and cut off point selection in three class classification problems using a generalization of the Youden index. Statistics in Medicine, 29, 2946-2955. https://doi.org/10.1002/sim.4044
  20. Nakas, C. T. and Yiannoutsos, C. T. (2004). Ordered multiple-class ROC analysis with continuous measurements. Statistics in Medicine, 23, 3437-3449. https://doi.org/10.1002/sim.1917
  21. Nelder, J. A. and Mead, R. (1965). A simplex method for function minimization. The Computer Journal, 7, 308-313. https://doi.org/10.1093/comjnl/7.4.308
  22. Patel, A. C. and Markey, M. K. (2005). Comparison of three-class classification performance metrics: A case study in breast cancer CAD. International Society for Optical Engineering, 5749, 581-589.
  23. Pepe, M. S. (2003). The statistical evaluation of medical tests for classification and prediction, Oxford UniversityPress, Oxford.
  24. Pepe, M. S., Cai, T. and Longton, G. (2005). Combining predictors for classification using the area under the receiver operating characteristic curve. Biometrics, 1, 221-229.
  25. Provost, F. and Fawcett, T. (2001). Robust classification for imprecise environments. Machine Learning, 42, 203-231. https://doi.org/10.1023/A:1007601015854
  26. Scurfield, B. K. (1996). Multiple-event forced-choice tasks in the theory of signal detectability. Journal of Mathematical Psychology, 40, 253-269. https://doi.org/10.1006/jmps.1996.0024
  27. Sherman, R. P. (1993). The limiting distribution of the maximum rank correlation estimator. Econometrics, 61, 123-137. https://doi.org/10.2307/2951780
  28. Sobehart, J. R. and Keenan, S. C. (2001). Measuring default accurately, Credit risk special report. Risk, 14, 31-33.
  29. Swets, J. (1988). Measuring the accuracy of diagnostic systems. Science, 240, 1285-1293. https://doi.org/10.1126/science.3287615
  30. Swets, J. A., Dawes, R. M. and Monahan, J. (2000). Better decisions through science. Scientific American, 283, 82-87.
  31. Wandishin, M. S. and Mullen, S. J. (2009). Multiclass ROC analysis. Weather and Forecasting, 24, 530-547. https://doi.org/10.1175/2008WAF2222119.1
  32. Zou, K. H., O'Malley, A. J. and Mauri, L. (2007). Receiver-operating characteristic analysis for evaluating diagnostic tests and predictive models. Circulation, 115, 654-657. https://doi.org/10.1161/CIRCULATIONAHA.105.594929