• Lee, Kyeong-Eun (Department of Statistics Kyungpook National University) ;
  • Lim, Johan (Department of Statistics Seoul National University)
  • Received : 2009.09.16
  • Published : 2011.05.31


A receiver operating characteristic (ROC) curve plots the true positive rate of a classier against its false positive rate, both of which are accuracy measures of the classier. The ROC curve has several interesting geometrical properties, including concavity which is a necessary condition for a classier to be optimal. In this paper, we study the nonparametric maximum likelihood estimator (NPMLE) of a concave ROC curve and its modification to reduce bias. We characterize the NPMLE as a solution to a geometric programming, a special type of a mathematical optimization problem. We find that the NPMLE is close to the convex hull of the empirical ROC curve and, thus, has smaller variance but positive bias at a given false positive rate. To reduce the bias, we propose a modification of the NPMLE which minimizes the $L_1$ distance from the empirical ROC curve. We numerically compare the finite sample performance of three estimators, the empirical ROC curve, the NMPLE, and the modified NPMLE. Finally, we apply the estimators to estimating the optimal ROC curve of the variance-threshold classier to segment a low depth of field image and to finding a diagnostic tool with multiple tests for detection of hemophilia A carrier.


  1. S. Boyd, S. Kim, L. Vandenberghe, and A. Hassibi, A tutorial on geometric programming, Optim. Eng. 8 (2007), no. 1, 67-127.
  2. S. Boyd and L. Vandenberghe, Convex Optimization, Cambridge University Press, Cambridge, 2004.
  3. G. Campbell and M. V. Ratnaparkhi, An application of Lomax distributions in receiver operating characteristic (ROC) curve analysis, Communication in Statistics 22 (1993), 1681-1697.
  4. D. D. Dorfman, K. S. Berbaum, C. E. Metz, R. V. Length, J. A. Hanlet, and H. A. Dagga, Proper receiver operating characteristic analysis: The bigamma model, Academic Radiology 4 (1996), 138-140.
  5. C. Feltz and R. Dykstra, Maximum likelihood estimation of the survival functions of N stochastically ordered random variables, J. Amer. Statist. Assoc. 80 (1985), no. 392, 1012-1019.
  6. P. A. Flach and S. Wu, Repairing concavities in ROC curves, Proceedings of 2003 UK Workshop on Computational Intelligence, 38-44, 2003.
  7. M. Grant, S. Boyd, and Y. Ye, CVX: Matlab software for disciplined convex programming, 2005; Available from
  8. S. Johansen, The product limit estimator as maximum likelihood estimator, Scand. J. Statist. 5 (1978), no. 4, 195-199.
  9. R. A. Johnson and D. W. Wichern, Applied Multivariate Statistical Analysis, Third edition. Prentice Hall, Inc., Englewood Cliffs, NJ, 1992.
  10. E. Kaplan and P. Meier, Nonparametric estimation from incomplete observations, J. Amer. Statist. Assoc. 53 (1958), 457-481.
  11. J. Lim, S.-J. Kim, and X. Wang, Estimation of stochastically ordered survival func- tions via geometric programming, To appear in Journal of Compuational and Graphical Statistics; Available from
  12. J. Lim, X. Wang, and W. Choi, Matimum likelihood estimation of ordered multinomial parameters by geometric programing, Compuational Statistics and Data Analysis 53 (2009), 889-893.
  13. C. Llyod, Regression models for convex ROC curves, Biometrics 56 (2000), 862-867.
  14. C. Llyod, Estimation of a convex ROC curve, Statist. Probab. Lett. 59 (2002), no. 1, 99-111.
  15. J. Lofberg, YALMIP. Yet Another LMI Parser, Version 2.4, 2003; Available from http: //
  16. M. W. McIntosh and M. S. Pepe, Combining several screening tests: optimality of the risk score, Biometrics 58 (2002), no. 3, 657-664.
  17. C. E. Metz and X. Pan, "Proper" binormal ROC curves: theory and maximum-likelihood estimation, J. Math. Psych. 43 (1999), no. 1, 1-33.
  18. A. Mutapcic, K. Koh, S. Kim, and S. Boyd, ggplab: A Matlab toolbox for geometric programming, 2006; Available from
  19. Y. Nesterov and A. Nemirovsky, Interior-Point Polynomial Methods in Convex Programming, volume 13 of Studies in Applied Mathematics, SIAM Philadelphia, PA, 1994.
  20. J. Norcedal and S. Wright, Numerical Optimization, Springer Series in Operations Research. Springer-Verlag, New York, 1999.
  21. X. Pan and C. E. Metz, The "proper" binomial model: Parametric receiver operating characteristic curve estimation with degenerate data, Academic Radiology 4 (1997), 380-389.
  22. M. S. Pepe, The Statistical Evaluation of Medical Tests for Classi cation and Prediction, Oxford University Press, Oxford, 2003.
  23. F. Preparata and S. J. Hong, Convex hulls of finite sets of points in two and three dimensions, Comm. ACM 20 (1977), no. 2, 87-93.
  24. J. Z. Wang, J. Li, R. M. Gray, and G. Wiederhold, Unsupervised multiresolution segmentation for images with low depth of field, IEEE Transaction on Pattern Analysis and Machine Intelligence 23 (2001), 85-90.
  25. C. Yim and A. C. Bovik, Multiresolution 3-D range segmentation using focused cues, IEEE Transaction on Image Processing 7 (1998), 1283-1299.