DOI QR코드

DOI QR Code

Variable Selection Via Penalized Regression

  • Published : 2005.12.01

Abstract

In this paper, we review the variable-selection properties of LASSO and SCAD in penalized regression. To improve the weakness of SCAD for high noise level, we propose a new penalty function called MSCAD which relaxes the unbiasedness condition of SCAD. In order to compare MSCAD with LASSO and SCAD, comparative studies are performed on simulated datasets and also on a real dataset. The performances of penalized regression methods are compared in terms of relative model error and the estimates of coefficients. The results of experiments show that the performance of MSCAD is between those of LASSO and SCAD as expected.

Keywords

References

  1. Fan, J. and Li, R. (2001), Variable selection via nonconcave penalized likelihood and its oracle properties, Journal of the American Statistical Association, 96, pp. 1348-1360 https://doi.org/10.1198/016214501753382273
  2. Friedman, J. (2001), Greedy function approximation: a gradient boosting machine, Annals of Statistics, 29, pp. 1189-1232
  3. Hastie, T., Tibshirani, R. and Friedman, J.H. (2001), Elements of Statistical Learning. Springer- Verlag, New York
  4. Hoed, A.E. and Kennard, R. (1970). Ridge regression: biased estimation for nonorthogonal problems, Technometrics, 12, pp. 55-67 https://doi.org/10.2307/1267351
  5. Mason, L., Baxter, J., Bartlett, PL. and Frean, M. (2000). Functional gradient techniques for combining hypotheses, In A. J. Smola, P.L. Bartlett, B. Scholkopf and D. Schuurmans, editors, Advances in Large Margin Classifiers, Cambridge: MIT press
  6. Stamey, T., Kabalin, J, McNeal, J., Johnstone, I., Freiha, F., Redwine, E. and Yang, N. (1989). Prostate specific antigen in the diagnostics and treatment of adenocarcinoma of the prostate II. radical prostatectomy treated patients, Journal of Urology, 16, pp. 1076-1083
  7. Tibshirani, R. (1996). Regression shrinkage and selection via the lasso, Journal of Royal Statistical Society B, 58, pp, 267-288