Communications for Statistical Applications and Methods

The Korean Statistical Society (한국통계학회)
권호 표지
DOI QR코드

DOI QR Code

Variable Selection in Sliced Inverse Regression Using Generalized Eigenvalue Problem with Penalties

  • Published : 2007.04.30
Download PDF
⟨ Previous Next ⟩

Abstract

Variable selection algorithm for Sliced Inverse Regression using penalty function is proposed. We noted SIR models can be expressed as generalized eigenvalue decompositions and incorporated penalty functions on them. We found from small simulation that the HARD penalty function seems to be the best in preserving original directions compared with other well-known penalty functions. Also it turned out to be effective in forcing coefficient estimates zero for irrelevant predictors in regression analysis. Results from illustrative examples of simulated and real data sets will be provided.

Keywords

References

  1. Aarts, E. and Korst, J. (1989). Simulated Annealing and Boltzmasui Machines. John Wiley & Sons
  2. Breiman, L. (1995). Better subset regression using the nonnegative garrote. Technometrics, 37, 373-384 https://doi.org/10.2307/1269730
  3. Cadima, J. and Jolliffe, I. T. (1995). Loadings and correlations in the interpretation of principal components. Journal of Applied Statistics, 22, 203-214 https://doi.org/10.1080/757584614
  4. Fan, J. and Li, R. (2001). Variable Selection via Nonconcave Penalized Likelihood and its Oracle Properties. Journal of The American Statistical Association, 96, 1348-1360 https://doi.org/10.1198/016214501753382273
  5. Hausman, R. E. Jr. (1982). Constrained multivariate analysis. Optimisation in Statistics (Zanckis, S. H. and Rustagi, J. S., eds.), 137-151, North-Holland: Amsterdam
  6. Jolliffe, I. T. (1972). Discarding variables in a principal component analysis. I: artificial data. Applied Statistics, 21, 160-173 https://doi.org/10.2307/2346488
  7. Jolliffe, I. T. (1973). Discarding variables in a principal component analysis. ii: real data. Applied Statistics, 22, 21-31 https://doi.org/10.2307/2346300
  8. Jolliffe, I. T. (1989). Rotation of Ill-defined principal components. Applied Statistics, 38, 139-147 https://doi.org/10.2307/2347688
  9. Jolliffe, I. T. (1995). Rotation of principal components: choice of normalization constraints. Journal of Applied Statistics, 22, 29-35 https://doi.org/10.1080/757584395
  10. Jolliffe, I. T., Trendafilov, N. T. and Uddin, M. (2003). A modified principal component technique based on the Lasso. Journal of Computational and Graphical Statistics, 12, 531-547 https://doi.org/10.1198/1061860032148
  11. Kirkpatrick, S., Gelatt, C. D. Jr. and Vecchi, M. P. (1983). Optimization by simulated annealing. Science, 220, 671-680 https://doi.org/10.1126/science.220.4598.671
  12. Li, K. C. (1991). Sliced inverse regression for dimension reduction. Journal of The American Statistical Association, 86, 316-342 https://doi.org/10.2307/2290563
  13. Li, K. C. (2000). High dimensional data analysis via the SIR/PHD approach. unpublished manuscript
  14. McCabe, G. P. (1984). Principal variables. Technometrics, 26, 137-144 https://doi.org/10.2307/1268108
  15. Tibshirani, R. (1996). Regression shrinkage and selection via the Lasso. Journal of the Royal Statistical Society, Ser. B, 58, 267-288
  16. Vines, S. K. (2000). Simple principal components. Applied Statistics, 49, 441-451