On Information Criteria in Linear Regression Model

  • Published : 2009.02.28


In the model selection problem, the main objective is to choose the true model from a manageable set of candidate models. An information criterion gauges the validity of a statistical model and judges the balance between goodness-of-fit and parsimony; "how well observed values ran approximate to the true values" and "how much information can be explained by the lower dimensional model" In this study, we introduce some information criteria modified from the Akaike Information Criterion (AIC) and the Bayesian Information Criterion(BIC). The information criteria considered in this study are compared via simulation studies and real application.


Linear regression;information criterion;model selection


  1. Akaike, H. (1973). Information theory and an extension of the maximum likelihood principle, Second International Symposium on Information Theory, 267-281
  2. Akaike, H. (1980). Likelihood and the Bayes procedure, Bayesian Statistics, University Press, Valencia Spain
  3. Belsley, D. A., Kuh, E. and Welsch, R. E. (1980). Regression Diagnostics: Identifying Influential Data and Sources of Collinearity, John Wiley & Sons, New York
  4. Breirnan, L., Friedman, J. H., Olshen, R. A. and Stone, C. (1984). Classification and Regression Trees, Chapman & Hall/CRC, New York
  5. Cavanaugh, J. E. (1999). A large-sample model selection criterion based on Kullback's symmetric divergence, Statistics and Probability Letters, 42, 333-343
  6. Foster, D. P. and George, E. I. (1994), The risk information criterion for multiple regression, The Annals of Statistics, 22, 1947-1975
  7. George, E. I. and Foster, D. P. (2000). Calibration and empirical Bayes variable selection, Biometrika, 87, 731-747
  8. Hurvich, C. M. and Tsai, C. L. (1989). Regression and time series model selection in small samples, Biometrika, 76, 297-307
  9. Kullback, S. and Liebler, R. A. (1951). On information and sufficiency, The Annals of Mathematical Statistics, 22, 79-86
  10. Rahman, M. S, and King, M. L. (1997). Probability of correct identification in model selection, In Proceedings of the Econometric Society Australasian Meeting, 631-640, University of Melbourne, Melbourne
  11. Rahman, M. S. and King, M. L. (1999). Improved model selection criterion, Communications in Statistics-Simulation and Computation, 28, 51-71
  12. Schwarz, G. (1978). Estimating the dimension of a model, The Annals of Statistics, 6, 461-464
  13. Theil, H. (1961). Economic Forecasts and Policy, (Second Ed.), North-Holland, Amsterdam
  14. Tibshirani, R. and Knight, K. (1999). The covariance inflation criterion for adaptive model selection, Journal of the Royal Statistical Society, Series B, 61, 529-546
  15. Wei, C. Z. (1992). On predictive least squares principles, The Annals of Statistics, 20, 1-42
  16. Wu, T. J. and Sepulveda, A. (1998). The weighted average information criterion for order selection in time series and regression models, Statistics and Probability Letters, 39, 1-10