Advanced SearchSearch Tips
A study on bias effect of LASSO regression for model selection criteria
facebook(new window)  Pirnt(new window) E-mail(new window) Excel Download
 Title & Authors
A study on bias effect of LASSO regression for model selection criteria
Yu, Donghyeon;
  PDF(new window)
High dimensional data are frequently encountered in various fields where the number of variables is greater than the number of samples. It is usually necessary to select variables to estimate regression coefficients and avoid overfitting in high dimensional data. A penalized regression model simultaneously obtains variable selection and estimation of coefficients which makes them frequently used for high dimensional data. However, the penalized regression model also needs to select the optimal model by choosing a tuning parameter based on the model selection criterion. This study deals with the bias effect of LASSO regression for model selection criteria. We numerically describes the bias effect to the model selection criteria and apply the proposed correction to the identification of biomarkers for lung cancer based on gene expression data.
LASSO;penalized regression;bias;model selection;information criterion;
 Cited by
Akaike, H. (1973). Information theory and an extension of the maximum likelihood principle. In Proceeding of the Second International Symposium on Information Theory, Budapest: Akademiai Kiado.

Andersen, P. and Gill, R. (1982). Cox's regression model for counting processes, a large sample study, Annals of Statistics, 10, 1100-1120. crossref(new window)

Benjamini, Y. and Hochberg, Y. (1995). Controlling the false discovery rate: a practical and powerful approach to multiple testing, Journal of the Royal Statistical Society. Series B (Methodological), 57, 289-300.

Belloni, A. and Chernozhukov, V. (2013). Least squares after model selection in high-dimensional sparse models, Bernoulli, 19, 521-547. crossref(new window)

Cantero-Recasens, G., Fandos, C., Rubio-Moscardo, F., Valverde, M. A., and Vicente, R. (2010). The asthma-associated ORMDL3 gene product regulates endoplasmic reticulum-mediated calcium signaling and cellular stress, Human Molecular Genetics, 19, 111-121. crossref(new window)

Danaher, P., Wang, P., and Witten, D. M. (2014). The joint graphical lasso for inverse covariance estimation across multiple classes, Journal of the Royal Statistical Society. Series B (Methodological), 76, 373-397. crossref(new window)

Demmel, J. W. (1997). Applied Numerical Linear Algebra, SIAM, Philadelphia.

Efron, B., Hastie, T., Johnstone, I., and Tibshirani, R. (2004). Least angle regression, Annals of Statistics, 32, 407-499. crossref(new window)

El-Telbany, A. and Ma, P. C. (2012). Cancer genes in lung cancer: racial disparities: are there any?, Genes & Cancer, 3, 467-480. crossref(new window)

Fan, J. and Li, R. (2001). Variable selection via nonconcave penalized likelihood and its oracle properties, Journal of the American Statistical Association, 96, 1348-1360. crossref(new window)

Fan, Y. and Tang, C. Y. (2013). Tuning parameter selection in high dimensional penalized likelihood, Journal of the Royal Statistical Society. Series B (Methodological), 75, 531-552. crossref(new window)

Friedman, J., Hastie, T., Ho ing, H., and Tibshirani, R. (2007). Pathwise coordinate optimization, Annals of Applied Statistics, 1, 302-332. crossref(new window)

Friedman, J., Hastie, T., and Tibshirani, R. (2008). Sparse inverse covariance estimation with the graphical lasso, Biostatistics, 9, 432-441. crossref(new window)

Irizarry, R. A., Bolstad, B. M., Collin, F., Cope, L. M., Hobbs, B., and Speed, T. P. (2003). Summaries of Affymetrix GeneChip probe level data, Nucleic Acids Research, 31, e15. crossref(new window)

Jemal, A., Siegel, R., Xu, J., and Ward, E. (2010). Cancer statistics, CA: A Cancer Journal for Clinicians, 60, 277-300.

Khare, K., Oh, S.-Y., and Rajaratnam, B. (2015). A convex pseudolikelihood framework for high dimensional partial correlation estimation with convergence guarantees, Journal of the Royal Statistical Society. Series B (Methodological), 77, 803-825. crossref(new window)

Meinshausen, N. and Buhlmann, P. (2006). High-dimensional graphs and variable selection with the lasso, Annals of Statistics, 34, 1436-1462. crossref(new window)

Nishii, R. (1984). Asymptotic properties of criteria for selection of variables in multiple regression, Annals of Statistics, 12, 758-765. crossref(new window)

Paige, C. C. and Saunders, M. A. (1975). Solution of sparse indefinite systems of linear equations, SIAM Journal on Numerical Analysis, 12, 617-629. crossref(new window)

Peng, J., Wang, P., Zhou, N., and Zhu, J. (2009). Partial correlation estimation by Joint sparse regression models, Journal of the American Statistical Association, 104, 735-746. crossref(new window)

Picard, R. R. and Cook, R. D. (1984). Cross-validation of regression models, Journal of the American Statistical Association, 79, 575-583. crossref(new window)

Pounds, S. and Morris, S. W. (2003). Estimating the occurrence of false positives and false negatives in microarray studies by approximating and partitioning the empirical distribution of p-values, Bioinformatics, 19, 1236-1242. crossref(new window)

Schwarz, G. (1978). Estimating the dimension of a model, Annals of Statistics, 6, 461-464. crossref(new window)

Shao, J. (1997). An asymptotic theory for linear model selection, Statistica Sinica, 7, 221-264.

Sterrenberg, J. N., Blatch, G. L., and Edkins, A. L. (2011). Human DNAJ in cancer and stem cells, Cancer Letters, 312, 129-142. crossref(new window)

Tang, H., Xiao, G., Behrens, C., Schiller, J., Allen, J., Chow, C. W., Suraokar, M., Corvalan, A., Mao, J., White, M. A., Wistuba, I., Minna, J. D., and Xie, Y. (2013). A 12-gene set predicts survival benefits from adjuvant chemotherapy in non-small cell lung cancer patients, Clinical Cancer Research, 19, 1577-1586. crossref(new window)

Tibshirani, R. (1996). Regression shrinkage and selection via the lasso, Journal of the Royal Statistical Society. Series B (Methodological), 58, 267-288.

Tibshirani, R. J. (2013). The lasso problem and uniqueness, Electronic Journal of Statistics, 7, 1456-1490. crossref(new window)

Tomida, S., Takeuchi, T., Shimada, Y., Arima, C., Matsuo, K., Mitsudomi, T., Yatabe, Y., and Takahashi, T. (2009). Relapse-related molecular signature in lung adenocarcinomas identifies patients with dismal prognosis, Journal of Clinical Oncology, 27, 2793-2799. crossref(new window)

Wang, H., Li, B., and Leng, C. (2009). Shrinkage tuning parameter selection with a diverging number of parameters, Journal of the Royal Statistical Society. Series B (Methodological), 71, 671-683. crossref(new window)

Wang, H., Li, R., and Tsai, C. L. (2007). Tuning parameter selectors for the smoothly clipped absolute deviation method, Biometrika, 94, 553-568. crossref(new window)

Wang, T. and Zhu, L. (2011). Consistent tuning parameter selection in high dimensional sparse linear regression, Journal of Multivariate Analysis, 102, 1141-1151. crossref(new window)

Yang, Y. (2005). Can the strengths of aic and bic be shared?: a conflict between model identification and regression estimation, Biometrika, 92, 937-950. crossref(new window)

Yoo, J., Lee, S.-H., Lym, K., Park, S. Y., Yang, S.-H., Yoo, C.-Y., Jung, J.-H., Kang, S.-J., and Kang, C.-S. (2012). Immunohistochemical expression of DCUN1D1 in non-small cell lung carcinoma: its relation to brain metastasis, Cancer Research and Treatment: Official Journal of Korean Cancer Association, 44, 57-62.

Yu, D., Son, W., Lim, J., and Xiao, G. (2015). Statistical completion of partially identified graph with application to estimation of gene regulatory network, Biostatistics, 16, 670-685. crossref(new window)

Zhang, C.-H. (2010). Nearly unbiased variable selection under minimax concave penalty, Annals of Statistics, 38, 894-942. crossref(new window)

Zhang, C.-H. and Huang, J. (2008). The sparsity and bias of the lasso selection in high-dimensional linear regression, Annals of Statistics, 36, 1567-1594. crossref(new window)

Zou, H. (2006). The adaptive lasso and its oracle properties, Journal of the American Statistical Association, 101, 1418-1429. crossref(new window)