Communications for Statistical Applications and Methods

The Korean Statistical Society (한국통계학회)
권호 표지
DOI QR코드

DOI QR Code

Ensemble approach for improving prediction in kernel regression and classification

  • Han, Sunwoo (Department of Applied Statistics, Yonsei University) ;
  • Hwang, Seongyun (Department of Statistics, Hankuk University of Foreign Studies) ;
  • Lee, Seokho (Department of Statistics, Hankuk University of Foreign Studies)
  • Received : 2016.06.25
  • Accepted : 2016.07.08
  • Published : 2016.07.31
Download PDF
⟨ Previous Next ⟩

Abstract

Ensemble methods often help increase prediction ability in various predictive models by combining multiple weak learners and reducing the variability of the final predictive model. In this work, we demonstrate that ensemble methods also enhance the accuracy of prediction under kernel ridge regression and kernel logistic regression classification. Here we apply bagging and random forests to two kernel-based predictive models; and present the procedure of how bagging and random forests can be embedded in kernel-based predictive models. Our proposals are tested under numerous synthetic and real datasets; subsequently, they are compared with plain kernel-based predictive models and their subsampling approach. Numerical studies demonstrate that ensemble approach outperforms plain kernel-based predictive models.

Keywords

References

  1. Breiman L (1996). Out-of-bag estimation, Technical report, Department of Statistics, University of California at Berkeley, CA, USA.
  2. Buhlmann P (2012). Bagging, boosting and ensemble methods. In Gentle JE, Hardle WK, Mori Y (Eds), Handbook of Computational Statistics (pp. 985-1022). Springer, Heidelberg.
  3. Han S (2016). A study on efficiency of kernel ridge regression using ensemble methods (Master's thesis), Hankuk University of Foreign Studies, Yongin, Korea.
  4. Hastie T, Tibshirani R, and Friedman J (2011). The Elements of Statistical Learning (2nd ed.), Springer, New York.
  5. Huh MH (2015). Kernel-trick regression and classification, Communications for Statistical Applications and Methods, 22, 201-207. https://doi.org/10.5351/CSAM.2015.22.2.201
  6. Hwang S (2016). A study on efficiency of kernel ridge logistic regression classification using ensemble method (Master's thesis), Hankuk University of Foreign Studies, Yongin, Korea.
  7. Lichman M (2013). UCI Machine Learning Repository [http://archive.ics.uci.edu/ml]. Irvine, CA: University of California, School of Information and Computer Science.
  8. Scholkopf B and Smola AJ (2002). Learning with Kernels, The MIT Press, Cambridge, MA.
  9. Zhou ZH (2012). Ensemble Methods: Foundations and Algorithms, CRC Press, Boca Raton, FL.